[Cuis] Canonical test cases?

[Juan Vuletich]

Hi Folks,

(below)
On 2/25/2015 12:22 AM, Phil (list) wrote:
> On Tue, 2015-02-24 at 17:51 -0800, Ken.Dickey wrote:
>> On Mon, 23 Feb 2015 14:43:26 -0500
>> "Phil (list)"<pbpublist at gmail.com>  wrote:
>>
>>> I started playing around with a couple of example test cases to see what
>>> I ran into and came up with a distinct class to store all of these per
>>> test category (i.e. under Test-Files a class ApiFile which could have a
>>> method testWriteTextFile)  The rationale was that it might make sense to
>>> keep these test cases separate from traditional test cases which are
>>> free to make calls that users of the class (i.e. who are calling the
>>> supported API) should not.
>> You bring up a good point.
>>
>> I suspect there are three basic cases:
>>    [1] A number of small test cases for a wide, shallow API.  E.g. Collection classes.  Many "small" usage tests.
>>    [2] A few deep calls which demonstrate a usage protocol.  E.g. open/read/close files, sockets; use library wrapper.
>>    [3] Sample/example UI code which illustrate how to build apps/applets/tools  [Color Picker, File List, ..]
>>
>> A test class, as you point out, is probably appropriate for case [1], where there would be a zillion test<collection><access>API kinds of things would swamp a name search.  A class naming convention would be good here.
>>
>> Individual test methods, with a searchable name convention, would be best for [2] where there would be few illustrative usage tests.
>>
>> We could call out example code in a README doc for [3] where examples might have auxiliary architectural documentation.
>>
> Agreed.
>
> 2 was what originally got me going on this subject as that's where my
> code seems to usually be breaking due to guessing wrong at what the
> longer term entry points were.  I wasn't envisioning a kitchen-sink
> approach to tests, just some basic 'here's the recommended way to use
> this area of functionality' tests.
>
> Unless Cuis evolves into a completely separate Smalltalk dialect, 1 can
> probably be handled via traditional unit tests / categorization and the
> numerous intro to Smalltalk tutorials out there.  3 seems like the point
> where written tutorials (along the lines of what Hannes pointed out
> earlier in the thread) and/or videos would best come into play.
>

I essentially agree with all you say. And obviously, I support any and 
all efforts to enhance Cuis (including docs, external tools, etc).

WRT using tests to document stable or standarized APIs, another reason 
for marking them or making them identifiable as such is managing change. 
When I change an API in Cuis, I update all senders in the image and core 
packages, including those in tests. So, I need to know that the test 
documents a standarized API, to avoid the change, or at least discuss it 
with you here, or move the old API to SqueakCompatibility.pck.st, etc.

Another alternative is to have some symbol, like #standardAPI or 
#stableAPI to be called from such methods. This makes it easy to spot 
them with 'senders', and it is obvious for anyone modifying them.

In any case, Phil, given that all this is not yet done, when you face 
such breakage, please don't just fix it yourself (unless you think 
that's best). Please use that frustration to open a specific discussion 
on that particular API. Ideally, we would agree on a reasonable API 
(maybe the old one, maybe a better new one), implement it, and document 
it. Just remember that we can and should discuss decisions.

Cheers,
Juan Vuletich