Domain Testing: Divide and Conquer - Testing Education

Perhaps this is in response to the "relationships" question (though in 2 of the 5 cases, it was
explicitly labeled as Part E (a2529s1) or part 5 (a2529s5)
Equivalence class table(s) that shows the complete equivalence class
and boundary value analysis for the function under test.
The equivalence class tables were similar across all subjects.
• All of the subjects included the enumerated variables in the equivalence class tables. This
makes little sense. The individual values are all "classes" of size 1. The risk descriptions
associated with these individual items are information-free, for example " Failure of Slides
sized for Letter" or " Failure to display the right slide" listed beside each possible value for
"Slide Sized For." This reflects a weakness in the instructional materials. Students were
encouraged to handle enumerated variables in exactly this way, up to and including risk
statements like "1) Failure to process option 1 correctly" and "1) Mishandling of values
outside the enumerated list of values" (Course materials, AppendixA.doc).
• The analyses for all of the other variables also marched through the table in lockstep,
virtually the same analysis by every tester, using almost the same order of cases, the same
verbose risk descriptions, etc. as each other and as the course notes.
It's hard to comment on this work.
If one individual drew this table and filled it in this way, I'd probably be impressed. The tables
are tidy and organized. The enumerated fields add noise, but if we ignore those, the analyses of
the integer and floating point fields make it appear as though the person is thinking creatively.
Whether or not these particular tests are likely to expose a bug is less interesting than the fact
that the tester is thinking beyond the literal wording of the problem.
The illusion evaporates when you see all of the answers together, and then review the course
notes. The two additional dimensions (number of characters and treatment of nondigits) are
prefabricated responses, laid out specifically in the course and applied mechanically. When I
look for other evidence of creativity, I see none. Here are a few examples of things that I would
expect from a sample of moderately experienced testers who claimed to have some experience
with domain testing:
• If I was considering the number of characters we could enter into a field, I'd paste several
variations of a huge number of characters, such as 65533 0's (or spaces) followed by a 1.
(If that yielded any response, I'd go to 2^32 characters or more.)
• I don't think it's unreasonable for the tester to make a working assumption that the
empirical limits on the variable (how many characters it accepted in testing) are the
intended limits, but I would expect the tester to raise the issue, recognize the uncertainty
involved, and try to check it or note that this should be checked.
• Even if the instructions hadn't instructed the tester to consider output variables, it is
common (and highly desirable) to check the result of a test. For example, when you set the
page size, you don't know whether it has been correctly set until you try to use the new