Domain Testing: Divide and Conquer - Testing Education

7.01.02 Minuses
Kaner (2003) also commented on the things the students failed at in the
performance test.
the students:
The students were also remarkably consistent in what they missed.
Examples: (a) If you enter a sufficiently large paper size, PowerPoint warns
that ‘the current page size exceeds the printable area of the paper in the
printer.’ No one identified a boundary related to this message. (b) When
you change page dimensions, the display of slides on the screen and their
font sizes (defaults and existing text on existing slides) change. No one
appeared to notice this. And (c) change the page size to a large page and try
a print preview. You only see part of the page. (p. 4)
Bach mentions the following problem with some of the risks identified by
Some of the risks identified by the students indicate a lack of insight about
the nature of the technology under test. For instance, student #1 suggested
using characters that are on either side of the ASCII code range for numeric
digits. That kind of test made sense in 1985, but it seems unlikely that the
programmers of PowerPoint are still writing their own input filter routines
based on ASCII code signposts in an IF statement. Modern languages
provide more sophisticated libraries and constructs to perform filtering on
input. I think a much better test would be to paste in every possible
character code. This is fairly easy to do and would trigger many more kinds
of faults in the filtering code. (Appendix Y: Performance Tests’ Evaluation
Report by James Bach, p. 4)
McGee commented that the students failed to achieve higher-order learning:
I believe that the problem presented was somewhat different from the
simple cases presented in the training. The problem presented had four
inter-related controls which basically represented a 2-D space with special
115