SEKE 2012 Proceedings - Knowledge Systems Institute

(a) printtok (b) printtok2 (c) schedule
(d) replace (e) schedule2 (f) totinfo
(g) space
(h) tcas
Fig. 2.
We can find that our Partial-CATESR can at least gain the
same reduction extent as HGS performs. At most of the cases,
we can get more reductions. Within subject space, for instance,
original test suite are reduced by around 99 percentage using
our Partial-CATESR. In this case, the size of reduced test suite
is much smaller than reduced by around 93 percentage using
pure HGS approach. For those versions on which our Partial-
CATESR performs as well as pure HGS, we seriously check
both the source codes and their test cases. We find that all
test cases are classified in the same category, because either
they all have a strong confidence to cover the changes, or they
are insensitive to the changes through our CFG-based change
analysis.
Note that the performance of Full-CATESR, including reduction
extent and FDA, will be discussed at the end of this
subsection.
3) Fault Detection Ability Analysis: To answer RQ2, we
also check the FDA using the metric FDA as we have
discussed above. Similar to the visualization of reduction
extent, we summarize FDA for each subject over all faulty
The size of test suite reduction for each subject
versions in Figure 3.
Clearly, FDA among all our experiments are not weakened
at all, and even strengthened many times. For some faults in
many subjects, such as replace and space, the probabilities of
being detected, used to be below 0.1 using pure HGS approach,
are increased to 1 when adopting our Partial-CATESR.
4) Partial Coverage Affect Analysis: To answer RQ3, we
design the Full-CATESR, to assure a 100 percentage coverage
of original test requirements. As we can see in Figure 2, the reduction
extent are decreased due to additional test cases added
to RT S, which is generated by Partial-CATESR. However,
the FDA among all experiments are not enhanced at all (See
Figure 3). Our experiment results show that, it is meaningless
to re-execute test cases who have few relevance with the
changes during regression testing, though they may cover some
irrelevant test requirements. Therefore, we conclude that the
FDA will not be weakened by the insufficient coverage of
original test requirements.
222