SEKE 2012 Proceedings - Knowledge Systems Institute

Algorithm 1 RegressionTesting
Input:
P : original program
T : test suite for P
E: a set of executions
C: a set of changed classes
Declare:
m: amethod
t: a test case
IS: a ranked list of potentially impacted methods
EXE: a set of methods executed by a test case
Use:
CIA(C): returns a ranked list of methods potentially impacted by the set of
changed classes C.
TCA(t): returns a set of methods covered by test case t.
Output:
A set of ordering test cases T ′
1: IS = CIA(C)
2: for each m in IS do
3: for each test case t in T do
4: if TCA(t) ∩ m == ∅ then
5: continue
6: end if
7: T ′ = T ′ ∪ t
8: T = T − t
9: EXE = TCA(t) ∩ IS
10: if EXE ≠ ∅ then
11: IS = IS − EXE
12: end if
13: end for
14: end for
15: return T ′
gorithm checks whether the test case t traverses m, if not,
continues to the next test case (Lines 4-6). If t traverses
m, this test case is added to the new test suite T ′ (Line 7).
Then, the methods in IS traversed by this test case are removed
(Lines 9-12) because they have been covered by this
test case. When all the methods in IS are processed, the w-
hole regression testing process is finished. And the new test
suite T ′ will traverse all the methods in the impact set, and
the test cases in T ′
are ordered according to the possibility
of the methods to be impacted. Thus, our approach directly
generates a new ordering test suite. In Algorithm 1, there
are two loops to scan the set of test cases and the impact set.
So the time to our approach is O(|T |×|M|), where |T | is
the size of the test suite, and |M| is the size of the impact
set. However, in most cases, the time of our approach is
less because the two loops may end at an earlier time when
some test cases can cover all the methods in the impact set.
We also give an example to exemplify our regression
testing approach. The test coverage information of the
above example program is assumed as Table 3. There are
six test cases to coverage these 10 methods. The √ in row
T and column M means that the test case can cover the
method. Then, we use the results in Table 2 and Table 3
to generate an ordered test suite. According to Algorithm
1, we should first select the test case which covers the
method with highest IF value in the new test suite. As
M2,M3,M5 have the highest IF value and the test case
T 1 covers these methods, we should first select T 1 into the
new test suite. Then we see which methods in the impact set
T 1
T 2
T 3
T 4
T 5
T 6
Table 3. Test coverage information
M1 M2
√
M3
√
M4 M5
√
M6
√
M7 M8 M9 M10
√
√ √ √ √ √ √
√ √ √
√
√
√ √ √ √
√
√
can be covered by this test case. As a result, these methods
{M2,M3,M5,M6,M10} are covered by T 1, and should
be removed from the impact set. At this time, we check the
method with the second highest IF value, we identify the
M1 method, and find T 2 can cover this method. Thus T 2
is added as the second test case in the new test suite. Similarly,
we again remove all the methods in the impact set
covered by this test case. After this, we select T 3 into the
new test suite, and find that there is no method in the impact
set. The whole regression testing process is finished. Ultimately,
we finish the process in three loops and three test
cases are generated in the new test suite, in which T 1 has
the highest prioritization to be run, then T 2, and followed
by T 3.
4 Initial Empirical Study
4.1 Setup
We use the Java Hierarchical Slicing and Applications
(JHSA) program as our research subject. JHSA is a tool
developed in our group, and used to construct hierarchical
dependence graphs from package level to statement level
[9]. We extracted six versions (V 0 to V 5 )ofJHSA from its
CV S repository, along with a test suite used to test the software.
There are about 13 classes, 121 methods for JHSA.
The test suite contains 72 test cases, which provide 80%
method coverage of the program. In addition, to perform
our study, we require fault data, and we utilized mutation
faults [3] to study the test case prioritization. The average
number of mutation faults of the program is 30.
In our study we use three measures to evaluate the proposed
regression testing approach. They are the percentage
of faults that the new test suite can identify (PF), the percentage
of test cases selected from original test suite (PTS)
and the weighted average of the percentage of faults detected
(AP F D) during the execution of the test suite, which is
defined as follows:
AP F D =1− TF 1+TF 2 +...+TF m
nm
+ 1
2n
In this Formula, n is the number of test cases, m is the
number of faults revealed by T , and TF i is the first test
case which reveals fault F i . The AP F D is a commonly
used metric to evaluate the regression test suite prioritization
[4]. And higher AP F D values imply better fault detection
rates.
455