SEKE 2012 Proceedings - Knowledge Systems Institute

B. Instrumentation
The experiment was supported by a set of artifacts: consent
forms, software product lin e specification, the SPLIT
documentation for the group that would execute them ,
worksheet to support defects and follow up questionnaire. The
software product line specification is composed by the
requirement specification, product map and feature model.
The software requirement artifact is a software product line
specification for a set of twitter client products with 13 features
from popular tw itter clients having 6 m andatory features, 3
optional features and 4 alternative features. It generates six
different products sharing the common features and each of
them having distinguished featur es. It describes each feature
for the software product line detailing its relationship to others
features, classifying it as mandatory, optional or alternative and
enumerating the products in which this feature would be
available.
The product map for the twitter client software product line
listed all the features available in softw are requirement
document and associates it to the product in which they are
available. It enumerates the six products from the twitter client
software product line and maps the features for each product.
The feature m odel for the software product line has been
created according to the softw are requirements for the tw itter
client and it would generate all the products, which are
specified in the product map. The feature model used in this
experiment has been seeded with defects by the researchers
based on deficiencies from feat ure models listed in [7]. It
contains defects from mode ling the feature model and
information inconsistencies from the softw are requirements
and the product map.
C. The Experiment Design
The subjects were divided in two groups, which would
inspect the same software pr oduct line model example: the
group A would evaluate using a defect type based inspection
approach and the group B would use the set of techniques
proposed in this paper. Sin ce no student had a previous
experience with SPL models, the subjects were assigned to
each technique using completely randomized design. Each
group was composed by 10 senior-level undergraduate students
chosen by convenience from Analysis and Design class in
Information System and Computer Science courses at Federal
University of Amazonas, Brazil.
D. Preparation
The subjects signed a consent form and they had a tutorial
about Software Product Line. It addressed an overview on
Software Product Line concepts and specifications including
feature models and product map. After the tutorial, the subjects
have been trained for the experiment execution. As the
classroom was divided in two groups, two different trainings
were prepared. The Group A has been trained in software
model inspection techniques. This training addressed the
objectives of model inspection for software product lines. It
described the defect types, which could be found by the
inspectors when conducting the inspection in this study. The
group B has been trained in the set of techniques proposed in
the SPLIT. The two trainings ha d different instructors due to
the fact that they occurred at the same time for avoiding
communication between the two groups. However both
materials have been prepared by the two instructors for having
balanced knowledge about softw are inspection for m itigating
the bias of prior knowledge of the subjects.
E. Execution
The experiment have been executed in a lim ited two hours’
time box and occurred at the same time for both groups. The
subjects were gathered in same room and then divided
randomly in groups. Group A and B have executed the
experiment in different rooms for avoiding any communication
between subjects from different groups.
V. RESULTS
At the end of the experim ent, the defect list form and the
evaluation questionnaire about the proposed set of techniques
were retrieved for the experim ent analysis. The num ber of
defects found per subject for each inspection is presen ted in
Table IV and Table V.
TABLE IV.
NUMBER OF DEFECTS FOUND IN DEFECT TYPE BASED
INSPECTION APPROACH
Subject
Number of Defects Found
Subject 01 4
Subject 02 9
Subject 03 7
Subject 04 8
Subject 05 4
Subject 06 9
Subject 07 1
Subject 08 3
Subject 09 7
Subject 10 7
TABLE V.
NUMBER OF DEFECTS FOUND USING SPLIT
Subject
Number of Defects Found
Subject 11 10
Subject 12 11
Subject 13 12
Subject 14 8
Subject 15 10
Subject 16 13
Subject 17 8
Subject 18 10
Subject 19 8
Subject 20 10
660