SEKE 2012 Proceedings - Knowledge Systems Institute

• External Validity: W e have considered two main
threats concerned with generalization of the results: the
subjects being undergraduate students and the software
specification used in the e xperiment not being a real
world example. For any academic laboratory
experiment the ability to generalize results to industry
practice is restricted by th e usage of students as study
participants. Although they we re not real inspectors, it
has been shown that empirical studies can have
benefits when using student s as subjects [16]. And the
example used in the experiment was based in real
world twitter clients. T he specification of the tw itter
client software product line gathered features from
several twitter clients available in the market.
• Conclusion Validity: The m ain threat concerned
between the treatment and outcome used in the
experiment is the small sample of data points for the
statistical analysis. It is not ideal since it som etimes
lacks statistical representation of phenomenon but it is
a known problem difficult to overcome [17].
• Construct Validity: We have used the number of defect
found by the inspector for m easuring and com paring
the two different inspection types used in the
experiment. It is the comm only measurement used for
evaluating inspection techniques as presented in [3;
18].
VII. CONCLUSION
In this paper, we have proposed and validated a set of
inspection techniques (SPLIT) to evaluat e software product
line specification. It address th e needs due to single system
inspections are insufficient to address the specific
characteristics of reusable systems [4]. SPLIT is composed by
three techniques that address to find defects in feature mod els
and product map based on the Software Product Line
requirements.
We have conducted a formal experiment for comparing
SPLIT against a defect type based inspection approach using a
twitter client Softw are Product Line specification. The
experiment result showed that the number of defects found
when using SPLIT was greater than when executed a defect
type based inspection approach . Founding a greater number of
defects in early stages of development reduces costs and
improves software quality.
Future work for this research should include: an
improvement of SPLIT based on this experiment results and a
replication of the experiment in the industrial environment.
ACKNOWLEDGMENT
We thank all the undergraduate students for their
participation in the experiment. The authors acknowledge the
support granted by CNPq and FAPESP to the INCT-SEC
(National Institute of Science and Technology – Critical
Embedded Systems – Brazil), processes 573963/2008 -8 and
08/57870-9; FAPEAM through process PRONEX-023/2009;
and CAPES process AEX 4982/12-6. One of the authors of this
work was partially supported by the National Institute of
Science and Technology for Software Engineering (INES),
funded by CNPq and FACEPE, grants 573964/2008 -4 and
APQ-1037-1.03/08 and CNPq grants 305968/2010-6,
559997/2010-8, 474766/2010-1.
REFERENCES
[1] Weiss, D.M, Lai, C.T.R.: Software Product-Line Engineering. Addison-
Wesley,Reading (1999).J. Clerk Maxwell, A Treatise on Electricity and
Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68–73.
[2] Pohl, K., Böckle, G. and Van Der Linden, F. Software Product Line
Engineering – Foundations, Principles, and Techniques. Springer, Berlin
(2005).
[3] Travassos, G. H., Shull, F., Fredericks, M ., Basili, V.: Detecting defects
in object-oriented designs: using reading techniques to increase software
quality. ACM SIGPLAN Notices, vol. 34, n. 10, pp. 47-56. (1999)
[4] Denger, C; Kolb, R.: Testing and inspecting reusable product line
components: first empirical results. In Proceedings of the 2006
ACM/IEEE international symposium on Empirical software engineering
(ISESE '06). ACM, New York, NY, USA, 184-193. (2006)
[5] Clements, P., Northrop, L.: Software Product Lines: Practices and
Patterns. Addison-Wesley, Longman (2001).
[6] Kang, K.: Feature-oriented domain analysis (FODA) - feasibility study.
Technical Report CMU/SEI-90-TR-21, SEI/CMU, Pittsburgh (1990).
[7] Massen, T., Lichter, H.H.: Deficiencies in Feature Models. In:
Workshop on Software Variability M anagement for Product Derivation-
Towards Tool Support. (2004).
[8] Lee, J., Kang, S., Lee D.: A Comparison of Software Product Line
Scoping Approaches. In International Journal of Software Engineering
and Knowledge Engineering. Vol. 20, No. 5. 637-663 (2010)
[9] Bayer, J.; Flege, O.; Knauber , P.; Laqua, R.; Muthig, D.; Schmid, K.;
Widen, T.; DeBaud, J: PuLSE: a methodology to develop software
product lines. In Proceedings of the 1999 symposium on Software
reusability (SSR '99). ACM, New York, NY, USA, 122-131. (1999)
[10] Kitchenham, B., Budgen, D., Brereton, P.:The value of mapping studies
– A participant-observer case study. In: Proceedings of Evaluation and
Assessment of Software Engineerin g - EASE´2010, Keele, UK, v. 56,
pp.638-651. (2010)
[11] Cunha, R.; Conte, T.: SPL Models Quality Assurance – a M apping
Study. Technical Report USES-TR-2011-004. Available at:
www.dcc.ufam.edu.br/uses (2011)
[12] Cunha, R.; Conte, T.; Maldonado, J.,C.: Inspection Technique for
Software Product Line Models. Technical Report USES-TR-2011-005.
Available in: www.dcc.ufam.edu.br/uses. (2011)
[13] Basili, V., Rombach, H., “The TAM E Project: Towards Improvem ent-
Oriented Software Environments”, IEEE Transactions on Software
Engineering, 14, (1988).
[14] Dyba, T.; Kampenes, V.; Sjoberg, D. A Systematic Review of Statistical
Power in Software Engineering Experiments.Information and Software
Technology. Elsevier. (2005).
[15] Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B. Wesslén,
A.: Experimentation in Software Engineering – An Introduction. Kluwer
Academic Publishers.(2000).
[16] Carver, J., Jaccheri, L., Morasca, S., and Shull, F., “Issues in using
students in empirical studies in software engineering education”,
Proceedings Ninth International Software Metrics Symposium, 3-5 Sept.
(2003).
[17] Conte, T., M assollar, J., M endes, E., Travassos, G.H.: Usability
Evaluation Based on WebDesign Perspectives. In: International
Symposium on Empirical Software Engineering and Measurement
(ESEM) Madrid, Spain. (2007).
[18] Shull, F., Rus, I., e Basili, V.R.: How Perspective-Based Reading Can
Improve Requirements Inspections. IEEE Computer, 33(7): 73-79.
(2000).
662