SEKE 2012 Proceedings - Knowledge Systems Institute

Fig. 5.
Single System Testing Tools Timeline
Fig. 6.
SPL Testing Tools Timeline
technical reports, white papers and manuals, in addition
to the tools executable.
• Research Questions. The questions defined could not
have covered the whole Testing Tools, which implies
that some one cannot find answers to the questions that
concern them. To mitigate this feasible threat, we had
several discussions with project members and experts in
the area, allowing the calibration of the question. Thus,
even if we had not selected the most adequate set of
questions, we attempted to address the most asked and
considered open issues in the field.
• Publication Bias. We cannot guarantee that all relevant
primary studies were selected. It is possible that relevant
studies were not chosen during the search process. We
mitigate this threat by following references in the primary
studies.
• Research Strings. The terms used in the research strings
can have many synonyms, and it is possible that some
work were overlooked. To mitigate this threat, we had
discussions with project members and experts in the area.
VII. CONCLUSION
This paper presented a mapping study on single system and
software product lines testing tools, whose main goal was to
identify how the available tools are supporting the SPL Testing
process. Through the review, it was possible to identify, which
current functionalities are being supported by the tools, and
which ones should be present in every tool based on their
priorities.
The research was conducted using techniques from mapping
study, a helpful approach for identifying the areas where there
is sufficient information for a systematic review to be effective,
as well as those areas where is necessary more research [12].
The great majority of the studies (60% of Solution Proposal)
were implemented to solve specific problems that avoid the
adaptability of the tools for the SPL context. For this reason,
it will be less costly create new tools than adapt the existing
ones.
Moreover, we identified an increasing number of automatic
testing case generation tools. As a result, one of the next
challenges will be learn how to automate this “generated” test
cases for the products of the SPL.
Finally, we noticed that publications describing industrial
experiences are rare in literature. The existing case studies
and experiments report only small projects, containing results
obtained from tools that solve specific problems related to
testing. Consequently, more experiments involving SPL testing
tools are needed.
As future work, we are incorporating the results of this
research in the development of a software product lines testing
tool.
ACKNOWLEDGMENT
This work was partially supported by the National Institute
of Science and Technology for Software Engineering
(INES), funded by CNPq and FACEPE, grants 573964/2008-
4 and APQ-1037-1.03/08 and CNPq grants 305968/2010-6,
559997/2010-8, 474766/2010-1.
REFERENCES
[1] F. J. van der Linden, K. Schmid, and E. Rommes, Software Product Lines
in Action: The Best Industrial Practice in Product Line Engineering.
Berlin: Springer, 2007.
633