SEKE 2012 Proceedings - Knowledge Systems Institute

B. Case study in a proprietary project
The tool was a lso used in a c ompany where capturing and
managing traceability is crucial. The tool was use d by an
engineering specialist who was in cha rge of evaluating
software artifacts and capturing traceability links to them.
Q1: The user was concerned with the am ount of overhead
needed to create the rules tha t will filter the traceability links.
Since the user may be switching between different projects, the
user may be v isiting artifacts that should not be traced to the
current project. Because the rules are currently expressed as
XSLT, it required m ore overhead than was acceptable to the
user. “I nee d an environm ent that allows me to m ake snap
decisions about individual pote ntial links, or whole groups of
links, quickly and on the fly.”
Q2: Centering links on the architecture was of limited value
because the organization has a requirem ents-centric approach
to traceability and it currently ha s no centralized architec ture
document for a project (i.e., the architecture is spread out over
multiple documents and different form ats). M oreover,
traceability capture starts before an architecture is created (e.g.,
among different levels of requirements document).
Q3: The user has expressed reservations with using ACTS,
because of its architecture-centric approac h and because of
usability issues, such as the background capture of links.
Again, because the user may be switching between diffe rent
projects and because the user wanted to ensure that traceability
links are inde ed captured, the user wanted a capability to
simply “point” to the artifact to capture t he traceability links,
instead of automatically capturing links in the background.
C. Case study in an open source project
Finally, to assess how the ACTS tool works in the context
of a gr oup of users, the tool wa s used to support a team of
developers adding functionality to the ArchStudio project. A
team of four undergraduate stude nts with i ndustry experience
(architect, programmer, technical support) were instructed to
capture links between the architecture and online refere nces
during the ten weeks of development. They were also provided
a pre-defined set of rules to use.
Q1: Three of the four students indicated that the time spent
in recording a nd using the pre-defined rules was accepta ble.
Using the same scale as in Table I, two students rated it a s 1
and one student rated it as 2.
Q2: The students indicated that linking to the architecture
was useful. Three students found it useful in the following
tasks: learning about the syste m, gathering requirements,
designing, implementing new features, and testing. Since t hey
coordinated their tasks am ong themselves, three students used
the traceability links to help answer questions from their
teammates. When asked how often they use d their traceability
links to perform these tasks, 3 students responded “Majority of
the time” while one responded “One fourth of the time”.
Q3: All the students stated that they would continue using
the tool. They pointed out that “[ t]he main hassle is the
installation and setup,” but once they learned how to us e the
tool, they found that it supported their development tasks. The
students commented, “Thanks. It’s very useful for me” and
“Excellent tool.”
D. Discussion
The case studies demonstrated that in development settings
where an a rchitecture is p redominantly used, the AC TS
technique was able to address difficulties from the three
perspectives, such as the cost in capturing links (with prespecified
rules), supporting traceability capture across different
tools, and inc reasing the motivation to perform traceability
tasks (indicated by the willingness to use the tool again).
Q1: There is a concern with regards to the overhead
involved in creating the rules, as expressed by the engineering
specialist in the proprietary project case study. Because XSLT
requires an initial learning curve, we previously planned that a
“traceability administrator” would be in charge of creating the
rules and simply providing them to the users, as we have done
in the lab usage trials and in the open s ource case study. It
turns out that there are users who would like to create their own
rules, and thus, making the rules access ible to the general user
is an important requirement to be addressed in future work.
During this study, we also encountered an une xpected
result—that users are willing to spend m ore time capturing
traceability links as long as they are in control of the
traceability task. In fact, some lab participants asked for the
ability to manually map artifacts to design via “drag and drop.”
One participant even sugges ted displaying a dialog box to
manually confirm the links to add after each recording session:
“I’d like to have more control over the links. I ’d like to have
checkboxes to m anually pick the one link I wanted.” One
participant also wanted to be able to manually enter labels for
the captured traceability links. T his finding is consistent with
our engineering specialist who works in a setting where it is
important not to have missing links. Thus, higher cost may be
acceptable to users as long as they have more control.
Q2: Our approach takes the assumption that there is an
architectural model whereby all the other artifacts can be
linked. The lab usage trial as well as open source case st udy,
where an architectural model exists, has demonstrated that
capturing traceability links work well. In fact, the tra ced
artifacts can support development tasks as demonstrated in the
open source project case study. However, in settings where
there is no dom inant architectural model, as in the propri etary
project case study, our approach provid es limited value. A
placeholder or “bare-bones” architectural model could be used
until a substantive model is developed.
Q3: Future tool usage was largely dependent on the user
experience during the trial usage and the context of usa ge.
Many of the lab users and the undergraduate team saw that they
could benefit from ACTS. T wo of the lab users, an architect
and a manager, opined that ACTS c ould be useful in the
context of thei r work: documents become more useful when
linked to the architecture and traceability is useful during
company audits. The undergraduate team was also able to save
time since they could orga nize their refer ences around the
architectural model. We also see that if ACTS does not match
the context of usage, as in th e proprietary project, it is unlikely
to be used in that setting.
416