Using Automated Tests to Document Software Architectures - ASPE

Using Automated Tests to
Document Software Architectures
A WHITE PAPER PREPARED FOR ASPE TECHNOLOGY BY JERRY OVERTON,
SENIOR SOFTWARE ENGINEER, OBJECT COMPUTING, INC. (OCI)
www.aspetech.com toll-free: 877-800-5221

Using Automated Tests to
Document Software
Architectures
Introduction
by
Jerry Overton, Senior Software Engineer
Object Computing, Inc. (OCI)
The book Extreme Programming Explained, by Kent Beck,
suggests that the architecture of an application is better
documented using automated tests rather than detailed
specifications. The book briefly describes an example of
documenting the required processing capacity of an application
using tests. Improvements to the architecture of the application
are made by updating the software to pass the tests. The
example by Beck was straightforward, but it also lacked detail.
For more detailed software quality requirements, is it possible to
write tests that can verify that an architecture satisfies these
complex requirements? This article presents a case study
designed to explore the viability of documenting more complex
architectural requirements using automated tests.
The Case Study
The design of the case study for this article is taken from the
book Essential Software Architecture by Ian Gorton. In the book,
Gorton describes a design for the Information Capture and
Dissemination Environment (ICDE). The ICDE is an application
that automatically captures events as a result of various actions
of workstation users. The information captured is made available
to third-party tools interested in analyzing various aspects of
user behavior.
This case study is based on an actual system and should provide
the kind of complexity needed to better test the viability of

automated tests as software architecture documentation. This
article attempts to partially re-create the architectural solution
given in the book, but to do so using automated tests rather
than detailed specifications.
Augmenting Source Code
Documentation
When comparing real examples of automated tests to the
architectural solution described in Essential Software
Architecture, it becomes clear that source code (written in Java)
alone is not expressive enough to replace every aspect of an
architecture specification. Even for an experienced software
developer reading well-written code, it is often necessary to
have additional background and context information to
understand what the code does.
Java source code does not provide a good mechanism for
documenting context or background for concerns that span
classes located in several different source files. This
documentation could be created as comments in the source files,
but the information would have to be repeated for all applicable
files or placed in a single location. Repeating the same
documentation in multiple files makes the documentation much
more difficult to maintain and placing the documentation in one
place assumes that the reader will know where to find the
information.
In this article, UML models are used to document context
information that cannot be easily described using source code
alone. In keeping with the agile style of automated testing, all
models were created using Agile Modeling. Models were created
using the simplest tools possible and contain only the elements
necessary to communicate context. Once a model was created, it
was updated only when absolutely necessary. The models are
not meant to be exact representations of the code.
How to Read the Rest of this Article
The reaminder of this article is written as an architecture
description of the ICDE application. The next section gives an
overview of the entire architecture and each major section after
that describes the solution for a specific architecturally

significant scenario. All scenarios are introduced in the
architecture overview, but for brevity, only one scenario is
described in detail.
The fact that the Notify Event test class both extends JUnit
TestCase and implements an interface from the architecture may
look weird at first. The test was written according to the The
Self-Shunting Unit Test Pattern. In this pattern, tests
impersonate collaborators in order to find out things that only a
collaborator could know. This trick was very useful for testing
some of the more complex architectural requirements.
Start with the Architecture Overview to get an idea of the
elements introduced and why. Read the Event Notification
section for an explanation of how the elements are used to
satisfy the quality requirements of that scenario. The Conclusion
section summarizes the most important lessons learned by
working through this exercise. A .zip file containing the full
source code shown below is available for review.
Architecture Overview
The ICDE application records events generated by workstation
users. Information from users is stored in the ICDE application
and that information is available for query by third-party tools.
Third-party tools can register to receive notifications of particular
events. When those events occur, the system notifies registered
listeners.

The overall design for the application uses a combination of the
Presentation-Abstraction-Control Pattern (PAC) and the Observer
Pattern. The user's state is modeled after the observer pattern
and the third party tool is modeled after the PAC pattern. The
listener class is the intersection between the two patterns and
allows updates from the user to be communicated to third-party
tools.

Key to the architecture solution is the user state. The user state
is responsible for maintaining a list of listeners and their
associated interests. The user state updates listeners with new
information as the state changes. Listeners are allowed to poll
for all relevant updates or query for specific ones.
package userFramework;
public class AbstractUserState implements UserState{
private HashMap events = new HashMap();
private HashMap register = new HashMap();
public void update(int eventType, String event){
events.put(eventType, event);
}
public String poll(StateListener listener){
//return all events that the listener has
registered for
String message = "";
for (int i=0; i <
register.get(listener).length; i++){
message +=
events.get(register.get(listener)[i]);
}
return message;
}
public String query(int eventType){
//return the events that match the given type
return events.get(eventType);

events){
}
}
public void register(StateListener listener, int[]
}
register.put(listener, events);
For the third-party tool solution, the most architecturally
significant class is the listener. The listener is registered with the
user state and is responsible for keeping the user interface and
local user view of the third-party tool up to date.
package thirdPartyToolFramework;
import userFramework.UserState;
public class AbstractStateListener implements StateListener{
private UserState userState;
private UserView userView;
private UI ui;
view){
}
public AbstractStateListener
(UserState state, UI userInterface, UserView
userState = state;
ui = userInterface;
userView = view;
}
public void update(){
String state = userState.poll(this);
userView.update(state);
ui.setState(state);
}
Event Notification
When the user performs an action of interest on the workstation,
the user state is updated accordingly. All interested listeners are
responsible for polling for updates at regular intervals. If the
user state has been updated with messages of interest to the
listener, that information is returned to the listener when the
listener polls. After the listener has received and update, the
listener is responsible for updating the user interface of the
third-party tool.

The quality requirements and satisfying architecture for the
Event Notification scenario is documented by the NotifyEventTest
class.
package architectureRules;
public class NotifyEventTest extends TestCase implements
UserState {
...
}
Location Transparency
To encourage adoption by third-party developers, the ICDE API
has to support location transparency for event notification.
Third-party tools should not have to be coupled to a particular
application distribution, nor should they have to rely on specific
users for their updates. The system should allow the event
generation mechanisms to be swapped out without disruption to
the third-party application.
public class NotifyEventTest extends TestCase implements
UserState {
private NotifyEventTest servant;
private StateListener listener;
...
public void testLocationTransparency(){

user
NotifyEventTest();
NotifyEventTest();
//the architecture must allow replacement of
//states to be transparent to the listeners
//create two versions of the same service
NotifyEventTest servantA = new
NotifyEventTest servantB = new
//use the test as a proxy and configure it
with a servant.
//use the proxy to query data and poll for
data
servant = servantA;
String firstPoll = this.poll(listener);
String firstQuery =
this.query(UserState.DEFAULT_EVENT);
//run the same test with a different servant
//and compare the results
servant = servantB;
String secondPoll = this.poll(listener);
String secondQuery =
this.query(UserState.DEFAULT_EVENT);
}
assertTrue(firstPoll.equals(secondPoll));
assertTrue(firstQuery.equals(secondQuery));
...
events){}
}
public String poll(StateListener listener){
return "Poll Successful";
}
public String query(int eventType){
return "Query Successful";
}
public String proxyPoll(StateListener listener){
return servant.poll(listener);
}
public String proxyQuery(int eventType){
return servant.query(eventType);
}
public void register(StateListener listener, int[]
public void update(int eventType, String event){}
Performance
Event notifications should be fast. Once an event notification is
sent out, it should be received rapidly by interested third-party
tools. Any mechanism responsible for disseminating information
must do so without unnecessary delay. The event notification

mechanism should provide sub-second message delivery
performance.
public class NotifyEventTest extends TestCase implements
UserState {
private NotifyEventTest servant;
private StateListener listener;
private static long MAX_TIME = 100;
...
until
and
();
public void testPerformance(){
//event notifications have to be fast.
//measure the time from when an event occurs
//the notification is received by a subscriber
//make sure that the time elapse is acceptable
List listeners = new ArrayList
UserState userState = new AbstractUserState();
listeners.add(newRegisteredListener(userState));
long time = timeToUpdateAndNotifyEvent(userState,
listeners);
assertTrue(time < MAX_TIME);
}
...
userState){
private long timeToUpdateAndNotifyEvent
(UserState userState, List
listeners){
//register all listeners with the given state
for (StateListener listener : listeners)
registerListener(userState, listener);
//update the user state and measure how long it takes
//the listeners to get an update
return timeToUpdate(userState, listeners);
}
private StateListener newRegisteredListener(UserState
UI userInterface = new AbstractUI();
UserView userView = new AbstractUserStateView();
AbstractStateListener listener =
new AbstractStateListener
(userState, userInterface, userView);
registerListener(userState, listener);
return listener;
}
private void registerListener
(UserState userState, StateListener listener){

int[] events = {UserState.DEFAULT_EVENT};
userState.register(listener, events);
}
private long timeToUpdate
(UserState userState, List
listeners){
Event");
long startTime = System.currentTimeMillis();
userState.update(UserState.DEFAULT_EVENT, "Test
for (StateListener listener : listeners)
userState.poll(listener);
long endTime = System.currentTimeMillis();
return endTime - startTime;
}
...
}
Scalability
The ICDE system must be capable of scaling to up to 150
concurrent users. A successful architecture will incur minimal
performance degradation due to additional users. The
performance of the event notification mechanism must be
capable of keeping up with a growth in the number of users, up
to the anticipated maximum.
public class NotifyEventTest extends TestCase implements
UserState {
private NotifyEventTest servant;
private StateListener listener;
private static long MAX_TIME = 100;
private static int MAX_USERS = 150;
...
users
should
to the
();
public void testScalability(){
//event notification must scale to 100-150
//the performance of the notification design
//scale linearly with the number of users up
//maximum expected capacity.
List listeners = new ArrayList
UserState userState = new AbstractUserState();
for (int i=0; i < MAX_USERS; i++){
listeners.add(newRegisteredListener(userState));
}

long time = timeToUpdateAndNotifyEvent(userState,
listeners);
assertTrue(time < MAX_TIME * MAX_USERS);
}
...
}
Conclusions
With help from a few agile models and some prose, automated
tests were capable of documenting some fairly detailed software
architecture requirements like location transparency and
scalability. Although it remains to be seen whether or not tests
are sufficient for other software qualities, the approach seems
viable so far.
Using tests to document architectural requirements helped
reduce the tendency to over-engineer a solution. The best
solution is the simplest solution that passes the test. Any
complexity beyond that is unnecessary. The tests provide an
objective measure for minimum simplicity required. Tests also
make it obvious where the solution's risks are. Any tests failing
indicate qualities that are at risk.
Tests make it much easier to estimate the cost of deferring a
design decision. When architecture is documented using tests,
the cost of changing a design later will be roughly proportional to
the cost of the refactoring required to make the change. Without
tests, the true cost of a design change is hidden by how easy it
is to update models and descriptions.
The certainty that comes with documenting architecture using
tests comes with a price. Generally, it takes longer to write a
test (and code that could pass the test) for a software quality
than it does to create UML depictions and explanatory text.
However, it is not clear if time saved with less formal
documentation would result in an increase in re-work due to
missed requirements later.
Also, it is not clear to what limit this technique can be extended.
The example presented in this article is more detailed than the
example given in Extreme Programming Explained, but certainly

does not represent the most complex and detailed example
possible.
References
• Beck, K., Extreme Programming Explained, Second Edition,
Pearson Education Inc., NJ, 205
• Gortin, I. Essential Software Architecture, Springer-Verlag
Berlin Heidelberg, 2006
• Ambler, S. W., The Elements of UML 2.0 Style, Cambridge
University Press, New York, 2005