Discussions on Accessibility in Industrial Automation Systems

<strong>Discussions</strong> on Accessibility in Industrial
Automation Systems
H. Vieritz*, F. Yazdi**, N. Jazdi**, D. Schilberg*, S. Jeschke*, and P. Göhner**
* RWTH Aachen University / ZLW-IMA, Aachen, Germany
** University of Stuttgart / IAS, Stuttgart, Germany
* {vieritz, daniel.schilberg, sabina.jeschke}@ zlw-ima.rwth-aachen.de
**{farzan.yazdi, nasser.jazdi, peter.goehner}@ ias.uni-stuttgart.de
Abstract— Importance of industrial automation systems [1],
also known as technical devices, has been a growing area
during the past decades. Web applications from one side
and industrial automation systems from the other side have
become a standard part of people’s daily life and more user
groups have to deal with them. Hence they must be
accessible to all users. Unfortunately, often by the
development of such systems, certain user groups are being
neglected. Therefore, a systematic concept is required to
support the development of industrial automation systems
and Web applications. In this paper, we will present the
concept, which is proposed in the context of a research
project funded by Deutsche Forschungsgemeinschaft
(DFG) a . We will discuss the suited methods to effectively
assess accessibility requirements.
I. INTRODUCTION
Pervasive computing as a growing phenomenon has
attracted many companies and manufacturers.
Progressively more industrial automation systems are
becoming a typical part of people’s daily life. On the other
hand, assessing accessibility to industrial automation
systems is an essential requirement, if they are to be used
by all members of the society. While accessibility has
been more considered in Web applications, industrial
automation systems have been less considered in this
respect. Nowadays in many cases, Web applications are
being combined into industrial automation systems. Many
Web-based technologies such as HTML or XML are so
common that are often being adopted for industrial
automation systems; e.g. for presentation of the user
interfaces (UIs) or in Automotive Open System Architecture
(AUTOSAR).
Clearly, all users should have free access to the
industrial automation systems, including those with
disabilities. Unfortunately, often certain user groups are
being neglected by the development. The needs of users
vary strongly, depending on their capabilities and
expectations etc. This causes the development process
very complex. Therefore, integration of accessibility
requirements into the development process is cumbersome.
User-centered Design (UCD) [2] is strongly
recommended for integrating none-functional requirements,
in our case accessibility [3], into the development
process, since none-functional requirements can strongly
affect the system architecture [4]. Hence, it is reasonable
to consider them early enough before the architecture has
been designed. The integration of UCD in Model-driven
a We highly appreciate Deutsche Forschungsgemeinschaft (DFG) for
funding this project.
Engineering (MDE) provides a formalized development
process.
For better understanding, we will explain the proposed
concept with an example. Our example is imaginary and
should only serve as a support throughout this paper.
Furthermore, with this example, we can better show the
application of the concept in real life. Our example
considers shopping in a futuristic supermarket by visually
impaired customers. Typically, supermarkets are not
accessible for visually impaired customers without the
support of another person. Non-visual guidance to
particular departments and items is missing. Apart from
degradation of the quality of life for visually impaired
customers, extra personal effort is required for helping
such customers around. These aspects are investigated in a
project running between RWTH Aachen University and
University of Stuttgart.
The example considers a supermarket with various
items available for the customers to buy and the
consulting service in the context of a supermarket.
Considering these two factors in the context of shopping,
the accessibility (see section Accessibility) of a pervasive
system refers to free access to the products and services
(considering its intended usage), for all users including
those with disabilities; i.e. all customers should be able to
access info materials or products in the supermarket and
obtain the offered services.
The proposed concept is UCD-based. Moreover, it
relies on MDE approaches. For more simplicity, we have
limited ourselves to the UI of an intelligent system that
helps guiding the visually impaired customers throughout
the supermarket and provide him with his/her required
information.
In this article, first we address basics of accessibility
and UCD, which are the basics for our concept. Furthermore,
we will demonstrate our conception using an
example. In the final section, we will provide a conclusion
and outlook in the field of accessibility development of
industrial automation systems.
II. RELATED RESEARCH
Along with MDE instructions and for the purpose of
integrating accessibility into the UI, the UI must be
modeled. However, modeling UI with conventional
methods, e.g. UML is not possible. The existing
approaches for modeling UI can be grouped into two
categories: 1) practical approaches, e.g. User Interface
Markup Language (UIML), User Interface Description
Language (UIDL), or Extensible Application Markup
Language (XAML) and 2) analytical approaches, e.g.
User Interface Extensible Markup Language (UsiXML) or

the Unified Modeling Language for Interactive
Applications (UMLi).
Practical approaches are focused on a usable platformindependent
development of UIs including development
tools. Recent implementations such as Silverlight or Flex
provide the separation of UI layout from UI behavior as
well as from system core functionalities. Whereas
analytical approaches are more abstract and includes a
meta model for all UI aspects, which describes UI
structure, components, behavior, navigation, domain, and
resource. However, the analytical approaches are still
under research and there is lack of tools for actual
development. The existing analytical approaches for the
modeling of UIs, e.g. useML [5], UMLi, Task Modeling
Language (TaskML) and Dialog Modeling Language
(DiaMODL) integrate the requirements of the modeling
process including analysis.
Nevertheless, rarely modeling solutions address
complete and systematic integration of accessibility in
model-driven design and more importantly in the
development process [6, 7, 8].
III.
BASICS
A. Accessibility
Accessibility in UI design means free perception and
control of all relevant information for all users, also for
those with disabilities. Here, the information refers to
those being needed by the user to follow his intended
workflow constituted by his activities, actions and their
relationships.
Industrial automation systems increasingly contain
more functionality; hence, they are getting more complex
in usage. Often not all functionalities are needed by all
users (or even should not be accessible) or some of the
users are being neglected in the development, e.g. modern
cell phones are mostly inappropriate for elderly. To
overcome these issues, UCD is often claimed to better
meet the requirements of usability and accessibility [3].
The existing accessibility solutions are focused on
accessibility guidelines, e.g. on the Web Content
Accessibility Guidelines (WCAG) [9]. Unfortunately, they
mostly aim at run-time behavior of the system, i.e. how
the system should work but not how it should be
developed. Moreover, they are specifically created for
Web applications. Whereas, industrial automation systems
have similar accessibility requirements as Web applications,
as long as human interaction with UI is concerned.
Fortunately, the guidelines can be adopted for industrial
automation systems and UCD is a promising approach for
the UI development of industrial automation systems.
Despite the importance of accessibility, it is still a big
challenge integrating it to the industrial automation
systems, due to numerous reasons. Few important features
are:
• Variety of the requirements of different user groups
• Lack of a systematic UCD-based development
process
• Limiting UI technologies e.g. cell phones with small
GUIs
• Difference between developer’s mindset with that of
the user
Guidelines issuing accessibility for UI, such as the
WCAG of the World Wide Web Consortium (W3C),
describe requirements of accessible Web content. They
are combined with other recommendations for user agents,
authoring tools and Rich Internet Applications (RIA) [10].
Although they are initially intended for Web applications,
there have been successful attempts in adopting these
guidelines for industrial automation systems. An example
for such accessibility guidelines, which has been adopted
from Web accessibility guidelines, is W3C extension of
the WCAG to fulfill the requirements of the elderly using
cell phones. The success in such attempt lies in the fact
that many requirements are equal or similar in both
systems. Moreover, as mentioned earlier, industrial
automation systems and Web applications are being often
combined together. Further examples of accessibility
guidelines are guidelines from IBM or Microsoft for
desktop applications.
B. User-centered Design
User-centered design is another important study field,
which crosses accessibility in many respects. It is a
development philosophy, which tries to integrate the end
users into the development process. User requirements are
integrated in the early stages of the development process.
As mentioned in the introduction, accessibility can
strongly impact system architecture; thus early
considerations can avoid late changes and additional costs.
Roughly speaking, system design, definition of
functionalities and UI design are parallelized in UCD.
Using an appropriate architecture pattern supports this
synchronization. The basic idea is to separate UI layout
and control from system core functionalities. This may be
realized by different alternatives, including the separation
of front- and backend, a client-server concept, the Modelview-control
(MVC) architecture pattern, or three-tier
architecture. We have decided upon MVC architecture
pattern for modeling and implementation, since it allows
focusing on the UI without any interference with the
system logic.
For integration of UCD process into the ordinary
development process, it is required to be able to translate
the results of each phase in a form that can be easily
merged with the development process. For this purpose,
we have chosen Model-driven Development (MDD),
which is well suited to integrate these needs into the
development process, since semantic annotations can be
easily extracted from models. Also various generating
tools exist for this purpose. Fig. 1 shows the relation
between UCD and MDD.
IV. THE CONCEPT
In our concept we have chosen a set of existing
methods and standards to gather accessibility requirements
of industrial automation systems. While accessibility
can relate to physical design of the products or architectural
construction patterns, we focus on the software
side and consider only the interaction interface, i.e. UI.
Appreciative
Inquiry
Requirements,
Use Cases
Structuring Maps,
Workflow Analysis
Task Model
Participant
Observation
Dialog Model
Prototype
Evaluation
Presentation
Model
Figure 1: Process view for UCD and MDD of UIs
Evaluation
Implementation

We refer to this set of methods as Accessibility
Requirements in Design Approaches (ARDA). ARDA can
be divided into two parts: 1) ARDA for the development
process and 2) ARDA for the run-time behavior of the UI.
Fig. 2 shows an overview of the ARDA. ARDA for the
development process consist of:
• Workflow identification: Knowledge about user’s
workflow is necessary to determine the relevant information
in the UI.
• Relevant information: Analysis of relevant information
depending from tasks and workflow. Information
is relevant if necessary for workflow.
• Separation of functionality/usage: Separation of
data model/core-functionality and UI-presentation/
control enables multimodal concepts for usage and
different layouts for particular input/output devices.
• Architecture pattern: Use of well-suited architecture
pattern as MVC, Presentation-Abstraction-Control
(PAC), Model-View-ViewModel (MVVM) or
Model-View-Presenter (MVP), which supports the
separation of core functionality and UI control.
ARDA for the run-time behavior of the UI consist of:
• Text-based annotations: Employment of descriptive
annotations (e.g. text-based) on interface components
and structure.
• Multimodal interaction: Deployment of multimodal
interactions.
• Enriched UI components: Realization of communication
amongst interface components on their role,
state, properties and changes of their role, state, and
properties.
• Serial order: Design of interface components in a
serial order – corresponding to user’s workflow – to
support Assistive Technology (AT) such as screen
readers.
• Clear navigation: Structuring the interface navigation
distinctly and avoiding deep hierarchical structures,
in order to support sitemaps and breadcrumbs
for instance.
The complex requirements of accessibility are one of
the major obstacles for their integration in pervasive
technology. To overcome this problem the developer-user
gap needs to be resolved. The classical development
model – according to DIN 15226, consists of four development
phases; namely analysis, design, realization, and
test (see fig. 3). In this model, the users are being mainly
considered in the analysis phase.
We adapt classical product development to UCD by
investigating user-centered methodologies and methods
for each development phase. We focus on the
development methodology and software parts of the system.
Mechanical or metallurgical aspects of such products
are not addressed.
Numerous approached are available out there to
enhance accessibility for the end users. Such methods
typically address the run-time behavior of the system and
are not suitable for analysis and design. Moreover, new
approaches are often still too abstract to be applied in real
production. For instance, UCD is a process model, which
is introduced with the objective to help better integrate the
user and his functional/none-functional requirements into
the development. Yet, it does not provide a concrete and
systematic process, thus, it is not widely accepted by firms
and companies.
Fig. 4 depicts an overall view on our concept. As one of
the important focuses of our research, the humantechnology
relation affects the accessibility factor.
Technology as an instrument of human action is always
a medium but not a purpose of design. Accessibility is not
a simple factor in the design of UI. Beside the physical
disabilities affecting user interactions, accessibility crosses
more study fields such as mental models, which differ
from person to person, and cultural backgrounds (e.g.
language, symbols, values, etc.). Mental Model is a
concept of cognitive psychology, which was introduced
by P.N. Johnson-Laird in 1983. The concept is based on
the assumption that internal models of reality are a vital
characteristic of human thinking. They are based on daily
experience, learned knowledge and reasoning.
The system needs detailed information about the place
and category for the existing products. Moreover, other
meta-information can be related with the products in the
database e.g. category, vendor or nutrition facts. This
information is also used for navigating user to the product
or comparing products. We assume that the infrastructure
for the information system exists, e.g. a wireless
technology-based and Radio Frequency Identification
(RFID) system.
Clear Navigation
Text-based
Annotations
Architecture
Pattern
Workflow
Identification
Relevant
Information
Separation Functionality/Usage
Multimodal
Serial Order
Interaction
Figure 2: Overview of ARDA
Enriched UI
Components
Analysis Design Realization Test
User is in the
focus of intention
User is not in the focus of intention
Figure 3: Developer focus during development process
Figure 4: Overview of the concept

Customers can use applications on cellular phones e.g.
barcode scanners or available devices on the basket for
navigation and product information. System
implementation is focused on Web technology as
Hypertext Markup Language (HTML) and Java since they
are widely used in implementing UIs in industrial
automation systems.
A. Analysis
Classical analysis starts with market research to find
out about potential markets or the public requirements. In
this process, there are usually certain classic methods
being used for gathering requirements of the system, e.g.
stakeholder identification and interviews. This process
attempts to cover overall aspects of content and
functionality as well as none-functional requirements.
However, classic interviews between experts and users
can only give one side of the facts. We use the
Appreciative Inquiry [9]. It enables detailed analysis with
user integration when it may be difficult for the users to
describe their needs or problems. Appreciative Inquiry is a
4-phase organizational development methodology. The
four phases are: discovery, dream, design and destiny. It
engages individuals within an organization or institution
in its change and renewal. In a first step, well-working
processes are identified with appreciative interviews
(discover phase). During dream phase, processes are
planned and redesigned. Finally, the destiny phase serves
for implementation and execution of the new design.
Here, interactions of users with and without visual
impairment are considered. The requirements are
documented as use cases with diagrams (see fig. 5) and
textual descriptions.
The next step is the analysis of user’s activities related
to the documented use cases and afterwards the modeling
of an appropriate workflow. Integrative design must
respect different ways of doing things. Users have mental
representations and expectations of workflow to follow
their activities. These mental models of workflow are
analyzed with an easy-to-learn structuring technique.
Structuring technique allows users to describe their
imagination with words and drawings and is therefore a
well-suited method to obtain user’s mental model of
activities. The workflow analysis can be extended with
participant observation and structured interviews. Also,
interview and observation are the first choice methods to
analyze the mental models of visually impaired users.
B. Task Modeling
The modeling process starts with workflow description
the task modeling. Use cases are detailed with UML
activity diagrams and user’s input comes from the
structuring technique maps. User’s side of the interaction
is described with modes. A mode of usage is a particular
context of an activity. The machine context will not be
changed within one mode. Modes may have submodes
e.g. for auxiliary functions or navigation. But only one
main mode is active at a certain time and deep nesting of
modes is avoided to facilitate user’s interaction. Modes
are well-suited to encapsulate the Human-machine
Interaction (HMI) and identifying user’s modes is a good
preparation for the dialog modeling later-on. A UML
activity diagram is well-suited to model user navigation
from mode to mode. Fig. 6 shows the modes for the use
case Buy item.
At first, the user moves towards a particular product
department. In the department, he searches the product. If
he can find his product, then he will take it. Otherwise, he
will compare similar products and take a suited
replacement.
Next step is a detailed description/modeling of the
supported workflow. The modes are refined with user’s
basic actions. Actions are the smallest (atomic) activities
of the user which still have motive and intention.
Generally, they are not physical operations. Physical
operations – such as moving a mouse or pressing a key –
define the most precise level in the workflow. Here, they
are not part of task modeling to keep it independent from
single input/output devices. Fig. 7 shows the resulting
activity diagram for the use case Buy Item.
Move to Department
Categorize item
Take Item
[Item found]
Search Item
[Similiar Items
found]
Figure 6: Modes of the use case Buy item
[Shop open]
[Shop closed]
[Department known]
Take item
Compare Items
Compare items
Supermarket Guide
Move and observe
Move
Obtain item information
Search
item
Buy item
Pay
purchase
Evaluate
item
Categorize observed items
[Have different
category]
[Have same
category]
Make
complaint
Search in department
[Item found]
[Similiar
items found]
Figure 5: Use cases for the super market guide
Figure 7: Actions of the use case Buy item

The user observes his environment and categorizes the
products. When the products are similar to the searched
one he starts to search the product itself in the department.
Action modeling can be facilitated by classifying the
basic activities of the user. Action classification is
domain-specific. Reuther’s [5] classification of atomic use
objects for UIs of automation systems includes input, edit,
select, activate and inform. The modeling of workflow
enables the determination of relevant information about
the related objects. Later on, this information must be
completely included in the device-independent description
of HMI dialogs to realize device-independence and
therefore accessible HMI.
The user observes his environment and categorizes the
products. When the products are similar to the searched
one he starts to search the product itself in the department.
Action modeling can be facilitated by classifying the
basic activities of the user. Action classification is
domain-specific. Reuther’s [5] classification of atomic use
objects for UIs of automation systems includes input, edit,
select, activate and inform. The modeling of workflow
enables the determination of relevant information about
the related objects. Later on, this information must be
completely included in the device-independent description
of HMI dialogs to realize device-independence and
therefore accessible HMI.
C. Dialog modeling
Dialog modeling connects the human side of tasks and
workflow with the applications part of UI actions and
components. It mediates between the task model and
presentation model. Accessibility has to be respected to
enforce all people in usage of the application. One basic
concept is the parallelization of HMI with different
modalities of in- and output. Typically, visual, acoustical
and haptical modalities are available. Using more than one
modality for the presentation and control of the same
information enables even users with sensory or physical
disabilities the use of the UI. Multimodal HMI models
allow the reuse of communication design for different UIs
including AT such as Braille devices or Speech
recognition.
The UI may have various states within the same user
mode, which are not always visible for the user. State
charts or activity diagrams in UML are appropriate to
model the behavior of interface components. Dialog
modeling can follow different strategies of HMI such as
universal or multimodal design. Universal design means
that perception and control of all relevant information in
the UI is independent from in- or output devices.
Multimodal design includes the use of different modalities
as visual, acoustical or haptical to realize the UI
functionalities. The example in fig. 8 shows the actions of
the UI and the system while guiding the customer to a
product department in the supermarket. Left-sided the
system actions are shown and right-sided the actions of
the UI. The vertical order is not chronically. The UI
actions are modeled device-independent.
D. Abstract Presentation Modeling
Abstract Presentation Modeling is the last part of the
modeling process and has to respect accessibility issues as
well. The components of the UI serve as mediators
between the user and the application. Therefore, they are
called mediators. Mediators have in- and output
connections for both sides – user and application (MVC
control). Modeling of internal UI states serves to detail UI
behavior within the application. The Abstract Presentation
Model (APM) describes the UI components and actions
semantically. The model annotations are used later-on to
extend UI components with text-based annotations for AT
support. Static UI hierarchy is modeled with class
diagrams in UML notation. Navigation is taken from the
model of modes (see fig. 6). Modes and mediators have
attributes, which are mapped to HTML elements, and
attributes later on to describe metadata and deviceindependent
properties. Device-independent event-handler
and WAI-ARIA-like [10] elements are used to describe
the behavior of the UI during runtime. Abstract UI
components are anchor, link, text, buttons, image, text
input, choice, container etc. Attributes are used to declare
necessary metadata for UI elements such as title, name,
type etc. Fig. 9 shows the APM for the navigation UI
including three pages to determine the destination, to
navigate and to announce that the customer has reached
the desired product department offering further
functionalities.
E. Concrete Presentation Design and Implementation
The presentation modeling and application layout themselves
are not part of the modeling process. For the
implementation of the declared models, an existing Web
frame-work is used. The frameworks for Web application
development support the usage of accessible UI
components.
Figure 8: System and UI actions for the navigation to the
department

Figure 9: Abstract presentation model for navigation UI
Two Java Web frameworks with MVC architecture are
Apache MyFaces (AMF) including the Trinidad and
Tobago subprojects and the Google Web Toolkit (GWT).
AMF Trinidad and GWT have built-in support for
accessible UI elements. GWT implements more clientside
functionality to obtain application behavior similar to
desktop applications. Furthermore, accessible notifications
about roles, states and changes of UI components are
included. AMF is an open source implementation of
SUN’s specification for Java Server Faces (JSF). Some
subprojects extend functionality of JSF such as Tobago,
Tomahawk or Trinidad. The Trinidad library - formerly
known as Oracle’s ADF Faces - enriches AMF with more
than 100 UI components including simpler components
like tables or trees as well as more complex components
like calendars. Additionally, accessibility for UI
components is supported and other features such as a
dialog framework or pageFlowScope – a bean scope
between the request and session scope of JSF. AMF
Trinidad uses its own markup elements. The markup of
the resulting HTML page is completely encapsulated. UI
modes from modeling are implemented as JSF-views.
Another important aspect of HMI is the modality of
interaction. In many cases, a multimodal interaction
provides more flexibility in interacting with an UI. It is
useful to overcome certain limitations of the users in
advance and corresponds to a higher accessibility. For
instance, providing the voice input along with classical
keyboard and mouse solutions is suitable for visually
impaired users. Development of software supporting
multimodal interaction requires a standard that supports
including various interactions channels. Currently,
XHTML+Voice (aka. X+V) is one of the pioneer
standards, and open source software, which supports
multimodality. VoiceXML 2.0, Speech Synthesis Markup
Language (SSML 1.0), Speech Recognition Grammar
Format (SRGF) and XML events can be integrated using
XHTML modularization to bring spoken interaction to the
Web. Using such technologies, a portable and accessible
technology can be employed. In the case of visually
impaired customers in a supermarket, this can be perfectly
combined with cell phones or other embedded systems for
online support of the customer during shopping.
V. CONCLUSIONS AND OUTLOOK
We have discussed the potential of UCD and MDE in
maintaining accessibility to UI, by assessing the
accessibility requirements to the development process.
Having focused on the early development stages, i.e.
analysis and design phase, we have investigated suitable
methods and technologies, which can assist developers by
designing an accessible UI for industrial automation
systems.
The proposed concept is based on the survey made on
the state of the art and consists of a set of suitable
methods. It provides a parallel process to the development
process, while considering human, technology, and
organization and their interconnections.
The parallel process is based on the UCD and uses
MDE instructions. It runs along the development and
feeds the development process with methods to assess
accessibility requirements of the end UI. The MDE nature
of the concept provides an easy integration of the results
in the main development process.
Our plan for the future is to evaluate the concept using a
real case study. Therefore, two scenarios have been
designed: accessible maintenance of a high bay rack
system and interaction of a blind user with a Web-based
travel information system.
REFERENCES
[1] Lauber, R. and Göhner, P.: Prozessautomatisierung I, 3rd
Edition, Springerverlag Berlin Heidelberg (1999)
[2] ISO13407:1999
[3] Henry, S. L.: Just Ask: Integrating Accessibility throughout
Design, http://www.Lulu.com (2007)
[4] Cysneiros, L. M., Werneck, V. and Kushniruk, A.: Reusable
Knowledge for Satisfying Usability Requirements. In: 13th
IEEE International Conference on Requirements
Engineering, Proceedings (2005)
[5] Reuther, A. 2003. useML – Systematische Entwicklung von
Maschinenbediensystemen mit XML. Technische
Universität Kaiserslautern
[6] Yesilada, Y. et.al. 2004. Dante – Annotation and
Transformation of Web Pages for Visually Impaired Users.
In: WWW Alt. '04: Proceedings of the 13th international
World Wide Web conference on Alternate track papers &
posters, ACM Press, New York, NY, 490-491
[7] Göhner, P. et al. 2008. Integrated Accessibility Models of
User Interfaces for IT and Automation Systems. In:
Proceedings of the 21st International Conference on
Computer Applications in Industry and Engineering. Cary,
NC 27511 USA
[8] Jeschke, S. and Pfeiffer, O. and Vieritz, H. 2008. Developing
Accessible Applications with User-centered Architecture. In:
Computer and Information Science. ICIS 08, Seventh
IEEE/ACIS International Conference on. IEEE Computer
Society, Los Alamitos, CA, pp. 684-689
[9] World Wide Web Consortium (W3C). Web Content
Accessibility Guidelines 2.0. 2008,
http://www.w3.org/TR/WCAG20/, last visited: 10/29/2010
[10] World Wide Web Consortium (W3C). Accessible Rich
Internet Applications (WAI-ARIA) 1.0. 2009,
http://www.w3.org/WAI/intro/aria, last visited: 10/29/2010