Automated PLC Software Testing Using Adapted UML Sequence Diagrams
Automated PLC Software Testing Using Adapted UML Sequence Diagrams
Abstract: Current production plants are highly customizable and flexible in their processes.
This flexibility is mainly realized by software. It causes an increasing complexity of control
software components and the need of new methods for comprehensive and automated testing
approaches to ensure a requested level of quality at high efficiency. A survey among mechanical
engineering companies of the industries aerospace, automotive and machine/plant automation
was conducted. Aspects of requirements engineering, testing, simulation, processes, tools, etc.
were addressed to reveal research demands of PLC (Programmable Logic Controller) control
software development in machine/plant automation. A system architecture embedding UML
sequence diagrams for testing is proposed. We further contribute transformation rules of
semantically adapted UML sequence diagrams to the cyclic execution logic of PLCs for reactive
systems. The approach can be applied to any IEC 61131-3 programming language of the
automation control software. A prototypical realization shows proof of concept and reveals
the potential for future work.
state space of modern software, complete testing would be questionnaires. The goal of the survey was to reveal the
time consuming and costly. Therefore a testing strategy lack of an integrated and continuous support of all de-
covering criticality and risks is considered to narrow down velopment related artifacts in machine and plant automa-
tests to those being relevant. These facts are known tion. The collected data includes aspects of requirements
for years in research and industry. Methods, tools and engineering, testing, simulation environment, description
processes have emerged, but software testing is still an of requirements / test specification, fault scenarios, pro-
underrepresented piece of a puzzle in system and software cesses, quality measurement and tools. Due to secrecy
development. The industrial sector machine and plant reasons, all collected data is made anonymous. Competing
automation finds itself in a constantly ongoing process companies not being part of this survey are invited to
of software becoming the central parameter for success. use these results as performance evaluation of their own
The necessity and importance of high quality methods, development strategies.
tools and processes for their software development hasn’t
Results of Survey: Table 1 states the current situation
arrived this mainly mechanical oriented industrial sector
of machine/plant automation. It is absolutely clear, that
yet.
especially software testing is poorly supported over all
Industrial Survey: To the author’s best knowledge, development stages. In contrast to aerospace and automo-
there is no citable publication revealing daily practical tive, there are quasi no standards or statutes, that need to
approaches, available tools, etc. in machine and plant au- be fulfilled. This leads to an unrepresented consideration
tomation to derive valid requirements for further research. of testing. Currently testing is reduced to spare manual
In 2011, the institute of automation and information sys- developer checks by randomly manipulating parameters
tems (AIS) consulted internationally operating companies of the control software, since there is no explicit time and
of the industrial sectors aerospace, automotive and ma- resource allocation for testing. But the mentioned testing
chine/plant automation with interviews, workshops and needs in Table 1 clearly states its necessity.
1616
INCOM 2012, May 23-25, 2012
Bucharest, Romania
Table 2. Comparison of industrial sectors Hametner et al. (2011) introduce an automated test case
generation approach for industrial automation applica-
tions which are specified by UML state chart diagrams.
Selecting UML models for requirement specification of
control automation systems is presented in Hametner et al.
(2010). The Testing and Test Control Notation (TTCN-3)
enables textual and graphical representation and execution
of test cases, see TTCN3 (2007). Its graphical language is
similar to UML Sequence Diagrams. Wang et al. (2009)
use TTCN-3 for the description of communication protocol
test cases.
A test process can be seen as a systematic execution of a
software program. Therefore specific data are used for the
system under test, which is an important part for this test
process. The goal of the test process is to plan, execute and
Need for Testing in Machine/Plant Automation
analyze the test run. Winkler et al. (2010) proposed a test
Sector: Table 2 sums up the main differences in soft-
framework for an automated test process which are used
ware testing of machine/plant automation compared to
for testing control automation applications based on IEC
aerospace and automotive. Since there is no demand in
61131-3 (2003) and IEC 61499-1 (2005). The challenges for
execution and documentation of tests, there is no explicitly
a useful test application are the test cases. Chevalley and
allocated time for testing, compared to a high testing
Thevenod-Fosse (2001) and Utting and Legeard (2007)
effort of 40-80 %. Therefore test cases are hardly given
used the system requirements definition based on UML
and mostly only existent in a textual manner. Neverthe-
model for the test case generation. Kim et al. (1999)
less developers of PLC application software are facing an
generate unit tests by transformation of manually specified
overwhelming complexity of software without having time,
UML state charts. It is being checked whether the system
resources or proper testing tools to verify the software’s
under test complies with its specification.
functional behavior. Since there is no common software
architecture or framework for PLC control software, those Brost (2010) introduces a test case generation approach for
applications are mostly composed out of reused software UML state chart based implementations of an automotive
components by a copy and modify approach. This leads to ECU by transforming existing hierarchies, concurrency
multiple variants of software components across projects into a flat automaton. Based on this automaton test cases
and complicates maintainability due to error propagation. are generated out of valid paths across the locations. The
The fact of not having testing tools and test cases for generated test cases are executed against a real system.
functional compliance checks results further drawbacks. A Schwarzl and Peischl (2010) propose test case generation
refactoring of all derivations of software isn’t pursued, due based on symbolic transition systems out of communicat-
to the risk of not reaching previous functionality, since it ing state chart models describing ECU behavior. The test
cannot be checked with the means of regression testing. definition phase distinguishes a systematic test, where a
predefined list of transitions is visited for test case gen-
We propose a system architecture for model-based soft-
eration. In a random walk through outgoing transitions
ware testing of PLC control software, which abstracts of
are chosen randomly, which reveals uncovered paths by
the application’s programming language and enables test
the systematic approach. The approach is applied to an
execution against real plants or simulation of any kind.
automotive flashing indicator application in a HiL envi-
The semantics of UML meta-model conform sequence di-
ronment.
agrams are adapted to the cyclic execution logic of PLC
applications for the specification of test cases. The verification of PLC applications demands a transfor-
mation of IEC 61131-3 code into model checker compli-
3. RELATED WORK ant models. Zoubek (2004) introduces a transformation of
logically concatenated operations, mainly done in ladder
diagram (LD) or continuous function chart (CFC). Bauer
An automatic test case generation approach to verify
et al. (2004) and L’Her et al. (2001) introduce a transfor-
the system behavior in erroneous situations using fault
mation process from SFC and physical models to UPPAAL
injection by simulating component (device) defects during
models and Petri net models for verification. Interlocking
runtime is proposed by Kormann and Vogel-Heuser (2011).
requirements are challenging to handle. The verification of
Special focus is applied on the generation of a reduced
such requirements demands transformation of control code
set of meaningful test cases to be executed in a simulated
and the requirement specification. The work of Wardana
environment to increase reliability.
et al. (2009) focuses on the automatic program verification
In order to evaluate the testability of a system, it must of CFC (Continuous Function Chart) and logic diagram
be verified whether the test case triggering system state based on Model-Checking.
is reachable. A transformation approach of the control
Dan and Hierons (2011) show a general test architecture
software model and the corresponding environment model
and transformation of message sequence charts (MSC)
(physics) to timed automata for model checking to exploit
onto executable test cases.
the power of the query language Timed Computation Tree
Logic for start state reachability analysis is proposed by
Kormann et al. (2011).
1617
INCOM 2012, May 23-25, 2012
Bucharest, Romania
1618
INCOM 2012, May 23-25, 2012
Bucharest, Romania
Table 4. Semantics of the PLCMessage template will be generated automatically for each test
case (Figure 2). The elements on this template cannot be
deleted. The name of the first call (object() on Figure 2),
and the names and types of control application lifeline
and simulation lifeline can be edited. The message TC OK
is responsible for the explicit definition of the test case
end point. The test case structure proposed above (with
TC INIT() and TC EXIT() functions) does not contradict
the terms of the PLC control software domain and can
be implemented easily using OO-IEC-61131-3 (object ori-
ented extension to the standard, currently being passed).
Each test case is a function block (FB), thus it is an
analogue of a class with internal methods TC INIT() and
(7) There are only SyncCall and Reply messages on each TC EXIT(). These methods can be implemented in any
sequence diagram. IEC-61131-3 language. The test case developer must select
(8) The set of messages is extended by the stereotype the control software object (object under test), enter the
PLCMessage. The semantics is summarized in Ta- appropriate simulation and describe the I/O time behavior
ble 4. Messages represent setting or checking of I/O (highlighted red areas on the Figure 2).
variables (sensor and actuator values).
(9) If there is no time interval (DurationConstraint) be- 4.3 Model Transformation / Code Generation
tween two consecutive message events on the lifeline,
the time interval is set to PLC cycle time. The previously introduced UML sequence diagrams fol-
(10) There are no simulation - TME-Engine, TME-Engine low a predefined syntactic and semantic specification.
- control software and self-messages allowed. The UML specification holds multiple description ele-
ments like Lifelines, Messages and many CombinedFrag-
ments (parallel block, etc.). The following definition of the
UML sequence diagrams used in our approach uses ba-
sic description elements, which are fundamentally needed
for test case specification and execution. A UML Se-
quence Diagram SD for PLC testing is a 9-tuple SD =
(L, M, O, E, C, S, δ, σ, τ ), where
• L is a set of Lifelines (Named Element), which repre-
sents a testing relevant participant with |L| = 3
• M is a set of Messages describing attributes and
communication kind (synchronous, etc.)
• O is a set of ordered Occurrence Specifications, in-
dicating beginning or end of message and execution
specifications
• E ⊂ O is a set of ordered Execution Specifications
Fig. 2. Test case template and developer activity areas describing a period of lifetime, while e.g. executing a
unit of behavior
The constraints listed above and the PLCMessage-stereotype • C is a set of State Invariants indicating runtime
form the PLCTestML profile, which is used to create constraints as internal or external state
executable test cases in the suggested testing approach. • S ⊂ OxM xO := {(o1 , m, o2 ); o1 < o2 } is a set of
signals, described by messages and their both end
Executable Test Cases: Template-based Creation points (sender and receiver)
and Structure: Different initializations often have to be • δ : O → L assigns Occurence Specifications to Life-
carried out, before a test case can be executed. Thus for
lines
each test case an initialization method TC INIT() must
• σ : C → O combines State Invariants to Occur-
exist, which will be executed at the beginning of the test rence Specifications, since State Invariants are being
case. This method can also be used for the simulation checked prior to evaluation of subsequent Occurrence
manipulations in order to bring it into the desired state Specification
(various test scenarios). At the end of the test case • τ : OxO := {(oi , oj ); oi < oj } → T xT := {(ti , tj ); ti ≤
execution, final activities may have to be performed, e.g.
tj } assigns time constraints (minimum duration, max-
resources must be shared, so that the entire system is not
imum duration) to any Occurrence Specification
blocked for further use. Therefore a finalization method
TC EXIT() must exist, that finalizes the execution of a A sequence of length n across all lifelines can therefore be
test case. The end point of a test case must also be specified denoted as: (o1 , m1 , m2 ), (o3 , m2 , o4 ), . . . , (o2n−1 , mn , o2n ).
(explicitly or implicitly).
Each UML sequence diagram describing an individual
The proposed testing approach uses a template-based test test case is being linearized and transformed into an
case creation, on the one hand to guarantee the fulfillment IEC 61131-3 compliant function block (FB), like illus-
of the requirements to the PLCTestML profile, on the trated in Figure 3. This FB is being called cyclically with
other hand to enable efficient test case development. A xExecute := true as input parameter until xDone becomes
1619
INCOM 2012, May 23-25, 2012
Bucharest, Romania
1620
INCOM 2012, May 23-25, 2012
Bucharest, Romania
diagram, code generation) has successfully been evaluated Kormann, B. and Vogel-Heuser, B. (2011). Automated test
on a typical application example. The efficient possibility case generation approach for plc control software excep-
of test case description and execution for documented tion handling using fault injection. Annual Conference
conformance verification purposes of PLC application sat- of the IEEE Industrial Electronics Society (IECON), 37.
isfies the requirements (chapter II) of the machine/plant Kormann, B., Vogel-Heuser, B., Hametner, R., and Zoitl,
automation domain, which supports development of sus- A. (2011). Engineering process for an online testing
tainable high quality systems/software. process of control software in production systems. In
Proceedings of Work-in-Progress Papers of Emerging
The future work will include an extension of the available Technologies and Factories and Factory Automation
description elements like PAR and ALT for test case (ETFA).
specification, because they are considered as an important L’Her, D., Le Parc, P., and Marcé, L. (2001). Proving se-
part for complex test cases. Due to the cyclic execution quential function chart programs using timed automata.
logic of PLC applications, the resulting code generation Theor. Comput. Sci., 267, 141–155. doi:10.1016/S0304-
of these enhanced elements will imply further research on 3975(00)00301-7.
adapted semantics. Schwarzl, C. and Peischl, B. (2010). Test sequence genera-
tion from communicating uml state charts: An industrial
ACKNOWLEDGEMENTS application of symbolic transition systems. In Pro-
ceedings of the 2010 10th International Conference on
The authors gratefully thank all companies having par- Quality Software, QSIC ’10, 122–131. IEEE Computer
ticipated in the survey. Their time spent, the profound Society, Washington, DC, USA.
and honest answering made the evaluation in chapter II TTCN3 (2007). Testing and Test Control Notation
possible. (TTCN3) V3.2.1. ETSI, ITU-T.
Utting, M. and Legeard, B. (2007). Practical Model-based
REFERENCES Testing. Morgan Kaufmann.
Wang, Z., Yin, X., Xiang, Y., Zhu, R., Gao, S., Wu, X.,
Bauer, N., Engell, S., Huuck, R., Lukoschus, B., and Sturs- Liu, S., Gao, S., Zhou, L., and Li, P. (2009). Ttcn-3
berg, O. (2004). Stursberg: Verification of plc programs based conformance testing of mobile broadcast business
given as sequential function charts. In In: Integration management system in 3g networks. In Proceedings of
of Software Specification Techniques for Applications in the 21st IFIP WG 6.1 International Conference on Test-
Eng., Springer, LNCS, 517–540. ing of Software and Communication Systems and 9th In-
Brost, M. (2010). Automatisierte testfallerzeugung auf ternational FATES Workshop, TESTCOM ’09/FATES
grundlage einer zustandsbasierten funktionsbeschrei- ’09, 163–178. Springer-Verlag, Berlin, Heidelberg.
bung für kraftfahrzeugsteuergeräte. In Schriftenreihe Wardana, A., Folmer, J., and Vogel-Heuser, B. (2009).
des Instituts für Verbrennungsmotoren und Kraftfahrwe- Automatic program verification of continuous function
sen der Universität Stuttgart. chart based on model checking. In The 35th Annual
Chevalley, P. and Thevenod-Fosse, P. (2001). Automated Conference of the IEEE Industrial Electronics Society -
generation of statistical test cases from UML state di- IECON 09.
agrams. In Computer Software and Applications Con- Winkler, D., Hametner, R., Östreicher, T., and Biffl, S.
ference, 2001. COMPSAC 2001. 25th Annual Interna- (2010). A framework for automated testing of automa-
tional, 205 –214. doi:10.1109/CMPSAC.2001.960618. tion systems. IEEE Conference on Emerging Technolo-
Dan, H. and Hierons, R. (2011). Conformance testing from gies and Factory Automation (ETFA), 1–4.
message sequence charts. In ICST, 279–288. Witsch, D., Ricken, M., and Kormann, B. (2010). Plc
Group, O.M. (2010a). Unified modeling language: Infras- state charts: An approach to integrate uml state charts
tructure, version 2.3. formal. in open-loop control engineering. In IEEE Information
Group, O.M. (2010b). Unified modeling language: Super- Technologies for Sustainable Development.
structure, version 2.3. formal. Witsch, D. and Vogel-Heuser, B. (2009). Close integration
Hametner, R., Kormann, B., Vogel-Heuser, B., Winkler, between uml and iec 61131-3: New possibilities through
D., and Zoitl, A. (2011). Test case generation ap- object-oriented extensions. In Proc. 14th IEEE Int.
proach for industrial automation systems. In The 5th Conf. Emerging Technologies and Factory Automation,
International Conference on Automation, Robotics and 1–6.
Applications, 6. ICARA, IEEE, New Zealand, IEEE. Witsch, D. and Vogel-Heuser, B. (2011). Plc-statecharts:
Hametner, R., Winkler, D., Östreicher, T., Surnic, N., and An approach to integrate uml-statecharts in open-loop
Biffl, S. (2010). Selecting uml models for test-driven control engineering – aspects on behavioral semantics
development along the automation systems engineering and model-checking. Proceedings of 18th IFAC World
process. In Proceedings IEEE Emerging Technologies Congress, 8.
and Factory Automation (ETFA 2010). Zoubek, B. (2004). Automatic verification of temporal
IEC 61131-3 (2003). IEC 61131-3 Standard - Pro- and timed properties of control programs. Ph.D. thesis,
grammable controllers - Part 3: Programming languages. University of Birmingham, England.
International Electrical Commission, 2 edition.
IEC 61499-1 (2005). Function blocks – Part 1: Architec-
ture. International Electrical Commission, Geneva.
Kim, Y., Hong, H., Cho, S., Bae, D., and Cha, S. (1999).
Test cases generation from uml state diagrams. In IEEE
Proceedings: Software, volume 146, 187–192.
1621