Systems and Control Theory

Lecture notes

Teacher: Prof. Franco Blanchini

Università degli Studi di Udine

the Italian version by

Luca Peretti

has been translated and revised by

Giulia Giordano, Flavio Pop, Franco Blanchini

November 6, 2023
Preface

These notes on System Theory are a revised version of the lecture notes first written by Dr. Luca
Peretti after the course taught by Prof. Franco Blanchini in the academic year 2002/2003. Through
the years, the notes have been deeply revised and integrated. Several students have pointed out
typos or errors (most of which are due to the teacher).
The English translation was first released in 2016. Should the reader have any comments or
find out errors, a message would be very appreciated. For this purpose, please contact

blanchini@uniud.it

The present notes will be updated/corrected in the future, tending to perfection in the very long
run (with a very slow convergence speed).
Given their nature, these notes SHOULD NOT be considered as a substitute of a good textbook.
They only provide a track of the arguments presented in the course.

The basic principles which inspire the work are:

• there are no error-proof manuscripts;

• no matter how big is your effort, someone will not approve your work;
• systems & control theory is as important as many other things in life;
• rule number one: have fun!

Franco Blanchini

i
Contents

1 Introduction 1

2 Systems of differential equations 5

2.1 Differential equations in normal form . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 General solution of a continuous-time linear time-invariant system . . . . . . . . 9
2.2.1 Natural response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.2 Forced response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.3 Complete solution of the system . . . . . . . . . . . . . . . . . . . . . . 11
2.3 The exponential eAt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.1 The case of multiple eigenvalues . . . . . . . . . . . . . . . . . . . . . . 15
2.4 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.5 Transfer functions and frequency response . . . . . . . . . . . . . . . . . . . . . 20
2.5.1 Frequency response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.6 Discrete-time systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.6.1 General solution of a discrete-time linear time-invariant system . . . . . 24
2.6.2 Discrete-time stability . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.7 State transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.8 Sampled-data systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.8.1 Analog-to-Digital conversion . . . . . . . . . . . . . . . . . . . . . . . . 27
2.8.2 Digital-to-Analog conversion . . . . . . . . . . . . . . . . . . . . . . . . 27
2.8.3 Aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.9 Discrete-time equivalent of a continuous-time system . . . . . . . . . . . . . . . 30

3 Dynamical systems 35
3.1 General consideration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.2 Causal dynamical systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3 Automata as dynamical systems . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4 Reachability and Observability 41

4.1 Reachability of continuous-time systems . . . . . . . . . . . . . . . . . . . . . . 41
4.2 Kalman decomposition for reachability . . . . . . . . . . . . . . . . . . . . . . . 44
4.3 Reachability of discrete-time systems . . . . . . . . . . . . . . . . . . . . . . . 48
4.4 Other reachability criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.5 Observability and detectability of continuous-time systems . . . . . . . . . . . . 55
4.6 Kalman decomposition for observability . . . . . . . . . . . . . . . . . . . . . . 58
4.7 Observability of discrete-time system . . . . . . . . . . . . . . . . . . . . . . . 59
4.8 Examples of reachability and observability analysis . . . . . . . . . . . . . . . . 60
4.9 Duality and dual system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.10 Joint Kalman form and transfer functions . . . . . . . . . . . . . . . . . . . . . 65

iii
iv CONTENTS

4.11 Cancellations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.11.1 External stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.12 Controllable canonical form . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

5 Realisation 71
5.1 The problem in general . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.2 Minimal realisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.3 Minimal realisation for SISO systems . . . . . . . . . . . . . . . . . . . . . . . 75
5.3.1 Other techniques for SISO systems realisation . . . . . . . . . . . . . . . 75
5.4 Realisation of discrete-time systems . . . . . . . . . . . . . . . . . . . . . . . . 76

6 Control Systems 77
6.1 General considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.2 State feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6.2.1 A simple algorithm for eigenvalue assignment . . . . . . . . . . . . . . . 83
6.3 State observer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.4 Synthesis of regulators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
6.5 External disturbances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

7 Nonlinear systems 97
7.1 General considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
7.2 Equilibrium points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
7.3 Lyapunov functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
7.3.1 Lyapunov stability theorem . . . . . . . . . . . . . . . . . . . . . . . . . 102
7.3.2 Instability theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.4 Lyapunov criterion for discrete-time systems . . . . . . . . . . . . . . . . . . . . 109
7.5 Lyapunov equations for linear systems . . . . . . . . . . . . . . . . . . . . . . . 110
7.5.1 Lyapunov equations for discrete-time systems . . . . . . . . . . . . . . . 112
7.6 Linearisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
7.7 Stability analysis of an equilibrium point . . . . . . . . . . . . . . . . . . . . . . 117
7.8 Stabilisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
7.9 Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.9.1 Lyapunov redesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

8 Optimal control: basic notions 123

8.1 General considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
8.1.1 Lagrange Multipliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
8.2 Variational calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
8.3 Dynamic programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
8.3.1 Linear quadratic control . . . . . . . . . . . . . . . . . . . . . . . . . . 129
8.3.2 Optimal observer: the Kalman-Bucy filter . . . . . . . . . . . . . . . . . 132
8.4 Connection filter-optimal control . . . . . . . . . . . . . . . . . . . . . . . . . . 132
8.5 Model predictive control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

9 Examples of dynamical systems 137

9.1 DC electrical machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
9.2 Robot arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
9.3 Mortgage payment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
9.3.1 Mortgage installment computation . . . . . . . . . . . . . . . . . . . . . 143
CONTENTS v

9.4 Positive linear systems and Markov chains . . . . . . . . . . . . . . . . . . . . . 143

9.4.1 Markov chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
9.5 Vibrating systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
9.6 Linear electrical circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
9.7 Operational amplifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
9.8 Two-tank system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
9.9 Magnetic levitator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
9.10 Cart–pendulum system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

A Elements of Mathematics 159

A.1 Vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
A.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
A.2.1 Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
A.2.2 Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
A.2.3 Linear systems of equations . . . . . . . . . . . . . . . . . . . . . . . . 165
A.2.4 Partitioned matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
A.2.5 Elementary row and column tranformations . . . . . . . . . . . . . . . . 167
A.2.6 Basis transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
A.2.7 Eigenvalues and eigenvectors . . . . . . . . . . . . . . . . . . . . . . . 172
A.2.8 Norms and inner product . . . . . . . . . . . . . . . . . . . . . . . . . . 175
A.2.9 Symmetric matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
A.2.10 Semidefinite and definite matrices . . . . . . . . . . . . . . . . . . . . . 178
A.3 Polynomials and power series . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
A.4 The impulse response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
A.5 Laplace transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
A.5.1 Transforms of rational functions . . . . . . . . . . . . . . . . . . . . . . 185
A.5.2 Rational functions and their inverse transform . . . . . . . . . . . . . . . 186
A.6 Zeta transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
A.7 Convex and least square optimization . . . . . . . . . . . . . . . . . . . . . . . 191
A.7.1 Application to system identification . . . . . . . . . . . . . . . . . . . . 193
List of Figures

1.1 General scheme of a dynamic system. . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 An example of linear time-invariant system: the balance. . . . . . . . . . . . . . 2
1.3 The balance system: a qualitative graph of the transient. . . . . . . . . . . . . . . 3

2.1 Model of a pendulum. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2 A system with m inputs and p outputs and its impulse response matrix. . . . . . . 12
2.3 Scheme of a simple elastic structure. . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4 Stability of a system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.5 Asymptotic stability: convergence of z(t) for t → +∞. . . . . . . . . . . . . . . . 18
2.6 Pendulum and inverted pendulum. . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.7 Phasors involved in the frequency response of a system. . . . . . . . . . . . . . . 22
2.8 Analog-to-Digital conversion. . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.9 Qualitative scheme of an Analog-to-Digital Converter. . . . . . . . . . . . . . . 28
2.10 Zero Order Hold. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.11 First Order Hold. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.12 Error with Zero Order Hold. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.13 Spectrum of a band-limited signal. . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.14 Discrete-time control of a continuous-time process. . . . . . . . . . . . . . . . . 31
2.15 Discrete-time system equivalent to the continuous-time system. . . . . . . . . . . 32

3.1 Two different types of systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.2 Flow graph for finite automata. . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.3 Example of automaton: recognition of a string. . . . . . . . . . . . . . . . . . . 40

4.1 Kalman decomposition for reachability. . . . . . . . . . . . . . . . . . . . . . . 46

4.2 Simplified block diagram of the Kalman decomposition for reachability. . . . . . 46
4.3 An electric circuit (example for discussing reachability). . . . . . . . . . . . . . 53
4.4 Kalman decomposition for observability. . . . . . . . . . . . . . . . . . . . . . . 59
4.5 Bode plot of a real differentiator, with a pole at ω = 103 rad
s . . . . . . . . . . . . 62

5.1 Block diagram of the minimal realisation with m = p = 1. . . . . . . . . . . . . . 75

6.1 Subdivision of inputs and outputs in a system. . . . . . . . . . . . . . . . . . . . 77

6.2 Linear system with feedback: block diagram. . . . . . . . . . . . . . . . . . . . 78
6.3 State feedback. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6.4 State observer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6.5 General scheme of a system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.6 Evolution of state and error with the trivial observer. . . . . . . . . . . . . . . . 86
6.7 Evolution of state, estimated state and error with the desired observer. . . . . . . 86
6.8 General scheme of a Luenberger observer. . . . . . . . . . . . . . . . . . . . . . 87

vii
viii LIST OF FIGURES

6.9 Positioning system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

6.10 Block diagram of an observer-based controller. . . . . . . . . . . . . . . . . . . 90

7.1 Evolution of a population for different initial conditions. . . . . . . . . . . . . . 98

7.2 Evolution of a population with external input. Line A: extinction threshold; line B:
survival threshold. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
7.3 Stable and unstable equilibrium points for a rolling sphere. . . . . . . . . . . . . 101
7.4 Level curves of a Lyapunov function. . . . . . . . . . . . . . . . . . . . . . . . . 102
7.5 Pendulum (basic). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.6 Dissipative pendulum: subspaces N and W and vector ẋ(t). . . . . . . . . . . . . 107
7.7 Trajectories of the dissipative pendulum. . . . . . . . . . . . . . . . . . . . . . . 108
7.8 Intuitive representation of Cetaev criterion. . . . . . . . . . . . . . . . . . . . . 108
7.9 Controller for the linearised system at an equilibrium point. . . . . . . . . . . . . 116
7.10 Linearisation and application of the regulator commute. . . . . . . . . . . . . . . 120

9.1 General model of a DC motor. . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

9.2 Series and parallel connections. . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
9.3 Independent connection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
9.4 Scheme of a robot arm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
9.5 Control of a robot arm: r and v are position and speed references. . . . . . . . . . 142
9.6 Buffer: graph of states and transitions. . . . . . . . . . . . . . . . . . . . . . . . 146
9.7 Buffer simulation for different values of P0 , P1 , P2 . . . . . . . . . . . . . . . . . 147
9.8 Building under seismic action. . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
9.9 Example of a linear circuit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
9.10 Operational amplifier. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
9.11 Operational amplifier in inverting feedback configuration. . . . . . . . . . . . . . 152
9.12 Feedback amplifier: adder. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
9.13 Two-tank system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
9.14 Example of segmentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
9.15 The magnetic levitator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
9.16 The cart–pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

A.1 Composition of linear maps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

A.2 Graph of the quadratic function V(x). . . . . . . . . . . . . . . . . . . . . . . . 179
A.3 Level curves of the quadratic function V(x). . . . . . . . . . . . . . . . . . . . . 179
A.4 The impulse can be seen as the limit of a sequence of functions. . . . . . . . . . 183
Chapter 1

Introduction

In the subject name, System Theory, an important aspect is actually omitted. To be more precise, in
fact, we should talk about the Theory of Dynamical Systems. The expression “dynamical system”,
or “dynamic system”, generically refers to any entity that evolves over time according to the laws
of causality, interacting with the environment in accord with the cause-and-effect principle.
From an external viewpoint, a dynamical system is characterised by a set of input functions,
representing the cause, and a set of output functions that represent the effect (Figure 1.1). A key
feature is that the link between input and output is not static, but is intrinsically linked to time and
to the past history.

inputs u(t) outputs y(t)

SYSTEM

Figure 1.1: General scheme of a dynamic system.

For example, consider a system whose output y(t) depends on the input u(t) as per the differential
equation1
ÿ(t) + aẏ(t) + by(t) = u(t) ,
fairly representing a balance (Figure 1.2). When u(t) is assigned, the solution of the equation
describes the time evolution of y(t). If we consider a constant input u(t) = ū, then this differential
equation has a constant (static) solution, which can be easily computed by setting ÿ(t) = ẏ(t) = 0.
This solution is
ū
ȳ = .
b
Typically, in order to determine the weight ū of the object on the balance, we observe the output y,
or a quantity proportional to y. Obviously, however, if we considered solutions of this type only, we
would neglect a fundamental aspect, which is the transient response of the system to a given input.
Indeed, a single given input may produce several (infinitely many) outputs, which is a peculiar
feature of dynamical systems.
Dynamical systems were well studied already at the beginning of the XX century, but systems
theory was deeply investigated and developed in the 1950s-1960s, with the main goal of explaining
the dynamic behaviour of a broad class of entities and processes. Actually, systems theory is able
1
We denote by f ′ (x), f ′′ (x) the first and second derivative of the function f (·) with respect to the generic variable x,
and by ġ(t), g̈(t) the first and second derivative of g(·) as a function of time.

1
2 Chapter 1. Introduction

to provide really meaningful explanations in the case of specific classes of systems, such as regular
dynamical systems (those that can be represented in terms of differential equations).
Linear Time-Invariant systems are an extremely interesting particular case, for which very
powerful tools have been proposed. Some of the techniques developed for studying this type of
systems can be applied to more general types of systems, thanks to a technique called linearisation.
This is an extremely important aspect and justifies the fact that most of the course will be devoted
to the study of Linear Time-Invariant (LTI) systems.

1111111111
0000000000
0000000000
1111111111
0000000000
1111111111
M
0000000000
1111111111

Figure 1.2: An example of linear time-invariant system: the balance.

Example 1.1. (The balance.) The balance (scales) is a simple example of a linear time-invariant
system. Consider Figure 1.2 and assume that (1) the mass of the balance plate is negligible
compared to the mass of the object to be weighed and (2) the viscous friction is a linear function of
the speed. Then, we can easily obtain the linear differential equation which governs the system:

M ÿ(t) + hẏ(t) + ky(t) = Mg ,

where y(t) represents the vertical displacement of the plate, while the constants M, h, k and g
respectively represent the mass of the object, the friction coefficient, the elastic constant of the spring
and the gravitational acceleration. When measuring the weight of an object, we are interested in
y(t) after the system has settled: assuming ÿ(t) = ẏ(t) = 0, we get

Mg
y= .
k

From this equation, expressing the steady-state value of y, we derive the weight Mg. However,
before reaching this static condition, the balance is subjected to a transient that depends on the
value of the friction coefficient, as is qualitatively shown in Figure 1.3.

Clearly, in the case of the scales, the most interesting aspect is the steady-state operation.
However, in other cases the dynamic behaviour of the system has a fundamental role. An example
is given by structural engineering. Buildings are normally designed assuming static loads. Limiting
the study to this case is correct as long as the structure is not affected by forces (or accelerations)
that vary over time. Yet, when the building is located in an earthquake region or when strong winds
are present2 , the building must be designed by modeling it as a dynamic system.
2
The Tacoma bridge is a famous example of this problem: http://www.youtube.com/watch?v=j-zczJXSxnw
 3

t
Figure 1.3: The balance system: a qualitative graph of the transient.

Outline
The course starts by introducing basic tools for modeling dynamical systems. These include
differential and difference equations, vector spaces and transfer functions. The theory of linear
systems and of their properties (superposition principle, free and forced responses, modes and
stability) are explained in detail. The problem of sampling and sampled-data systems, along with
the techniques for digital implementation, are then introduced.
A formal definition of a general dynamical system is given along with some examples of
non-regular systems (i.e., not expressed by differential or difference equations), such as automata.
Then, basic structural properties of linear systems are presented, in particular reachability and
observability and their connection with the transfer function analysis.
The theory of realisation of transfer functions is subsequently introduced. The basic problem
is how we can associate a (minimal) state representation with a transfer function (or a transfer
function matrix).
These notions are preliminary and essential3 to understand the fundamental concepts of regula-
tion. The regulation problem via eigenvalue assignment with state feedback is initially presented
for linear systems. The output feedback problem is then solved via state observer design.
The course then presents fundamental notions on the analysis and the control of nonlinear
systems. The tools presented for the linear case can be successfully applied by means of the
linearisation technique (a key point of the course). A strong support is given by Lyapunov theory,4
which allows us to deal in an efficient and elegant way with systems whose solutions are impossible
to compute analytically, or even numerically.
Finally, the course presents in a very simple way fundamental topics that cannot be discussed
in depth. These include robustness and control optimality. Typically, the final lectures also
present short seminars about specific topics, such as model-predictive control (MPC) and system
identification.
About one third of the course is devoted to exercises. These include a survey of mathematical
notions, in particular linear algebra. Many models of linear and nonlinear systems are described.
Video of laboratory experiences are shown. Many exercises to better understand the theory are
proposed, including three sessions of test exercises.5

3
Although perhaps tedious...
4
As you will notice, this is the part that is most appreciated by the teacher.
5
The teacher strongly approves human cooperation in all occasions but these...
Chapter 2

Systems of differential equations

This chapter will briefly present the basics concept required as a technical support for what follows.
The proposed theory should be the subject of previous courses. However, given its importance,
some fundamental aspects are summarised here.

2.1 Differential equations in normal form

In this course, we will mainly consider is the class of regular systems, namely, systems represented
by differential equations. A regular system has a number (m) of variables named inputs (associated
with the causes) and a number (p) of variables named outputs (associated with the effects); note
that the numbers m and p can be different in general. Input and output variables can be grouped in
two vectors. We will denote by u(t) and y(t) the input vector and the output vector, respectively:

 u1 (t)   y1 (t) 

   
 u (t)   y (t) 
 2  2 
u(t) =  .  ∈ Rm y(t) =  .  ∈ R p

(2.1)
.
 .   .. 
   
um (t) y p (t)

These vectors provide an external representation of the system behaviour. A dynamical system is
internally represented by a number n of variables named states, or state variables, which are also
grouped in a vector x(t):
 x1 (t) 
 
 x2 (t) 
x(t) =  .  ∈ Rn (2.2)
 .. 
 
xn (t)

Example 2.1. (Electric motor.) Consider the system describing an electric motor, illustrated in
Section 9.1. The input vector for this system is
 
 v f (t) 
u(t) =  va (t)  ,
 
 
Cm (t)

whose components represent the excitation voltage, the armature voltage and the external torque.
As an output vector, we can take
φ(t)
" #
y(t) = ,
ω(t)

5
6 Chapter 2. Systems of differential equations

representing the angle and the speed. The state vector is

 
 I f (t) 
 I (t) 
x(t) =  a  ,
 ω(t) 
φ(t)

whose components are the excitation current, the armature current, the angular speed and the
angle.
Note that some state variables can correspond to output variables (as in the example above).
Conversely, a state variable cannot be an input.
In general, the relation between state variables and inputs is given by differential equations. In
most cases, these equations are of the first order and the overall system can be expressed in the form

ẋ1 (t) = f1 (t, x1 (t), . . . , xn (t), u1 (t), . . . , um (t))



 .




 .
. (2.3)

 ẋ (t) = f (t, x (t), . . . , x (t), u (t), . . . , u (t))

n n 1 n 1 m

This is called the normal form, since the derivatives of the state are explicit functions of all of the
other variables.
Remark 2.1. A formal definition of state will be given later on. For regular systems in normal form,
the states are the variables xk (t) whose derivatives appear on the left hand side of the differential
equations.
It is desirable to seek a compact form that allows us to easily manipulate the equations. To this
aim, we consider the derivative of the state vector
 
 ẋ1 (t) 
d  ẋ (t) 
ẋ(t) = x(t) =  2  ∈ Rn , (2.4)
dt  ... 
ẋn (t)

so that the above equations reduce to the following compact vector representation:

ẋ(t) = f (t, x(t), u(t)). (2.5)

This equation represents a generic regular system of n first order differential equations, with m
inputs. It represents an instant relation between the state variables x(t) and inputs u(t) of the system.
Example 2.2. (Chemical reactions.) Consider the following system of differential equations

ẋ1 (t) = −αx1 (t)x2 (t) + u1 (t)




ẋ2 (t) = −αx1 (t)x2 (t) + u2 (t)

(2.6)



 ẋ (t) = αx (t)x (t) − βx (t)

3 1 2 3

that involves three state variables (x1 (t), x2 (t) and x3 (t)) and two input variables (u1 (t) and u2 (t)).
If x1 , x2 and x3 are the concentration of the chemical species A, B and C, this system represents the
time evolution of the concentrations when the chemical reactions
u1 u2 α β
∅ −−⇀ A, ∅ −−⇀ B, A + B −
⇀ C, C ⇀
− ∅

are occurring: u1 and u2 are injections of A and B; C is produced from A and B with reaction
speed α and C is consumed with speed β.
2.1. Differential equations in normal form 7

According to the theory of differential equations, in order to solve the equation (2.5) for a given
value of the input vector u(t), we need to know the initial conditions x(t0 ), namely, the value of
the state at a given (initial) time instant t0 . Under regularity conditions, there will be a solution
x(t) = φ(x(t0 ), u(t)) that depends on the initial conditions x(t0 ) of the system and on the input u(t).
In the following, we will always assume that the solution exists and is unique. This is true if the
function f has some regularity property (for instance, it is a Lipschitz function) and u is sufficiently
regular (e.g., piecewise-continuous).

Cm

θ
mg

Figure 2.1: Model of a pendulum.

Example 2.3. (The pendulum.) Consider Figure 2.1, where Cm is the torque (system input) and
y = θ is the angle with respect to the vertical reference frame (system output). The equation
governing this system in the absence of friction is
g 1
θ̈(t) = − sin(θ(t)) + 2 Cm (t)
l ml
Denoting by ω(t) = θ̇(t), this equation can be written in normal form as the system

θ̇(t) = ω(t)
g 1
ω̇(t) = − sin(θ(t)) + 2 Cm (t)
l ml
where θ and ω are the state variables.
In order to find the value of output at the generic time instant t, it is not enough to know the time
evolution of the torque: we also need to know the initial position and the initial speed (namely, the
initial state). Given these quantities, one can determine the time evolution of the position θ(t) and
of the angular speed ω(t) = θ̇(t) by integrating the differential equation, thus obtaining a complete
description of the trajectory.

The dynamical system described by equation (2.5) is said invariant o autonomous if the
function f does not directly depend on time. Note, however, that it always indirectly depends on
time, since t is an argument of both x(t) and u(t). A time invariant system can then be written as

ẋ(t) = f (x(t), u(t)) (2.7)

Instead, if time enters directly in the equation, the system is said non-autonomous.
8 Chapter 2. Systems of differential equations

Example 2.4. The following system

ẋ1 (t) = −t x2 (t)
(

ẋ2 (t) = u(t)

is non-autonomous.
The system is called linear if the function f is linear with respect to x(t) and u(t). In this case,
it is expressed as:
ẋ(t) = A(t)x(t) + B(t)u(t) (2.8)
A system is said to linear time-invariant (LTI) if it is both time invariant and linear:

ẋ(t) = Ax(t) + Bu(t) (2.9)

In this case, then, the matrices A (dimension n × n) and B (dimension n × m) does not depend on
time.
The output variables of a system are given by the components of a vector y(t) whose components
are statically1 determined by the following algebraic equations
y1 (t) = g1 (t, x1 (t), . . . , xn (t), u1 (t), . . . , um (t))


 ..





 . (2.10)
 y (t) = g (t, x (t), . . . , x (t), u (t), . . . , u (t))

p p 1 n 1 m

which, again, can be written in a compact form:

y(t) = g(t, x(t), u(t)). (2.11)

The above equation is called the output transformation. If the system is time-invariant, the output
transformation is of the form:
y(t) = g(x(t), u(t)). (2.12)
Conversely, if the system is linear, we have:

y(t) = C(t)x(t) + D(t)u(t). (2.13)

Finally, if the system is both linear and time-invariant, the output transformation is of the form:

y(t) = Cx(t) + Du(t), (2.14)

where the matrices C (dimension p × n) and D (dimension p × m) are independent of time.

Putting together state and output equations, we achieve the following system:
ẋ(t) = f (t, x(t), u(t))
(
(2.15)
y(t) = g(t, x(t), u(t))
With few exceptions, this is the most general form of dynamical system we will consider in the
course.
Remark 2.2. There are systems involving partial derivatives, such as that given by the heat
equation
∂
ϕ(z, t) = a2 ∇2 ϕ(z, t),
∂t
where z is a space variable. In this case, the state at each time t is a function ϕ(z, t), which can
be seen as a vector in a proper (infinite dimensional) space. These systems are denoted as infinite
dimensional systems, or distributed parameter systems, and will not be considered in this course.
1
Namely, no derivatives are involved.
2.2. General solution of a continuous-time linear time-invariant system 9

2.2 General solution of a continuous-time linear time-invariant system

During the course, we will intensively discuss the case of linear time-invariant systems, whose the
equations can be written in the form:

ẋ(t) = Ax(t) + Bu(t)

(
(2.16)
y(t) = Cx(t) + Du(t)

where matrix A has dimension n × n, B has dimension n × m, C has dimension p × n and D has
dimension p × m. We will sometime use the notations

Σ(A, B, C, D),

or " #
A B
Σ ,
C C
or even (A, B) or (A, C) when the properties we are discussing concern these matrices only.
For this class of systems, we can determine an analytic solution (while for most systems it
is impossible to obtain a solution without resorting to numerical techniques). The solution we
obtain will not be used numerically, but will be analysed qualitatively and will provide fundamental
information about the system.
Suppose that u(t) and x(0) are known (since the system is time-invariant, we can choose t0 = 0
without loss of generality). If we carefully look at the equations, we can easily see that, in order
to understand the entire system behaviour, we only need to determine x(t): then, y(t) can be
immediately determined from the algebraic linear output transformation. Therefore, our main aim
is to solve the system:
ẋ(t) = Ax(t) + Bu(t)
(
(2.17)
x(0) known
A fundamental property of this class of systems2 is the superposition principle.
For the vectors u(t) and x(0) representing input and initial conditions respectively, the symbol
−→ means “give rise to the solution”:

x(0), u(t) −→ x(t).

Assume that we have two different solutions with the different inputs and initial conditions

x∗ (0), u∗ (t) −→ x∗ (t)

x∗∗ (0), u∗∗ (t) −→ x∗∗ (t)
(2.18)

and assume we combine inputs and initial vectors as follows

u(t) = αu∗ (t) + βu∗∗ (t)

x(0) = αx∗ (0) + βx∗∗ (0) (2.19)

with coefficients α and β. Then,

αx∗ (0) + βx∗∗ (0), αu∗ (t) + βu∗∗ (t) −→ αx∗ (t) + βx∗∗ (t)

We can use this general principle for the natural-forced response decomposition: the problem
of computing the solution can be decomposed into two simpler problems, and the overall solution
2
Actually, of all linear systems, including heat equation, waves equation, transmission lines. . .
10 Chapter 2. Systems of differential equations

is the sum of the solutions of the two simpler problems. Any given initial condition and input x(0)
and u(t) can be decomposed as follows:

x(0) : x∗ (0) = 0, x∗∗ (0) = x(0)

u(t) : u∗ (t) = u(t), u∗∗ (t) = 0

We then have two-problems:

• natural response (assigned initial conditions, null input):

ẋL (t) = AxL (t)
(
(2.20)
xL (0) = x(0)

• forced response (null initial conditions, assigned input):

ẋF (t) = AxF (t) + Bu(t)
(
(2.21)
xF (0) = 0

Applying the superposition principle with α = 1 and β = 1, we get:

x(t) = xL (t) + xF (t). (2.22)

2.2.1 Natural response

The natural response is the solution of the system
ẋL (t) = AxL (t)
(

xL (0) = x(0)
Consider the scalar case first. Then, we have
ẋL (t) = axL (t)
(
⇒ x(t) = x(0) eat
xL (0) = x(0)

In fact, for t = 0 the exponential is equal to 1, and ẋ = a x(0) eat = ax(t). The solution in Rn is
similar to that in the scalar case. Let us define the exponential of a matrix:
X Ak tk +∞
1 1
eAt  I + At + A2 t2 + A3 t3 + . . . = (2.23)
2 3! k=0
k!

This is a convergent series (the factorial dominates the exponential for k → +∞).3 We can then
verify that the natural response of the system is equal to:

x(t) = eAt x(0). (2.24)

In fact, for t = 0,
eA0 x(0) = I x(0) = x(0). (2.25)
Moreover, if we write the derivative, we get
 +∞   +∞ 
d At d X Ak tk  X Ak−1 tk−1 
(e x(0)) =   =  A  = A(eAt x(0)) ⇒ ẋL (t) = AxL (t) (2.26)
dt dt k=0 k!   k=1 (k − 1)! 

The problem of computing eAt will be considered later.

3
Note that the expression is different from the matrix obtained by taking the exponential of each element of the
matrix.
2.2. General solution of a continuous-time linear time-invariant system 11

2.2.2 Forced response

The forced response of the system is the solution of

ẋF (t) = AxF (t) + Bu(t)

(
(2.27)
xF (0) = 0

This solution is equal to the convolution product

Z t
xF (t) = eA(t−σ) Bu(σ)dσ (2.28)
0

This can be verified by applying the Laplace transform to the system equations:

s xF (s) − xF (0) = AxF (s) + Bu(s)

⇒ (sI − A) xF (s) = Bu(s)
⇒ xF (s) = (sI − A)−1 Bu(s) (2.29)

Since the Laplace transform of a convolution is equal to the product of the individual Laplace
transforms, we have:
L
u(σ) −→ u(s)
L
eAt −→ (sI − A)−1
Z t 
eA(t−σ) Bu(σ)dσ = L eAt B L[u(t)] =
 
⇒L
0
= (sI − A)−1 B u(s) (2.30)

Note also that the Laplace transform of the matrix exponential is nothing but a generalisation of the
Laplace transform of the scalar exponential:
L
eAt −→ (sI − A)−1

2.2.3 Complete solution of the system

Since x(t) = xL (t) + xF (t), by adding the obtained expressions we get:
Z t
x(t) = e x(0) +
At
eA(t−σ) Bu(σ)dσ (2.31)
0

Then, the system output is

Z t
y(t) = Ce x(0) + C
At
eA(t−σ) Bu(σ)dσ + Du(t) (2.32)
0

Let us now introduce two assumptions: x(0) = 0 and D = 0 (the latter assumption is only for
brevity). In this case we get: Z t
y= CeA(t−σ) Bu(σ)dσ (2.33)
0
This is the input-output relationship, which is very important and leads to the definition of impulse
response matrix W(t), having dimension p × m:

W(t) = CeAt B. (2.34)

12 Chapter 2. Systems of differential equations

The reason of the name can be explained by considering the Dirac delta (impulse) as the system
input. The Dirac delta δ(·) is a distribution, having the property that, for any continuous function f
defined on an interval [a, b] and any t0 in this interval, we have
Z b
f (t)δ(t − t0 )dt = f (t0 ). (2.35)
a

Then, if u(t) = δ(t) (the impulse function at zero),

Z t
CeA(t−σ) Bδ(σ)dσ = CeAt B. (2.36)
0

The meaning of the elements of matrix W(t) is then the following: each element Wi j (t −τ) represents
the ensuing response of the i-th output at time t, due to an impulse applied at time τ to the j-th
input (Figure 2.2).

1 1
2 2
j W(t− τ) i
m p

Figure 2.2: A system with m inputs and p outputs and its impulse response matrix.

In the general case with D non-zero, we have

W(t) = CeAt B + Dδ(t),

where δ(t) is the impulse function at τ = 0.

Figure 2.3: Scheme of a simple elastic structure.

2.3. The exponential eAt 13

Example 2.5. (Impulse on an elastic structure.) Figure (2.3) represents a simple elastic structure,
consisting of three storeys that can shift horizontally relatively to one another. The lower storey
is placed on a cart. Take as inputs the forces u1 , u2 , u3 acting on the storeys and as outputs the
positions of the storeys, y1 , y2 , y3 . An impulsive input u1 (t) = δ(t − t0 ) applied to the first storey
(illustrated by an object hitting the first storey) causes the output W31 on the third storey.

2.3 The exponential eAt

In order to obtain qualitative information on the general solution of the system, we need to analyse
the exponential matrix eAt . This is a matrix of size n × n.
Computing the eigenvalues and the right and left eigenvectors of A, we can derive an expression
for eAt without using the power series that was previously introduced. Consider the case of A
with n distinct eigenvalues, so that there are t1 , . . . , tn distinct right eigenvectors that are linearly
independent and then form a basis. Let

T  [t1 , . . . , tn ]

be the invertible matrix formed by the right eigenvectors. Let also

 ⊤ 
 s1 
 s⊤ 
S = T −1 =  2 
 : 

s⊤
n

k A = λk sk , i.e., the rows of matrix S are left eigenvectors of A).

be its inverse matrix (note that s⊤ ⊤

Denoting by Λ the diagonal matrix of the eigenvalues of A, we can verify that

A = T ΛT −1 .

By noticing that Ak = T ΛT −1 T ΛT −1 T ΛT −1 . . . T ΛT −1 = T Λk T −1 , we can derive

s⊤
 
+∞
 +∞  n n
1
T Λk S tk X Λk tk    X
eAt = T eΛt S = Λt .. λi t
Zi eλi t
X X
= T   S = [t1 , . . . , tn ]e   = ti s ⊤ =
 
. e
 
i
k! k!  
k=0 k=0 s⊤
n
i=1 i=1

(note that eΛt is diagonal). Note that

Z i = ti s ⊤
i ,

where ti is a column of matrix T = [t1 t2 . . . tn ] and s⊤

i is a row of matrix S = T , are square
−1

matrices of size n × n, named components of A.

Remark 2.3. The product of a column vector and a row vector (unlike the product of a row vector
and a column vector) forms a matrix (and not a scalar). For example, if we take the vectors
" #
−1 h i
ti = s⊤
i = 1 2 ,
2

we obtain " #
−1 −2
Z i = ti s ⊤
i = ,
2 4

i ti = [3].
while s⊤
14 Chapter 2. Systems of differential equations

Example 2.6. (Computing eAt .) Consider the matrix

" #
−4 −1
A=
2 −1

Its characteristic polynomial is given by

s+4
" #
1
det(sI − A) = det = s2 + 5s + 6 = (s + 2)(s + 3)
−2 s + 1

Hence, the eigenvalues are λ1 = −2, λ2 = −3 and their respective eigenvectors can be computed as

−4 + 2
" #" # " #
−1 t1 1
(A − λ1 I)t¯1 = 0 ⇒ = 0 ⇒ t¯1 =
2 −1 + 2 t2 −2

and
−4 + 3
" #" # " #
−1 t1 1
(A − λ2 I)t¯2 = 0 ⇒ = 0 ⇒ t¯2 =
2 −1 + 3 t2 −1

At this point we can easily find the matrices T and S = T −1 :

" # " #
1 1 −1 −1
T= , S =T −1
=
−2 −1 2 1

Finally, the exponential can be expressed as

" # " −2t #" #
1 1 e 0 −1 −1
eAt = T eΛt S = =
−2 −1 0 e−3t 2 1
" # " #
1 h i 1 h i
= −1 −1 e + −2t
2 1 e−3t =
−2 −1
" # " #
−1 −1 −2t 2 1
= e + e−3t =
2 2 −2 −1
−e−2t + 2e−3t −e−2t + e−3t
" #
=
2e−2t − 2e−3t 2e−2t − e−3t

The expression
n
Zi eλi t ,
X
e At
= (2.37)
i=1

valid in the case of distinct eigenvalues, reveals that the elements of matrix eAt are linear combina-
tions of the functions eλi t , where λi are the eigenvalues of A.
The scalar functions eλi t are called the modes of the system. They are of fundamental impor-
tance for the system analysis.
Note that, in general, the eigenvalue of a real matrix can be complex. If the i-th eigenvalue is
complex, then
λi = ξi + ji ωi ⇒ eλi t = e(ξi + jωi )t = eξi t (cos(ωi t) + j sin(ωi t)) (2.38)
If there is a complex eigenvalue λ, then also its complex conjugate λ∗ is an eigenvalue. Moreover, if
λ is associated with the matrix Zi , then λ∗ is associated with the conjugate matrix Zi∗ . Therefore, the
2.3. The exponential eAt 15

exponential eAt can be decomposed into two summations that include, respectively, the contribution
of real eigenvalues and the contribution of complex eigenvalues:
r n−1
Zi eλi t + (Zi eλi t + Zi∗ eλi t ),
X X ∗
eAt = (2.39)
i=1 i=r+1, step 2

where r is the number of real eigenvalues. Replacing Zi = Mi + jNi and λ = ξ + jω gives:

r n−1
Zi eλi t + 2 eξi t (Mi cos(ωi t) − Ni sin(ωi t))
X X
eAt = (2.40)
i=1 i=r+1, step 2

From this expression one can see that the exponential of a real matrix is always a real matrix, as
expected.
In conclusion, from the analysis of the eigenvalues of a matrix we can obtain important
information about the natural response of the system, which depends on the modes eλi t .

2.3.1 The case of multiple eigenvalues

If the matrix A has distinct eigenvalues, then it can be diagonalised
 λ1 0 0 0
 

 0 λ 0 0 
2
A = T ΛT ,
−1
Λ =  .
 
 0 0 .. 0 


λn
 
0 0 0
This is no longer true in general when there are coincident eigenvalues: in this case, we can only
“block-diagonalise” A in the following way
 J1 0 0 0 
 
 0 J 0 0 
2
A = TJT , −1
J =   ,
 0 0 . . . 0 

(2.41)
 
0 0 0 Jn
where the blocks Jk are of the form
 λk 1 0 0
 

 0 λ 1 0 
k
Jk =  ..  .
 
(2.42)
 0 0 . 1 
λk
 
0 0 0
Then, we say that the matrix J is in Jordan canonical form, or in Jordan normal form.
Example 2.7. (Jordan form.) The following matrix is in Jordan form
 
 2 0 0 0 0 0 0 0 
 0 2 1 0 0 0 0 0 
 
 0 0 2 1 0 0 0 0 
 
 0 0 0 2 0 0 0 0 
A = 
 0 0 0 0 3 0 0 0 

 0 0 0 0 0 3 0 0 
 
 0 0 0 0 0 0 3 1 
 
0 0 0 0 0 0 0 3
16 Chapter 2. Systems of differential equations

In the presence of multiple eigenvalues, a fundamental parameter is the ascent deg(λk ) of the
eigenvalue λk , defined as the maximum size of the block associated with λk in the Jordan form. In
Example 2.7, we have
deg(2) = 3 and deg(3) = 2.

In this case, the exponential eAt has the following expression:

M deg(λk )−1

Zk j t j eλk t
X X
eAt = (2.43)
k=1 j=0

where M is the number of distinct eigenvalues and deg(λk ) the ascent of each of them.
The function
t j eλk t

is the most general form of a mode.

Example 2.8. The modes associated with matrix A in Example 2.7 are:

• λ1 = 2: e2t , t e2t , t2 e2t ;

• λ2 = 3: e3t , t e3t

Example 2.9. Given the matrix in the Jordan form

 
 −4 1 0 0 0 0 
0 −4 0 0 0 0
 
 
 0 0 −4 1 0 0 
A =   ,
 0 0 0 −4 1 0 

0 0 0 0 −4 1
 
 
0 0 0 0 0 −4

the associated modes are e−4t , t e−4t , t2 e−4t , t3 e−4t .

We now have an expression for the impulse response matrix. In the case of distinct eigenvalues,

n
Qi eλi t ,
X
W(t) = CeAt B = Qi = CZi B. (2.44)
i=1

In the case of multiple eigenvalues,

m deg(λk )−1

Qk j t j eλk t
X X
W(t) = CeAt B = Qk j = CZk j B. (2.45)
k=1 j=0

The modes appear in the impulse response matrix as well, hence they affect not only the natural
response, but also the forced response of the system.
2.4. Stability 17

2.4 Stability
Consider a linear time-invariant system with given initial condition and input:

ẋ(t) = Ax(t) + Bu(t)

(

x̄(0) known

Denote by x̄(t) the corresponding solution. Assume now to perturb the initial condition: x(t) , x̄(t)
will then be the new solution of the new system

ẋ(t) = Ax(t) + Bu(t)

(

x̄(0) + ∆x(0) known

The main question is how the two solutions are related: do they tend to stay close, to get closer, or
to split apart? To investigate this problem, we can study the difference of the two solutions,

z(t)  x(t) − x̄(t),

and its behaviour as t → +∞. If we take the derivative of z(t), we get the following relation:

ż(t) = ẋ(t) − x̄˙(t) = Ax(t) + Bu(t) − A x̄(t) − Bu(t) = A(x(t) − x̄(t)) = Az(t) (2.46)

Observation 2.1. The difference between the nominal solution and the perturbed solution evolves
as the natural response of the system, regardless of the input u(t).

We can now introduce the concept of stability of a system.

Definition 2.1.

• The system is stable if ∀ ε > 0 ∃ δ > 0 such that, if ∥x(0) − x̄(0)∥ = ∥z(0)∥ < δ, then
∥x(t) − x̄(t)∥ = ∥z(t)∥ ≤ ε.

• The system is asymptotically stable if the previous condition holds and moreover, for ∥z(0)∥ <
δ, we have lim ∥z(t)∥ = 0;
t→+∞

• The system is marginally stable if it is stable, but not asymptotically stable.

• The system is unstable if it is not stable.

Graphically, the situation is shown in Figure 2.4. For any sphere Sε of radius ε > 0, there
must exist a smaller sphere Sδ of radius δ > 0 such that, for any perturbation inside Sδ , the system
solution is confined within Sε . If, on top of this, z converges to 0, then the system is asymptotically
stable (see Figure 2.5).
Stability can be defined as a property of the system for linear systems only. In the nonlinear
case, it is only possible to discuss stability of a given equilibrium point. A nonlinear system may
have both stable and unstable equilibrium points, such as the pendulum in the upper and lower
positions.

Example 2.10. (The pendulum.) Figure 2.6 represents the pendulum in the lower equilibrium
position (on the left) and the inverted pendulum in the upper equilibrium position (on the right).
For small oscillations and small perturbations, the systems corresponding to the two cases can be
considered linear. The “linearised” system associated with the lower position is stable. Conversely,
the “linearised” system associated with the upper position is unstable, because for any disturbance,
no matter how small, the difference between nominal solution and perturbed solution will leave a
small sphere of radius ε.
18 Chapter 2. Systems of differential equations

z1
ε
δ
z(0)
z2

Figure 2.4: Stability of a system.

z1
ε
δ
z(0)
z2

Figure 2.5: Asymptotic stability: convergence of z(t) for t → +∞.

Figure 2.6: Pendulum and inverted pendulum.

2.4. Stability 19

Since the difference between nominal solution and perturbed solution evolves as the natural
response, stability can be analysed by considering the modes of the system.

Definition 2.2. The spectrum of A is the the set of eigenvalues of A:

σ(A) = {λ1 , . . . , λn }.

We have to remind that any eigenvalue can be written as λ = ξ + jω, where ω = 0 if the
eigenvalue is real; we denote as ℜ(λ) the real part of a complex number. Hence, the most general
expression of a mode is
tk eλt = tk eξt [cos(ωt) + j sin(ωt)]

If we consider the magnitude, we have the following cases:

• if ℜ(λ) < 0, then |tk eλt | → 0;

• if ℜ(λ) > 0, then |tk eλt | → ∞;

• if ℜ(λ) = 0, then |tk eλt | → ∞, unless k = 0, since in this case we have |t0 eλt | = |eλt | = 1.

To have asymptotic stability, all of the modes must converge to 0; to have stability, they must be
bounded. Then, the following fundamental properties hold.

Theorem 2.1. • The system is asymptotically stable if and only if, for any λ ∈ σ(A), we have
ℜ{λ} < 0.

• The system is stable if and only if, for any λ ∈ σ(A), we have ℜ{λ} ≤ 0 and, moreover, if
some λ ∈ σ(A) is such that ℜ{λ} = 0, then it must be deg(λ) = 1.

• The system is unstable if either there exists λ ∈ σ(A) with ℜ{λ} > 0 or there exists λ ∈ σ(A)
with ℜ{λ} = 0 and deg(λ) > 1.

We stress that marginal stability should be carefully considered and it cannot be enough for
engineering applications. Indeed, infinitely small perturbations of the system parameters can bring
the eigenvalues with zero real part to the unstable (right) half-plane where ℜ(λ) > 0. In addition,
marginal stability does not guarantee the damping of the modes, i.e., convergence, but only their
boundedness.

Example 2.11. Consider the following matrix in Jordan form:

 
 −2 0 0 0 0 0 
0 −3 0 0 0 0
 
 

 0 0 − j4 1 0 0 


 0 0 0 − j4 0 0 

 0 0 0 0 j4 1 
 
0 0 0 0 0 j4

The ascent of the eigenvalues having zero real part is equal to 2: hence, the system is unstable. If
we replace the “1” entries with two zero entries, the system becomes stable (marginally).
20 Chapter 2. Systems of differential equations

2.5 Transfer functions and frequency response

The Laplace transform is a fundamental tool to analyse linear time-invariant dynamical systems.
Applying the Laplace transform to a LTI system equations gives

ẋ(t) = Ax(t) + Bu(t) s x(s) − x(0) = Ax(s) + Bu(s)

( (
L
−→ (2.47)
y(t) = Cx(t) + Du(t) y(s) = Cx(s) + Du(s)

We stress that, for computing the transform, it is essential that the matrices A, B, C, D are
constant over time, meaning that the system is linear and time-invariant. Then we get

(sI − A)x(s) = x(0) + Bu(s)

⇒ x(s) = (sI − A)−1 x(0) + (sI − A)−1 Bu(s) (2.48)

which expresses the system state in terms of the Laplace transform. If we compare this with the
result obtained in the time domain, we can observe the following matches:
L
eAt −→ (sI − A)−1
Z t
L
eA(t−σ) Bu(σ)dσ −→ (sI − A)−1 Bu(s) (2.49)
0

The transformed output is given by

y(s) = C(sI − A)−1 x(0) + C(sI − A)−1 B + D u(s)

 
(2.50)

If we consider the forced response only, namely we set x(0) = 0, we have

y(s) = C(sI − A)−1 B + D u(s) = W(s)u(s),

 

N(s)
where W(s) = is a matrix of rational transfer functions (where each entry expresses the ratio
d(s)
between the Laplace transform of an output and the Laplace transform of an input). The term d(s)
is the characteristic polynomial of matrix A.
The elements of matrix W(s) are proper transfer functions, that is, the degree of numerator is
less than or equal to the degree of the denominator:

deg(ni j ) ≤ deg(d).

The strict inequality holds if and only if Di j = 0:

deg(ni j ) < deg(d) ⇔ Di j = 0.

In this case, the functions are called strictly proper. To summarise, the matrix of transfer functions
W(s) is proper, and it is strictly proper (it has all strictly proper components) if and only if D = 0.
The transfer function can be computed as follows. Consider the case with a single input and a
single output m = p = 1

1. Compute d(s) = det(sI − A).

2. Compute the numerator as follows

" #
(sI − A) −B
n(s) = det
C D
2.5. Transfer functions and frequency response 21

To prove the last equation let ψ(s)  (sI − A)−1 B and note that

n(s)
= C(sI − A)−1 B + D = Cψ(s) + D
d(s)

then
ψ(s)
" # " #  
(sI − A) −B 0
=  where o = 1
 
n(s) 
C D o d(s)

Now let us pretend to ignore the value of o = 1 and let us apply Cramer’s rule to determine o
 
 (sI − A) 0 
det  n(s) 
C d(s)
o= " #
(sI − A) −B
det
C D

Now we suddlently remind that o = 1 and we notice that the numerator is the determinant of a
block–triangular matrix, hence
  " #
 (sI − A) 0 n(s) (sI − A) −B
 = det(sI − A) = n(s) = det

det  n(s)
C d(s) d(s) C D

It is not difficult to see that for generic m and n the numerators polynomials are
" #
(sI − A) −B j
ni j (s) = det
Ci Di j

2.5.1 Frequency response

Consider a system with a single input and a single output (SISO) and assume x(0) = 0. For brevity,
we also assume D = 0. The system output evolves as
Z t
L
y(t) = W(t − σ)u(σ)dσ −→ y(s) = W(s)u(s)
0

Now suppose that we apply to the system an input of the form

( σt
e if t ≥ 0
u(t) = (2.51)
0 if t < 0

where σ is a complex number. Note that, if σ = 0, this input signal is the unit step. If σ = jω, we
have
e jωt = cos(ωt) + j sin(ωt).
Both are meaningful signals to test the system.
From the well-known properties of the Laplace transform, we have:

1
uσ (s) =
s−σ
Assuming σ < σ(A) (σ is not in the spectrum of A, namely, it is not an eigenvalue), we obtain:

1 1 1
y(s) = W(s) = W(σ) + (W(s) − W(σ)) (2.52)
s−σ s−σ s−σ
22 Chapter 2. Systems of differential equations

It can be shown that the second term does not have poles at s = σ, since the term s − σ cancels out.
Then
1
y(s) = W(σ) + W̃(s), (2.53)
s−σ
ñ(s)
where W̃(s) = and p(s) is the characteristic polynomial of matrix A. Hence, the poles of W̃(s)
p(s)
are in the spectrum of matrix A. By applying the inverse Laplace transformation, we get:
y(t) = W(σ)eσt + W̃(t), (2.54)
where W̃(t) is a linear combination of the modes. In the case of distinct eigenvalues, we have
n
σt
q̃i eλi t .
X
y(t) = W(σ)e + (2.55)
i=1

The first term is called steady-state response, while the second is the transient response and
depends on the modes only. If the system is asymptotically stable, the transient response tends to
zero as t → +∞.
In particular, the steady-state response to the test signal for σ = jω is
y jω (t) = W( jω)e jωt = |W( jω)|e jωt+φ( jω) =
= |W( jω)| cos(ωt + φ( jω)) + j|W( jω)| sin(ωt + φ( jω)). (2.56)
This means that, if the input is a sine or cosine wave of pulsatance ω, the resulting output is a sine
or cosine with the same pulsatance, amplified by a factor |W( jω)| and shifted by an angle equal to
φ( jω)) = arg{W( jω)}. This can be represented in terms of phasors, as shown in Figure 2.7.

Im
jωt
W(jω ) e

t
ϕ ejω

Re

Figure 2.7: Phasors involved in the frequency response of a system.

2.6 Discrete-time systems

Discrete-time systems are similar to continuous-time systems. The substantial difference is that
they are not described by differential equations, but by difference equations, in which input, output
and state u(k), y(k), x(k) are sequences defined for k ∈ Z.
Example 2.12. (Fibonacci sequence.)
y(k + 2) = y(k + 1) + y(k)
is a difference equation that describes a discrete-time system. For y(0) = y(1), this difference
equation gives the Fibonacci sequence.
2.6. Discrete-time systems 23

As in the case of continuous-time systems, the variables can be expressed in a vector form,
x(k) ∈ Rn , u(k) ∈ Rm , y(k) ∈ R p . Then, the discrete-time equations are
x(k + 1) = f (k, x(k), u(k))
(
(2.57)
y(k) = g(k, x(k), u(k))
Again, this notation is equivalent to the following set of equations
xi (k + 1) = fi (k, x1 (k), . . . , xn (k), u1 (k), . . . , um (k)) for i = 1, . . . , n
y j (k) = g j (k, x1 (k), . . . , xn (k), u1 (k), . . . , um (k)) for j = 1, . . . , p

Example 2.13. (Euler approximating system.) The following equations

ẋ1 (k + 1) = x1 (k) + τ[−αx1 (k)x2 (k) + u1 (k)]



ẋ2 (k + 1) = x2 (k) + τ[−αx1 (k)x2 (k) + u2 (k)]




 ẋ (k + 1) = x (k) + τ[ αx (k)x (k) − γx (k)]

3 3 1 2 3

where τ is a small parameter, represent a nonlinear discrete-time system. If we assume that the
time variables are taken from a continuous time axis that has been discretised with step τ,
t = kτ, x(k) = x(kτ), u(k) = u(kτ),
and τ is omitted to have a simpler notation, then this system is the Euler approximating system of
the equations (2.6). For τ small enough, the sequence x(kτ) approximates the exact solution of the
system (2.6).
In general, given any system ẋ(t) = f (x(t), u(t)), we can numerically approximate its solution
by means of the Euler approximating system
x((k + 1)τ)) = x(kτ)) + τ f (x(kτ), u(kτ))
This is the simplest method to numerically compute the solution of a continuous-system (of course,
there are more sophisticated methods that can be used).

A discrete-time system is invariant or autonomous if the functions f and g do not depend

directly on time k:
x(k + 1) = f (x(k), u(k))
(
(2.58)
y(k) = g(x(k), u(k))
Example 2.14. The system
! !
k 1
x(k + 1) = x(k) + u(k)
k+1 k+1
provides at each time k the average of all the past input values:
u(k − 1) + u(k − 2) + · · · + u(0)
x(k) = .
k
This system is not autonomous. Conversely, the system providing the Fibonacci sequence is
autonomous (time-invariant).
A discrete system is linear if f and g are linear with respect to x and u:
x(k + 1) = A(k)x(k) + B(k)u(k)
(
(2.59)
y(k) = C(k)x(k) + D(k)u(k)
A discrete system is linear and time-invariant if it satisfies both conditions:
x(k + 1) = Ax(k) + Bu(k)
(
(2.60)
y(k) = Cx(k) + Du(k)
24 Chapter 2. Systems of differential equations

2.6.1 General solution of a discrete-time linear time-invariant system

The solution of a discrete-time system can be recursively computed. However, we are interested in
qualitative information. As we have done for continuous-time systems, in the linear case we split
the problem into two distinct parts:

• natural response (assigned initial conditions, null input):

xL (k + 1) = AxL (k)
(
(2.61)
xL (0) = x(0) known

• forced response (null initial conditions, assigned inputs):

xF (k + 1) = AxF (k) + Bu(k)

(
(2.62)
xF (k) = 0

We have
x(k) = xL (k) + xF (k)
It is not difficult to verify that the general solution is:
k−1
X
x(k) = Ak x(0) + Ak−h−1 Bu(h)
h=0
k−1
X
y(k) = CAk x(0) + CAk−h−1 Bu(h) + Du(k) (2.63)
h=0

Assume that matrix A has n distinct eigenvalues. Then there are n linearly independent eigenvectors
that we can group in an invertible matrix T

T = [t1 , t2 , . . . , tn ]

Let us denote  ⊤ 
 s1 
 s⊤ 
S = T −1 =  2 
 : 

s⊤
n

as before. We remind that Ak = T Λk S . Hence, along the same lines as in the previous computation
for the continuous-time case, we find
n
X
Ak = Zi λki , (2.64)
i=1

where Zi = ti s⊤
i . The sequences λi are called modes of the discrete-time system, or discrete-time
k

modes.
As expected, the modes affect both the natural response and the forced response. Indeed, for
any eigenvalue λ ∈ C we have
λ = |λ|e jθ ,
where θ = 0 if the eigenvalue is real. Then

λk = |λ|k e jθk = |λ|k cos(θk) + j sin(θk)

 
2.6. Discrete-time systems 25

Note that the magnitude of this sequence is increasing if |λ| > 1 or decreasing if |λ| < 1 and
constant if |λ| = 1. Then we can already see the similarity of the theory for continuous and discrete
systems: the only difference is that the real and imaginary part of eigenvalues are considered in
the continuous-time case, while the magnitude and phase of the eigenvalues are considered in the
discrete-time case.
In the case of distinct eigenvalues, we can write the following expression for Ak
r
X n−1
X
A =
k
Zi λki +2 |λi |k (Mi cos(θi k) − Ni sin(θi k)) (2.65)
i=1 i=r+1, step 2

in the presence of r real eigenvalues.

If matrix A has multiple eigenvalues, we have a more general formula:
m deg(λ
Xi )−1 !
X k
Ak = Zih λki (2.66)
h
i=1 h=0

Note that
k(k − 1) . . . (k − h + 1)
!
k
= = ph (k)
h h!
is a polynomial in k of degree h. The maximum degree is the ascent of the eigenvalue.
The expression for the modes can be obtained by taking into account that, for any matrix A,
A = T J k S , where J is in Jordan form. Raising to a power k a matrix in Jordan form, we get
k

J k = blockdiag{J1k , J2k , . . . , J sk }

a block diagonal matrix with the powers of the Jordan blocks. For each Jordan block
k !
X k
(λI + J0 )k = λk−h J0h
h
h=0

where J0 is
...
 
 0 1 0 0 
0 0 1 ... 0
 
.. .. .. ..
 
J0 =  ..
.
 
. . . . 
...
 
 0 0 0 1 
...
 
0 0 0 0
It is easy to check that, for h larger than the ascent of the eigenvalue, we have J0h = 0.

Example 2.15. Consider of a matrix with a single eigenvalue in Jordan form

 
 3 1 0 0 0 
 0 3 0 0 0 
J =  0 0 3 1 0 
 

 0 0 0 3 1 
 
0 0 0 0 3

The three modes associated with the corresponding discrete-time system are
! !
k k
3,k
3 = p1 (k)3 ,
k k
3k = p2 (k)3k .
1 2
26 Chapter 2. Systems of differential equations

We can also define the impulse response matrix as follows:

W(0) = D, and W(k) = CAk B, k = 1, 2, . . .

Each element Wi j (k) = Ci Ak B j is the response of the i-th output when the discrete impulse
{1, 0, 0, . . . , 0} is applied as the j-th input.

2.6.2 Discrete-time stability

To analyse the stability of discrete-time linear systems, note that the general expression of a mode is

ph (k)λk = ph (k)|λ|k [cos(θk) + j sin(θk)]

! only if |λ| < 1 and is bounded if and only if |λ| ≤ 1 and

Therefore, a mode converges to zero if and
k
the polynomial has degree 0: p0 = = 1. We conclude the following.
0
Theorem 2.2. A discrete system is
• asymptotically stable if and only if ∀ λ ∈ σ(A) it holds |λ| < 1;

• stable if and only if ∀ λ ∈ σ(A) we have |λ| ≤ 1 and, if |λ| = 1, then deg(λ) = 1;

• unstable if and only if there exists either λ ∈ σ(A) with |λ| > 1, or λ ∈ σ(A) with |λ| = 1 and
deg(λ) > 1.

2.7 State transformation

For reasons that will become clear later on, it is often useful to introduce a change of variables in
the natural representation of a system. Given an invertible matrix associated with a new basis for
the system state space,
T = [t¯1 , t¯2 , . . . , t¯n ] ,
we define the transformation
x̂ = T −1 x
(

x = T x̂
Applying this transformation to a linear autonomous system, we get:

ẋ(t) = Ax(t) + Bu(t) ˙ = AT x̂(t) + Bu(t)

( (
T x̂(t)
⇒
y(t) = Cx(t) + Du(t) y(t) = CT x̂(t) + Du(t)
˙ = T −1 AT x̂(t) + T −1 Bu(t)
(
x̂(t)
⇒ (2.67)
y(t) = CT x̂(t) + Du(t)
So the system has a new representation:

ẋ(t) = Âx(t) + B̂u(t)

(
(2.68)
y(t) = Ĉx(t) + D̂u(t)
where the matrices have the following expressions:

 = T −1 AT (2.69)
B̂ = T −1
B (2.70)
Ĉ = CT (2.71)
D̂ = D (2.72)
2.8. Sampled-data systems 27

Note that matrix D is unchanged. This was expected, because D represent the relationship between
y(t) and u(t), which are not transformed.
Note also that, in the transition to new representation, matrix A undergoes a similarity transfor-
mation. Characteristic polynomial and (thus) eigenvalues are invariant under this transformation,
hence the modes and the stability properties of the system are unchanged. Also the system
transfer function matrix is unchanged, as expected since input and output are unaltered by the
transformation.

2.8 Sampled-data systems

Nowadays, signal processing occurs almost always through the use of computers, which are known
to evolve in discrete time. The theory for discrete- and continuous-time systems has strong analogies
that can be exploited. Clearly, when a continuous-time system must interact with a discrete-time
system, a fundamental aspect is to understand how it is possible to convert continuous-time signals
(coming from the outside world into the digital system) in discrete-time signals (evolving within
the computer) and vice versa.

2.8.1 Analog-to-Digital conversion

The conversion of an analog signal into a digital signal occurs according to the block diagram
depicted in Figure 2.8.

t kTs

Figure 2.8: Analog-to-Digital conversion.

The block marked with A/D is an analog-to-digital converter and the implemented operation
is called sampling. It is assumed here that the signal is sampled with a fixed step T s . In general
it is also possible that a vector signal (multiple input channels) is sampled with different rates
for different components. These are called multi-rate systems, a case we will not consider. The
sampling operator is linear, therefore it does not compromise the linearity of the system:
A/D
f (t) −→ f (kT s )
A/D
α f (t) + βg(t) −→ α f (kT s ) + βg(kT s ) (2.73)

A basic scheme of an analog-to-digital converter is depicted in Figure 2.9. When the START
signal switches to 1, it enables a ramp generator and a counter. The counter is stopped when the
value r reaches the signal s. The counter provides then an integer number s∗ proportional to s (with
a roundoff error). The time required for conversion must be much smaller than the sampling time
and it must be very small compared to the inverse of the signal bandwidth.

2.8.2 Digital-to-Analog conversion

This type of operation is certainly less straightforward than analog-to-digital conversion. In
principle, given N data points, interpolation allows us to find a function (for example, a polynomial
28 Chapter 2. Systems of differential equations

LEVEL
SIGNAL COMPARATOR

RAMP
COUNTER
GENERATOR

Figure 2.9: Qualitative scheme of an Analog-to-Digital Converter.

of degree N − 1) that passes through these N data points. In the case of polynomial interpolation,
this reduces to the following linear equations

a0 + a1 xi (kT s ) + a2 xi2 (kT s ) + . . . + an xin−1 (kT s ) = f (kT s ) ∀ i = 1, . . . , N (2.74)

that allow us to determine the coefficients of the interpolating polynomial. However, in order to
apply this method and find the polynomial, we must know all of the N samples. Then this method
has to be adopted off-line. This is obviously not feasible in real time applications. It is necessary
to find a way to convert on-line the samples, to generate the continuous-time signal in real time.
An easy way to convert on-line is to use the so-called zero order hold (Z.O.H.). This method
simply creates a piecewise constant function, whose value is equal to the value of the last available
sample (see Figure 2.10).

kTs t

Figure 2.10: Zero Order Hold.

This operation creates from the samples f (kT s ) a piecewise constant function whose value is

fˆ(t) = f (kT s ) for kT s ≤ t < (k + 1)T s .

There are also higher-order holders (see Figure 2.11 for the case of order 1), but nowadays the
sampling time can be assumed so small that the information lost by adopting zero order holders is
practically negligible (given that the error decreases when T s decreases, as shown in Figure 2.12).
In fact, methods based on higher order holders are seldom used.

kTs t

Figure 2.11: First Order Hold.

2.8. Sampled-data systems 29

Figure 2.12: Error with Zero Order Hold.

2.8.3 Aliasing
We now briefly recall the problem of aliasing, which has to be carefully considered when we use
a sampling device. We consider the case of a sinusoidal signal. This is not restrictive, since it
is known that periodic functions can be described by a Fourier series expansion that consists of
harmonic functions, and the most reasonable signals admit a Fourier transform that expresses their
harmonic content.
Let us denote by
2π
Ωc =
Ts
the sampling pulsatance. Consider the continuous-time signal cos(ω1 t): by sampling the signal
with step T s , we obtain a sequence cos(ω1 kT s ). It is easily seen that the same sequence can be
obtained if we sample, with the same step T s , a signal cos(ω2 t) with a different pulsatance ω2 ,
provided that the following equality holds:

cos(ω2 kT s ) = cos(ω1 kT s + mk2π)

2π
⇒ ω2 = ω1 + m = ω1 + m ΩC , with m ∈ Z (2.75)
Ts

Therefore, if the expression

ω2 − ω1 = m ΩC , m∈Z

holds, the two signals are indistinguishable under sampling. In general, if a signal includes two
components of this kind, it is impossible to correctly reconstruct the signal.

|f|

ω
−ω ω

Figure 2.13: Spectrum of a band-limited signal.

30 Chapter 2. Systems of differential equations

Signals processed in real problems typically are band-limited: their Fourier transform, or their
spectrum, has a bounded support, as is qualitatively shown in Figure 2.13. A band-limited signal can
always be fully reconstructed from its samples, provided that a fundamental condition is satisfied
(Nyquist-Shannon sampling theorem). For a band-limited signal whose band is between −ω̄ and
ω̄, reconstruction is possible after sampling as long as

2ω̄ < ΩC , (2.76)

because in this case (2.75) can be satisfied only for m = 0, hence ω2 = ω1 . Replacing ω̄ = 2π f¯ and
Ωc = 2π fc , where f¯ is the maximum signal frequency and fC is the sampling frequency, we get
f¯ < f2C or
fC > 2 f¯, (2.77)
namely, to be able to fully reconstruct the signal after sampling, the sampling frequency must
exceed twice the maximum frequency in the band-limited signal. The minimum sampling frequency
fCmin = 2 f¯ is called the Nyquist rate. This fundamental result is a consequence of the sampling
theorem, stating that the continuous-time signal is completely recoverable, at least in principle,
starting from the individual samples provided that the condition above is satisfied.
The proof of sampling theorem, due to Shannon in 1949, was a fundamental step for signal
theory and telecommunications, because it allowed the transition from analog to digital signals
processing.
fC
One final consideration concerns the condition f¯ < and its applicability. In reality, being
2
the signals unknown and unpredictable, we cannot know the signal bandwidth in advance. In
these cases, however, once the sampling frequency of the devices is known, the received signal is
pre-filtered in order to eliminate all of the spurious spectral components whose frequency is higher
fC
than . This filtering effect often occurs naturally in many devices, as microphones or sensors.
2
Otherwise, the introduction of an analog filter is necessary.

Example 2.16. The frequencies audible by human ears are up to about 20 kHz. This means
that each spectral component whose frequency is above this value is not detectable by humans.
Microphones have a cutoff frequency of about 20 kHz, so basically greater frequencies are not
recorded. In view of the condition fC > 2 f¯, the sampling frequency for digital music is usually
chosen as fC = 44, 1 kHz.

2.9 Discrete-time equivalent of a continuous-time system

Consider the scheme shown in Figure 2.14, representing a typical situation where a continuous-time
process is controlled by means of a discrete-time device (e.g., a computer). A/D and D/A converters
in the figure are used to allow the communication between continuous-time and discrete-time
subsystems.
To study the entire system, composed of both the process and the controller, it is convenient to
have consistent equations (defined on the same time domain). Then, there are two possibilities:
a) compute a discrete-time system that is equivalent to the continuous-time process;
b) compute a continuous-time system that is equivalent to the discrete-time controller.
We now concentrate on case a) and compute the discrete-time equivalent of the continuous-time
process. We assume that the digital-to-analog converter is a zero order holder, hence the signal it
generates as an output is piecewise constant:

u(t) = u(kT s ) for kT s ≤ t < (k + 1)T s .

2.9. Discrete-time equivalent of a continuous-time system 31

PROCESS

D/A A/D

REGULATOR

Figure 2.14: Discrete-time control of a continuous-time process.

The continuous-time linear process governed by equations

ẋ(t) = Ax(t) + Bu(t)

(

y(t) = Cx(t) + Du(t)

has then a constant input during the sampling interval. However, this does not mean that x(t) and
y(t) are constant during the sampling interval as well. We can only observe that x(t) is continuous
and is differentiable everywhere, with the exception of the points kT s (because u is not continuous at
these points). The output y(t) of the continuous-time process enters the analog-to-digital converter,
and for this reason we can consider it to be relevant at instants kT s that are multiples of the sampling
time T s . So, we look at the relationship between u(kT s ), x(kT s ), x((k + 1)T s ) and y(kT s ).
If u(kT s ) and x(kT s ) are known, the continuous-time system solution at a time t > kT s is
Z t
x(t) = e A(t−kT s )
x(kT s ) + eA(t−σ) Bu(σ)dσ (2.78)
kT s

The time value kT s has now been taken as the initial time, instead of t = 0 (this is allowed because
the system is time invariant). If we evaluate this expression for t = (k + 1)T s , we get:
Z (k+1)T s
x((k + 1)T s ) = e AT s
x(kT s ) + eA((k+1)T s −σ) Bu(kT s )dσ, (2.79)
kT s

where we have set u(σ) = u(kT s ) because the input is constant in the integration interval [kT s , (k + 1)T s ).
Hence, u(kT s ) can be taken outside the integral
"Z (k+1)T s #
x((k + 1)T s ) = e AT s
x(kT s ) + e A((k+1)T s −σ)
Bdσ u(kT s ) (2.80)
kT s

and the term in brackets can be simplified by means of the change of variables (k + 1)T s − σ = ξ
Z (k+1)T s Z 0 Z Ts
e A((k+1)T s −σ)
Bdσ = e B(−dξ) =
Aξ
eAξ Bdξ (2.81)
kT s Ts 0

to see that the integral does not depend on k. Finally, we get:

"Z Ts #
x((k + 1)T s ) = e AT s
x(kT s ) + Aξ
e Bdξ u(kT s )
0
32 Chapter 2. Systems of differential equations

Therefore, we can write

x((k + 1)T s ) = AD x(kT s ) + BD u(kT s )

(
(2.82)
y(kT s ) = C D x(kT s ) + DD u(kT s )

where Z Ts
AD = eAT s , BD = eAξ Bdξ, C D = C, DD = D (2.83)
0

The graphic representation of the equivalent discrete-time system is shown in Figure 2.15.

s s
,D)

Figure 2.15: Discrete-time system equivalent to the continuous-time system.

Observe that the discrete-time system we have found is exactly equivalent to the continuous-
time system to which the D/A and A/D converters have been applied (before the system and after
the system, respectively), resulting in the digital input u(kT s ) and the digital output y(kT s ). Also,
note that the matrices C and D are unchanged.
These results are very important in systems and control theory, since they allow us to design
a regulator indifferently in the continuous- or in the discrete-time domain. Nowadays, a digital
implementation is highly preferred for its flexibility, reliability and cheapness.
It can be shown that asymptotic stability of the continuous-time system represented by ma-
trices A, B, C, D is equivalent (necessary and sufficient condition) to asymptotic stability of the
discrete-time system with matrices AD , BD , C D , DD . This is true because, if A has eigenvalues
{λ1 , λ2 , . . . , λn }, then AD = eAT s has eigenvalues {eλ1 T s , eλ2 T s , . . . , eλn T s } and, if λk has a negative
real part, then the modulus of eλ1 T s is smaller than 1 (and vice-versa).
RT
Example 2.17. (Calculation of the integral 0 s eAξ Bdξ.)
Using the expression eAξ = T eΛξ S that we have previously derived, we get:
Z Ts Z Ts "Z Ts #
Λξ Λξ
e Bdξ =
Aξ
T e S Bdξ = T e dξ S B
0 0 0

and then we need only to calculate the individual integrals of the exponential eΛt , which are placed
on the diagonal.
Z Ts h i
Example 2.18. (Formula for A invertible.) If A is invertible, we have eAt dt = A−1 eAT s − I .
0
This can be verified by considering that
+∞ +∞ k +∞
d At d X (At)k X A X Ak
e = = kt k−1
= tk−1 =
dt dt k=0 k! k=0
k! k=1
(k − 1)!
+∞ +∞
X Ak−1 k−1 X (At)h
=A t = A = A eAt
k=1
(k − 1)! h=0
h!

d h −1 At i
In fact , we have that A e = eAt , hence the definite integral between 0 and T s is the proposed
dt
expression.
2.9. Discrete-time equivalent of a continuous-time system 33

Example 2.19. (Discrete-time realisation of a controller.)

Let us compute the discrete-time equivalent of the continuous-time PI (proportional-integral)
controller:
1
KP + KI .
s
Assuming that
! the input is e(t) and output u(t), in terms of Laplace transform we have u(s) =
1
KP + KI e(s). A continuous-time state representation is
s

ẋ(t) = e(t)
(

u(t) = KP e(t) + KI x(t)

associated with the matrices

A = [0] , B = [1] , C = [KI ] , D = [KP ]

Therefore, the equivalent discrete-time representation requires the following matrices:

h i h i
AD = eAT s = e0T s = [1]
"Z T s #
BD = eAξ Bdξ = [T s ]
0
C D = [KI ]
DD = [KP ]

and has the form

x((k + 1)T s ) = x(kT s ) + T s e(kT s )
(

u(kT s ) = KI x(kT s ) + KP e(kT s )

To actually implement this controller on the computer, we can adopt the following algorithm.

1. Initialise x;

2. read e;

3. compute u = KI x + KP e;

4. write u in the digital-analog converter;

5. update x := x + T s e;

6. wait for the next sampling instant;

7. return to 2 if the job is still active, otherwise STOP.

Chapter 3

Dynamical systems

3.1 General consideration

Although the systems considered in the course are mainly described by differential equations, or
difference equations, it is appropriate to introduce a more general definition of dynamical system.
Before giving such a definition, we discuss two simple examples.

v R v R

Figure 3.1: Two different types of systems.

Example 3.1. Consider Figure 3.1, in which two different types of electrical circuit are represented,
the first including just a resistor R and the second formed by a simple R-L circuit. The laws
governing the two circuits are:
v(t)
• for the first system, i(t) = ;
R
Z t
di(t) 1
• for the second system, L = v(t) ⇒ i(t) = i(t0 ) + v(σ)dσ.
dt L t0

In both cases, we can take as input (cause) the voltage and as output (effect) the current. To
compute the output of the R circuit, we only need to know the function v(t). However, the R-L
system is governed by a differential equation and the knowledge of v(t) is not enough to uniquely
determine the value of the current. In fact, we also need the value of i(t0 ), the initial condition of
the system, for some initial time t0 . These type of systems, for which it is necessary to know the
initial conditions at a given initial time instant, are called dynamical.
For a dynamical system, the evolution after a time t0 depends on both the input and the past
history of the system, and not only on u(t). One question we might ask is: besides u(t), do we
need to know the entire past history of the evolution of the system in order to calculate its future
evolution? The answer is no: if we are interested in what happens after an initial time t0 , it is

35
36 Chapter 3. Dynamical systems

enough to know the state of the system at t0 : information concerning the previous instants is
redundant. This concept is the notion of state of the system.
State of the system (meta-definition) The state of a system at time τ is given by all the
information we need in order to uniquely determine the future evolution for t > τ, given the input
function u(t).

Example 3.2. In the case of the R-L circuit, a state variable is the current i(t). Indeed we can write
Z t Z t0 Z t Z t
1 1 1 1
i(t) = v(σ)dσ = v(σ)dσ + v(σ)dσ = i(t0 ) + v(σ)dσ (3.1)
L −∞ L −∞ L t0 L t0

where we assumed i(−∞) = 0. Note that we can arrive at the same value i(t0 ) via completely
different trajectories, but this does not affect the future evolution.

3.2 Causal dynamical systems

In this section we provide a formal definition of a dynamical system. The main purpose is to make
it clear that under the term “Dynamical Systems” we include objects which are much more general
than the “simple” systems we analyse in the course.
Consider the following sets.

• A set of times T , which is totally ordered (that is, if t1 , t2 , then either t1 < t2 or t2 < t1 ). In
our case, we mainly consider T = R or T = Z. Clearly, any sequence of ordered instants tk ,
not necessarily equi-spaced, is also appropriate.

• A set of states X. It can be any set (finite or not). We will typically use Rn .

• a set of inputs U. It can be any set (finite or not). We will mainly adopt Rm .

• A set of outputs Y. We will mainly adopt R p .

• A set of input functions Ω = {U(. . .) : T → U| condition}. As an example, in the case of

an actuator with a minimum and a maximum value, this set will be the set of continuous
functions defined on an interval.

• A set of output functions Γ = {Y(. . .) : T → Y| condition}. In the case of a sensor, output

functions will take values in an interval between the minimum and the maximum values.

Then we introduce the transition function φ as

φ:T ×T ×X×Ω→ X
(
(3.2)
x(t) = φ(t, τ, x(τ), u(·))

where t is the current time, τ the initial instant, x̄ the initial state. The notation u(·) indicates the
dependence on the entire function and not only on its value at time t. To define a dynamic system,
the transition function must satisfy the following axioms.

• Consistency:
φ(τ, τ, x̄, u(·)) = x̄ ∀ τ ∈ T, x̄ ∈ X, u ∈ Ω (3.3)

In essence, at the initial time t = τ the function must give the initial state x(τ) = x̄.
3.2. Causal dynamical systems 37

• Composition: Given three time instants t1 ≤ t2 ≤ t3 ,

x(t3 ) = φ(t3 , t1 , x1 , u(·)) = φ(t3 , t2 , x2 , u(·)) (3.4)
= φ(t3 , t2 , φ(t2 , t1 , x1 , u(. . .)), u(. . .))
∀ t1 ≤ t2 ≤ t3 ∈ T, x ∈ X, u ∈ Ω (3.5)
This means that, if the state x(t2 ) is given, there is no need to know the history of the system
before t2 to determine the future x(t) for t ≥ t2 .
• Forward definition: the function φ must be defined for all t ≥ τ ∀ x ∈ X, u ∈ Ω. For t < τ,
the transition function may be not defined. However, it may be defined for all t as well; this
can occur in many physical systems in which it is possible to trace the past evolution of a
system: systems for which this happens are called reversible.
• Causality: given two inputs u1 (t) and u2 (t), overlapping (u1 (t) = u2 (t)) on the interval
t1 ≤ t ≤ t2 (but they may differ outside of this range), we have:
φ(t, t1 , x, u1 (. . .)) = φ(t, t1 , x, u2 (. . .)), ∀ t in the interval (3.6)
In practice, if we are interested in the range of time [t1 , t2 ], given the state in t1 , information
on previous times is not required since it is included in x(t1 ), while information after t2 (the
future) is not relevant since it does not affect the past.
As far as the output of a system is concerned, we introduce the following function:

η:T ×X×U →Y
(
(3.7)
y(t) = η(t, x(t), u(t))
If y(t) = η(t, x(t)), the system is said strictly proper (in the case of linear systems, this means that
matrix D is equal to zero).

Example 3.3. The axioms above can be checked for the known expression of the solution of a
linear time-invariant system
Z t
x(t) = e A(t−t0 )
x(t0 ) + eA(t−σ) Bu(σ)dσ
t0

This is a transition function. In fact, we have

• x(t0 ) = I x(t0 ) because eA(t0 −t0 ) = I and the integral vanishes, so we have consistency;
• given the instants t1 ≤ t2 ≤ t3 , we have
Z t3
x(t3 ) = e A(t3 −t1 )
x(t1 ) + eA(t3 −σ) Bu(σ)dσ
t1
Z t2 Z t3
= e A(t3 −t2 ) A(t2 −t1 )
e x(t1 ) + e A(t3 −σ)
Bu(σ)dσ + eA(t3 −σ) Bu(σ)dσ
t1 t2
Z t2 Z t3
= eA(t3 −t2 ) eA(t2 −t1 ) x(t1 ) + eA(t3 −t2 ) eA(t2 −σ) Bu(σ)dσ + eA(t3 −σ) Bu(σ)dσ =
t t
" Z1 t2 # Z t3 2
= eA(t3 −t2 ) eA(t2 −t1 ) x(t1 ) + eA(t2 −σ) Bu(σ)dσ + eA(t3 −σ) Bu(σ)dσ
t1 t2
Z t3
= eA(t3 −t2 ) [x(t2 )] + eA(t3 −σ) Bu(σ)dσ
t2
38 Chapter 3. Dynamical systems

• x(t) is surely defined for t ≥ t0 because the integral is defined for these values; in this case
the system is reversible, so it is defined for t < t0 as well;
• if u1 (t) = u2 (t) in an interval [t1 , t2 ], then the integral does not change by taking different
values u1 (t) , u2 (t) outside the interval, so we have causality.
The properties of the transition function are also verified for discrete-time systems. In this case,
the exact calculation of φ is possible by recursion. However, for discrete-time systems the property
of reversibility, typically verified in continuous-time, may fail. For example, the system described
by the equation
x1 (k + 1)
" # " #" #
0 1 x1 (k)
=
x2 (k + 1) 0 0 x2 (k)
is non-reversible, because the state matrix (A) is not invertible.

We may wonder when the system of equations

ẋ(t) = f (t, x(t), u(t))
(

x(τ) known
(which is not a transition function) can be associated with a dynamical system. Under the assump-
tions of existence and uniqueness of the solution, x(t) is uniquely determined by the input and by
x(τ). Therefore, in principle there exists a function x(t) = φ(t, τ, x(τ), u(. . .)), which is precisely
the transition function, even though most of the times it is impossible to know or to calculate
analytically, without resorting to some numerical technique.
A mathematical problem that arises when we deal with differential equations is that, in patho-
logical cases the uniqueness of the solution might be not insured. For example, the differential
equation √
ẋ(t) = x(t)
(

x(0) = 0
has infinite solutions. This happens because the considered function f is not Lipschitz. This initial
value problem is therefore not a candidate to describe a dynamic system. The differential equation
ẋ(t) = x(t)2
(

x(0) = 1
has a unique solution but it is not defined for all t > 0. Hence, this is not a candidate to describe a
dynamic system if we take as T the real axis.

3.3 Automata as dynamical systems

A class of dynamical systems of interest are the finite automata, where the sets X, U, Y are finite:

U = {u1 , u2 , . . . , un }
Y = {y1 , y2 , . . . , y p }
X = {x1 , x2 , . . . , xm } (3.8)

An automaton is typically represented by a function of the form

x(k + 1) = F(k, x(k), u(k)) (3.9)

3.3. Automata as dynamical systems 39

that provides the state at the next time instant given the current time, the current state and the
current input. In the case of time-invariant automata, the expression has the form

x(k + 1) = F(x(k), u(k)). (3.10)

There are basically two ways to effectively represent an automaton: a table or a graph that represent
the function F.
The table is a m × n matrix, whose n columns are associated with the system states, and whose
m rows with the inputs. Every element Fi j of this matrix, called transition table, represents the
future state in the case the current automaton state is x = j and the current input is u = i.

x1 x2 ... xn
u1 F11 F12 ... F1n
u2 F21 F22 ... F2n
.. .. .. .. ..
. . . . .
um Fm1 Fm2 ... Fmn

The complete description of the automaton can be also given, as said, in terms of flow graph.
Consider for example Figure 3.2, where X = {1, 2, 3, 4}, U = {1, 2, 3}, Y = {0, 1}. This graph

1 1,2 2
1,2 1 1 3

3 3
1,3

1 3 4
0 2 0
2

Figure 3.2: Flow graph for finite automata.

corresponds to the table

x1 x2 x3 xn
u1 1 1 3 2
u2 1 1 4 4
um 3 2 2 2

The four possible states are represented as circles divided in two parts: the upper part shows the
state index, the lower shows the output value associated with that state. Every input changes state
of the system in accordance with the direction of the arcs. Each arch is associated with the value
of input that causes the transition. It is important to note that, in this case, the output is linked to
the current state only by an equation of the type y = g(x), and then the system is proper. It is also
possible to consider automata where the output depends on both the state and the input y = g(x, u).
The output of such a system can be represented by a matrix as well. The representation by means
of flow graphs can be quite complex if the system has a high number of states.
40 Chapter 3. Dynamical systems

Example 3.4. (Recognition of a string.) Suppose we want to realise a finite state machine that
recognises the string 358 within a longer string. The sets U, Y, X are:

U = {0, 1, 2, . . . , g}
Y = {0, 1}
X = {1, 2, 3, 4}

where the output is 0 if the string is not recognised and 1 if the string is recognised. The flow graph
is shown in Figure 3.3.

U−{3} 3
3 5 8
0 0 0 1

U−{5,3} U−{8}
U

Figure 3.3: Example of automaton: recognition of a string.

Example 3.5. (Exercise: sum of two binary sequences.) Suppose we want to calculate the sum of
two sequences of bits that enter the machine, starting from the least significant digits. Sets U, Y, X
are:
( ! ! ! !)
0 1 0 1
U = , , ,
0 0 1 1
Y = {0, 1}
X = {A, B, C, D}

As an exercise, derive the graph of the automaton.

Chapter 4

Reachability and Observability

The reachability and the observability problem are concerned with the relationships between input
and state and between state and output, respectively. Hence, these two problems affect the input-
output relationship. To understand the importance of these aspects, consider the following problem.
Assume that matrices A (2 × 2), B (2 × 1) and C (1 × 2) are given and produce the input-output
relation
s−1 1
y(s) = u(s) = u(s)
(s + 2)(s − 1) (s + 2)
expressed in terms of transfer function. The zero-pole cancellation reveals something pathological
in the system. Although λ = 1 appears as a pole in the first expression, hence it is an eigenvalue of
A, the cancellation implies that this pole does not affect the input-output behaviour. Indeed, the
impulse response is
W(t) = e−2t
At the end of the chapter we will understand that, in this situation, “something wrong” must occur
in either the relation between input and state or the relation between state and output (or both).

4.1 Reachability of continuous-time systems

Consider a continuous-time linear autonomous system

ẋ(t) = Ax(t) + Bu(t)

(

y(t) = Cx(t) + Du(t)

In this section we will consider the following basic problem: given an initial state x̄1 and a target
state x̄2 , is there any choice of the input u that allows us to reach x̄2 ? If the answer is yes, we
say that x̄2 is reachable from x̄1 and that x̄1 is controllable to x̄2 . Formal definitions are given
below for continuous-time systems. These definitions consider the reachability from zero and the
controllability to zero. As we will see, they allow for a complete characterisation of the problem.

Definition 4.1. The state x̄ is reachable (from zero) in the time interval [0, τ] if ∃ u(·) such that
x(0) = 0 ⇒ x(τ) = x̄.

Definition 4.2. The state x̄ is controllable (to zero) in the time interval [0, τ] se ∃ u(·) such that
x(0) = x̄ ⇒ x(τ) = 0.

Definition 4.3. Given two states x̄1 and x̄2 , we say that x̄1 is controllable to x̄2 and that x̄2 is
reachable from x̄1 in the time interval [0, τ] if ∃ u(·) such that x(0) = x̄1 ⇒ x(τ) = x̄2 .

41
42 Chapter 4. Reachability and Observability

Note that, if x̄1 is controllable to zero and x̄2 is reachable from zero, then x̄2 is reachable from
x̄1 . Later, we will show that reachability and controllability properties are equivalent.
We now define the following set

Xr (τ) = x̄ ∈ Rn : x̄ is reachable (from 0) in [0, τ]



The set Xr (τ) is a subspace of Rn . Indeed if x̄1 is reachable in [0, τ] with input u1 (·) and x̄2 is
reachable in [0, τ] with input u2 (·), then α x̄1 + β x̄2 is reachable in [0, τ] with input αu1 (·) + βu2 (·),
given the linearity of the system.
This property can be verified by observing that the set of states that are reachable from zero can
be expressed as follows: Z τ
x̄(τ) = 0 + eA(t−σ) Bu(σ)dσ (4.1)
0
Given the linearity of the integral, we notice that the whole set Xr (τ) is actually a subspace of Rn .
The reachable states form a subspace
( Z τ )
Xr (τ) = x̄ = e A(t−σ)
Bu(σ)dσ
0

In the same way we can define the set of controllable states

Xc (τ) = x̄ ∈ Rn : x̄ is controllable (to 0) in [0, τ]



The elements x̄ of this set are such that:

Z τ
0 = e x̄ +
Aτ
eA(τ−σ) Bu(σ)dσ
0

Considering the invertibility of eAt (whose inverse is equal to e−At ) we have

Z τ Z τ
x̄ = −e −Aτ
e A(τ−σ)
Bu(σ)dσ = − e−Aσ Bu(σ)dσ (4.2)
0 0

So we see that Xc (τ) is a subspace of Rn as well.

Let us now introduce the reachability matrix R as the following partitioned matrix
h i
R = B|AB|A2 B| . . . |An−1 B (4.3)

If B has dimension n × m and A has dimension n × n, then R has dimension n × (n · m). We remind
that the range or image of a matrix M is the subspace Ra(M) = {y : y = Mx}. The following
fundamental theorem holds.
Theorem 4.1. Xr (τ) = Xc (τ) = Ra(R) for each τ > 0.
Proof To prove the statement of the theorem, we equivalently demonstrate Xr⊥ (τ) = Ra(R)⊥ .1
Reachable states can be expressed as
Z τ
x̄ = eA(τ−σ) Bu(σ)dσ
0

We have: ( Z τ )
Xr⊥ (τ) = z: z ⊤
e A(τ−σ)
Bu(σ)dσ = 0, ∀ u(·)
0
1
This equivalence holds in finite-dimensional spaces in which the orthogonal of the orthogonal of X is (X ⊥ )⊥ = X.
This is not true in infinite-dimensional spaces.
4.1. Reachability of continuous-time systems 43

We can characterise all of the elements of the orthogonal space using the following result:
Z τ
z⊤ eA(τ−σ) Bu(σ)dσ = 0 ∀u(·) ⇐⇒ z⊤ eA(τ−σ) B ≡ 0 (4.4)
0

(≡ means identically 0 for all σ ∈ [0, τ]). The implication right to left is obvious. The implication
left to right is not. Then denote with W(σ) = z⊤ eA(t−σ) B. Since the equality on the left is true for
all inputs, can choose in particular u(σ) = W ⊤ (σ), to get:
Z τ Z τ Z τ
W(σ)u(σ)dσ = W(σ)W (σ)dσ =
⊤
∥W(σ)∥2 dσ = 0 (4.5)
0 0 0

Since the value ∥ · ∥ is always non negative and is zero only if the argument is the zero vector, we
have that W(σ) = z⊤ eA(τ−σ) B = 0, ∀ σ ∈ [0, τ], the condition on the right.
Considering the expression of the matrix exponential, we can write:
 +∞  +∞ ⊤ k
X Ak (τ − σ)k  X z A B
⊤ 
z   B ≡ 0 ⇐⇒

 (τ − σ)k ≡ 0 (4.6)
k!
k=0
k! k=0

In view of the principle of identity for power series this equality is true if and only if

z⊤ A k B = 0 ∀k ≥ 0 (4.7)

From Cayley-Hamilton identity, the powers Ak with k ≥ n can be expressed as a linear combination
of the first n powers of A. Therefore the previous condition holds true if and only if it is verified for
k = 0, 1, . . . , n − 1. Putting these n zero terms together we have
h i
z⊤ B| z⊤ AB| z⊤ A2 B| . . . | z⊤ An−1 B = 0
h i
⇐⇒ z⊤ B|AB|A2 B| . . . |An−1 B = 0
⇐⇒ z⊤ R = 0 (4.8)

This is equivalent to saying that any vector z⊤ , belonging to the orthogonal subspace of Xr (τ), is
orthogonal to the columns of R, hence orthogonal to Ra(R). On the other hand, all the previous
implications are reversible and therefore any vector z⊤ , belonging to the orthogonal subspace of R,
is orthogonal to Xr (τ). Hence Ra(R)⊥ = Xr⊥ (τ) and therefore Xr (τ) = Ra(R).
Along the same lines, using (4.2), we can arrive to the conclusion that Xc (τ) = Ra(R). □
Key points that emerge from the theorem are:

• Xr and Xc are equal;

• they are the image of R;

• they are independent of τ.

The fact that reachability does not depends on the time horizon is suspect. This would mean, for
example, that any state reachable by an aircraft can be reached in an arbitrarily small amount of
time. This is obviously not true: it is in contradiction with the physical limits of the actuators in the
plane. The apparent contradiction can be explained by the fact that the theorem does not take into
account upper limits for the magnitude of the system input. If such limits are taken into account
(for instance, ∥u∥ ≤ umax ), then one can see that Xr (τ) is a bounded convex set that depends on τ.
44 Chapter 4. Reachability and Observability

Example 4.1. Let " # " #

0 1 0
A= , B=
0 0 1
and |u| ≤ 1. These equations represent the motion of an object, where x1 is the position, x2 the
speed and u a bounded force. The reachable set is the set of all states [x1 (τ) x2 (τ)] with
Z τ Z τ
x1 (τ) = σu(σ)dσ, x2 (τ) = u(σ)dσ,
0 0

It is not difficult to see that (as is physically obvious) this set is bounded if u is bounded and it gets
larger as we increase τ.

We have just seen that the subspaces Xr and Xc do not depend on τ, and are the image of
reachability matrix R.

Definition 4.4. A system is said reachable (resp. controllable) if all of the possible states are
reachable (resp. controllable).

In view of the previous theorem, since the reachable (controllable) space is the image of R, we
have the following.

Theorem 4.2. The system is reachable (controllable) if and only if

Rank(R) = n (4.9)

This means that we have reachability of the whole space: all possible state vector for the system
can be generated by the columns of R, namely, the columns of R are a set of generators for Rn ).
This result is remarkable, because it transforms an analytical problem (finding the u to put in the
integral) into an algebraic problem (easily solvable via standard software).

4.2 Kalman decomposition for reachability

If the system is not reachable, we can analyse the situation by means of the Kalman Decomposition
for reachability.
Suppose that R has rank r < n; therefore, there are r linearly independent columns that form
a basis of Xr . We place them in a matrix T r = [t¯1 , t¯2 , . . . , t¯r ]. To form a basis, consider now its
complement T nr = [t¯r+1 , t¯r+2 , . . . , t¯n ]. Putting together the columns of T r and T nr , we get a matrix
T of size n × n,
T = [t¯1 , t¯2 , . . . , t¯r |t¯r+1 , t¯r+2 , . . . , t¯n ] = [T r |T nr ] , (4.10)
whose columns are a basis of Rn . The space generated by T r is the reachable subspace and the space
generated by T nr is the unreachable subspace. Notice that T nr is any subspace that is complementary
to T r , so there is some freedom in this choice.
Each vector x ∈ Rn can then be expressed as a linear combination of the basis vectors, as
follows
h i " x̂r #
x = T r T nr (4.11)
x̂nr
and, in particular, each vector belonging to the reachable subspace can be written as:
h i " x̂r #
x = T r T nr (4.12)
0
4.2. Kalman decomposition for reachability 45

If we apply the state transformation T to the system we get the new representation

˙
x̂(t) = Â x̂(t) + B̂u(t)
y(t) = Ĉ x̂(t) + D̂u(t)

where

 = T −1 AT
B̂ = T −1 B
Ĉ = CT
D̂ = D

We can write the system in the form

" # " #" # " #
d x̂r (t) Ar Ar,nr x̂r (t) Br
= + u(t) (4.13)
dt x̂nr (t) ϕ1 Anr x̂nr (t) ϕ2

We demonstrate now that ϕ1 and ϕ2 are zero matrices. If we start from zero initial conditions,
x̂r (0) = 0 and x̂nr (0) = 0, then after a time t > 0, by construction, only reachable states are reached.
In the new representation, these are of the form
" #
x̂r (t)
x=T
0

This means that x̂nr (t) = 0 for all t > 0 and then x̂˙nr (t) = 0 for all t > 0. Consider the second
equation in (4.13)

x̂˙nr = ϕ1 x̂r + Anr x̂nr + ϕ2 u (4.14)

⇒ 0 = ϕ1 x̂r + ϕ2 u (4.15)

and write it in the form

h i " x̂r (t) #
ϕ1 ϕ2 =0 (4.16)
u(t)
This relationship is valid ∀ u(t) and ∀ x̂r (t). Note that x̂r (t) can be assumed arbitrary, because it is
the reachable component and t is arbitrary. The only matrix which gives zero when multiplied by
any vector is the zero matrix, hence ϕ1 and ϕ2 are zero. Then the system can be written in the form:
" # " #" # " #
d x̂r (t) Ar Ar,nr x̂r (t) Br
= + u(t)
dt x̂nr (t) 0 Anr x̂nr (t) 0
h i " x̂r (t) #
Y(t) = Cr Cnr (4.17)
x̂nr (t)

The first subsystem is the reachable subsystem, the second the unreachable subsystem. Note that
the unreachable subsystem influences the reachable one through the block Ar,nr .
The block diagram corresponding to the whole system is reported in Figure 4.1.
If a matrix is block-triangular, then its determinant is equal to the product of the determinants
of the diagonal blocks:
det(sI − Â) = det(sI − Ar ) det(sI − Anr ) (4.18)
The eigenvalues of Â, which are the same eigenvalues as A, are thus those of Ar and those of Anr ;
more precisely, we can split the eigenvalues in two distinct sets:
46 Chapter 4. Reachability and Observability

• {λ1 , λ2 . . . , λr } = σ(Ar ) are the reachable eigenvalues, associated with reachable modes;

• {λr+1 , λr+2 . . . , λn } = σ(Anr ) are the unreachable eigenvalues, associated with unreachable
modes.

This result has a fundamental consequence when the system has to be controlled. The evolution
of the second subsystem is completely independent of the input u and evolves in natural response
with modes σ(Anr ). As we will see, a feedback controller may change the reachable eigenvalues,
but not the unreachable ones (if any). Therefore, if the system has unstable unreachable modes, no
controller will be able to stabilise it.

u x&ˆr x̂r y
Br ∫ Cr

Ar,nr Ar

x̂nr
x&ˆnr ∫ Cnr

Anr

Figure 4.1: Kalman decomposition for reachability.

From the diagram in Figure 4.1, we can notice the absence of the interaction between the
input and x̂˙nr , which should be given by ϕ2 , and of the interaction between x̂r and x̂˙nr , which
should be given by ϕ1 . The part consisting of (Ar , Br , Cr ) is called reachable subsystem, while
that consisting of (Anr , 0, Cnr ) is called unreachable subsystem. A simplified block diagram is
depicted in Figure 4.2.
Both in the block diagram reported in Figure 4.1 and in its simplified version in Figure 4.2, there
is no arrow from u to the unreachable subsystem, in accordance with the definition of unreachable
state. Also, there is no arrow from the reachable subsystem to the unreachable one: in fact, this
would indirectly cause reachability of some states of the unreachable subsystem. Instead, no
information can reach the unreachable subsystem transiting through the reachable one. However,
note that the unreachable subsystem is able to influence the reachable subsystem (and the output of
the system). This motivates us to study stability of the unreachable subsystem as well (as said, if
the unreachable subsystem is not asymptotically stable, it is impossible to stabilise the system by
means of a feedback controller).

reachable
subsystem +

unreachable
subsystem

Figure 4.2: Simplified block diagram of the Kalman decomposition for reachability.
4.2. Kalman decomposition for reachability 47

It can be shown that the transfer function depends on the reachable subsystem only
W(s) = C(sI − A)−1 B + D = Cr (sI − Ar )−1 Br + D (4.19)
because cancellations occur related to eigenvalues associated with the unreachable subsystem. This
is obvious, because the transfer function represents the link between the system input and the
system output when the initial conditions are zero and, by definition, the unreachable subsystem is
not affected by the applied input.
Example 4.2. Consider the system with the following matrices:
" # " #
−1 1 1 h i
A= , B= , C = 1 0 , D = [0]
1 −1 1
The reachability matrix is equal to
" #
h i 1 0
R= B AB =
1 0
and its rank is 1 (hence, less than n = 2), therefore the system is not completely reachable. The set
of reachable vectors can be easily interpreted in a geometric fashion: it is bisector of the first and
third quadrant of the Cartesian plane.
We can complete the basis for R2 by choosing, together with the only linearly independent
column of R, any vector that is not linearly dependent on that column. For example, we can choose:
" #
1 1
T= ,
1 0
but also " # " #
1 0 1 −1
T= , or T=
1 1 1 1
are valid choices. Any vector of the state space, the plane R2 , can be expressed as a linear
combination of the two vectors in T .
Before applying T , let us consider the system transfer function.
d(s) = det(sI − A) = s2 + 2s
 s + 1 −1 −1 
 
" #
sI − A −B j
n(s) = det = det  −1 s + 1 −1  = s + 2
 
Ci Di j  
1 0 0
s+2 1
⇒ W(s) = =
s(s + 2) s
We see that there is a cancellation of the factor (s + 2). We expect that this is due to the lack of
reachability.
We can easily verify that matrix  = T −1 AT , calculated with a T chosen as above, gives
" #
0 #
 =
0 −2
where the underlined 0 is expected, and corresponds to the identically zero ϕ1 matrix, while the 0
and −2 on the diagonal are the eigenvalues of matrix A. The element # in the upper right position
depends on matrix T , and in particular on the choice of the second column. Similarly, B̂ = T −1 B
has the form " #
$
B̂ =
0
where, again, the zero 0 is expected since it corresponds to the identically zero matrix ϕ2 , while $
depends the chosen second column of T . The reader is invited to perform the computation with two
different T matrices among those proposed above.
48 Chapter 4. Reachability and Observability

4.3 Reachability of discrete-time systems

The previous definitions of reachability and controllability for continuous-time systems can be
extended to the case of discrete systems
x(k + 1) = Ax(k) + Bu(k)
(

y(k) = Cx(k) + Du(k)

Definition 4.5. A state x̄ is said reachable from zero in an interval [0, T ] if ∃ u(·) such that
x(0) = 0 ⇒ x(T ) = x̄.
Definition 4.6. A state x̄ is said controllable to zero in an interval [0, T ] if ∃ u(·) such that
x(0) = x̄ ⇒ x(T ) = 0.
There are strong analogies with the continuous-time case, but some important differences as
well.
Consider the expression of the discrete time solution from 0 to k:
k−1
X
x(k) = Ak x(0) + Ak−h−1 Bu(h)
h=0

If the state x̄ is reached at time T from x(0) = 0 we have:

T −1
X
x(T ) = x̄ = 0 + AT −h−1 Bu(h) (4.20)
h=0

This expression can be written in matrix form:

 
 u(T − 1) 
u(T − 2)
 
i  ..
h 
x̄ = B AB . . . AT −1 B   = R(T ) U(T )

. (4.21)

 
 u(1) 

u(0)
Since vector U(T ) can be arbitrary, the set of reachable states in T steps is given by:
Xr (T ) = Ra(R(T ) ) (4.22)
A first difference emerges between continuous-time systems and discrete-time systems: the reach-
ability set, which is a subspace of the state space, depends on the time T , and gets bigger as T
increases:
Xr (1) = Ra ([B])
h i
Xr (2) = Ra B AB (4.23)
..
. (4.24)
Note that Xr (T ) ⊆ Xr (T + 1) ∀ T . This is obvious, because if a state can be reached in T steps, then
it is reachable in T + 1 steps as well.2
The key point is that the reachability subspace of the system does not increase beyond T = n,
where n is the system size. This happens because, by the identity of Cayley-Hamilton, the columns
Ak B with k ≥ n are a linear combination of the first n powers (k = 0, 1, . . . , n − 1). In other words,
matrix Rn has the maximum achievable rank, equal to n, and any reachable state can be reached in
at most n steps. Hence, the range of matrix R = Rn is the set of all reachable states.
2
You can just do nothing for the first step and then use the remaining T steps to reach the state.
4.3. Reachability of discrete-time systems 49

Theorem 4.3. In a discrete-time system, the set of all reachable states is

Xr = Ra(R). (4.25)

In this sense there is a strong analogy with the continuous-time case.

Example 4.3. (Inventory with ageing.) Consider the system with the following matrices:
   
 0 1 0   0 
A =  0 0 1  , B =  0 
   
   
0 0 0 1

The quantity u(k) is the supply of goods at time k, x3 (k) is the quantity of goods arrived the previous
day, x2 (k) is the quantity of goods arrived two days before, x1 (k) is the quantity of goods arrived
three days before. Then, the goods are eliminated due to ageing (in xh (k), h represents the number
of days for which the goods can survive). It is not difficult to see that the natural response Ak x(0)
goes to zero in finite time, as expected in a perishable goods inventory.
The set of states that are reachable in one step is
 
 0 
Xr (1) = Ra ([B]) = Ra  0 
 
 
1

and can be geometrically interpreted as the set of vectors R3 on the z axis. The set of states that
can be reached in two steps is
 
h i  0 0 
Xr (2) = Ra B AB = Ra  0 1 
 
 
1 0

and, geometrically, is the set of vectors in the yz plane. Finally, the set of states that are reachable
in three steps is
 
h i  0 0 1 
Xr (3) = Ra B AB A2 B = Ra  0 1 0 
 
 
1 0 0
namely the whole R3 .

To summarise, in discrete time (A, B) is fully reachable if and only if Ra(R) = Rn , hence, if
and only if rank(R) = n. In addition, either a state can be reached in n or it cannot be reached at all.
The controllability problem in discrete time requires more caution. In fact, the set of controllable
states, for a discrete-time system, is different from the set of reachable sets, as can be seen from the
following example.

Example 4.4. Given the system with matrices

" # " #
0 1 0
A= , B=
0 0 0

the reachability matrix is equal to

" #
h i 0 0
R= B AB = ,
0 0
50 Chapter 4. Reachability and Observability

from which it is clear that the set of reachable states is the null set (only 0 can be reached). However,
the set of controllable states is equal to all R2 . Indeed, for u(k) = 0,
" #
0 0
x(k) = A x̄ = 0 for k ≥ 2 because A =
k k
for k ≥ 2.
0 0

Hence, any x(0) can be driven to zero in at most 2 steps. Matrix A in this example is nilpotent.3

The above example highlights the discrepancy between the sets Xr and Xc in discrete time. In
general, in fact, we have
Xr ⊆ Xc .
It can be proved that, if A is invertible (i.e., it has no null eigenvalues), then Xr = Xc in the
discrete-time case as well.
There is no a simple formula to express the set of all controllable states in T steps. This set is
the subspace of all vectors x̄ for which
T −1
X
0 = A x̄ +
T
AT −h−1 Bu(h)
h=0

for some choice of u. If we take T = n, this set is the kernel of

h i
An An−1 B An−2 B . . . AB B

projected on the subspace of the first n components.

4.4 Other reachability criteria

We return now to the case of continuous-time systems.

Theorem 4.4. The following implications are equivalent

• System (A, B) is reachable (or, equivalently, controllable)

h i
• Rank λI − A B = n ∀ λ ∈ C
h i
• Rank λI − A B = n ∀ λ ∈ σ(A)

Proof. It is very easy prove that the third condition is equivalent to the second. Obviously the
second implies the third. Now, if the third holds, then the rank condition holds even for λ < σ(A),
because in this case the matrix [λI − A B] has the full block λI − A having full rank.
We prove by contradiction that the first statement
h implies
i the third: if the third statement fails,
the first must fail as well.4 Assume that Rank λI − A B < n, namely, the n rows of this matrix
are not linearly independent. Therefore there exists a vector z⊤ , 0 such that:
h i
z⊤ λI − A B = 0

By the properties of the block matrix, we can therefore write

z (λI − A) = 0
( ⊤

z⊤ B = 0
3
Matrix A is nilpotent if Ak = 0 for some k. This is true if and only if all of its eigenvalues are zero.
4
{A} ⇒ {B} if and only if not{B} ⇒ not{A}.
4.4. Other reachability criteria 51

and then it appears that z⊤ is the left eigenvector of matrix A associated with the eigenvalue λ. We
can write

z⊤ A = λz⊤
z⊤ A2 = z⊤ A · A = λz⊤ A = λ2 z⊤
z⊤ A3 = z⊤ A · A2 = λz⊤ A2 = λ3 z⊤
..
.
z⊤ Ak = λk z⊤

and also
z ⊤ A k B = λk z ⊤ B = 0
Therefore,
h i h i
z⊤ B AB . . . An−1 B = z⊤ B z⊤ AB . . . z⊤ An−1 B = 0,

hence the reachability matrix R does not have full rank n, and the system is not reachable.
To prove that the third condition implies the first, again we proceed by contradiction. Assume
that the system is noth reachable and
i write ithin Kalman form.i This can be done, since it can be
easily seen that Rank λI − A B = Rank λI − Â B̂ . We get the matrix

λI − Ar −Ar,nr
" #
h i Br
λI − Â B̂ =
0 λI − Anr 0

So, if the system is not reachable, then the last block-row only has the square matrix [λI − Anr ]
of dimension n − r that is not identically zero. Then, if we take λ ∈ σ(A) among the unreachable
eigenvalues, matrix [λI − Anr ] is singular and so the last n×r rows are linearly dependent. Therefore,
the rank must be less than n. □
h i
From the previous proof, we have the Popov criterion: Rank λnr I − A B < n for all the
unreachable eigenvalues λnr ∈ σ(A). This criterion has several applications. It is very useful to test
if an eigenvalue λ (and the corresponding mode) is reachable or not.
For instance,
h assumei that matrix A has an eigenvalue λ̄ with positive or nonnegative real part.
Consider λ̄I − A B and observe that:

• if its rank is less than n, this eigenvalue λ̄ is unreachable, hence there is no way to design a
feedback controller that asymptotically stabilises the system;

• if its rank is equal to n, this eigenvalue is reachable, and therefore we can hope to realise the
controller (we will deal with this problem in one of the following chapters).

The corresponding discrete-time theorem is the following.

Theorem 4.5. The discrete system (A, B) is
h i
• reachable if and only Rank λI − A B = n ∀ λ ∈ σ(A);
h i
• controllable if and only if Rank λI − A B = n ∀ λ ∈ σ(A), λ , 0.
Example 4.5. We want to find the set of all reachable states for the system with matrices
   
 0 −2 1   2 
A =  −3 1 −3  , B =  0 
   

−2 2 −3
 
−1

52 Chapter 4. Reachability and Observability

The reachability matrix is:  

 −1 −1 −1 
R =  0 −3 3 
 

2 −1 5


To compute the rank of this matrix, we can use the Gauss method for triangulation. By adopting
elementary operations that do not change the matrix rank, we can reduce the matrix to a triangular
form. In this case, multiplying the first row by 2 and adding it to the last row, we have:
 
 −1 −1 −1 
 0 −3 3 
 
0 −3 3
and multiplying the second row by −1 and adding it to the last we get
 
 −1 −1 −1 
 0 −3 3 
 
0 0 0
The number of rows that are not identically zero represents the rank of matrix. In this case, it
is equal to 2. A basis of the reachable subspace is then obtained by considering two linearly
independent columns of the original matrix R. We can take, for instance,
  

  2   −1  
Xr = 
 
 0   −3 
 


 −1    

−1 


This selection may be completed with an additional column that must be linearly independent to
the other two, so as to form a basis of R3 :
 
 2 −1 1 
T =  0 −3 0 
 

−1 −1 0


This matrix is invertible and we have:

 
 0 −1 3
1 

T −1 =  0 1 0

−3 

−3 3 −6
We can easily check that
   
 0 2 1   1 
 = T −1 AT =  1 −1 1  , B̂ = T −1 B =  0 
   
  
0 0 −1 0
The underlined zeros are not a coincidence: they result from the Kalman decomposition. In fact,
since the reachability matrix has rank 2, there must be a matrix Anr of size 1 × 1, and also a row of
zeros in the first two positions of the last row of Â.
As mentioned earlier, the eigenvalues of  are same as those of A and are equal to the union of
the eigenvalues of Ar and those of Anr :
 
 s −2 −1 
det(sI − A) =  −1 s + 1 −1  =
 
s+1
 
0 0
= (s + 1)(s(s + 1) − 2) = (s + 1)(s2 + s − 2)
4.4. Other reachability criteria 53

In this case the eigenvalues are λ1 = 1, λ2 = −2, λ3 = −1; the eigenvalue λ3 = −1, which is
asymptotically stable, is associated with the unreachable part of the system. If we had known the
eigenvalues and wanted to check the feasibility of a feedback controller to be applied to system, we
could just have checked the reachability of the eigenvalue λ1 , which is responsible of the system
instability. To this aim, we can use the Popov criterion explained
h above.i According to this criterion,
to test reachability of the unstable eigenvalue, the matrix λI − A B must have rank n for λ = 1.
We have:  
h i  1 2 −1 2 
λI − A B =  3 0 3 0 
 
λ=1 
2 −2 4 −1


We use again Gauss elimination to triangularise the system. Multiplying the first row by −2 and
adding it to the last row, and multiplying the first row by −3 and adding it to the second, we obtain
 
 1 2 −1 2 
 0 −6 6 −6 
 
0 −6 6 −5
Then, multiplying the second row by −1 and adding it to the last one, we get
 
 1 2 −1 2 
 0 −6 6 −6 
 
0 0 0 1
Finally, the matrix rank is equal to 3 = n, hence the eigenvalue λ1 = 1 is reachable. Therefore we
expect that, in principle, there is a controller able to stabilise the system.

Figure 4.3: An electric circuit (example for discussing reachability).

Example 4.6. The circuit in Figure 4.3 is governed by the following equations:
v(t) = R1 I1 (t) + v1 (t) = R1C1 v̇1 (t) + v1 (t)
v(t) = R2 I2 (t) + v2 (t) = R2C2 v̇2 (t) + v2 (t)
Set v(t) = u(t) and consider the state vector:
" # " #
x1 (t) v1 (t)
=
x2 (t) v2 (t)
54 Chapter 4. Reachability and Observability

The system can be written in the following matrix form

− R11C1 1
" # " #" # " #
ẋ1 (t) 0 x1 (t)
= + R1 C1
u(t)
ẋ2 (t) 0 − R21C2 x2 (t) 1
R2 C2

The reachability matrix is

1
− (R C1 )2
 
R = 
 R1 C1

1 1
1
− (R C1 )2

R2 C2 2 2

In order to have full rank (equal to 2), matrix R must have a non-zero determinant
1 1 1 1
det(R) = − 2
+ =
R1C1 (R2C2 ) R2C2 (R1C1 )2
!
1 1 1 1
= − , 0,
R1C1 R2C2 R1C1 R2C2
hence R1C1 , R2C2 . This condition means that, if the circuit presents a symmetry (R1C1 = R2C2 ),
it is not possible to independently control both of the capacitor voltages. This situation is very
similar to the case of two identical tanks with identical drainpipes, and filled by a single tap that
supplies both of the tanks with the same amount of water. It is intuitive that, if the tanks start from
the same level, by symmetry, we cannot unbalance the level of one tank relatively to the other. If,
instead, the tanks are different, with an appropriate flow input function one can bring both tanks at
the desired levels at some time τ.
The situation of the previous example can be generalised. Consider two systems (A1 , B1 ), (A2 , B2 ),
where B1 and B2 are columns (single input), and consider the parallel of the two. The parallel
system equation is " # " #" # " #
ẋ1 (t) A1 0 x1 (t) B1
= + u(t) (4.26)
ẋ2 (t) 0 A2 x2 (t) B2
We can prove the following:
Theorem 4.6. Consider two systems with a single input variable. The parallel system is reachable
if and only if the two systems are reachable and they have no common eigenvalues.
Proof. (⇒) Assume that A1 has size n1 × n1 and A2 has size n2 × n2 . If the parallel system is
reachable, then the matrix
λI − A1
" #
0 B1
0 λI − A2 B2
has full rank equal to n1 + n2 , according to the Popov criterion. This means that the number of
linearly independent rows of the matrix is n1 + n2 . So, if we take the submatrix
h i
λI − A1 0 B1

it must have n1 linearly independent rows (and the same number of linearly independent columns).
Removing the 0 column, we have that
h i
λI − A1 B1

has rank n1 , which implies the reachability of system (A1 , B1 ). The same reasoning applies to the
system (A2 , B2 ).
It is also necessary that (A1 , B1 ) and (A2 , B2 ) have no common eigenvalues. Indeed, if there
were a common eigenvalue λ, the matrices
λI − A1
" # " #
0
,
0 λI − A2
4.5. Observability and detectability of continuous-time systems 55

would have rank less than n1 and than n2 , respectively. This means that, in the most favourable
case, the sub matrix
λI − A1
" #
0
0 λI − A2
would have at most n1 + n2 − 2 = n − 2 linearly independent columns, so that the addition of the
column " #
B1
B2
would bring the rank at most to to n1 + n2 − 1. As a consequence,the parallel system would be
unreachable, in contradiction with the hypothesis.
(⇐) Assume that the two systems are reachable and do not have common eigenvalues. Take λ
eigenvalue of A1 . Then, h i
λI − A1 B1
has rank n1 because system 1 is reachable. On the other hand, matrix
h i
λI − A2

has rank n2 , because λ is not an eigenvalue of A2 ; hence, matrix

λI − A1
" #
0 B1
0 λI − A2 B2

has full rank n1 + n2 = n, because, in view of how the blocks are arranged, the first n1 rows are
independent and none of them can be generated as a combination of the last n2 , and vice versa. A
similar reasoning applies to any eigenvalue of A2 ; therefore, the Popov criterion is satisfied. □

4.5 Observability and detectability of continuous-time systems

The problem of observability amounts to studying the relationship between the system state and the
system output. Consider a linear autonomous continuous-time system with matrices A, B, C, D.
The assumption is that the signals y(t), u(t) are known in an interval [0, τ]. Is it possible to
determine the initial state of the system at time t = 0? Is it possible to determine the final state of
the system at time t = τ?

Definition 4.7. The system is observable in the interval [0, τ] if, given known u(t) and y(t) for
t ∈ [0, τ], it is possible to uniquely determine x(0).

Definition 4.8. The system is detectable in the interval [0, τ] if, given known u(t) and y(t) for
t ∈ [0, τ], it is possible to uniquely determine x(τ).

Before proceeding further, we set matrix D to zero. In fact, y(t) = Cx(t) + Du(t) reads as
y(t) − Du(t) = Cx(t), so we can consider the new “output” ỹ(t) = y(t) − Du(t). Hence, it is not
restrictive to choose D = 0.
A first question is whether the problem of observability can be solved by adopting the punctual
relationships at time t = 0:
y(0) = Cx(0).
The answer is clearly negative, because, most of the times, matrix C is not square and therefore not
invertible, therefore it is impossible to uniquely determine x(0) from the condition y(0) = Cx(0).
Then, we need more information and we have to observe the system evolution in an interval.
56 Chapter 4. Reachability and Observability

The system general solution is

Z t
x(t) = e x(0) +
At
eA(t−σ) Bu(σ)dσ
0
Z t
y(t) = Ce x(0) + At
CeA(t−σ) Bu(σ)dσ
0

From this expression we can deduce the following fact.

Observation 4.1. Since the continuous-time system is reversible, if we can find x(0), then we can
calculate x(τ), and vice versa. Therefore, in the case of continuous-time systems, the observability
problem is equivalent to the detectability problem.

Note also that the observability problem is conceptually different from the reachability problem
discussed earlier. The reachability problem mainly concerns the existence of an input that allows us
to reach a certain state. Here, the existence is a certain fact, because the system has indeed been in
some initial state x(0). The issue now is uniqueness. In fact, given the same input and observed
output, there can be several initial states that are consistent with the observation.
To analyse this problem, we define
Z t
g(t) = CeA(t−σ) Bu(σ)dσ
0

and we discuss the uniqueness of the solution x(0) of the equation

y(t) = CeAt x(0) + g(t),

where y(t) and g(t) are known on the observation interval [0, τ].
Suppose that there is no uniqueness, namely there are two different states x̄1 and x̄2 such that:

y(t) = CeAt x̄1 + g(t) (4.27)

y(t) = Ce x̄2 + g(t) At
(4.28)

that is, two different initial states that produce the same output when the same input is applied.
Subtracting the first equation from the second and denoting by x̄ = x̄2 − x̄1 we obtain:

0 = CeAt x̄ (4.29)

Then uniqueness is ensured if there are no vectors x̄ , 0 which satisfy (4.29). Indeed if (4.27) and
(4.28) are true, then x̄ = x̄2 − x̄1 , 0 is one of such vectors. The opposite implication is quickly
verified: assume that x̄ , 0 satisfies (4.29). If one considers the equations:

0 = CeAt x̄
y(t) = CeAt x(0) + g(t) (4.30)

then their sum is

y(t) = CeAt (x(0) + x̄) + g(t) (4.31)
and both the vectors x(0) and x(0) + x̄ produce the same output for any g: then, uniqueness is
missing. We conclude the following.

Observation 4.2. The problem of observability is solvable if and only if there are no vectors x̄ , 0
such that CeAt x̄ = 0 for all t ∈ [0, τ].
4.5. Observability and detectability of continuous-time systems 57

Definition 4.9. A vector x̄ ∈ Rn , x̄ , 0 is called not observable (indistinguishable from zero) in the
interval [0, τ] if CeAt x̄ = 0 ∀ t ∈ [0, τ].
We have established that the problem has no solution if and only if there are non-observable
vectors. From equation (4.29), it is easy to realise that any of these vectors, if chosen as an initial
condition with zero input u(t) = 0, produces a zero output. The set of non-observable vectors Xno is
a subspace of the state space, because it is the kernel of CeAt , and any kernel is a subspace. In the
light of this definition, the observability problem has solution if and only if such a kernel is trivial,
namely, Xno = {0}.
The expression CeAt x̄ can be expanded using the exponential series
 +∞  +∞
X Ak tk  X CAk x̄ k
Ce x̄ = C 
At  x̄ = t = 0 ∀ t ∈ [0, τ]
k=0
k! k=0
k!
Using once again the principle of power identity, this is equivalent to
CAk x̄ = 0 ∀k ≥ 0 ⇔ CAk x̄ = 0 for k = 0, 1, . . . , n − 1
in view of the Cayley-Hamilton theorem. Similarly to what was done for the reachability problem,
we can incorporate all of these terms in a matrix:
C x̄ = 0

 C 
 


 CA x̄ = 0



  CA 

  x̄ = 0 ⇔ O x̄ = 0
.. ⇔  ..
. .


 

 
 
 CAn−1 x̄ = 0


CAn−1
where the matrix
C
 
 
 CA 
..  = O
 
(4.32)
.

 
CAn−1
 

is the observability matrix. Hence, the set of all vectors such that CeAt x̄ = 0 in [0, τ] is the kernel
of O. We conclude the following.
Theorem 4.7. The system is observable if and only if ker(O) = {0}.
Matrix O has dimension (n · p) × n, and it is square if p = 1 (single output). If ker(O) = 0, then
it must be rank(O) = n. In fact, if it happened that O x̄ = 0 with x̄ , 0, this would mean that the
columns of O are linearly dependent, namely, that rank(O) < n. Therefore we have the following.
Theorem 4.8. The system is observable if and only if rank(O) = n.
Once again, an analytical problem has been turned into an algebraic problem. Note the following
facts:
• matrix B has no role in determining the observability of the system;
• the horizon length τ does not influence observability.
The previous considerations are somewhat analogous to what was found for the reachability problem.
In particular, the second fact means that, if the initial state of a system can be determined, this is
possible in an arbitrarily small observation period. This has to do with the ideal hypothesis of the
theorem, and not with real-world situations. Each measure y(t), in fact, is certainly affected by
disturbances that are not always exactly quantifiable. As a result, the smaller is the observation
period, the higher is the influence of the error. In reality, therefore, to determine the state we need
to filter the measured signal and this requires the observation over a period of time that is not
infinitesimal.
58 Chapter 4. Reachability and Observability

4.6 Kalman decomposition for observability

Analogously to the reachability problem case, if ker(O) , {0} (the system is not observable), then
we can take the vectors forming a basis of ker(O) and group them in a matrix

T no = [t¯1 , t¯2 , . . . , t¯r ] .

It is then possible to complete this basis with other n − r linearly independent vectors

T o = [t¯r+1 , t¯r+2 , . . . , t¯n ]

so as to get a basis of Rn , given by the columns of

h i
T = T no T o . (4.33)

Each vector in Rn can thus be expressed as:

h i " x̂no #
x= T no T o (4.34)
x̂o

In particular, each vector that is not observable can be expressed as

h i " x̂no #
x = T no T o , (4.35)
0

from which it follows that

( h i " x̂no # )
Xno = x : x = T no To , x̂no arbitrary .
0

We denote the subspace generated by T o as the observable subspace. This nomenclature is purely
conventional, because, if there exists unobservable vectors, the entire observability problem has no
solution.
Note the duality with respect to the reachability problem: in that case, the basis of the reacha-
bility subspace is obtained by selecting the linearly independent columns of R and then the basis
is completed (to get a basis of the whole state space) by adding new vectors that generate the
“non-reachable” subspace (if the system is reachable, R is a basis of Rn ). Here, the “non-observable”
subspace is determined as the kernel of O and the basis is completed (to get a basis of the whole
state space) with vectors that generate the observable subspace.
If we transform the system by means of T , we get
" # " #" # " #
d x̂no (t) Ano Ano,o x̂no (t) Bno
= + u(t)
dt x̂o (t) ϕ1 Ao x̂o (t) Bo
h i " x̂no (t) #
Y(t) = ϕ2 Co (4.36)
x̂o (t)

where the matrices ϕ1 and ϕ2 are identically zero. To see this, consider u(t) = 0 and an unobservable
initial vector equal to " #
x̂no (t)
x(0) =
0
By definition of unobservable vector, y(t) = 0 ∀ t > 0, and this is possible only if ϕ2 is a null matrix.
Moreover, if ϕ1 were not zero in the same conditions we would have that, at some t, x̂o (t) , 0, and
this would produce a nonzero output.
4.7. Observability of discrete-time system 59

u x&ˆno x̂no
Bno ∫

Ano,o Ano

x&ˆo x̂o y
Bo ∫ Co

Ao

Figure 4.4: Kalman decomposition for observability.

The structural scheme of the system is shown in Figure 4.4.

The part consisting of (Ao , Bo , Co ) is called observable subsystem, while that composed of
(Ano , Bno , 0) is called non-observable subsystem. The observable subsystem is, as mentioned,
the only one that is able to influence the output. Indeed, the unobservable subsystem does not
affect the output y (this would be the contribution of submatrix ϕ2 , which is zero). Moreover, the
unobservable subsystem states do not influence observable states (this would be the contribution
of ϕ1 , which is zero), because otherwise we would have an indirect influence of the unobservable
states on the output, through the observable subsystem.
The spectrum of matrix Â, obtained after the Kalman decomposition, includes the eigenvalues of
Ano and those of Ao , because  is a block-triangular matrix. As in the reachability case, observability
leads to cancellations. Indeed, the transfer function matrix depends on the observable subsystem
only
W(s) = C(sI − A)−1 B = Co (sI − Ao )−1 Bo (4.37)
and the unobservable subsystem has no role in the input-output relationship.

4.7 Observability of discrete-time system

The observability problem for discrete-time systems is stated exactly as in the continuous-time case.
The output of the system is now
k−1
X
y(k) = CAk x(0) + CAk−h−1 Bu(h) = CAk x(0) + g(k)
h=0

and, assuming to perform T observations at different times. with the purpose of uniquely determin-
ing x(0), we obtain the following equations:

y(0) C g(0)
     
     
 y(1)   CA   g(1) 
..  =  ..  x(0) +  ..
     
(4.38)
. . .
 
     
CAT −1
     
y(T − 1) g(T − 1)

In a more compact form, we can write:

Y(T ) = O(T ) x(0) + G(T ), (4.39)

60 Chapter 4. Reachability and Observability

which represents an observation experiment in T steps. It is intuitive that, if T is large, then we

get more information on the system and more constraints on x(0), hence we are more likely to
determine it uniquely. It may happen that we do not obtain a unique solution in T − k steps, while
uniqueness holds in T − k + 1 steps. So, the question is how many steps are needed. For sure it is
not necessary to continue for T > n, because the k-th power Ak of A is a linear combination of I, A,
. . . An−1 . Another interpretation is that, when we increase the number of steps, we try to increase the
rank of matrix O, in order to decrease its kernel dimension, with the hope of reducing it to the trivial
subspace. This is hopeless after n steps. In conclusion, the problem of observability either has
solution in n steps or has no solution. Therefore, we can consider n steps and the matrix O = O(T ) ,
and conclude that the problem has solution if and only if O has trivial kernel or, equivalently, has
rank n. The following theorem holds.

Theorem 4.9. The system is observable if and only if rank(O) = n.

The condition is thus the same obtained for continuous-time systems.

Analogously to reachability and controllability, also observability and detectability are not
equivalent in general for discrete-time systems. They are equivalent if the system is reversible, i.e.,
the matrix A is invertible, because in this case for any initial state we can determine the final one,
and vice-versa.

Example 4.7. Consider the following discrete-time system:

x1 (k + 1)
" # " #" #
0 1 x1 (k)
=
x2 (k + 1) 0 0 x2 (k)
h "
i x1 (k) #
y(k) = 0 0
x2 (k)

Since we get no information from y(k) = 0, it is impossible to derive x1 (0) and x2 (0). However, A is
a nilpotent matrix. Hence, for k > 2, the state vector is zero and the problem of detectability has
a solution. From this simple case, we observe that the detectability problem is different from the
observability problem for discrete-time systems.

4.8 Examples of reachability and observability analysis

Example 4.8. Consider the following model of a direct-current electrical machine:
  R kī

 ia (t)   − Laa − Lae 0   ia (t)   L1a 
   
d 
 ω(t)  =  kīe − Jf 0   ω(t)  +  0  va (t)
    
dt    J
φ(t)  φ(t) 0
 
0 1 0

For simplicity, the external load torque is assumed to be zero. We first perform a reachability
analysis. We have a single input u. The reachability matrix is

− RL2a
 1 
 La ∗ 
a
R =  0
 kīe

JLa ∗ 
kīe 
 
0 0 JLa

where the determination of the entries (∗) is left as an exercise. This matrix is upper triangular,
with non-zero diagonal elements, hence it has full rank and the system is reachable.
This systems can be used as an electric drive, but to this aim we need to measure the position,
and possibly the speed and the current.
4.8. Examples of reachability and observability analysis 61

We first assume that an encoder is available as a sensor, namely, the angle is measured. The
output matrix C takes the form h i
0 0 1
from which we get the observability matrix
 
 0 0 1 
O =  0 1 0
 

− Jf
 kīe
0

J

This matrix has full rank, so we can find φ(0) and the initial values of all of the other variables,
based on measurement of the angular position φ(t).
Conversely, assume that we choose the angular speed as an output. Then matrix C is
h i
0 1 0

and the observability matrix is

 
 0 1 0 
O =  kJīe − Jf 0 
 
(∗) (∗) 0
 

where (∗) are non-zero numbers. Now, the matrix rank is equal to 2, hence the kernel has dimension
1 and its unique basis vector is
 
 0 
x̄ =  0 
 
 
1
This means that x(0) and x(0) + α x̄ produce the same output for any α ∈ R, namely, we can change
the initial value of the angle and we will get the same output ω. Consequently, from the angular
velocity we cannot determine the position. This is expected, because the angle can be determined
from the speed by integration but we must know the initial value.5 Also if we measure the armature
current, we cannot determine the angular position (this is left as an exercise). In conclusion, given
the angle, we can find speed and current, but given any of these last two we cannot determine the
position.

Example 4.9. One way to determine the angular velocity based on the angle (which is theoretically
possible) is to use a pseudo-differentiator. Assuming y(0) = 0 (without restriction, since we measure
its value), in time and frequency (Laplace) domain we have

d
ω(t) = φ(t)
dt
ω(s) = sφ(s)

The transfer function above is not physically implementable, because it is not proper. However, we
can alter the high-frequency Bode diagram by inserting two poles at higher frequencies, far away
from the system bandwidth. For instance, we can take the strictly proper transfer function

s 1 s s
W(s) = = 2 = λ2
(1 + τs) 2 τ (s + τ )
1 2 (s + λ2 )

What we get is a Bode diagram of the type shown in Figure 4.5.

5
Recall that any primitive is defined up to a constant...
62 Chapter 4. Reachability and Observability

Figure 4.5: Bode plot of a real differentiator, with a pole at ω = 103 rad
s .

A continuous-time state space representation corresponding to this transfer function is given

(as we will show in the next chapter) by the matrices
" # " #
0 1 0 h i
A= 2 , B = , C = 0 λ2
−λ −2λ 1

We can then realise the corresponding discrete-time system by means of the equivalent discrete-time
matrices

AD = eAT
Z ⊤
BD = eAξ dξ B
0
CD = C

This type of pseudo-differentiator satisfactorily operates on a digital computer.

A different possibility is to choose

s ω20 s
W(s) = = ,
1 + 2 ωξ0 + s2
ω20
s2 + 2ξω0 s + ω20

which has a sharper frequency cut. A suitable state space representation is

" # " #
0 1 0 h i
A= , B= , C = 0 ω20
−ω20 −2ξω0 1

Example 4.10. A method based on accelerometers allows us to track the position based on the
acceleration, by means of a double integrator: acceleration → speed → position. This seems
to be in contradiction with the observability notion, and impossible. Indeed, the position is not
recoverable from the acceleration. As an exercise, one can write the state space representation of
the force-acceleration equation
ÿ(t) = u(t)

and check non-observability. Where is the trick?

4.9. Duality and dual system 63

In these cases, as opposed to the observability problem, the position and the speed at the initial
instant are assumed to be known. The differential problem to be solved is
Z t
v(t) = a(σ)dσ + v(0)
0
Z t
x(t) = v(t)dt + x(0)
0

and requires an integrator.

A weakness of the methods is that a small measurement error affecting a(t) has repercussions
on the estimated position ŷ(t). Assuming that x(0) = x′ (0) = 0 (this is not restrictive), we actually
obtain
ÿ(t) = u(t) + ω(t),
where ω(t) is the measurement noise. After a double integration, we have an estimated position
equal to
Z tZ ξ Z tZ ξ
ŷ(t) = u(ξ)dξ dσ + ω(ξ)dξ dσ = y(t) + e(t),
0 0 0 0
where the error is e(t). The double integration amplifies the noise effect, causing a drift. For this
kind of systems, no longer in use after the introduction of satellite systems, the accelerometer must
be reset at regular intervals, corresponding to positions where the location of the system is known
exactly (in other ways, of course).

4.9 Duality and dual system

The reachability problem and the observability problem are quite similar. This analogy can be
formalised by introducing the concept of dual system, which is a pure mathematical artifice, without
physical meaning, but is very helpful. Given a system
X
(A, B, C, D)

where the matrices are of dimensions n × n, n × m, p × n, and p × m respectively, we define dual

system represented by
∗
X X
= (A⊤ , C ⊤ , B⊤ , D⊤ )
Namely, the primal and the dual system have the following equations
∗
ẋ(t) = Ax(t) + Bu(t) ż(t) = A⊤ z(t) + C ⊤ v(t)
X ( X (
= and = (4.40)
y(t) = Cx(t) + Du(t) w(t) = B⊤ z(t) + D⊤ v(t)

An easy way to get the dual system matrices is take the transpose
" ⊤
A C⊤
" # #
A B T
−→ (4.41)
C D B⊤ D⊤

Note that the number of inputs to the dual system is p, equal to the number of outputs of the primal
system, while the number of outputs of the dual system is equal to m, the number of inputs to the
primal system. The state of the dual system, instead, always belongs to Rn , as in the primal system
case. Also observe the following relationships:

• reachability concerns matrices (A, B) for the primal system, (A⊤ , C ⊤ ) for the dual;
64 Chapter 4. Reachability and Observability

• observability concerns matrices (A, C) for the primal system and (A⊤ , B⊤ ) for the dual.
The reachability matrix R∗ of the dual system,
h i
R∗ = C ⊤ A⊤C ⊤ (A⊤ )2C ⊤ . . . (A⊤ )n−1C ⊤ (4.42)

is equal to the transpose of the observability matrix O⊤ of the primal system: R∗ = O⊤

  
 C   C 
 (A⊤C ⊤ )⊤   CA 
⊤
R∗ =  ..  =  ..  = O
   
(4.43)
 .   . 
 
⊤ n−1 ⊤
 ⊤   n−1 
(A ) C CA

The observability matrix of the dual system is the transpose of the reachability matrix of the primal:
O∗ = R⊤
B⊤
 
 
 B⊤ A⊤ 
 ⇒ O∗⊤ = B AB . . . An−1 B = R
h i
O∗ =  .. (4.44)
.
 

 ⊤ ⊤ n−1 
B (A )
Thus, fundamental properties hold:
• is reachable ⇐⇒ ∗ is observable;
P P

• is observable ⇐⇒ ∗ is reachable
P P

Observation 4.3. Observability and reachability problems are dual.

This duality property is useful from the mathematical standpoint, because every algorithm that
we use for reachability is valid, with some modifications, for solving observability problems by
means of the dual system.
A first application of the newly introducedh duality concerns
i the Popov criterion. We have
already seen that is reachable if and only if λI − A B has rank n for all eigenvalues λ of A.
P
P
By duality, we can then say that is observable if and only if:
λI − A
" #
rank =n (4.45)
C
for all eigenvalues λ of A. It is sufficient to apply the Popov criterion to the dual system and to
note that the rank of a matrix is equal to the rank of its transpose, and that A and A⊤ have the same
spectrum.
Another application concerns the parallel of systems. It has been shown that, given two systems
(A1 , B1 ) and (A2 , B2 ), with B1 and B2 column matrices, the parallel system is reachable if and only
if the two systems are reachable and have no common eigenvalues. Applying the same theorem
to the dual system and exploiting duality properties, we can say that given two systems (A1 , C1 )
and (A2 , C2 ), with C1 and C2 row matrices, the parallel system is observable if and only if the two
systems are observable and have no common eigenvalues.
Note also that, as far as the transfer function matrices are concerned,

W(s)∗ = W ⊤ (s),

namely, the dual transfer function matrix is the transpose. Clearly, if m = p = 1, the primal
system and the dual system have the same transfer function. The only difference in this case is
that cancellations due to the unreachable modes in the primal system become cancellations due to
unobservable modes in the dual system.
4.10. Joint Kalman form and transfer functions 65

4.10 Joint Kalman form and transfer functions

Suppose that reachability and observability of a system have already been investigated, and the
(uniquely determined) subspaces Xr = Ra(R) and Xno = ker(O) have been found. We can introduce
the following subspaces :
• X1 = Xr Xno ;
T

• X2 , such that Xr = X1
L
X2 (completion of X1 in Xr );
• X3 , such that Xno = X1
L
X3 (completion of X1 in Xno );
• X4 , such that X1 X4 = Rn .
L L L
X2 X3
X3 = Xr + Xno . Furthermore, since the subspaces Xnr and Xo are
L L
We also have that X1 X2
complements of Xr and Xno in Rn ,
• Xnr = X3
L
X4
• Xo = X2
L
X4 .
The relations between these subspaces areLsummarised in Table 4.1 (arrow means direct sum: e.g.,
the arrow in the first row means Xr = X1 X2 , and so on). It is now possible to choose a matrix T

reachable → X1 X2
non reachable → X3 X4
↑ ↑
non observable observable

Table 4.1: Relations between subspaces.

whose columns provide a basis for Rn and that is formed as follows:

h i
T = T1 T2 T3 T4 , (4.46)

where the columns of the matrix T k form a basis of the corresponding subspace Xk . Each vector
x ∈ Rn can be represented as:
 
 x̂1 
h i  x̂ 
x = T 1 T 2 T 3 T 4  2  (4.47)
 x̂3 
x̂4
Each reachable vector can be expressed as:
 
 x̂1 
h i  x̂ 
xr = T1 T2 T3 T 4  2  (4.48)
 0 
0
Each non-observable vector can be expressed as:
 
 x̂1 
h i  0 
xno = T1 T2 T3 T 4   (4.49)
x̂
 3 
0
66 Chapter 4. Reachability and Observability

Similarly, the unreachable vectors are expressed as:

 
 0 
h i  0 
xnr = T1 T2 T3 T 4   (4.50)
 x̂3 
x̂4

While the observable vectors are expressed as:

 
 0 
h i  x̂ 
xo = T1 T2 T3 T 4  2  (4.51)
 0 
x̂4

Using the transformation T , we get  = T −1 AT, B̂ = T −1 B, Ĉ = CT , D̂ = D, hence

      
 x̂1   A1 A12 A13
A14   x̂1   B1 
    
d  x̂2   0 A
2 0
A24   x̂2   B2 
 =  + u
dt  x̂3   0 0 A34   x̂3   0 
A3
 
   
x̂4 0 0 0A4 x̂4 0
 
 x̂1 
h i  x̂ 
y = 0 C2 0 C4  2  + Du (4.52)
 x̂3 
x̂4

All zeros in this matrix are structural. We can recognise the following subsystems:
• 1 (A1 ) is the reachable and unobservable subsystem;
P

• 2 (A2 ) is the reachable and observable subsystem;

P

• 3 (A3 ) is the unreachable and unobservable subsystem;

P

• 4 (A4 ) is the unreachable and observable subsystem;

P

Theorem 4.10. The transfer function depends on the reachable and observable subsystem only

W(s) = C(sI − A)−1 B = C2 (sI − A)−1 B2 (4.53)

The proof of the theorem requires tedious (not difficult) computations. This result is intuitive.
Unreachable components, in fact, are not affected by the input, while unobservable components do
not give any contribution to the output.

4.11 Cancellations
Consider the simplest case of a dynamic single input-single output (SISO) system, in which
n(s)
m = p = 1 and D = 0, hence W(s) = C(sI − A)−1 B = d(s) . The denominator d(s) is the
characteristic polynomial of matrix A, hence it has degree n.
The polynomials n(s) and d(s) are co-prime if they do not have common roots, namely, ∄ λ ∈ C
such that n(λ) = d(λ) = 0. Note that, if the two polynomial are not co-prime, then n(s) = (s − λ)ñ(s)
and d(s) = (s − λ)d̃(s), hence we have a zero-pole cancellation and
n(s) ñ(s)
W(s) = C(sI − A)−1 B = =
d(s) d̃(s)
4.11. Cancellations 67

Theorem 4.11. Given a linear system (A, B, C) with m = p = 1, the numerator n(s) and the
P
denominator d(s) of the transfer function are co-prime if and only if (A, B, C) is reachable and
observable.
Proof. We prove the implication (⇒) by contradiction. Assume that n(s) and d(s) are co-prime
and that the system is not reachable and observable. We have seen that the transfer function depends
only on the reachable and observable part of the system:

W(s) = C(sI − A)−1 B = C2 (sI − A2 )−1 B2

However, dim(A2 ) < dim(A). Then the denominator of the transfer function has degree at most
n2 = dim(A). This means that there is at least one cancellation. But then n(s) and d(s) cannot be
co-prime: a contradiction.
We now prove the implication (⇐). We assume that the system is reachable and observable,
with n(s) and d(s) not co-prime. We obtain a transfer function that is reducible through cancellations.
This means that, for some λ,

d(s) = det(sI − A) = 0 if s = λ (4.54)

" #
sI − A −B
n(s) = det = 0 if s=λ (4.55)
C 0
This implies that the matrix in (4.55) is singular and has a non-trivial kernel. We can write

λI − A −B
" #" #
x̄
=0 (4.56)
C 0 −ū
where x̄ has n components and ū is a scalar. The same applies to the matrix in (4.54): [sI − A] is
singular. We can write
h i h i h i " x̃ #
λI − A x̃ = λI − A x̃ + B · 0 = λI − A B =0 (4.57)
0
where the last equality is always true due to the properties of partitioned matrices. We have two
possible cases
• ū , 0: in this case the two vectors
" # " #
x̄ x̃
,
ū 0
h i
are linearly independent, therefore dim(ker λI − A B ) ≥ 2. Since the number of columns
h i
of λI − A B is equal to n + 1, it follows that the rank of this matrix is less than n, hence
the system is not reachable (contradiction).

• u = 0: in this case
λI − A −B λI − A
" #" # " # " #
x̄ B
= x̄ + 0 (4.58)
C 0 0 C 0
thus, if we consider only the first term, we get

λI − A
" #
x̄ = 0 (4.59)
C
It follows that the matrix has rank less than n, and therefore, in view of the Popov observability
criterion (see the section about the duality), the system is not observable: a contradiction.
68 Chapter 4. Reachability and Observability

The case of systems with m > 1 and p > 1 is much more involved. Consider the following
example.

Example 4.11. Given the system with matrices

" # " # " #
−1 0 1 0 1 0
A= , B= , C=
0 −1 0 1 0 1

the reachability and the observability matrices are

h i h i
R = I −I , O = I −I

and they both have rank n, hence the system is reachable and observable. However, the transfer
function matrix is
0
 s+1 
 (s+1)2 (s+1)2 
W(s) =  0

s+1 

(s+1)2 (s+1)2

and we can see that there is a cancellation in each term. Therefore, it is no longer true that a
reachable and observable system has no cancellations.

The case of multidimensional systems will not be analysed in detail here. However, we state
some fundamental results:

• if either reachability or observability of the system fails, there must be cancellations in the
transfer function matrix (but not vice versa, as already seen);

• if matrix A has distinct eigenvalues, then the system is reachable and observable if and only
if there are no cancellations.

4.11.1 External stability

The input-output behaviour and the state representation of a dynamical system can be studied, as
far as the asymptotic stability property is concerned, by introducing the following definition.

Definition 4.10. Let x(0) = 0. We say that a system (A, B, C, D) is externally stable or BIBO
P
(Bounded Input, Bounded Output) stable if, assuming that ∥u(t)∥ ≤ µ, then there exists ν such that
∥y(t)∥ ≤ ν.

We have the following result.

Theorem 4.12. A system is externally stable if and only if the reachable and observable subsystem
is asymptotically stable. If the whole system is reachable and observable, then asymptotic stability
is equivalent to BIBO stability.

We will not prove the theorem, which is intuitive. Indeed, non-reachable and non-observable
parts (if any) do not affect the input-output behaviour. Conversely, reachable unstable (or marginally
stable) modes can become arbitrarily large due to a bounded input and, if they are observable, they
will produce an arbitrarily large output.
We will prove the last part of the theorem, to get a very interesting formula, in the case
m = p = 1. Assume that the system is reachable and observable, and that A is asymptotically stable.
We show that the system is BIBO stable. Let x(0) = 0 and ∥u(t)∥ ≤ µ. We have:
Z t
y(t) = W(t − σ)u(σ)dσ
0
4.12. Controllable canonical form 69

hence
Z t Z t Z t
|y(t)| = W(t − σ)u(σ)dσ ≤ |W(t − σ)u(σ)| dσ = |W(σ)| |u(t − σ)| dσ ≤
0 0 0
Z t Z +∞ Z +∞
≤ µ |W(σ)| dσ ≤ µ |W(σ)| dσ = µ CeAσ B dσ = µ∥(A, B, C)∥1 = ν
0 0 0

where we have denoted by Z +∞

∥(A, B, C)∥1  CeAσ B dσ
0
It can be shown that this quantity is a norm and can be quite useful to compute an output limitation
in the presence of a limited input noise (typically due to non-quantifiable phenomena of which we
know, at least, some bounds on the amplitude).

Observation 4.4. The property of BIBO stability does not offer any kind of warranty on the proper
functioning of the system, if the system is not reachable or is not observable. In fact, unstable
unobservable modes do not affect the output, but may lead to exponentially increasing internal
variables. Unstable unreachable modes theoretically remain at zero, because in the definition of
BIBO stability we take x(0) = 0, but they may grow large when x(0) , 0 (no matter how small).
Non-observability and non-reachability is a pathology.

4.12 Controllable canonical form

P
Consider a system (A, B, C) with A of size n × n, B of size n × 1 (i.e., scalar input), and C of size
1 × p (i.e., scalar output). We can check that, if the pair (A, B) is reachable and the pair (A, C) is
observable, then there exists a transformation T such that ÂF = T −1 AT , B̂F = T −1 B and Ĉ F = CT
have the form:

0 ...
   
 0 1 0   0 
0 0 1 . . . 0  0 
   
 . . . .  . 
 
ÂF =  ..  .. .. . .. ..  , B̂F =  .. 

 (4.60)
0 ...
   
 0 0 1   0 
−a0 −a1 −a2 . . . −an−1

1
h i
Ĉ F = c0 c1 c2 . . . cn−1 (4.61)

This is called the Frobenius form. It is not difficult to prove the following

• the numbers ak are the coefficients of the characteristic polynomial;

• the numbers ck are the coefficients of the transfer function numerator.

The state transformation T can be computed as follows.

First notice that, given any transformation T , the reachability matrix in the new state form is
h i h i
R̂ = B̂ Â B̂ . . . Ân−1 B = T −1 B T −1 AB . . . T −1 An−1 B = T −1 R

hence
R̂ = T −1 R (4.62)
By duality we also have
Ô = O T (4.63)
70 Chapter 4. Reachability and Observability

(in brief, R and O transform as B and C). If, as assumed, m = 1 and the system is reachable, the
reachability matrix is square and invertible. Hence, T R̂ = R and
T = RR̂−1 . (4.64)
Therefore, to determine the transformation T , we have to compute R and R̂. We know that
the similarity transformation operation does not change the eigenvalues of the matrix A, so the
characteristic polynomial of  is the same as that of A. To determine the unknown elements of Â,
we need to compute the coefficients of the characteristic polynomial of A. Then, the transformation
T can be computed through the following steps:
1. compute det (sI − A) and form matrix  with its coefficients;
2. compute R;
3. compute R̂;
4. compute T = RR̂−1 .
As mentioned, Â is said to be in Frobenius form. This form is very important for three reasons,
two of which will be analysed later. The remaining reason is that a matrix in Frobenius form is
the simplest option to represent in a state form (n differential equations of the first order) a generic
linear differential equations of order n. If, for example, we have
y(n) (t) + an−1 y(n−1) (t) + . . . + a1 y(1) (t) + a0 y(t) = u(t), (4.65)
then we can define the state variables
x1 (t) = y(t)
x2 (t) = y(1) (t)
..
.
xn (t) = y(n−1) (t) (4.66)
and write the equivalent system of n first order differential equations, thus obtaining
ẋ1 (t) = x2 (t)
ẋ2 (t) = x3 (t)
...
ẋn−1 (t) = xn (t)
ẋn (t) = −a0 x1 (t) − a1 x2 (t) − . . . − an−1 xn (t) + u(t), (4.67)
which is equivalent to the Frobenius form (check!).
Clearly, in view of duality, we can define the (Frobenius) observability canonical form, where
A and C are the transposed matrices of A and B in the controllable canonical form.
 0 0 0 . . . −a0 
   
 b0 
 1 0 0 . . . −a1   b1 
   

Â∗F =  0 1 0 . . . −a2  , B̂∗F =  ... 

   
(4.68)
 .. .. .. . . ..
 . . . . .
  

  b2 
 
0 0 . . . 1 −an−1
   
bn−1
h i
Ĉ F∗ = 0 0 0 ... 1 (4.69)
Again, ak are the coefficients of the characteristic polynomial and bk the coefficients of the numerator
of the transfer function.
Chapter 5

Realisation

5.1 The problem in general

The realisation problem, in general, is the problem of associating a state-space representation with
a given input-output description. We will consider here the case of linear time-invariant systems,
whose input-output behaviour is described in terms of the transfer function matrix. For a linear
autonomous system (in the time domain), the solution of the equations is given by
R +∞
x(t) = eAt x(0) + 0 eA(t−σ) u(σ)dσ
(

y(t) = Cx(t) + Du(t)

By applying the Laplace transform, the expression becomes
x(s) = (sI − A)−1 x(0) + (sI − A)−1 Bu(s) + Dδ(t)
(

y(s) = C(sI − A)−1 x(0) + C(sI − A)−1 Bu(s)

If x(0) = 0, the input-output relation expressed by the transfer function matrix W(s):
y(s) = [C(sI − A)−1 B + D]u(s) = W(s)u(s).
Setting x(0) = 0 is not a strong restriction, because the essential information on the free evolution
of the system, namely the modes, is preserved and can be found in the denominator of W(s) (the
characteristic polynomial of the matrix A, if there are no cancellations).
Based on a given state space representation, we can easily determine the corresponding transfer
function matrix. Now the discussion concerns the opposite problem: can we derive a state
representation from an assigned transfer function matrix? This is the realisation problem. Studying
the realisation problem is useful from both a theoretical standpoint, to understand the link between
the realisation A, B, C, D and W(s), and from a practical standpoint, to physically implement filters
or controllers designed in the frequency domain (as explained in the course of Automatic Control).
Given A, B, C, D, respectively with size n × n, n × m, p × n and p × m, the transfer function
matrix is
N(s) N0 + N1 s + N2 s2 + . . . + Nν sν
W(s) = = C(sI − A)−1 B + D = ,
d(s) d0 + d1 s + d2 s2 + . . . + dν sν
where Nk are numeric matrices of dimension p × m. Note that the elements of W(s),
h i " ni j (s) #
Wi j (s) = ,
d(s)
are rational functions. Hence, the realisation problem can provide a state-space representation of
the form Σ(A, B, C, D) only if we start from a rational transfer function matrix. Moreover, the
given rational transfer function matrix has to be proper.

71
72 Chapter 5. Realisation

Example 5.1. It is legitimate to ask how we can realise the transfer function W(s) = e−τs (delay),
but we will not find matrices Σ(A, B, C, D) that provide a realisation.
Example 5.2. The PID controllers given by the transfer functions
KI K D s2 + K P s + K I
G(s) = G D (s) + KD + =
s s
are not directly implementable, because the above rational function is not proper. Then, it does not
admit a realisation A, B, C, D. A physical implementation is possible if we introduce one or two
additional poles, such as
K I + K P s + K D s2
G̃(s) = .
s(1 + τs)2
These artificially introduced poles are located at high frequency, far from the band of frequencies of
the system. They are not only necessary for the realisation, but also beneficial, since they have a
low-pass effect.
Definition 5.1. Given a proper rational matrix W(s), we say that Σ(A, B, C, D), is a realisation if
W(s) = C(sI − A)−1 B + D.
The realisation problem corresponds to finding Σ(A, B, C, D) for a given W(s).
The first step for solving the realisation problem is the determination of matrix D. We have that
• if W is strictly proper, then D = 0;
• if W is weakly proper, then D , 0 and its value can be obtained by division.
Division means writing the transfer matrix as the sum of a constant matrix and a strictly proper
transfer function:
Nν sν + Nν−1 sν−1 + . . . + N0
W(s) = =
d0 + d1 s + . . . + sν
Nν (d0 + d1 s + . . . + sν ) + (Nν−1 − Nν dν−1 )sν−1 + (Nν−2 − Nν dν−2 )sν−2 + . . . + (N0 − Nν d0 )
=
d0 + d1 s + . . . + sν
ν−1
Ñν−1 s + . . . + Ñ0
= Nν + = D + W̃(s) (5.1)
d0 + d1 s + . . . + sν
Then, the constant matrix D is obtained (which is obviously zero in the strictly proper case).
The next step is finding the matrices A, B, C starting from the strictly proper part of the transfer
function.
So let W(s) be strictly proper and write it as
N0 + N1 s + . . . + Nν−1 sν−1
(5.2)
d0 + d1 s + . . . + sν
Determining a realisation is straightforward. We can just adopt the following matrices (generalised
Frobenius form):
...
 
 0 Im 0 0
 0 
  
 0 0 I . . . 0

m   0 

. . .
 
A =   0 0 0 I m  , B =  .  ,

 .. 
.. .. .. .. .. 
. . . . .
   

Im
−d0 Im −d1 Im −d2 Im . . . −dν−1 Im
 
h i
C = N0 N1 N2 . . . Nν−1 , (5.3)
5.2. Minimal realisation 73

where Im is the identity matrix of dimension m × m. To demonstrate that this realisation returns
W(s) we just write:
ϕ(s) = (sI − A)−1 B ⇒ (sI − A)ϕ(s) = B. (5.4)
By direct verification we can check that ϕ(s) is given by
 
 I 
 Is
 

1  s2 I
ϕ(s) =

(5.5)
d(s)  ..
 
 .


 ν−1 
s I

and then check the equation

W(s) = Cϕ(s). (5.6)
This realisation has a matrix A of dimension mν × mν, where mν is greater than ν (the degree of the
denominator of the transfer functions). Therefore, we suspect that this is not a minimal realisation,
in terms of state-space dimensions, and in general it is not.

Remark 5.1. By duality, one can find a realisation as follows. Consider the transfer function matrix
W ⊤ (s) of the dual system and find a realisation (A∗ , B∗ , C∗ , D∗ ), of dimension pν × pν. Consider
the dual realisation (A⊤ ∗ , C ∗ , B∗ , D∗ ): it is a realisation for W. This procedure is convenient if
⊤ ⊤ ⊤

p < m (and vice-versa).

Example 5.3. As a preliminary example, we show that the realisation found as above can be non
minimal. Take i h [02] [ 1 0 ]s
h i
W(s) = (s+1)(s+2)
2 s
(s+1)(s+2) = (s2 +3s+2) (s2 +3s+2)
The realisation has state dimension 2m = 4.
   
 0 0 1 0 
  0 0 
 0 0 0 1   0 0  h i
A =   B =   C = 0 2 1 0
 −2 0 −3 0 


 1 0 

0 −2 0 −3 0 1

Let us now consider the dual W ⊤ . The realisation is now p × 2 = 1.

" # " # " #
0 1 0 0 1
A∗ = B∗ = C∗ =
−2 −3 1 2 0

∗ , C ∗ , B∗ ) is a smaller realisation of order 2. We will see that this is minimal, in terms of

Then (A⊤ ⊤ ⊤

number of state variables.

5.2 Minimal realisation

In the problem of finding a state-space realisation for W(s), the input dimension m and the output
dimension p are the dimension of the transfer function matrix W(s), hence they are fixed. However,
the state space dimension n is not known. Since n is a nonnegative integer, there will be a minimal
possible value of n that allows us to provide a realisation.

Definition 5.2. Matrices (A, B, C, D) are a minimal realisation of W(s) if they are a realisation,
namely C(sI − A)−1 B + D = W(s), and there is no other realisation (A′ , B′ , C ′ , D) with dim(A′ ) <
dim(A).
74 Chapter 5. Realisation

Finding a minimal realisation amounts to minimising the size of matrix A, namely n. Once
again, the number of columns B (equal to m) and the number of lines of C (equal to p) are fixed.
The following fundamental theorem characterises minimal realisations.

Theorem 5.1. The realisation (A, B, C, D) of W(s) is minimal if and only if (A, B, C) is reachable
and observable.

The implication ⇒ has already been discussed, because if the realisation is minimal then
there are cannot be unreachable or unobservable parts in the system. Otherwise, adopting Kalman
decomposition, we could just take the reachable (or observable) subsystem which has the same
transfer function, but a smaller dimension. The implication ⇐ is quite difficult to prove: its proof
can be found in specialised texts. The theorem tells us something interesting.

Observation 5.1. If (A, B, C, D) is a realisation, but not a minimal one, then the reachable and
observable subsystem (A2 , B2 , C2 , D) gives us a minimal realisation (because it has the same
transfer function).

The next question is the following. Given two minimal realisations, is there any difference
between the two?
Assume to have obtained two minimal realisations, (A1 , B1 , C1 , D1 ) and (A2 , B2 , C2 , D2 ), of
course with the matrices A1 and A2 having the same size. There is a fundamental link between the
two solutions.

Theorem 5.2. Let (A1 , B1 , C1 , D1 ), with state dimension n, be a minimal realisations of W(s).
Then (A2 , B2 , C2 , D2 ) is a minimal realisations of W(s) if and only if: if has state dimension n,
D1 = D2 and there exists T , invertible such that T −1 A1 T = A2 , T −1 B1 = B2 C1 T = C2 .

The previous theorem states that all minimal realisations are equivalent up to a state transforma-
tion.
The implication ⇐ is simple, because if (A1 , B1 , C1 ) is a minimal realisation and we apply a
state transformation we obtain a system with the same transfer function and a state matrix having
the same size as A1 . The implication ⇒ is much more difficult to prove and is not discussed.
We can say that if a given linear system (A, B, C, D) is modelled by means of transfer functions,
any information about the unreachable and unobservable parts is lost. If we assume that the system
is reachable and observable, we still loose some information because we will not be able to
determine the true realisation we started from, but we will be able to determine a realisation which
is related to the original one by a state transformation.
We conclude with the general procedure to derive a minimal realisation.

1. Given W(s) rational and proper, determine D by division: W(s) = D + W̃(s), with W̃(s)
strictly proper.

2. Consider the strictly proper part W̃(s) and take any realisation, not necessarily minimal (for
instance the generalised Frobenius one).

3. Remove the unobservable and unreachable part to get a minimal realisation.

It is worth mentioning that the generalised Frobenius realisation shown above is always reachable.
So, in this case, we need to remove the unobservable part only. By duality, the dual generalised
Frobenius realisation is always observable and in this case we only need to remove the unreachable
part.
5.3. Minimal realisation for SISO systems 75

5.3 Minimal realisation for SISO systems

The transfer function, in the case of single-input and single-output (SISO) systems (m = p = 1),
can be written as
n0 + n1 s + . . . + nν−1 sν−1
W(s) = +D (5.7)
d0 + d1 s + . . . + sν
where D is zero if and only if W is strictly proper. As discussed earlier, one possible tool for the
realisation and implementation is given by the Frobenius form

0 ...
 
 0 1 0 
 0 
 
 0 0 1 ... 0 
 
 0 
A =  . . .  , B =  ..  ,
 0 
0 0 1   
 .. .. .. ..
.
..  . 
 . . . .

 1
−d0 −d1 −d2 . . . −dν−1
h i
C = n0 n1 n2 . . . nν−1 (5.8)

From Theorem 4.11, since m = p = 1, this realisation is minimal if and only if there are no
cancellations in W(s).
A nice property of the Frobenius form is that it can be implemented not only by a digital device,
but also by an analog device, by adopting the block diagram presented in Figure 5.1.

+ + + +
y
D n ν−1 n ν−2 n1 n0

u xν xν x ν−1 x2 x1
+
−

d ν−1 d ν−2 d1 d0

+ + +

Figure 5.1: Block diagram of the minimal realisation with m = p = 1.

The diagram is easy to interpret if one considers that, given the form of the matrices, we obtain
the following expressions

ẋi = xi+1 ∀ i = 1, . . . , n − 1
 ν−1 
X 
ẋn = −  di xi+1  + u
i=0
ν−1
X
y = n j x j+1 + Du (5.9)
j=0

Each block in Figure 5.1 can be implemented by means of operational amplifiers.

5.3.1 Other techniques for SISO systems realisation

There are many different techniques for the realisation of systems, especially for SISO systems.
76 Chapter 5. Realisation

For instance, if we know the poles of the transfer function,

n0 + n1 s + . . . + nν−1 sν−1
W(s) =
(s − λ1 )(s − λ2 ) . . . (s − λn )
if the poles are distinct we can compute the fraction expansion
n
X r1
W(s) =
i=1
(s − λi )
and notice that a realisation is
λ1 0 0 ... 0
   
   1 
0 λ2 0 ... 0 1
   
   
A =  0 0 λ3 . . . 0  ,

B = 
 1  ,

.. .. .. . . . ..
. . . . .. .
   
   
0 . . . λn
   
0 0 1
h i
C = r1 r2 r3 . . . rn (5.10)
If there are complex poles, the previous realisation is complex. To deal with this case, one can
consider that pairs of complex poles give rise to real terms in the sum of degree two
r r∗ αs + β
+ =
s−λ s−λ ∗ (s − ξ)2 + ω2
This partial term can be realised by means of a 2 × 2 Frobenius realisation or by
ξ ω
" # " #
0 h i
A= , B= , C= µ ν
−ω ξ 1
where µ and ν can be determined from α and β.

5.4 Realisation of discrete-time systems

The problem of the realisation of discrete-time systems is analogous to the realisation problem
for continuous-time systems. The only difference is that the transfer function is obtained by using
the Z-transform. Recall that the Z-transform is an operator that associates with a sequence f (k) a
function of complex variable F(z) defined as follows
+∞
Z X 1
f (k) −→ F(z) = f (k) k
k=0
z
The operator is linear and it has following property
Z
f (k + 1) −→ zF(z) − z f (0)
If we consider the discrete-time system
x(k + 1) = Ax(k) + Bu(k), y(k) = Cx(k) + Du(k)
and we set x(0) = 0, we have
y(z) = W(z)u(z)
where W(z) is the discrete-time transfer function
W(z) = C(zI − A)−1 B + D,
which has the same expression as the transfer function for continuous-time systems. Then, the
realisation problem is solved in exactly with the same approach adopted in the continuous-time
case.
Chapter 6

Control Systems

6.1 General considerations

The main goal of a control system, in general terms, is to govern a system in order to achieve the
desired behaviour, by suitably choosing the control input u. There can be many special goals in
designing a control. The main are

• regulation: steer a system to a desired state and keep it there;

• tracking: steer a system along a desired trajectory;

• disturbance rejection: mitigate the effect of disturbances.

There are two main types of control.

• Open loop control: the input u(t) applied to the system is synthesised by means of some a
priori criterion based on the knowledge of the initial state and does not require the knowledge
of x(t) or y(t) in real time.

• Feedback control: the input to the process is generated in real time (instant by instant) by a
control algorithm that takes into account the current output (output-feedback) or the current
state (state-feedback) of the system.

A simple example of open-loop control is given by a temporised oven, which heats a cake for a
certain amount of time (decided at the beginning) and then stops.1 The main feature of a feedback
control is that it is able to modify the dynamics of a system to make it stable, or faster in terms
of convergence. A feedback control makes the system less sensitive to disturbances and assures
efficient tracking. In this chapter, we consider the case of feedback control for linear systems. This
technique will be applied to nonlinear systems later on.
In general, the inputs and outputs of a process can be divided according to Figure 6.1.

d e
u y

Figure 6.1: Subdivision of inputs and outputs in a system.

1
Inconveniences of this approach are often experimented by inexperienced bakers.

77
78 Chapter 6. Control Systems

• Vector d(t) represents exogenous, external inputs not modifiable by the user. This category
includes, for example, noise and external loads.

• Vector u(t), instead, represents the control inputs, on which we can actually act in real time,
in order to control the process.

• Vector e(t) is the performance output, on which design specifications are assigned.

• Vector y(t) is the measured output, namely, the signal measured in real time by sensors,
which can be used to generate a feedback control law.

The input u(t) corresponds therefore to the action of the actuators, while the output y(t) originates
from the sensors monitoring the system. The process needs to be equipped with these devices in
order to be governed by a feedback controller.
Example 6.1. (Damper.) Consider the damping control problem for a vibrating systems. The goal
is to keep the coordinate θ(t) small in the presence of a disturbance d(t). The measured output is
the derivative θ̇(t)

θ̈(t) = −µθ(t) + d(t) + u(t)

e(t) = θ(t)
y(t) = θ̇(t)

A control can be any system with input y and output u, such as a simple passive damper: u(t) =
−ky(t). Needless to say, the eigenvalues play a fundamental role. For instance, in the case of the
damper, the closed loop eigenvalues are the solutions of

s2 + κs + µ = 0

The eigenvalues are constrained on the so called root locus [λ1 (κ), λ2 (κ)]. It can be shown that the
√
real part is minimised by κ∗ = 2 µ. This control has one degree of freedom only. By adopting
more sophisticated techniques, we will be able to arbitrarily place the eigenvalues of the system.

SYSTEM

REGULATOR

Figure 6.2: Linear system with feedback: block diagram.

In general, a feedback-regulated process can be described by the scheme in Figure 6.2. The first
purpose of the feedback is the stabilisation of the process, if the system is not already stable. Even
in the case of stable processes, feedback can have beneficial effects in improving the dynamics
(yielding, for instance, faster convergence). In the case of linear systems, this essentially means
changing the position of the eigenvalues in the complex plane, in order to modify the system
transient. The allocation of the eigenvalues is possible only by means of feedback controllers
(open-loop controllers do not modify the eigenvalues).
6.1. General considerations 79

The generic linear process to be controlled has the form

ẋ(t) = Ax(t) + Bu(t) + Ed(t)




= Cx(t)

y(t) (6.1)



 e(t) = Lx(t)


The controller is, in turn, a dynamical system

ż(t) = Fz(t) + Gy(t)

(
(6.2)
u(t) = Hz(t) + Ky(t)

By connecting the two systems, the overall system becomes

A + BKC BH
" # " #" # " #
d x(t) x(t) E
= + d(t)
dt z(t) GC F z(t) 0
h i " x(t) #
e(t) = L 0 (6.3)
z(t)

The closed-loop state matrix is

A + BKC BH
" #
ACL =
GC F

and the modes of the closed-loop system are given by the eigenvalues of ACL . The first goal of
a controller is to modify the original eigenvalues of matrix A and get new eigenvalues that are
chosen appropriately. A more advanced task is to optimise the performance, under the constraints
of closed-loop stability.
The basic problem, which we face for the moment, is the following.
Problem 6.1. Eigenvalues assignment problem. Given a set ΛCL of N = dim(A) + dim(F)
complex numbers, ΛCL = {λ1 , λ2 , . . . , λN }, determine the unknown matrices F, G, H, K so that the
closed-loop eigenvalues are the element of ΛCL :

A + BKC BH
" #!
σ = ΛCL
GC F

In the following, we will always assume that the chosen set ΛCL satisfies the conjugate
eigenvalue constraint (CEC):
λ ∈ ΛCL ⇒ λ∗ ∈ ΛCL .
This means that either λ is real or, if it is complex, then also λ∗ belongs to ΛCL .2 If we do not
respect this condition, then the solution F, G, H, K will be given by complex matrices, hence
useless from a practical standpoint.
A fundamental principle in dealing with the allocation problem is the following. The eigenvalues
of the unreachable subsystem and of the unobservable subsystem cannot be changed.
Theorem 6.1. The unreachable eigenvalues, given the pair (A, B), and the unobservable eigenval-
ues, given the pair (A, C), are invariant under feedback.
There is no need to prove this theorem. Unreachable an unobservable subsystems can be
removed without changing the input-output behaviour of the system, hence no regulator can change
them. A fundamental consequence is the following.
2
For instance, −1, −2+ j3, −2− j3, −3+2 j, −3−2 j, −4 is a proper choice, while −1, −2+ j3, −2− j3, 6+2 j, −3−2 j, −4
is not.
80 Chapter 6. Control Systems

Theorem 6.2. A process can be asymptotically stabilised if and only if the unreachable and the
unobservable eigenvalues (if any) are asymptotically stable, namely, they have negative real part.
Necessity follows from Theorem 6.1. The sufficiency part will be proved in a constructive way:
we show how to arbitrarily modify reachable and observable modes (under CEC).
To study the allocation of the eigenvalues we will work, without restrictions, under the assump-
P
tion that (A, B, C) is reachable and observable. Its unreachable and/or unobservable parts (whose
eigenvalues are fixed) can be eliminated after having reduced the system in Kalman form.
We will solve the eigenvalue assignment problem in three steps.

• State-feedback design: we assume3 that any state variable is measured and we solve the
allocation problem.
• Observer design: we design a device that estimates the state variables based on output
measurement.
• Pole placement regulator: we put together these two steps and we apply the state feedback
to the estimated state variables.

In the first step, state feedback design, we consider the following equations:
ẋ(t) = Ax(t) + Bu(t)
(
(6.4)
u(t) = K x(t)
The corresponding diagram is shown in Figure 6.3.

u x x
B

Figure 6.3: State feedback.

Note that a direct practical implementation of the state feedback u(t) = K x(t) requires a sensor
for each state variable, and this may be unrealistic, expensive and even impossible in some cases.
This leads to the second step, state estimation, in accordance to diagram shown in Figure 6.4.
The estimation block returns an estimated value x̂(t) of the state x(t) of the process. This value
is subsequently used to operate the feedback u(t) = K x̂(t). This is convenient from the economic
point of view, since it does not require sensors for all state variables. In the following sections we
will study separately and in detail the two design phases above.

6.2 State feedback

We now assume that the measured output is the state x. This is actually true in some applications.
To design a state feedback controller, we assume that a choice has been made for the closed loop
system eigenvalues
Λc = {λ1 , λ2 , . . . , λn } ∈ C
3
We pretend, because this is not true in general.
6.2. State feedback 81

OBSERVER

Figure 6.4: State observer.

where the subscript c stands for “compensator” and the CEC constraint are satisfied (so as to deal
with real problems). We look for a real matrix K such that the system

ẋ(t) = Ax(t) + Bu(t)

(
⇒ ẋ(t) = (A + BK)x(t) (6.5)
u(t) = K x(t)

has the desired set of eigenvalues

σ(A + BK) = Λc

The matrix K has dimension m × n, hence there are m · n free parameters to be assigned in order to
solve the problem.
Here we have much more freedom with respect to classical techniques, such as root locus and
PID design, where the number of free parameters is limited.

Theorem 6.3. The problem has a solution with Λc arbitrarily taken (provided that CEC constraints
P
are satisfied), if and only if the system (A, B) is reachable. Instead, if the system is not reachable,
then the problem has solution if and only if Λc includes all of the unreachable eigenvalues.

Proof. If the system is not reachable, we can turn it into Kalman form. The equation u(t) = K x(t)
turns into u(t) = KT x̂(t) = K̂ x̂(t).4 We get
" # " #h
Âr Âr, nr B̂r i
 + B̂K̂ = + K̂1 K̂2 =
0 Ânr 0
Âr + B̂r K̂1 Âr, nr + B̂r K̂2
" #
= (6.6)
0 Ânr

This is a block-triangular matrix, whose spectrum is given by σ(Âr + B̂r K̂1 ) ∪ σ(Ânr ). The resulting
spectrum includes the eigenvalues of Ânr , the unreachable ones, which are invariant and cannot be
changed with the feedback. So we cannot assign Λc if it does not include the eigenvalues of Ânr .
We now assume that the system is reachable and we show how we can arbitrarily assign the
eigenvalues to the system. We now consider the case m = 1, while we will discuss the general case
later. The proof is constructive.
If the system is reachable, we can always find T such that the transformed system AF = T −1 AT ,
BF = T −1 B is in the Frobenius form (note that the new feedback matrix is transformed as well,

4
Note that matrix K is transformed as matrix C.
82 Chapter 6. Control Systems

K̂ = KT ). The closed-loop matrix is

...
 
 0 1 0 0 
0 0 ... 0
 
.. .. .. ..
 
AF + BF K̂ =  ..
. . . . .
 
 (6.7)
...
 
 0 0 0 1 
−a0 + k̂1 −a1 + k̂2 −a2 + k̂3 . . . −an−1 + k̂n
 

and it is therefore still a matrix in Frobenius form. Then the new coefficients of the closed-loop
characteristic polynomial are the opposite of the entries of the last row of the matrix. These elements
can be changed by selecting the individual elements k̂i . The problem of the arbitrary eigenvalues
assignment can be then reduced to the assignment of the characteristic polynomial whose roots
are the desired eigenvalues, which is p̂(s) = ni=1 (s − λ̄i ).
Q
The steps to solve the problem are the following.

1. Compute the coefficients a0 , a1 , . . . , an−1 of the characteristic polynomial of matrix A (with

no feedback).

2. Choose Λc = {λ̄1 , λ̄2 , . . . , λ̄n } and compute the new characteristic polynomial
n
Y
p̂(s) = (s − λ̄i ) = ā0 + ā1 s + . . . + ān−1 sn−1 + sn
i=1

3. Compute k̂ = [ k̂1 k̂2 . . . k̂n ], whose elements are derived as follows

ā0 = a0 − k̂1 ⇒ k̂1 = a0 − ā0

ā1 = a1 − k̂2 ⇒ k̂2 = a1 − ā1
...
ān−1 = an−1 − k̂n ⇒ k̂n = an−1 − ān−1

4. Return to the original form applying the inverse transformation: K = K̂T −1 .5

If m > 1, the proof is more involved. The idea is to use one input at the time and proceed as
follows.

• If the system is reachable from a single input, for instance u1 , then there is nothing to prove.
The closed loop matrix will be A + B1 K1 where B1 is the first column of B.

• Alternatively, we assign all of the eigenvalues of the subsystem that are reachable from the
first input, achieving the matrix A∗ + B1 K1 . The system becomes

(A∗ + B1 K1 , [B2 . . . Bm ]) = (A(1) , B(1) )

where A1 has the eigenvalues already assigned. The first input is no longer considered,
namely, we set m := m − 1, A := A∗ + B1 K1 , B := [B2 . . . Bm ].

• If there are still eigenvalues to assign, then go back to the previous step with m − 1 inputs,
in order to assign all of the eigenvalue we can (by means of some K2 ) and leave unchanged
those already assigned. Otherwise STOP.
5
This step is necessary because Frobenius form is only a tool to facilitate the solution of the problem, so we cannot
apply k̂ directly.
6.2. State feedback 83

• The final closed-loop matrix will be

K1
 
 
 K2 
K =  ..
 
.

 
 
Km

Example 6.2. Consider

" # " #
−1 1 1 −1
A= B=
1 −1 1 −1
The system is reachable (why?), but not by means of a single input. A has eigenvalues 0 and −2.
We wish to assign the eigenvalues −4 and −4.
Consider the first input. 0 is reachable with B1 , while −2 is not (use Popov tho check). So let us
move 0 to −4. With the feedback matrix K1 = [−2 − 2] we get
" # " # " #
−1 1 1 −3 −1
A + B1 K1 = + [−2 − 2] =
1 −1 1 −1 −3

having eigenvalues −4 and −2.

Consider the second input. −4 is not reachable while −2 is reachable (use Popov to check). So
let us move −2 to −4 in the new A (−4 remains unchanged!)
" # " # " #
−3 −1 −1 −4 0
A + B1 K1 + B2 K2 = + [ 1 − 1] =
−1 −3 1 0 −4

The eigenvalues are the desired ones. The overall feedback matrix is
" #
−2 −2
K=
1 −1

Please check!6

In general it can be proved that, if the system is reachable, the procedure is successful. These
considerations are valid from a mathematical point of view. In practice, this procedure might not
distribute the work among the different actuators in a proper way. For instance, if the system is
reachable from the first input, the procedure would provide a feedback which uses the first input
only. This multi-input control problem will be reconsidered in the optimal control theory.

6.2.1 A simple algorithm for eigenvalue assignment

P
Now we suggest a very simple algorithm for eigenvalue assignment for a reachable system (A, B),
with m = 1 and matrix B having the form

b1
 
 
 b2 
B =  ..
 
(6.8)
.

 
 
bn
6
This is the English “please”, which basically means that the reader is strongly invited to check...
84 Chapter 6. Control Systems

We consider the unknown K and we write the characteristic polynomial of the feedback system

det(sI − (A + BK)) =
 s − a11 − b1 k1 −a12 − b1 k2 . . . −a1n − b2 kn
 

 −a − b k
21 2 1 s − a22 − b2 k2 . . . −a2n − b2 kn 
= det  .. .. .. ..  =
 
 . . . . 
−an2 − bn k2 . . . s − ann − bn kn
 
−an1 − bn k1
= a0 (k1 , k2 , . . . , kn ) + a1 (k1 , k2 , . . . , kn )s + . . . + an−1 (k1 , k2 , . . . , kn )sn−1 + sn (6.9)

The characteristic polynomial coefficients are functions of the unknown elements ki . It turns out
that these functions are affine.7 Now we simply match the coefficients of the desired polynomial,
ai , with those desired after feedback, āk : p̄(s) = ā0 + ā1 s + ā2 s2 · · · + sn . Hence

a0 (k1 , k2 , . . . , kn ) = ā0



 a1 (k1 , k2 , . . . , kn ) = ā1




(6.10)
...




 a (k , k , . . . , k ) = ā


n−1 1 2 n n−1

The solution of this linear system provides the elements ki , hence solving the state feedback
problem.

Observation 6.1. The reachability test has to be performed before the algorithm. If the system is
not reachable, then the matrix of the system is singular. In this case, we have a solution only if we
choose a polynomial p̄(s) that has the non-reachable eigenvalues among its roots.

Example 6.3. Consider the unstable system described by the equations:

" # " #
2 1 1
A= , B=
1 2 0

Then the closed-loop characteristic polynomial is

" #
s − 2 − k1 −1 − k2
det(sI − (A + BK)) = det = s2 − (4 + k1 )s + 3 + 2k1 − k2
−1 s−2

Suppose that the desired new eigenvalues are λ̄1 = −1 and λ̄2 = −2. Then, the characteristic
polynomial of the closed-loop system should match

p̂(s) = (s + 1)(s + 2) = s2 + 3s + 2

We get the following linear system:

−4 − k1 = 3 k1 = −7
( (
⇒
3 + 2k1 − k2 = 2 k2 = −13

6.3 State observer

A state observer is a fundamental device when the state variables are not all accessible, or are not
measured for economic reasons (more sensors imply additional cost). Consider the scheme in
Figure 6.5.
7
An affine function is the sum of a constant and a linear function: e.g., 2k1 + 3k2 + 1 is affine.
6.3. State observer 85

u x x y
B C

Figure 6.5: General scheme of a system.

The most favourable case that can occur is when C is the identical matrix (or has full row
rank): this means that the outputs are precisely the state variables (or that the state variables can
be immediately computed based on the outputs). Then, we can straightforwardly apply a state
feedback controller. However matrix C, in general, has less rows than columns (often it has a single
row). Hence, we have to proceed differently.
A first idea (which will not lead us to the solution) could be to replicate the system by
implementing a device with the same equations, namely, with the same matrices of the process:
˙ = A x̂(t) + Bu(t)
x̂(t) (6.11)

This is a trivial observer (open-loop), in which x̂(t) is an estimate of the system state variables. To
evaluate the effectiveness of this solution (or of any other estimator), we monitor the estimation
error
e(t) = x̂(t) − x(t)
Since the system is described by the equations
˙ = Ax(t) + Bu(t)
(
x̂(t)
y(t) = Cx(t)

we can subtract the state equation from the estimated state equations. We obtain
d
( x̂(t) − x(t)) = A( x̂(t) − x(t)) + Bu(t) − Bu(t) = A( x̂(t) − x(t)) (6.12)
dt
The estimation error is then described by the equation

ė(t) = Ae(t) (6.13)

and therefore it evolves in natural response. We immediately understand that this open-loop observer
is not applicable to unstable systems, because the error diverges. Even for asymptotically stable
systems, trivial observers do not provide good results. Indeed, the state x̂(0) must be initialised in
some way and cannot be initialised exactly to x(0), which is unknown. In the absence of specific
information, for the symmetry of the problem, the most obvious choice is to set x̂(0) = 0. This
means that e(0) = x̂(0) − x(0) = −x(0). Assuming, for example, u(t) = 0, we obtain

ẋ = Ax(t),
ė(t) = Ae(t), (6.14)

with e(0) = −x(0), so the error has opposite initial conditions with respect to the state. By linearity,
we get
e(t) = −x(t)
Then, the error (shown in Figure 6.6) has the same amplitude of the state ∥x(t)∥ = ∥e(t)∥: the relative
error is 100 %. We would like to design an observer able to provide an estimate of the state that
86 Chapter 6. Control Systems

Figure 6.6: Evolution of the state (solid line) and of the error (dotted line) with the trivial observer.

Figure 6.7: Evolution of the state (solid line), the estimated state (dashed line) and of the error
(dotted line) with the desired observer.

converges to x(t), also in the case of unstable systems. If the system is stable, we would like the
observer to have a faster convergence than the natural convergence speed assured by the system
modes, as qualitatively shown in Figure 6.7.
Then, the right solution is given by the Luenberger observer. The trivial observer uses
information about u(t), but not about y(t). On the contrary, the Luenberger observer uses information
about both the input and the output of the process. The differential equation that governs the state
estimate is then
˙ = A x̂(t) + Bu(t) − LC( x̂(t) − x(t)),
x̂(t) (6.15)
where L is a matrix to be determined. Note that the difference is the correction term

LC( x̂(t) − x(t)) = L[ŷ(t) − y(t)]

The quantity ŷ(t) is the estimated output, which is compared with the true output y(t) and fed back.
This device is often called feedback observer. Note that the state estimation error x̂(t) − x(t)
is unknown, but the output estimation error ŷ(t) − y(t) = C( x̂(t) − x(t)) = C x̂(t) − y(t) is known
on-line, because x̂(t) is the known observer state and y(t) is the measured output. For the observer
implementation, we use the equivalent equation

˙ = (A − LC) x̂(t) + Bu(t) + Ly(t)

x̂(t) (6.16)

Matrix L, having size n × p, contains the free design parameters. For L = 0 we recover the
trivial observer. The general scheme for the Luenberger observer is represented in Figure 6.8.
6.3. State observer 87

u x x y
B + C

x x
B + C +

Figure 6.8: General scheme of a Luenberger observer.

To see how this new device works, we compute the error expression, subtracting the state
derivative from the estimated state derivative:
d
( x̂(t) − x(t)) = (A − LC) x̂(t) + Bu(t) + Ly(t) − Ax(t) − Bu(t) =
dt
= (A − LC) x̂(t) − Ax(t) + LCx(t) =
= (A − LC) x̂(t) − (A − LC)x(t)
= (A − LC)( x̂(t) − x(t))

and we get
ė(t) = (A − LC)e(t) (6.17)
Therefore, the error does not depend on u(t): it evolves in natural response according to the
modes of matrix (A − LC). Hence, the role of matrix L is similar to that of matrix K in a state
feedback controller. In analogy to the problem of eigenvalue placement via state feedback, we
would like to impose a set of eigenvalues Λo (O stands for observer) to matrix (A − LC), so as to
ensure fast converge of the error to zero. More formally, the problem is as follows.
Problem 6.2. Observer eigenvalues assignment Given the set of complex numbers

Λo = {λ̄1 , λ̄2 , . . . , λ̄n }

satisfying CEC (conjugate eigenvalues constraint), find matrix L so that the set of eigenvalues of
A − LC is
σ(A − LC) = Λo
Then we have the following result.
Theorem 6.4. The set Λo (satisfying CEC) can be arbitrarily assigned if and only if (A, C) is
observable. If the system is not observable, then Λo can be assigned if and only if it contains all of
the eigenvalues of the unobservable subsystem.
Proof. To prove the theorem, we exploit duality. Consider then the dual of the system that
provides matrix (A − LC):

A∗ = A⊤
B∗ = C ⊤
K ∗ = −L⊤ (6.18)
88 Chapter 6. Control Systems

The dual of matrix (A − LC) is

(A − LC)⊤ = (A⊤ − C ⊤ L⊤ ) = (A∗ + B∗ K ∗ ), (6.19)

where K ∗  −L.
Then assigning the spectrum of (A − LC) is equivalent to assigning the spectrum of (A∗ + B∗ K ∗ ),
a problem we have already studied and solved. The proof follows by duality since observability
of a system is equivalent to reachability of the dual system. Then, the problem is solvable with
arbitrary Λo if and only if the dual system is reachable, hence, if and only the primal is observable.
In the presence of unobservable modes, these correspond to unreachable modes in the dual, and the
last part of the proof follows. □
The analogy between state feedback design and Luenberger observer design, due the duality,
allows us to use the same algorithms to derive the matrices K and L. In fact, if we have any
procedure that, given the input data A, B, Λc , returns matrix K of a state feedback controller, we
can use the same procedure for observer design: by providing the input A⊤ , C ⊤ , Λo , we obtain
matrix K ∗ (the state feedback matrix for the dual system) and finally L = (−K ∗ )⊤ .

• State feedback gain: Procedure[A, B, Λc ] → K

• Observer gain: Procedure[A⊤ , C ⊤ , Λo ] → K ∗ = −L⊤

Cm

Figure 6.9: Positioning system.

Example 6.4. Consider a servo system as in Figure 6.9. Assuming a reference angle θ = 0, let us
solve the problem of bringing the system to the zero state θ = θ̇ = 0. The equation describing this
system is
Cm (t)
J θ̈(t) = Cm (t) ⇒ θ̈(t) = = u(t)
J
If we set x1 (t) = θ(t) and x2 = θ̇(t), we obtain a state representation with matrices
" # " #
0 1 0
A= , B=
0 0 1

Suppose we have sensors for both the position and the speed. We can implement a state feedback
h i " x1 (t) #
u(t) = k1 k2
x2 (t)
6.4. Synthesis of regulators 89

and we get the following closed-loop matrix

" #
0 1
(A + BK) =
k1 k2

which is already in Frobenius form. Then, we just have to require that −k1 and −k2 are equal to the
coefficients of the desired characteristic polynomial. If we choose p(s) = (s + 1)(s + 2) = s2 + 3s + 2,
we immediately get k1 = −2 and k2 = −3.
Employing both position and speed sensors is not convenient from an economic point of view.
We can then expect to have a single sensor, for just one of the state variables, along with an observer
estimating the variable that is not measured.
If we adopt a speed sensor, matrix C is
h i
C= 0 1

By means of a simple calculation, we see that the system is not observable (as is known, from
the speed it is not possible to determine the position). In addition, the eigenvalue of the non-
observable subsystem is zero, hence it is not asymptotically stable and the spectrum of (A − LC)
must necessarily contain the zero eigenvalue. This is not good news, because the error system
would not be asymptotically stable.
If, instead, we consider the position sensor, matrix C takes the form
h i
C= 1 0

and in this case the observability matrix has full rank. We can then compute (A − LC):
" # " # i " −l1 1 #
0 1 l1 h
(A − LC) = − 1 0 =
0 0 l2 −l2 0

The characteristic polynomial is:

s + l1 −1
" #
det(sI − A + LC) = det = s 2 + l1 s + l2
l2 s

Suppose that the desired eigenvalues are λ̄3 = −3 and λ̄4 = −4. We have p(s) = s2 + 7s + 12, hence
l1 = 7 and l2 = 12.

Remark 6.1. The observer generates an estimate of vector x̂(t) whose components are the estimated
position (which we already have as an output, thanks to the sensor) and the estimated speed. Hence,
we could avoid estimating the position, since it is already measured, and estimate the speed only
by means of a reduced order observer, which is not treated in the present course for two reasons.
First, in view of the current technology, the complexity of the full order observer is not a problem in
terms of implementation. Second, using the estimated position instead of the measured one has the
advantage that the observer has a filtering (noise-suppressing) action.

The further step for the control of the servo system is to feed the estimated state back:
h i " x̂1 (t) #
u(t) = k1 k2
x̂2 (t)

Intuitively, this idea should work because, after all, the estimated state converges to the true one.
We will prove that this scheme is successful in the following.
90 Chapter 6. Control Systems

u x x y
B C

A
L
x x
B C

Figure 6.10: Block diagram of an observer-based controller.

6.4 Synthesis of regulators

After designing state feedback and state estimator (the latter is necessary if some state variables
are not measured, that is, C is not the identity matrix), the next step is the overall (observer-based)
controller synthesis. This is based on the feedback of the state estimate, as shown in Figure 6.10.
It is intuitive enough that everything is working properly. However, to verify the effectiveness
of this strategy, we analyse the obtained overall system. The process equations are:
ẋ(t) = Ax(t) + Bu(t)
(

y(t) = Cx(t)
while those of the observer and the controller are
˙ = (A − LC) x̂(t) + Bu(t) + Ly(t)
(
x̂(t)
u(t) = K x̂(t)
The overall system has n + n = 2n state variables (those of the process, together with those we
introduce with the observer-based controller).
" #
x(t)
xoverall (t) = ∈ R2n (6.20)
x̂(t)
The closed-loop matrix is " #
A BK
ACL =
LC A − LC + BK
To carry out the computations, a state transformation is useful to obtain a state vector that includes
both process state and estimated error. Mathematically, we have:
" # " #" #
x(t) I 0 x(t)
= (6.21)
e(t) −I I x̂(t)
The transformation matrix from the new to the old representation is then:
" # " #
I 0 I 0
T= T =
−1
(6.22)
I I −I I
(T is invertible). The overall system is governed by the following equations:
ẋ(t) = Ax(t) + Bu(t)



 y(t) = Cx(t)




˙ = (A − LC) x̂(t) + Bu(t) + Ly(t) (6.23)
x̂(t)




 u(t) = K x̂(t)


6.4. Synthesis of regulators 91

Applying the state transformation (i.e., manipulating the equations, with x̂(t) = x(t) + e(t)) we get

ẋ(t) = (A + BK)x(t) + BKe(t)

(
(6.24)
ė(t) = (A − LC)e(t)

In matrix form, we have

A + BK
" # " #" #
d x(t) BK x(t)
= (6.25)
dt e(t) 0 A − LC e(t)
| {z }
=Acl

Matrix ACL (CL means Closed Loop) is block triangular of size 2n × 2n. Therefore, its spectrum is
equal to the union of the spectra of its diagonal blocks:
[
σ(ACL ) = σ(A + BK) σ(A − LC) (6.26)

Under reachability and observability assumptions, these spectra can be arbitrarily assigned.
Hence, we can assign all of the eigenvalues of ACL , as follows:

1. design a state feedback that assigns the eigenvalues σ(A + BK) = Λc ;

2. design an observer for state estimation that assigns the eigenvalues σ(A − LC) = Λo ;

3. design the overall controller as the feedback of the estimated state

˙
x̂(t) = (A − LC) x̂(t) + Bu(t) + Ly(t)
u(t) = K x̂(t)

Observation 6.2. The assignment of the eigenvalues of the state feedback and of the observer is
performed in two independent stages. This independence is called separation principle.

The procedure leaves us complete freedom on how to choose the eigenvalues in Λc and Λr .
As a guideline, the convergence of the observer must often be faster than the convergence of the
system modes. Therefore, the negative real part of the eigenvalues of (A − LC) is usually required
to be larger (in magnitude) than the real part the eigenvalues of (A + BK) (typically 4 times larger).
The observer-based controller is thus governed by the equations:

˙ = (A − LC) x̂(t) + Bu(t) + Ly(t)

(
x̂(t)
u(t) = K x̂(t)

Replacing the second equation in the first yields a useful mathematical simplification that is very
important from a physical standpoint:

˙ = (A − LC + BK) x̂(t) + Ly(t)

(
x̂(t)
(6.27)
u(t) = K x̂(t)

In fact, we notice that we do not need to have sensors capable of measuring the input u(t), in order
to send it to the observer, because the input itself is generated as an output of the regulator system.
The regulator matrices are

(F, G, H, K) = (A − LC + BK, L, K, 0)

hence, the regulator is strictly proper.

92 Chapter 6. Control Systems

The controller thus obtained can be implemented in practice by means of an analog circuit, or,
much more realistically, can be implemented digitally on a computer, by exploiting the equivalent
discrete-time system having matrices

F D = eFT = e(A−LC+BK)T
Z T Z T
GD = eFσ Ldσ = e(A−LC+BK)σ Ldσ
0 0
HD = K

For a computer implementation, we can adopt an algorithm of the following type.

1. Fix T and compute F D , G D , HD .

2. Initialise x̂(0).

3. If ONLINE = ON, start.

4. Compute u := HD x̂.

5. Write the value of u in a memory location that will be read from the D/A converter.

6. Read the value of y written in a memory location by the A/D converter.

7. Compute the updated estimated state x̂+ = F D x̂ + G D y.

8. Set x̂ = x̂+ .

9. Return to step 4 unless ONLINE = OFF.

10. Quit.
A final consideration concerns the effect of the observer. If we cannot completely assign
the eigenvalues via state feedback, we need an observer. Hence, we need to assign additional
modes: these affect the system with their transient (which is typically fast). To analyse the problem,
consider an input of the form
u(t) = K x̂(t) + v(t), (6.28)
where v(t) is a reference signal (often constant). It can be seen that the transfer function between
v(t) and y(t) is the same in both the state feedback case and the estimated state feedback case,
because the error e is not a reachable variable (this is immediate, since e evolves as ė = (A − LC)e).
Example 6.5. Reconsider the previous example, where we have obtained the parameters of the
matrices K and L for the servo-system. The controller implementation is immediate:
" # " #h i " 7 #h
0 1 0 i
F = A + BK − LC = + −2 −3 − 1 0 =
0 0 1 12
" #
−7 1
=
−14 −3
" #
7
G = L=
12
h i
H = K = −2 −3

Then, the resulting modes of the overall feedback system correspond to

a) the eigenvalues λ1 = −1, λ2 = −2: modes of the regulator;
b) the eigenvalues λ3 = −3, λ4 = −4: modes of the observer.
6.5. External disturbances 93

A final remark concerns the discrete-time systems case. The discrete-time observer-based
control writes as
u(k) = K x(k)
x̂(k + 1) = (A − LC) x̂(k) + Bu(k) + Ly(k) (6.29)
As in the continuous-time system case, the error is governed by the equation:
e(k + 1) = (A − LC)e(k) (6.30)
Once again, the theory for discrete-time systems is parallel to that for continuous-time systems.
The only caution is that the eigenvalues must be assigned with the condition |λ̄k | < 1 so as to ensure
stability of the modes.

6.5 External disturbances

Any system is affected by disturbances. A noisy system can be described by equations of the form
ẋ(t) = Ax(t) + Bu(t) + Ed(t) (6.31)
y(t) = Cx(t) + w(t) (6.32)
where we have two external inputs (exogenous disturbances)
• d(t), called process noise;
• w(t), called measurement noise.
Process noise is typically an external signal affecting the system evolution, such as turbulence
in aeronautics, voltage fluctuations in electrical circuits, external torque in mechanical systems.
Measurement noise is due to the sensors (quantisation or electric noise). In this case, the error
equation becomes
ė(t) = (A − LC)e(t) − Ed(t) + Lw(t) (6.33)
If we have a bound for the noises
∥w(t)∥ < β, ∥d(t)∥ < δ
in general we still can achieve an error bound of the type
∥e(t)∥ < γ
provided that A − LC is asymptotically stable.
The overall controlled system is
" # " #" # " #" #
d x(t) A BK x(t) E 0 d(t)
= + (6.34)
dt z(t) LC A − LC z(t) −0 L w(t)
or, if we use e instead of z
A + BK
" # " #" # " #" #
d x(t) BK x(t) E 0 d(t)
= + (6.35)
dt e(t) 0 A − LC z(t) −E L w(t)
It interesting to notice that there is a trade-off. If we attempt to assign the eigenvalues of A + BK
with very large negative parts to have a fast convergence, this requires a large K and results in a
strong actuator exploitation, because u = K x. If we attempt to assign the eigenvalues of A − LC
with very large negative part, this requires a large L and we amplify the effect of measurement
noise.
94 Chapter 6. Control Systems

6.6 Examples
Example 6.6. (Two tank system.) As we will see in the examples chapter, the linearised equation
of the two-tank system has matrices
" # " #
−p p −1
A= B=
p −(p + δ) 0

The system poles are −2p ± 4p2 − 4pδ. Since δ is positive and small, one (negative) root is quite
p

close to 0.
To speed up the system, we can assign the eigenvalues: in particular, we can move the “slow”
pole far from the imaginary axis (to the left).
s + p − k1
" #
−p − k2
det[sI − A + BK] = det = s2 + (+2p − δk1 )s − (p + δ)k1 − pk2 + δp
−p s + (p + δ)
If the polynomial p̄(s) = s2 + p̄1 s + p̄0 is to be assigned, we get the equations
(+2p − δk1 ) = p̄1
−(p + δ)k1 − pk2 + δp = p̄0
which the reader is invited to solve.
Example 6.7. (Electrical machine control.) Consider the following model of a direct-current
electrical machine
  x1 (t)   ϵ 
      
 x1 (t)   −α −β 0
 x2 (t)  =  γ −δ 0   x2 (t)  +  0  u(t)
       
x3 (t) 0 1 0 x3 (t) 0
where x1 (t) = ia , x2 (t) = ω and x3 (t) = φ are the armature current, the angular speed and the
angle, respectively, and where the input u(t) = va (t) is the armature voltage. We assume that
y(t) = x3 (t) = φ is the measured output, while α, β, γ, δ and ϵ are positive constants. Observability
and reachability have been tested for this system. To assign the eigenvalues, we write the matrix
 s + α − ϵk1 β − ϵk2 −ϵk3 
 
[sI − (A + BK)] =  −γ s+δ 0 
 

−1

0 s
Its determinant is
s3 + [α − ϵk1 + δ]s2 + [αδ − δϵk1 + γβ − γϵk2 ]s + ϵk3 γ
Given the desired characteristic polynomial for the state feedback s3 + ā2 s2 + ā1 s + ā0 , the solution
is found by equating term by term
α − ϵk1 + δ = ā2 , αδ − δϵk1 + γβ − γϵk2 = ā1 , ϵk3 γ = ā0
therefore k1 = (α + δ − ā2 )ϵ, k3 = (ā0 /γ)ϵ and k2 = (αδ − δ(α + δ − ā2 ) + γβ − ā1 )/(ϵγ).
For the state observer we proceed in the dual way and we consider the determinant of matrix
 s + α β
 
l1 
[sI − (A − LC)] =  −γ s + δ l2 
 
−1 s + l3
 
0
which we equate to the desired observer characteristic polynomial s3 + â2 s2 + â1 s + â0 . The reader
is invited to complete the exercise. The resulting regulator is
ż(t) = [A + Bk − LC]z + Ly, u(t) = Kz(t),
where z(t) is a state estimate.
6.6. Examples 95

Example 6.8. (Unknown constant signal.) As we have seen, the presence of an external signal or
disturbance may prevent the estimation error from converging to zero. This is true in the absence of
information on the disturbance. An interesting case is that of constant disturbances.
Assume that we have the system

ẋ(t) = Ax(t) + Bu(t) + Ed, y(t) = Cx(t) (6.36)

where d is unknown but constant. This allows us to write an additional equation, ḋ = 0. We get
" # " #" # " # i " x(t) #
x(t) A E x(t) B h
= + u(t) y(t) = C 0
d(t) 0 0 d(t) 0 d(t)

If the augmented system is observable, we can reconstruct the constant signal by means of an
observer.8 Consider the servo system subject to a constant external torque (e.g., a lift with an
unknown load)
" # " # " #
0 1 0 0 h i
A= , B= , E= , C= 1 0
0 0 1 1

Then the augmented system is

   
 0 1 0   0  h i
Aaug =  0 0 1  , Baug =  1  , Caug = 1 0 0
   
   
0 0 0 0

This system is observable. It is essential to notice that the augmented system is not reachable with
input B. In fact, by assumption, the external torque is constant and cannot be modified. Still, we
can design an observer that estimates position, speed and torque. Then we can use state feedback
to control the system position and speed.

8
If (A, E) is reachable and (A, C) is observable, then the augmented system is observable. The proof is not easy: it
can be done via Popov criterion.
Chapter 7

Nonlinear systems

7.1 General considerations

Most of the systems encountered in nature are nonlinear. Nonlinear systems are much harder to
study than linear systems. Indeed, in the case of nonlinear systems, it is in general impossible
to find analytic solutions. This is a limitation of system theory. However, systems that cannot
be studied analytically can be approached numerically, to obtain an approximated solution of the
corresponding differential equations. A powerful tool, which is fundamental in the analysis of
nonlinear systems and very important for control purposes, is linearisation. In this chapter, we will
describe this approach in detail.
From a conceptual standpoint, a substantial difference between linear and nonlinear system is
behavioural homogeneity. For linear systems, in fact, it is possible to consider modal functions
and frequency analysis, and properties such as stability, oscillations and exponential decay are a
property of a system. For nonlinear system, instead, every property is typically referred to a certain
equilibrium point (or, more in general, to a certain working condition, such as a periodic orbit). In
other words, a nonlinear system can be stable or not, sensitive to noise or not, oscillating or not
(and so on), depending on the equilibrium we are considering. Therefore, the global theory that
holds for linear systems cannot, in general, be extended and we most often need to resort to local
theories.

Example 7.1. (Population evolution.) Population dynamics are interesting to analyse, for instance,
when studying colonies of bacteria in a laboratory or animal species in a natural environment.
These phenomena can be modeled by more or less complicated equations. We consider a very
simple model, which describes the evolution of a single population x(t). In the simplest case, a
population evolves according to the following linear equation: ẋ(t) = ax(t), where a > 0 is a scalar
coefficient. This represents the fact that the population grows proportionally to the population itself.
The solution is then a classic exponential x(t) = eat x(0). However, this model does not take into
account the limited resources of the surrounding environment. A well accepted model that includes
the dependence on the available resources is represented by the following nonlinear system:
" #
x(t)
ẋ(t) = a 1 − x(t)
C

x(t)
Note that, for x(t) ≪ C, the term is negligible and the model is almost linear. On the other
C
hand, if x(t) increases, the same term represents a limitation for the birth rate, due to the shortage
of resources. Even more, if x(t) > C, we have a trend inversion, and the number of births is less
than the number of deaths.

97
98 Chapter 7. Nonlinear systems

This is one of the simple and rare cases in which is possible to give an analytical solution,
which is the following:
eat x(0)
x(t) =   x(0)
1 − x(0)
C + C e
at

For general nonlinear systems, there are few chances of finding analytic solutions like this one.
The evolution of the population with different initial conditions is reported in Figure 7.1.

Figure 7.1: Evolution of a population for different initial conditions.

To present the spirit of the investigation of nonlinear systems, we show that the same qualitative
considerations that we can derive from the closed-form solution can be drawn without solving the
equation, but just analysing it. Indeed, from the equation we immediately see that

• for x(t) > C, ẋ(t) < 0;

• for x(t) < C, ẋ(t) > 0;

• both x(t) = C and x(t) = 0 yield ẋ(t) = 0.

Then, there are two equilibria characterised by ẋ = 0. Above the upper equilibrium the population
decreases, between the two equilibria the population increases, in 0 the population remains at 0.
The solution x(t) = C is a stable equilibrium point, because all the solutions with initial condition
grater than zero converge to x(t) = C. Vice versa, x(t) = 0 is an unstable equilibrium point, because
any small perturbation brings the solution far from zero. Actually, what we can see is that all the
conditions that we obtained from the analytical solution can be deduced based on a qualitative
analysis as well. Such a qualitative analysis is the main subject of this chapter.

We will consider autonomous continuous-time systems of the form

ẋ(t) = f (x(t), u(t))

(
(7.1)
y(t) = g(x(t), u(t))

and autonomous discrete-time systems of the form

x(k + 1) = f (x(k), u(k))

(
(7.2)
y(k + 1) = g(x(k), u(k))

With few exceptions, we neglect the case where there is a direct time dependence of ẋ(t) =
f (x(t), u(t), t) or x(k + 1) = f (x(k), u(k, k)), on time (non-autonomous systems), since it is much
more complicated and less important in practice.
7.2. Equilibrium points 99

7.2 Equilibrium points

Among all possible trajectories of a system, constant trajectories are of particular interest. These
are called equilibria. Almost all dynamical systems (mechanical electrical, thermal, aeronautical)
we encounter in practice operate at an equilibrium most of the time.

Definition 7.1. Given a vector x̄ ∈ Rn , we say that it is an equilibrium point for the system (either
(7.1) or (7.2)) if there exists ū, called equilibrium input, such that, for x(0) = x̄ and u(t) = ū, we
have that x(t) ≡ x̄. In this case, the pair ( x̄, ū) ∈ Rn × Rm is called equilibrium pair.

The equilibrium conditions in a nonlinear system are easily obtained. Indeed, we have that, if
x(t) ≡ x̄ and u(t) = ū, then it is also true that ẋ = 0 for continuous-time systems, and x(k + 1) = x(k)
for discrete-time systems. Then, the equilibrium conditions are

f ( x̄, ū) = 0 (7.3)

the continuous-time systems and

x̄ = f ( x̄, ū) (7.4)
for discrete-time systems. The equilibrium pairs are therefore the vector pairs that satisfy one of
the above conditions (depending on the continuous- or discrete-time nature of the system).

x
B

A
t

Figure 7.2: Evolution of a population with external input. Line A: extinction threshold; line B:
survival threshold.

Example 7.2. (Population evolution, continued.) Consider again the equation for population
evolution in the previous example. We can easily find out that the equilibrium points are
 x̄ 
1− x̄ = 0 ⇒ x̄ = 0, x̄ = C
C
Is is interesting to observe what happens if we add to the equation a term that represents an external
input (for instance hunting or predation):
 x̄ 
1− x̄ + ū = 0
C
where ū < 0. In this case, the equilibrium points are the solutions of the equation above for a given
ū:  r 
C  a a 2 a 
x̄1, 2 =  ± + ū
a 2 2 C
100 Chapter 7. Nonlinear systems

The new situation is depicted in Figure 7.2.

We can observe that, for initial conditions greater than x̄2 , the death rate is larger than the
birth rate, due to resource shortage. In the range between x̄1 and x̄2 , repopulation occurs, while
for initial conditions below x̄1 we have the species extinction. The extinction threshold x̄1 has
been introduced by the presence of a negative external input. Moreover, ū can have a different
nature: if it is related to predation, then it is for sure influenced by x(t). This typically requires
the introduction of a new variable representing the predator population (prey-predator models).
Conversely, if ū is related to hunting, it will be an independent term.
Example 7.3. (Euler discrete-time system.) The Euler approximating system of a continuous-time
system is obtained by approximating the derivative ẋ(t) as an incremental ratio:
x(t + τ) − x(t)
ẋ(t) ≃ (7.5)
τ
By replacing this expression in the general formula of the continuous-time system and evaluating it
only for t = 0, τ, 2τ, . . ., we obtain the discrete-time system:

x(t + τ) = x(t) + τ f (x(t), u(t)) (7.6)

This method, known as Euler explicit method, represents a good approximation if the step τ is small
enough, while, if τ is too large, the discrete-time solution may not converge to the continuous-time
solution.
To assess the effectiveness of this method, we can check if the equilibrium conditions of the
Euler system correspond to those of the original continuous-time system. Indeed, it is very easy
to check that the equilibrium points of the approximating discrete-time system are the same of the
continuous-time system:
x̄ = x̄ + τ f ( x̄, ū) ⇔ f ( x̄, ū) = 0
Obviously, nothing can be said about the stability. In fact, stable equilibrium points of the
continuous-time system could be unstable for the discrete-time approximation (in particular if τ > 0
is not small enough).
Additional examples about equilibrium points will be presented in further sections.
Note that, for continuous-time linear systems, the equilibrium condition is

0 = A x̄ + Bū,

namely, the equilibrium pairs are the kernel of matrix [A B]. The reader is invited to find out the
equilibrium condition for discrete-time linear systems.

7.3 Lyapunov functions

A nonlinear system can have several equilibrium points, as said. We are interested in analysing
their local stability, namely, in understanding what happens for small perturbations around the
equilibrium point: does the system return to the equilibrium or does the trajectory escape? We stress
(this concept will be repeated) that for nonlinear system stability is referred to an equilibrium of the
system, and not to the system itself. Unless we consider very special cases, a nonlinear system can
have both stable and unstable equilibria. Some examples of stable and unstable equilibrium points
of the same “rolling sphere” system are shown in Figure 7.3.
Consider a nonlinear system admitting an equilibrium point characterised by the condition

0 = f ( x̄, ū) (7.7)

7.3. Lyapunov functions 101

UNSTABLE

STABLE

Figure 7.3: Stable and unstable equilibrium points for a rolling sphere.

We initially assume that ū is fixed, so we can study the system:

ẋ(t) = f (x(t), ū) = Fū (x(t)) (7.8)

which is an autonomous system without inputs, with equilibrium point in x̄. We can apply a
coordinate shift: we introduce then the new variable z(t) = x(t) − x̄ and we get

ż(t) = Fū (x(t)) = Fū (z(t) + x̄) (7.9)

With this new definition, the equilibrium point is z̄ = 0. We define G x̄, ū (z(t)) = Fū (z(t) + x̄).
Therefore can study the following system:

ż(t) = G x̄, ū (z(t)) (7.10)

with equilibrium point in 0 = G x̄, ū (0). Thanks to a suitable change of variables (as done above), we
can always bring us back to the study of autonomous systems without inputs, of the form

ẋ(t) = f (x(t)), f (0) = 0.

In the sequel we will be also concerned with the stabilisation problem: we will use a control that
stabilises the equilibrium (if necessary). In this case, a change of variable is applied to the input as
well:
v(t) = u(t) − ū

Hence,
ẋ(t) = ż(t) = f (z(t) + x̄, v + ū) = G x̄, ū (z(t), v(t))

and
ż(t) = G x̄, ū (z(t), v(t)) (7.11)

is the equation we use for stabilisation. Again, the stabilisation problem can be approached by
considering a system for which (0, 0) is an equilibrium pair

ẋ(t) = f (x(t), u(t)), f (0, 0) = 0

To recap we have

• stability analysis: fix u = ū and check local stability;

• stabilisability analysis: use v (generated by an algorithm on-line) to stabilise the system.

102 Chapter 7. Nonlinear systems

7.3.1 Lyapunov stability theorem

Let us consider the stability analysis problem first. To this aim, we consider the system
ẋ(t) = f (x(t)) (7.12)
with equilibrium x̄ = 0 and f continuous.
Definition 7.2. System (7.12) is stable in the equilibrium point (I.E.P.) x̄ = 0 if, given ε > 0,
∃ δ > 0 such that, if ∥x(0)∥ ≤ δ ⇒ ∥x(t)∥ ≤ ε.
System (7.12) is asymptotically stable in the equilibrium point (I.E.P.) x̄ = 0 if is stable I.E.P. and,
furthermore, for ∥x(0)∥ ≤ δ, we have that ∥x(t)∥ → 0 for t → +∞.
The study of the stability of the equilibrium points of a nonlinear system is mainly based on the
theory introduced by Aleksandr Mikhailovich Lyapunov at the beginning of XX century. Lyapunov
functions generalise the concept of “energy of a system”. We know (without resorting to equations)
that dissipative systems encountered in physics, in the body, in circuit theory, and so on, without
an external energy supply, reach an equilibrium where the system energy has a minimum. If we
generalise this concept, we can have an efficient general tool for assessing stability of nonlinear
systems.
Consider a continuously differentiable function V (V ∈ C1 ) defined on the state space, V :
n
R → R. Assume that V is a positive definite function in a neighborhood W of the origin, i.e.,
• V(0) = 0;
• V(x) > 0 for all x ∈ W, x , 0.
Observe that all the sub-level sets of this function, {x : V(x) ≤ k}, for k > 0, include the origin (see
Figure 7.4).

V=1 x(0)
V=2
V=4

Figure 7.4: Level curves of a Lyapunov function.

The concept introduced by Lyapunov, which we will shortly formalise with a theorem, is based
on an observation: if a trajectory x(t) of the system, with x(0) sufficiently small, is such that V(x(t))
is a non increasing function, then x(t) is forced to stay inside a sub-level set. And, if V(x(t)) is
decreasing, then the trajectory must converge to 0.
The fundamental problem is that, in most of the cases, the trajectory x(t) of a nonlinear system
is unknown, so in principle we are not able to tell whether V(x(t)) is a non increasing (or decreasing)
function. However, to avoid this obstacle, we can consider the derivative of the function:
d
V(x(t)) = V̇(x(t)) = ∇V(x) ẋ(t) = ∇V(x) f (x(t)), (7.13)
dt
7.3. Lyapunov functions 103

h ∂V ∂V ∂V
∂x2 . . . ∂xn . We notice that, if at a certain instant t the system has state
i
where ∇V(x) = ∂x 1
x(t) = x̃, then the derivative is easily computable

d
V(x(t)) = V̇(x(t)) = ∇V( x̃) f ( x̃) (7.14)
dt x(t)= x̃ x(t)= x̃

Hence it is no longer necessary to know the full trajectory x(t) in order to evaluate the derivative
of V. This derivative, also known as Lyapunov derivative, is useful to establish the stability of
a system: if V̇(x) ≤ 0, then V(x) is a non increasing function, while if V̇(x) < 0, then V(x) is a
decreasing function.

Remark 7.1. The Lyapunov derivative V̇(x) = ∇V(x) f (x) is the derivative of V(x(t)) if x(t) = x
along the system trajectory: it depends on both V and the system equation ẋ = f (x).

We can then formalise these ideas in the following theorem.

Theorem 7.1. Given a neighborhood W of the origin, assume that a function V ∈ C1 , V : Rn → R

is defined and such that V(0) = 0 and V(x) > 0 for x ∈ W\{0}. Then

• if V̇(x) ≤ 0 for x ∈ W, then the system is stable I.E.P. x̄ = 0 ;

• if furthermore V̇(x) < 0 for x ∈ W\{0}, then the system is asymptotically stable I.E.P. x̄ = 0.

Proof. We prove the first part of the theorem only: if V̇(x) ≤ 0, then the system is stable I.E.P.
in zero. Let us take the sphere S ε = {x : ∥x∥ ≤ ε} in the state space. It is not restrictive to choose
S ε ⊂ W. Let m be the minimum value of V on the external surface of the sphere, ∥x∥ = ε,

m  min V(x).
x∈∂S ε

Now we take a new sphere S δ with radius δ < ε. Define M as the maximum of V inside S δ

M  max V(x),
x∈S δ

We can take δ small enough to assure that (by continuity of V, since V(0) = 0)

M < m.

By contradiction, assume that the system is not stable and that, for some initial condition inside
S δ , there is a trajectory that escapes from the sphere S ε . Then there exists a time instant t′ such that
x(t′ ) ∈ ∂S ε . Let t1 be the first positive instant in which x(t) hits the boundary of S ε

t1 = min{t ≥ 0 : x(t1 ) ∈ ∂S ε } (7.15)

Between the instants t = 0 and the instant t = t1 , the trajectory is contained in S ε , and therefore
also in W. Then we have that V̇(x(t)) ≤ 0 for all t ∈ [0, t1 ]. On the other hand, by construction and
by the definitions of m and M, we have

V(x(t1 )) ≥ m > M ≥ V(x(0)), (7.16)

hence V(x(t1 )) > V(x(0)), in contradiction with the fact that V̇(x(t)) ≤ 0 in the interval.
The second part of the problem, which concerns asymptotic stability, will not be proved: the
interested reader is referred to specialised books. However, the proof can be sketched as follows.
First we notice that V̇ = ∇V(x) f (x) is continuous, since f is continuous. If x(0) ∈ {x : V(x) ≤ κ} ⊂
W, namely, if we start inside a sphere that is contained in the κ sub-level set, then V(x(t)) ≤ κ for
104 Chapter 7. Nonlinear systems

all t > 0, because V̇ ≤ 0. For the same reason, if at some time t1 the trajectory reaches a smaller
sub-level set, x(t1 ) ∈ {x : V(x) ≤ κ1 }, κ1 < κ, then it will be trapped inside this new sub-level
set after t1 : V(x(t)) ≤ κ1 for all t > t1 . The proof can be carried out by showing that, no matter
how small κ1 is taken, the set {x : V(x) ≤ κ1 } will be ultimately reached. This can be proved
by contradiction. If the set is not reached, then the solution has to stay in the intermediate set
x(t) ∈ {x : κ1 ≤ V(x) ≤ κ}, namely,
κ1 ≤ V(x(t)) ≤ κ
for all t ≥ 0. In this closed set, V̇ has a maximum (−µ) that is strictly negative. Then

V̇(x(t)) ≤ −µ.

By integration Z t Z t
V(x(t)) − V(x(0)) ≤ V̇(x(σ))dσ ≤ −µdσ = −µt,
0 0
but this means that V(x(t)) becomes negative for t > V(x(0))/µ, which is not possible. □
Example 7.4. Consider the system

ẋ1 = −2x1 + x22 + x14

(

ẋ2 = 3x13 − 2x2 + x12

and take V(x1 , x2 ) = x12 + x22 . The Lyapunov derivative is:

h i " −2x1 + x2 + x4 #
V̇(x) = ∇V f (x) = 2x1 2x2 2 1 =
3x13 − 2x2 + x12
= −4x12 − 4x22 + 2x1 x22 + 2x15 + 6x13 x2 + 2x12 x2 ≈ −(x12 + 4x22 )

In fact, in a sufficiently small neighborhood of the equilibrium point (0, 0), the quadratic terms
dominate over the others. Therefore, the function V̇(x1 , x2 ) is negative semi-definite and this proves
the asymptotic stability of the system (I.E.P.).
Example 7.5. (The pendulum.) Given the pendulum shown in Figure 7.5, we want to study stability
of its equilibrium points. The system is described by the equation
2
lm ϑ̈(t) = −lm g sin ϑ(t)

Denoting by x1 (t) = ϑ(t) and x2 (t) = ϑ̇(t), and assuming all constants equal to 1, the system is:

ẋ1 (t) = x2 (t)

(

ẋ2 (t) = − sin x1 (t)

The natural candidate Lyapunov function, from physical considerations, is mechanical energy,
equal to the sum of kinetic energy and potential energy:
1 2
V(x1 , x2 ) = x + 1 − cos x1
2 2
This function is positive definite in a neighborhood of the equilibrium, hence is a candidate
Lyapunov function. We obtain:
h i" x2
#
V̇(x1 , x2 ) = sin x1 x2 = x2 sin x1 − x2 sin x1 = 0
− sin x1

Hence, V̇(x1 , x2 ) is negative semi-definite, and this guarantees the system stability I.E.P.
7.3. Lyapunov functions 105

Mg
Figure 7.5: Pendulum (basic).

Example 7.6. Two-tank system Let us consider the system with two water tanks, whose equations
will be presented in Chapter 9.8.
√
ḣ1 = − αS
1
√ h1 − h2 +1u √
(
ḣ2 = αS h1 − h2 − βS h2
1

The system admits a generic equilibrium point in (h̄1 , h̄2 ). By translating the system with x1 = h1 −h̄1
and x2 = h2 − h̄2 ,1 we have:
 p
 ẋ1 = − αS

 1
p x1 − x2 + h̄1 − h̄2 + u p
 ẋ2 =

 1
x1 − x2 + h̄1 − h̄2 − βS1 x2 + h̄2
αS

Consider the function V(x1 , x2 ) = x12 + x22 , V̇(x1 , x2 ). Then, with some manipulation we get
!
1
q
V̇(x1 , x2 ) = x1 x1 (t) + h̄1 − x2 (t) − h̄2 − q̄
αS
!
1 1
q q
+ x2 − x1 (t) + h̄1 − x2 (t) − h̄2 + x2 (t) + h̄2 ± x2 q̄
αS βS
! !
1 1
q q
= (x1 − x2 ) x1 (t) + h̄1 − x2 (t) − h̄2 − q̄ − x2 x2 (t) + h̄2 − q̄
αS βS
p p
If we consider the equilibrium conditions, αS 1
+h̄1 − h̄2 − q̄ = 0 and βS1 h̄2 − q̄ = 0, we see that
the sign of the terms in round brackets is opposite to that of x1 − x2 and x2 , hence the products are
negative. Then, V̇(x1 , x2 ) < 0 (for x1 , x2 ) , (0, 0). This proves asymptotic stability I.E.P.
1
The equilibrium conditions are derived in Chapter 9.8.
106 Chapter 7. Nonlinear systems

It is important to notice that a wrong choice of the function V(x) does not lead us to any
conclusion. If the Lyapunov derivative is possibly greater than zero in the neighborhood W, we
cannot say anything about the system stability in the equilibrium point.
Example 7.7. Consider system ẋ = Ax, where
−1 α
" #
A=
−1 −1

with α ≥ 0. This system is asymptotically stable (since det(sI − A) = s2 + 2s + 1 + α). If we take

V(x) = x12 + x22 , this is a Lyapunov function only for certain values of α. For example, it can be
easily seen that, for α = 1, the conditions of Lyapunov theorem are satisfied, while for large α > 0
they might be not satisfied. Other examples are
" # " #
0 1 0 1
A1 = , A2 =
−5 −2 +5 −2
The linear system associated with the first matrix is asymptotically stable, the one associated with
the second is unstable. In both cases, if V(x) = x12 + x22 , then V̇ assumes both positive and negative
values. Hence, if we take a positive definite V whose Lyapunov derivative V̇ is sign indefinite, the
only possible information we can get is: V is not a good candidate.
Then, the fundamental problem is the choice of a Lyapunov function that can prove stability
of the system. Such a function must have particular properties: its level curves need to fit the
trajectories of the system. There are inverse theorems that prove, under certain hypothesis, the
existence of a Lyapunov function for asymptotically stable systems. Those results are theoretical
and, in general, they are not of any help to actually determine V(x).2
Example 7.8. (Dissipative pendulum.) Let us consider again the pendulum equations, to which we
add a term due to the friction and proportional to the speed:
ẋ1 (t) = x2 (t)
(

ẋ2 (t) = − sin x1 (t) − αx2 (t)

The system is dissipative, so we might think that the energy, taken as a Lyapunov function, will
prove the asymptotic stability of the system. Unfortunately, we obtain:
h i" x2
#
V̇(x1 , x2 ) = sin x1 x2 = −αx22
− sin x1 − αx2
This function is a negative semi-definite function: it is null in the origin and in the x1 -axis, x2 = 0,
and negative otherwise. This means that the energy variation when the velocity is null is zero,
because there is no dissipation in the maximum elongation point. So, we can confirm stability of
the system, but not asymptotic stability I.E.P., because V̇(x) is not negative definite.
There exists another criterion to ensure asymptotic stability of a system: the Krasowskii
criterion.
Theorem 7.2. We consider the same assumptions of the (weak) Lyapunov theorem: ∃ V ∈ C1 , V :
Rn → R with V(x) > 0 ∀ x ∈ W\{0} and V(0) = 0, with V̇(x) ≤ 0 ∀ x ∈ W. Consider now the set
N = {x , 0 : V̇(x) = 0}. If there are no trajectories entirely contained in N ∩ W (in other words,
x(t) < N ∩ W ∀ t ≥ 0) then the system is asymptotically stable.
The criterion shows that, if the system does not stop in the region where there is no dissipation,
sooner or later it will reach an equilibrium point. This criterion can be applied to the pendulum
example.
7.3. Lyapunov functions 107

x2

N
x1

Figure 7.6: Dissipative pendulum: subspaces N and W and vector ẋ(t).

Example 7.9. Let us re-examine the dissipative pendulum and consider Figure 7.6.
The figure shows the neighbourhood W of the origin (a circle), the set N (the set of points with
x2 = 0 and x1 , 0) and a trajectory of vector ẋ(t) starting from an initial condition in N. For any
point in N, such that x2 = 0 and x1 , 0, the derivative is
" # " #
ẋ1 0
=
ẋ2 − sin x̄1

Hence, it must be ẋ2 , 0 and x2 (t) cannot remain 0, hence the trajectory will leave the set N. In
geometric terms, we see that the derivative vector has the direction of the x2 -axis, and the trajectory
is forced to exit from N, leaving the set of states where there is no dissipation. Thanks to the
Krasowskii criterion, we have then proved asymptotic stability. Figure 7.7 shows the trajectories of
the dissipative pendulum.

7.3.2 Instability theorems

There are also instability criteria, which complement stability criteria.

Theorem 7.3. (Cetaev criterion.) Suppose that a function V(x) ∈ C1 is defined in a neighbourhood
W of the origin. Assume that there exists an open set A such that the origin is contained inside the
closure of A (in other words, 0 is either in A or on the border of A, ∂A). Assume also that

• V(x) = 0 ∀ x ∈ ∂A ∩ W;

• V(0) = 0;

• V(x) > 0, V̇(x) > 0 ∀ x ∈ A ∩ W

then the system is unstable I.E.P.

We can give an intuitive explanation of this theorem. Suppose, to simplify, that the origin
belongs to the border of A, which is the most interesting case, as shown in Figure 7.8. We see that
every trajectory that starts from W near the border of A, ∂A ∩ W, since V and V̇ are positive and
continuous, has to exit from W, so we cannot have local stability.
2
Mathematicians often publish a scientific article when they find a good Lyapunov function...
108 Chapter 7. Nonlinear systems

Figure 7.7: Trajectories of the dissipative pendulum.

δA

V=0
VV>0

Figure 7.8: Intuitive representation of Cetaev criterion.

7.4. Lyapunov criterion for discrete-time systems 109

Example 7.10. Consider the magnetic levitator shown in Figure 9.15, Chapter 9.9. If we neglect
the electric equation (which means that we consider a permanent magnet), the system is described
by following equations:
 ẋ1 (t) = x2 (t) 2



 ẋ2 (t) = g − mk xi 2


1

The equilibrium point of the system is:

 0 = x2



k i2
 0=g−


m x2
1

This means having null velocity and magnetic force equal and opposite to the gravity force. We
want to prove the instability of the equilibrium point obtained with a constant value of the current
i = ī. First of all, we shift the intersection of the axes to the equilibrium point:

z1 = x1 − x̄1
z2 = x − x̄2
2
 ż1 = z2 + x̄2 = z2


⇒  2
 ż2 = g − mk (z +ī x̄ )2

1 1

Now the equilibrium point is (0, 0). As a candidate Cetaev function, we take V(x1 , x2 ) = z1 z2 . We
choose the set A as A = {z1 > 0, z2 > 0}. We have that:

ī2
  !
h i  z2 k
V̇(z1 , z2 ) = z2 z1  = z2 + z1 g −
2

 2
g − mk (z +ī x̄ )2  m (z1 + x̄1 )2
 
1 1

V is null on the boundary (if either z1 = 0 or z2 = 0). Both V and V̇ are positive in the interior of A.
In view of Cetaev criterion, the system is unstable I.E.P.

7.4 Lyapunov criterion for discrete-time systems

The formulation of the Lyapunov criterion for discrete-time systems is similar to that given for
continuous-time systems, with only one essential change: the Lyapunov derivative is replaced by
the Lyapunov difference.
The discrete-time system has the following form:

x(k + 1) = f (x(k), u(k)) (7.17)

and the equilibrium points are found as the solutions of the equation

x̄ = f ( x̄, ū) (7.18)

The shifting is done as for continuous-time systems, assuming z = x − x̄, and assuming that
v(k) = u(k) − ū we can study a system of the following type:

z(k + 1) = F x̄, ū (z(k), v(k)) (7.19)

with equilibrium pair (0, 0).

Let us start with the analysis of system

x(k + 1) = f (x(k)), f (0) = 0,

110 Chapter 7. Nonlinear systems

in the 0 equilibrium state. For discrete-time systems, we take in consideration the Lyapunov
difference defined as follows:

∆V(x(k))  V(x(k + 1)) − V(x(k)) = V( f (x(k))) − V(x(k)) (7.20)

Observe how in the last formula x(k + 1) has been replaced by the function f (x(k)), in view of the
equation which describes the system: we do not need to know the trajectory. The main idea is that,
if difference is not increasing, then the system is stable.

Theorem 7.4. Assume that f is continuous and there exists a function V(x) that is continuous in a
neighborhood W of 0 and positive definite in W (i.e., V(0) = 0 and V(x) > 0 ∀ x ∈ W\{0}). Then:

• if ∆V(x) ≤ 0 ∀ x ∈ W, then the system is stable I.E.P.;

• if ∆V(x) < 0 ∀ x ∈ W\{0}, then the system is asymptotically stable I.E.P..

We do not prove the theorem.3

7.5 Lyapunov equations for linear systems

As we have previously seen, there is no systematic method to find a Lyapunov function for a
nonlinear system. Normally, the determination of a good candidate is left to intuition and is
typically based on the knowledge of the system (for example, physical considerations). Linear
systems are an exception: for asymptotically stable linear systems, there is a systematic method to
find a suitable Lyapunov function (to certifies asymptotic stability).
We consider the following quadratic function:

V(x) = x⊤ Px (7.21)

If matrix P is symmetric and positive definite, then V(x) is a candidate Lyapunov function. It can
be easily verified that the gradient is the row vector

∇V(x) = 2x⊤ P. (7.22)

Quadratic functions with symmetric and positive definite matrices are, without any doubt, the most
used Lyapunov functions. Now, if we consider a linear system ẋ(t) = Ax(t) (the input is not relevant
to stability) and we compute the derivative of the Lyapunov function that we have just introduced,
we obtain:
V̇(x) = ∇V(x) · f (x) = 2x⊤ PAx = x⊤ PAx + x⊤ PAx (7.23)
The last two terms are scalars, therefore we can transpose the first one without changing the result.
Then we have:

V̇(x) = x⊤ A⊤ P⊤ x + x⊤ PAx = x⊤ A⊤ Px + x⊤ PAx =

= x⊤ (A⊤ P + PA)x  −x⊤ Qx (7.24)

where we have defined Q = −(A⊤ P + PA). It can be easily seen that Q is a symmetric matrix,
because it is the sum of −A⊤ P and its transpose (−A⊤ P)⊤ = −PA. We obtain the following
fundamental equation
A⊤ P + PA = −Q (7.25)
3
The proof is more involved than in the continuous-time case, because discrete time can “jump”, so we cannot define
the instant t1 as we did the previous proof.
7.5. Lyapunov equations for linear systems 111

called Lyapunov equation.

Now, suppose that the system is asymptotically stable. Then, we could think about finding a
matrix Q that is positive definite, so that V̇(x) is negative definite and verifies the conditions of
Lyapunov theorem. This does not work in general because, again, it is not easy to find a matrix P
such that Q is positive definite. For instance, if we consider matrix A1 in Example 7.7 (which is
asymptotically stable) and we take P = I, Q is not positive definite. Instead, it is appropriate to
proceed in the other way: we fix matrix Q and then we compute the corresponding P, as a solution
of the equation (7.25). This procedure is supported by the following theorem.
Theorem 7.5. The following conditions are equivalent.
1. ẋ(t) = Ax(t) is asymptotically stable;
2. for any symmetric and positive definite matrix Q, the corresponding matrix P that solves the
Lyapunov equation A⊤ P + PA = −Q is symmetric and positive definite.
Proof. We start by showing that 2 implies 1. If we arbitrarily fix a matrix Q and by solving the
Lyapunov equation we find a matrix P that is symmetric and positive definite, then the quadratic
function V(x) = x⊤ Px is a valid Lyapunov function and we have that V̇(x) < 0 ∀ x , 0, because Q
is symmetric and positive definite. Therefore, the conditions of Lyapunov theorem are verified.
Now we prove that 1 implies 2. Assume that the system ẋ(t) = Ax(t) is asymptotically stable,
and fix a symmetric and positive definite matrix Q. We prove that the solution of the Lyapunov
equation is the following: Z +∞
⊤
P= eA t QeAt dt. (7.26)
0
Indeed, by replacing this expression in the equation A⊤ P + PA = −Q, we obtain:
Z +∞ Z +∞
⊤ ⊤
A⊤ P + PA = A⊤ eA t QeAt dt + eA t QeAt dt A =
Z +∞0h 0
⊤ ⊤ i
= A⊤ eA t QeAt + eA t QeAt A dt (7.27)
0

d At
Noticing that e = AeAt = eAt A,4 the function to be integrated is a derivative, hence:
dt
Z +∞ h i+∞
d A⊤ t At i h ⊤
A P + PA =
⊤
e Qe dt = eA t QeAt = 0 − Q, (7.28)
dt 0
0

where in the last equality we have used the fact that the system is asymptotically stable, hence
eAt = 0.
t→+∞
Matrix P given by the previous expression is
• symmetric, since the function
 to  be integrated is symmetric, because we can see (based on
At ⊤ A⊤ t
the exponential series) that e =e ;
• positive definite, because for any x̄ , 0 we have:
Z +∞ Z +∞ Z +∞
A⊤ t ⊤ A⊤ t
x̄ ⊤
e Qe dt x̄ =
At
x̄ e Qe x̄ dt =
At
x⊤ (t)Qx(t) dt (7.29)
0 0 0

where we have denoted by x(t) = eAt x̄ , 0. Then the function to be integrated is positive,
because by assumption Q is positive definite, therefore also the integral is positive, thus
proving that P is positive definite.
4
This is one of the few cases in which the product commutes and the position of matrix A is not important in the
derivative.
112 Chapter 7. Nonlinear systems

□
The Lyapunov equation is linear in P, therefore we can avoid the computation of the integral
and solve an algebraic linear system, where the unknowns are the elements of P. Actually, only the
entries of the upper (or lower) triangle of P have to be considered, because P is symmetric. The
(n + 1)n
total number of unknowns and equations is then n + (n − 1) + (n − 2) + . . . + 1 = .
2
Example 7.11. Consider the state matrix
" #
0 1
A=
−1 −1

which yields an asymptotically stable system. Take Q = I, the Lyapunov equation is

α β α β
" # " # " # " # " #
0 −1 0 1 1 0
+ =−
1 −1 β γ β γ −1 −1 0 1

namely
α−β−γ
" # " #
−2β 1 0
=−
α−β−γ 2(β − γ) 0 1
Equating term-by-term, we get β = 1/2, γ = 1 and α = 3/2. The resulting matrix P is
" 3 1
#
P= 1 2 2 ,
2 1

which is positive definite, as expected.

7.5.1 Lyapunov equations for discrete-time systems

The case of discrete-time systems x(k + 1) = Ax(k) is similar. The Lyapunov function candidate
is again V(x) = x⊤ Px, but instead of the derivative we have to consider the Lyapunov difference.
Therefore we obtain that:

∆V(x) = V(x(k + 1)) − V(x(k)) = V(Ax) − V(x) =

= (Ax)⊤ P(Ax) − x⊤ Px = x⊤ A⊤ PAx − x⊤ Px =
= x⊤ (A⊤ PA − P)x = −x⊤ Qx (7.30)

The fundamental equation is

A⊤ PA − P = −Q (7.31)
named the discrete Lyapunov equation. The following theorem holds.

Theorem 7.6. The following conditions are equivalent:

1. x(k + 1) = Ax(k) is asymptotically stable;

2. for any positive definite matrix Q, the solution P of the discrete Lyapunov equation A⊤ PA −
P = −Q is positive definite.

Proof. It is similar to the proof in the continuous-time case, but we just have to consider, instead
of the integral, the sum
X∞
P= (A⊤ )k Q(A)k
k=0
□
7.6. Linearisation 113

The previous discussion allows us to perform a test to verify the asymptotic stability of a system.
However, since we consider linear systems, such a test could not seem really useful, because
stability can be simply studied by analysing the eigenvalues of matrix A. Moreover, almost all of
the algorithms that compute matrix P by solving either continuous or discrete Lyapunov equations
go through the computation of the eigenvalues of A, making the test pointless. Nevertheless, these
results are of fundamental importance for their consequences, including the possibility to give a
rigorous support to the linearisation theory.
A final consideration concerns the choice of matrix Q in order to find P. Choosing a positive
definite matrix is easy: a possibility is the identity matrix. As an alternative, we can consider a
square and invertible matrix R and compute Q = R⊤ R, which is

• symmetric, because R⊤ R ⊤ = R⊤ R;

• positive definite, because ∀ x , 0 we have x⊤ (R⊤ R)x = (Rx)⊤ (Rx) = ∥Rx∥2 > 0, because R is
invertible (nonsingular).

7.6 Linearisation
The topics discussed in this section and in the next ones are of high importance, because they will
justify our careful development of the theory for linear systems.
We start by reminding elementary notions of differential calculus. Consider a nonlinear function
f (x) in one dimension. It is well known that, if f is continuously differentiable, we can apply
Taylor theorem and write the function as follows:

f (x) = f (x0 ) + f ′ (x0 )(x − x0 ) + R(x − x0 ), (7.32)

where R(x − x0 ) is the residual and is an infinitesimal of order greater that one:

|R(x − x0 )|
→ 0 as x → x0 (7.33)
|x − x0 |

The first two terms form the linear approximation:

f¯(x) = f (x0 ) + f ′ (x0 )(x − x0 ) (7.34)

They represent a line passing through the point x0 with slope f ′ (x0 ).
For nonlinear functions in two scalar variables, the situation is similar. With a Taylor expansion,
we obtain:
" #
x − x0
f (x, u) = f (x0 , u0 ) + ∇ f + R(x − x0 , u − u0 ) =
u − u0
∂f ∂f
= f (x0 , u0 ) + (x − x0 ) + (u − u0 ) +
∂x x0 , u0 ∂u x0 , u0
+R(x − x0 , u − u0 ) (7.35)

In this case, the linear approximation is the equation of the plane that passes through (x0 , u0 ), with
equation:
f¯(x, u) = f (x0 , u0 ) + α(x − x0 ) + β(u − u0 ) (7.36)
The partial derivatives computed in the point are the coefficients of the plane that locally approxi-
mates the curve f = f (x, u).
114 Chapter 7. Nonlinear systems

This approximation can be applied to the case of nonlinear systems ẋ(t) = f (x(t), u(t)), where
the linear approximation can be determined with reference to a particular equilibrium point identified
by 0 = f ( x̄, ū):

∂f ∂f
" # " #
f (x, u) = f ( x̄, ū) + (x − x̄) + (u − ū) + R(x − x̄, u − ū) (7.37)
| {z } ∂x x̄, ū ∂u x̄, ū
=0

The terms
∂f ∂f
" # " #

∂x x̄, ū ∂u x̄, ū

are Jacobian matrices, whose (i, j) terms are equal to

∂ fi ∂ fi
" # " #

∂x j x̄, ū ∂u j x̄, ū

and become simple numbers once the derivatives are evaluated at the equilibrium point. Computing
the derivatives is a simple operation.
If an output transformation is given,

y(t) = g(x(t), u(t))

is possible to linearise it at the equilibrium point. Let us assume

ȳ = g( x̄, ū)

as the equilibrium output. Then

∂g ∂g
" # " #
y(t) = G(x(t), u(t)) = ȳ + (x(t) − x̄) + (u(t) − ū) + S (x(t) − x̄, u(t) − ū) (7.38)
∂x x̄, ū ∂u x̄, ū

where S is the residual, which is an infinitesimal of order greater than one.

Now is possible to translate the variables as

z(t) = x(t) − x̄
v(t) = u(t) − ū
w(t) = y(t) − ȳ

This transformation is nothing else than an axis shifting, in order to have the equilibrium point in
(z, v) = (0, 0) corresponding to the output w = 0. This means that, if in a thermal system T = 20oC
is the equilibrium temperature, in the new reference the desired temperature will be 0oC.5
Then the series expansion becomes:

∂f ∂f
" # " #
ż(t) = ẋ(t) = f ( x̄, ū) + z(t) + v(t) + R(z(t), v(t))
∂x x̄, ū ∂u x̄, ū
= 0 + A x̄, ū z(t) + Bx̄, ū v(t) + R(z(t), v(t)) (7.39)

The residual has the property:

∥R(z, v)∥
→ 0 as (∥v∥, ∥z∥) → 0
∥z∥ + ∥v∥
5
With no complaints.
7.6. Linearisation 115

Therefore, for little shifts in the neighbourhood of the equilibrium point, we can neglect the residual
term and obtain a linearised system of the form:

ż(t) = A x̄, ū z(t) + Bx̄, ū v(t) (7.40)

Now, by considering the variable w(t) = y − ȳ, we obtain the equation:

∂g ∂g
" # " #
w(t) = z(t) + v(t) + S (z(t), v(t))
∂x x̄, ū ∂u x̄, ū

where
∥S (z, v)∥
→ 0 as (∥v∥, ∥z∥) → 0
∥z∥ + ∥v∥
by neglecting the residual
w = C x̄, ū z(t) + D x̄, ū v(t)

To summarise, the nonlinear system written with reference to the equilibrium point is

ż(t) = Az(t) + Bv(t) + R(z(t), v(t))

(
(7.41)
w(t) = Cz(t) + Dv(t) + S (z(t), v(t))

where the matrices depend on the chosen equilibrium, although we can remove the indices x̄, ū
for a softer notation

A = A x̄,ū B = Bx̄, ū , C = C x̄, ū , D = D x̄, ū v(t)

and the linearised system (in the equilibrium point) is:

ż(t) = Az(t) + Bv(t)

(
(7.42)
w(t) = Cz(t) + Dv(t)

The matrix entries are equal to:

∂ fi
[A]i j = ( x̄, ū)
∂x j
∂ fi
[B]i j = ( x̄, ū)
∂u j
∂gi
[C]i j = ( x̄, ū)
∂x j
∂gi
[D]i j = ( x̄, ū) (7.43)
∂u j

We will use the linearised system at the equilibrium point for two different purposes.

• Stability analysis: we analyse stability of the approximated system ż(t) = A x̄, ū z(t).

• Synthesis problem: by considering the linear approximation, we will build a regulator based
on the techniques studied for linear systems. This regulator will be applied to the nonlinear
system, as shown in Figure 7.9.
116 Chapter 7. Nonlinear systems

SYSTEM

Figure 7.9: Controller for the linearised system at an equilibrium point.

Example 7.12. Let us consider the magnetic levitator described in Chapter 9.9. The equations that
describe the system are:
ẋ1 (t) = x2 (t)




x2 (t)

ẋ2 (t) = g − mk x32 (t)




 1
 ẋ3 (t) = − R x3 (t) + 1 u(t)


L L
The output e(t) is the voltage of a photo-diode that detects the position of the metallic sphere, based
on the light intensity that crosses a diaphragm and that is not covered by the sphere itself. The
equilibrium points of the system are determined by:
0 = x̄2



 2
k x̄3

=

0 g −


 m x̄2
 1
 0 = − R x̄3 + 1 ū


L L

In this case, it is natural to fix the position x̄1 , in which we want to keep the sphere, and find x̄3 and
ū consequently. Setting z1 = x1 − x̄1 , z2 = x2 − x̄2 e z3 = x3 − x̄3 , we have the linearised system
  
 ż1 (t)   0 2 1 0
   z1 (t)   0 
  k x̄     
 ż2 (t)  =  2 m x̄33 0 −2 m x̄32   z2 (t)  +  0  v(t)
 k x̄  
1 1  1
ż3 (t)  z3 (t)

0 0 − RL L

which can also be written in the following way:

      
 ż1 (t)   0 1 0   z1 (t)   0 
 ż2 (t)  =  α2 0 −β   z2 (t)  +  0  v(t)
δ
       
ż3 (t) 0 0 −γ z3 (t)
where
k x̄32 k x̄3 R 1
α =2
2
3
, β=2 2
, γ= , δ=
m x̄1 m x̄1 L L
The equilibrium point is unstable, because the eigenvalues are −γ, −α, +α. A simple reachability
test proves that the system is reachable, hence it is possible to design a state feedback control that
stabilises the linear system in a neighbourhood of the equilibrium point.
Concerning the output of the system, we have that e(t) = φ(y(t)), therefore:
 
  z1 
∂φ

e(t) − ē = φ(y(t)) − φ(ȳ) = ∂x1 0 0  z2  + R(y(t) − ȳ)
x¯1 , x¯2 , x¯3  
z3
7.7. Stability analysis of an equilibrium point 117

The derivative inside matrix C x̄1 , x̄2 , x̄3 can be found from the slope of the photo-diode curve given
by the constructor. Matrix C is of the type
h i
µ 0 0

with µ , 0, hence the system is observable, as it can be easily checked. Therefore the linearised
system is stabilisable. The guess is that, if we can stabilise the linearised system, then we can also
stabilise the nonlinear system around the equilibrium point. We will see that this is the case.

7.7 Stability analysis of an equilibrium point

The linear approximation is referred to a particular equilibrium point. Therefore, is intuitive that
the linearised system can give informations exclusively about stability in such an equilibrium point,
which is a local property.
Suppose that v(t) = u(t) − ū = 0, namely, u(t) = ū. Then

ẋ(t) = f (x(t), ū) (7.44)

We have seen that, by means of the coordinate shift z(t) = x(t) − x̄ and v(t) = u(t) − ū, we can bring
the equilibrum point to 0 and obtain the equivalent system

ż(t) = f (z(t)) = Az(t) + R(z(t))

where A is the Jacobian evaluated at x = x̄ (we should have written A x̄ , but for brevity we will not
write any index from now on).

Observation 7.1. Remember: the linear approximation is valid only in a neighborhood of the
considered equilibrium.

Now we can associate with the nonlinear system

ẋ(t) = Ax(t) + R(x(t)), (7.45)

such that f (0) = 0, the corresponding linearised model

ẋ(t) = Ax(t) (7.46)

The fundamental question is now: by analysing the stability of (7.46), which information can
we obtain on the stability of (7.45) in the equilibrium point (I.E.P.) x̄ = 0? The following is a
fundamental result.

Theorem 7.7. The two implications hold.

• If the linear system (7.46) is asymptotically stable, then the nonlinear system (7.45) is
asymptotically stable I.E.P.

• If the linear system (7.46) is exponentially unstable (namely, ∃ λ ∈ σ(A) such that ℜ{λ} > 0),
then the nonlinear system is unstable I.E.P.

Proof. We prove only the first claim, since the second one is more difficult. To this aim, we
need two important properties. We remind that the norm of a matrix is defined as
∥Mx∥  1
∥M∥  sup = max σ(M ⊤ M) 2 (7.47)
∥x∥,0 ∥x∥
118 Chapter 7. Nonlinear systems

Hence ∥Mx∥ ≤ ∥M∥ ∥x∥ (the implication is quite easy to see) and ∥MN x∥ ≤ ∥M∥ ∥N x∥ ≤
∥M∥ ∥N∥ ∥x∥. Moreover, we remind Schwartz inequality: if we denote by (x, y) the scalar product
of two vectors x and y, then
|(x, y)| ≤ ∥x∥ ∥y∥. (7.48)
In view of Theorem 7.5, if the linear system is asymptotically stable, for any choice of Q
positive definite, matrix P, solution of the Lyapunov equation, is positive definite. Then take Q = I,
the identity matrix, and the solution P of the equation

A⊤ P + PA = −I (7.49)

provides the Lyapunov function V(x) = x⊤ Px for the linear system. We now show that V(x) if a
Lyapunov function for the nonlinear system as well, thus proving the stability of (7.45) I.E.P.
Compute V̇(x) for the nonlinear system:

V̇(x) = ∇V(x) f (x) =

= 2x⊤ P(Ax + R(x)) = 2x⊤ PAx + 2x⊤ PR(x) = x⊤ PAx + x⊤ PAx + 2x⊤ PR(x) =
= x⊤ A⊤ Px + x⊤ PAx + 2x⊤ PR(x) = x⊤ (A⊤ P + PA)x + 2x⊤ PR(x) =
= −x⊤ I x + 2x⊤ PR(x) ≤ −∥x∥2 + 2|x⊤ PR(x)| (7.50)

We obtain, for x , 0,
2|x⊤ PR(x)|
!
V̇(x) ≤ −∥x∥ + 2|x PR(x)| = −∥x∥ 1 −
2 ⊤ 2
∥x∥2
We show now that there exists a neighborhood W of zero in which
2|x⊤ PR(x)|
<1 (7.51)
∥x∥2
In fact, by applying Schwartz inequality and the property of matrix norms,

2x⊤ PR(x) x⊤ PR(x) |(x, PR(x))|

= 2 = ≤
∥x∥2 ∥x∥2 ∥x∥2
2∥x∥ ∥PR(x)∥ 2∥PR(x)∥ ∥R(x)∥
≤ 2
= ≤ 2∥P∥
∥x∥ ∥x∥ ∥x∥
∥R(x)∥
We remind that R(x) is an infinitesimal of order greater than one, hence → 0 for x → 0.
∥x∥
Therefore, there exists a neighbourhood W of zero in which
∥R(x)∥ 1
<
∥x∥ 2∥P∥
(for x , 0). Hence, the inequality (7.51) holds in a set W for x , 0. In this set W, the derivative
V̇(x) is less then zero (for x , 0), hence the conditions of Lyapunov theorem are satisfied, thus
proving the asymptotic stability I.E.P. of the nonlinear system. □
The theorem does not take into consideration the case in which all of the eigenvalues of A
have non-positive real part, but some have 0 real part. In this case, in fact, the linear system is not
asymptotically stable: it can be either marginally stable or unstable, but not exponentially unstable.
Then, there is no relation between the behaviour of the nonlinear system and of its linearisation:
everything can happen.
Example 7.13. System ẋ(t) = αx3 (t) can have different behaviours based on the values of α:
7.8. Stabilisation 119

• if α > 0, then for x(t) > 0 we have ẋ(t) > 0, and for x(t) < 0 we have ẋ < 0: the system is
unstable I.E.P.;

• if α < 0, then for x(t) > 0 we have ẋ(t) < 0, and for x(t) < 0 we have ẋ > 0: the system is
asymptotically stable I.E.P., as we can see also by means of the Lyapunov function V(x) = x2 ;

• if α = 0, the system is marginally stable.

If we linearise the system at the equilibrium point x̄ = 0, we obtain that:

∂f
" #
ẋ(t) = x + R(x) = 0 + R(x)
∂x 0

Then the matrix of the linearised system is zero independently of α (A = [0], and so its eigenvalue is
equal to zero for all α), in all of the three cases above. Therefore, it is obvious that the linearisation
criterion does not give us any information about the stability of the nonlinear system.

Remark 7.2. In control theory, if we find some eigenvalues with zero real part (while the others
have negative real part), we can come to a conclusion anyway. This is a situation that we should
avoid. Indeed, small perturbations on the system parameters can lead the eigenvalues to have a
real part greater than zero, thus leading to instability. Would you take a flight if the aircraft had
some poles at ± jω?

7.8 Stabilisation
Suppose that we have already linearised a nonlinear system at the equilibrium point. To avoid
complicated computations, we assume that y = g(x) does not depend on u (strictly proper system).
This immediately implies that D = 0. Hence,

ż(t) = Az(t) + Bv(t) + R(z(t), v(t))

(
(7.52)
y(t) = Cz(t) + S (z(t))

By neglecting the residuals and working only with the approximated linear systems, we know how
to design a linear regulator. A legitimate question is whether the regulator that has been designed
based on the linear system is able to stabilise the nonlinear system in the equilibrium point as well.
The regulator, determined by means of any of the available methods for linear systems, will be
a system of the form:
żC (t) = FzC (t) + Gw(t)
(
(7.53)
v(t) = HzC (t) + Kw(t)
Assume that it stabilises the approximated linear system. The matrix of the closed-loop linear
system is
A + BKC BH
" # " #" #
ż(t) z(t)
= (7.54)
żC (t) GC F zC (t)
and is stable if the regulator is properly designed.
This regulator, however, is applied to the true system, which is a nonlinear system. By joining
(7.52) and (7.53), we get

ż = Az + BHzC + BKw + R(z, w) =

= Az + BHzC + BK(Cz + S (z)) + R(z, v) =
= (A + BKC)z + BHzC + R(z, v) + BKS (z)
żC = FzC + G(Cz + S (z)) = FzC + GCz + GS (z) (7.55)
120 Chapter 7. Nonlinear systems

which can be written in the following matrix form:

A + BKC BH R(z, v) + BKS (z)

" # " #" # " #
ż(t) z(t)
= + (7.56)
żC (t) GC F zC (t) GS (z)
| {z }
infinitesimal of order greater than 1

where the last term is the residual, which is an infinitesimal of order greater than 1. So, if we
apply the theorem about stability analysis at the equilibrium point, we can study the stability of the
linearised system (neglecting the residual). But the matrix of such a linear system has eigenvalues
with negative real part, because it is exactly the matrix obtained when designing the chosen regulator

A + BKC BH
" #
Acl =
GC F

Hence, the closed-loop system turns out to be stable at the equilibrium point. This confirms that the
design of a regulator for the approximated linear system allows us to obtain a nonlinear system
that is asymptotically stable I.E.P., as shown in Figure 7.10. In the figure, solid arcs represent
linearisation, while dashed arcs represent the application of the regulator: by commuting the
operations of linearisation and application of the regulator, we reach the same stable linear system.

linearized system

nonlinear system linear feedback systems

nonlinear feedback system

Figure 7.10: Linearisation and application of the regulator commute.

7.9 Robustness
All systems in practice are affected by uncertainties. Uncertainties are typically due to the following
factors:

• unknown parameters, typically because they are not measurable;

• parameters that vary in time in an unpredictable way;

• model approximation.

In these cases, stability has to be ensured in a robust way, namely, under all possible cases and
variations. This is the problem of robust control design and is fundamental in control applications.
We provide a general definition of robustness and then we give just a simple condition based on
Lyapunov theory in a specific case.
First of all, to formulate a robustness problem we need

• a family F of systems, which can be all the possible actual models among which a single
one will be the exact “true” system (we do not know which one);
7.9. Robustness 121

• a property P of the system, which can be, for instance, stability, or maximum frequency
response peak, maximum amplification of the input, non-overshoot, no oscillations,. . .
Definition 7.3. Property P is robust if any system f ∈ F of the family has the property P.
Example 7.14. Consider a linear system with constant unknown parameters and transfer function
s
W(s) = 2 ,
s + as + b
where we know that a− ≤ a ≤ a+ , b− ≤ b ≤ b+ , for some known bounds a− , a+ , b− , b+ . This is a
family. The considered property can be asymptotic stability. Another property can be
sup |W( jω)| ≤ µ
ω≥0
for some µ, and so on.
Remark 7.3. The definition seems a trivial tautology. A deeper insight shows that it points out two
facts:
• we must have a family that is properly defined;
• we must consider a specific property.
For instance, in the example, the unknown parameters are constant. Completely different con-
clusions about stability can be drawn if we consider time-varying parameters a− ≤ a(t) ≤ a+ ,
b− ≤ b(t) ≤ b+ . We must also specify the property. For instance, a system may remain stable for
any parameter variation, but may have a bad frequency response for some parameters.
Here we consider a simple case of robustness analysis. In the model
ẋ(t) = [A + E(t, x(t), w(t)]x(t) (7.57)
A represents the nominal state matrix, while E is a perturbation matrix that can depend on time t,
on the state and on some external signal w. We may assume that E is unknown, but has a norm that
is bounded as follows:
∥E(t, x(t), w(t)∥ ≤ ρ (7.58)
The following theorem ensures that, if the nominal system is stable, then there exists a margin in
terms of the size ρ of the perturbation E for which robust stability is guaranteed.
Theorem 7.8. If the nominal system
ẋ(t) = Ax(t) (7.59)
is asymptotically stable, then there exists ρ > 0 such that the perturbed model (7.57) is asymptoti-
cally stable for any E as in (7.58).
Proof. If (7.59) is asymptotically stable, then there exists P positive definite such that
A⊤ P + PA = −I
We apply the Lyapunov function V(x) = x⊤ Px to the system (7.57). The Lyapunov derivative is
then
V̇(x) = 2x⊤ P[A + E]x = x⊤ (A⊤ P + PA) + 2x⊤ PEx = −x⊤ I x + 2x⊤ PEx
≤ −x⊤ x + |2x⊤ PEx| ≤ −∥x∥2 | + 2|x⊤ PEx| ≤ 2∥x∥ ∥PEx∥
≤ −∥x∥2 | + 2∥x∥ ∥P∥ ∥E∥ ∥x∥ = −∥x∥2 (1 − 2∥P∥ ∥E∥)
In view of Lyapunov theorem, the derivative is negative definite if 2∥P∥ ∥E∥ < 1, namely
1
ρ< ,
2∥P∥
which is a robustness bound. □
122 Chapter 7. Nonlinear systems

7.9.1 Lyapunov redesign

This technique is quite important in the control of mechanical systems, including robots, which
present model uncertainties. For instance, in equation (9.18) we have the term Ω that is due to
errors in the model, which cause a non exact cancellation.
Consider the system
ẋ = Ax + Bu + B∆x
(for brevity we assume Ω = ∆x), with ∥∆∥ ≤ δ an uncertain term. We assume that A is asymptotically
stable, because it is the closed-loop matrix A = A0 + BK0 achieved by applying a stabilising control
K0 to the nominal system (with ∆ = 0). Then, we want to deal with the uncertain term ∆. The
nominal closed-loop system satisfies the Lyapunov equation

A⊤ P + PA = −I

We consider a “robustifying” control of this form

u = −γ2 B⊤ Px,

where γ is a parameter. Then

V̇ = x⊤ (A⊤ P + PA)x + 2x⊤ PBu + 2x⊤ PB∆x = −x⊤ x − 2γ2 x⊤ PBB⊤ Px + 2x⊤ PB∆x
x⊤ ∆⊤ ∆x x⊤ ∆⊤ ∆x
" #
= −∥x∥ − 2 γ x PBB Px − x PB∆x +
2 2 ⊤ ⊤ ⊤
+ 2
4γ2 4γ2
#⊤ "
∆x ∆x x⊤ ∆⊤ ∆x
" #
= −∥x∥2 − 2 γB⊤ Px − γB⊤ Px − +2
2γ 2γ 4γ2
∆x 2 ∥∆x∥2 ∥∆x∥2 2 ∥x∥
2
= −∥x∥2 − 2 γB⊤ Px − + ≤ −∥x∥ 2
+ 2 ≤ −∥x∥ 2
+ δ
2γ 2γ2 4γ2 2γ2
δ2
" #
= −∥x∥2 1 − 2 < 0
2γ
√
if we take γ > δ/ 2.
Chapter 8

Optimal control: basic notions

8.1 General considerations

When we apply a feedback control, typically we would like to assign eigenvalues with a real
part that is “very negative” (negative and large in absolute value), in order to have fast transients.
However, this implies that the elements of the state feedback matrix K must have large entries. For
instance, if in a scalar system we assign λ = −µ with µ > 0 large, by means of the feedback control
u = kx, then in the equation
ẋ = ax + bu = (a + bk)x
we have k = (−a − µ)/b, so the larger is µ > 0, the larger is the magnitude of k. Too large values
are not suitable, due to physical limitations introduced by the actuators:
• there is a trade-off between speed of convergence and control effort.
Moreover, the control action and/or the state variables are often constrained, such as
|u| ≤ ūmax
This kind of problems can be faced with optimal control theory.
Essentially, there are two fundamental theories.
• Pontryagin’s maximum principle;
• Bellman’s dynamic programming;
The first theory provides an open-loop optimal control, while the second theory provides a closed-
loop optimal control, which would be preferable in principle, but unfortunately is often intractable
from a numerical point of view. For brevity, we will introduce the theory of dynamic programming
only, and we will discuss the mentioned difficulties. Still, we will see that, for the fundamental
linear-quadratic problem, Bellman’s theory leads to an explicit, elegant and very efficient solution.
We will also briefly sketch Pontryagin’s approach in connection to the calculus of variation.

8.1.1 Lagrange Multipliers

We start the section with a simple consideration about the constrained optimality of functions.
Consider the optimization problem
min l(θ)
where l(θ) is a continuously differentiable function of a vector θ ∈ Rn defined on a domain θ ∈ Θ.
A necessary condition for a minimum in θ∗ internal to Θ is
∇l(θ∗ ) = 0

123
124 Chapter 8. Optimal control: basic notions

This condition is known as the stationary condition. For instance the minimum of l = θ12 + θ22 −
2θ1 − 4θ2 is in the point where

∇l = [2θ1 − 2 2θ2 − 4] = 0

namely θ = (1, 2)⊤ .

Consider now the constrained optimization problem

min l(θ) (8.1)

s.t.
h(θ) = 0 (8.2)

where l is a functional and h : Rn → Rm , is a constraint function with m < n. Assume that both h
and l are continuously differentiable. Then a necessary condition for a local minimum is that the
Lagrangian
L(θ, λ)  l(θ) + λ⊤ h(θ)
has a stationary point. Vector λ ∈ Rm is called the Lagrange multiplier. Namely the constrained
minima are computed by applying the stationary condition to L(θ, λ). Differentiating with respect
to λ, we derive h(θ) = 0, namely (8.2), conversely differentiating with respect to θ we derive the
equations
∂L(θ, λ)
=0
∂θ
Example 8.1. Compute the minimum Euclidean norm solution of Aθ = b, A = m × n matrix, with
m < n. Assume rank[A] = m. The problem is

min ∥θ∥2 /2 s.t. Aθ − b = 0

The 1/2 is added for clarity in the sequel. The Lagrangian is

∥θ∥2 /2 + λ⊤ [Aθ − b]

If we differentiate with respect to θ we get θ⊤ = λ⊤ A. We transpose it

θ = A⊤ λ (8.3)

and replace it in Aθ = b to get

A(A⊤ λ) = [AA⊤ ]λ = b
Since AA⊤ is invertible, we have λ = [AA⊤ ]−1 b, in view of the expression (8.3),

θ∗ = A⊤ [AA⊤ ]−1 b

which is the expression of the minimum norm solution of the system of equations.

8.2 Variational calculus

As a sketch of Pontryagin’s principle, we consider the standard problem of finding the scalar
function x that minimises the cost
Z T
min l(x(t), ẋ(t))dt
0
8.2. Variational calculus 125

under the boundary conditions x(0) = a and x(T ) = b on the fixed interval [0, T ]. The idea is that,
if x̄ is optimal, then any small variation

x → x + δ,

with δ(t) small, does not provide any advantage. Hence,

Z T h i
l( ẋ(t) + δ̇(t), x(t) + δ(t)) − l( ẋ(t), x(t)) dt ≥ 0
0

The use of the first order approximation (assuming differentiability) provides

∂l(x, ẋ) ∂l(x, ẋ)

Z T " #
δ̇ + δ ≥0
0 ∂ ẋ ∂x

Integrating by parts, to get rid of the term δ̇, we get

#T
T
∂l(x, ẋ) ∂l(x, ẋ)
Z " # "
d l(x, ẋ)
δ+ δ dt − δ ≥0
0 dt ∂ ẋ ∂x ∂ ẋ 0

The inequality above has to be true for any δ, and in particular for those that are zero at the extrema
δ(0) = δ(T ) = 0. Hence, considering these perturbations, it must be

d l(x, ẋ) ∂l(x, ẋ)

Z T " #
− δdt ≥ 0
0 dt ∂ ẋ ∂x

Now the condition must be true if we replace δ by −δ, therefore the inequality is actually an equality

d l(x, ẋ) ∂l(x, ẋ)

Z T " #
− δdt = 0
0 dt ∂ ẋ ∂x

Since δ is arbitrary (with δ(0) = δ(T ) = 0), the function between brackets must be 0:

d ∂l(x, ẋ) ∂l(x, ẋ)

− =0 (8.4)
dt ∂ ẋ ∂x
This is the fundamental Euler-Lagrange equation of the calculus of variations, which has to be
solved along with the conditions x(0) = a and x(T ) = b.
The relation with the optimal control is the following. We can rewrite the problem as a special
case of the optimal control problem
Z T
min l(x(t), u(t))dt (8.5)
0
ẋ(t) = u(t) (8.6)
x(0) = a, x(T ) = b (8.7)

In general, in the optimal control problem, we have

Z T
J= l(x(t), u(t)) dt (8.8)
0

with the bound u(t) ∈ U, a given set, and

ẋ(t) = f (x(t), u(t)),

126 Chapter 8. Optimal control: basic notions

with x(0) = x0 and x(T ) = xT . We can conclude that the optimal control problem is more general
and includes the calculus of variations as a special case, with the particular equation ẋ = u.
In this optimal control problem, the solution can be found by considering the Pontryagin
equations as follows. Define the Hamiltonian function as
H(x, u, λ)  l(x, u) + λ⊤ f (x, u),
where λ(t) is a vector function with values in Rn , called the co-state. Then the Pontryagin equations
are
∂H(x, ū, λ)
ẋ = (8.9)
∂λ
∂H(x, ū, λ)
λ̇ = − (8.10)
∂x
ū = arg min H(x, u, λ) (8.11)
u∈U
x(0) = x0 , x(T ) = xT (8.12)
These equations are typically solved numerically. For ẋ = u scalar (under suitable assumptions),
we recover Euler-Lagrange equations.
We now derive the Pontryagin equations. As a first step we replace the cost function by
Z T Z Th i
J¯ = l(x(t), u(t)) + λ(t)⊤ − ẋ(t) + f (x(t), u(t)) dt = H(x(t), u(t)) − λ(t)⊤ ẋ(t) dt (8.13)
 
0 0

and we notice that the problem is unchanged, because − ẋ(t) + f (x(t), u(t)) = 0. Then we apply the
following principle.
Maximum principle. A pair (u, x) is optimal if for any infinitesimal perturbation δu
u(t) + δu(t) → x(t) + δx(t), ẋ(t) + δ ẋ(t)
compatible with the constraints
δx(0) = δx(T ) = 0
the cost does not decrease, namely the cost variation is nonnegative
Z Th i
δ J¯ = H(x + δx, u + δu, λ) − H(x, u, λ) + λ⊤ δ ẋ dt ≥ 0
0

Note that Z T Z T h iT Z T
λ δ ẋdt = −
⊤
λ̇ δxdt + λ δx
⊤ ⊤
=− λ̇⊤ δxdt
0
0 0 0

because λ⊤ δx T0 = 0 since no variations of x are possible at the extrema (constraints (8.12)). Then
 
we have
Z Th i
δ J¯ = H(x + δx, u + δu, λ) − H(x, u, λ) + λ̇⊤ δx dt
0
Z Th i Z T
= H(x + δx, u + δu, λ) − H(x, u + δu, λ) + λ̇ δx dt +
⊤
[H(x, u + δu, λ) − H(x, u, λ)] dt
0 0
∂H(x, u, λ)
Z T" # Z T
≈ + λ̇ δx dt +
⊤
[H(x, u + δu, λ) − H(x, u, λ)] dt ≥ 0
0 ∂x 0

Due to the optimality principle, if u is optimal the last inequality must be satisfied, as a necessary
condition, for any admissible small variation and for any λ(·). If we take λ(t) as the solution of the
differential equation
∂H(x, u, λ)
+ λ̇⊤ = 0
∂x
8.3. Dynamic programming 127

which is (8.10), then the only condition to be ensured is

Z T
[H(x, u + δu, λ) − H(x, u, λ)] dt ≥ 0
0

which means that u must minimize the Hamiltonian, namely (8.11).

Example 8.2. (Optimal height reaching.) A craft has to reach the optimal-cruise height h after
taking off. Assuming that the horizontal speed is constant, we can decide the slope of the trajectory.
A height increase ẋ(t), with x(0) = 0, implies a fuel cost. This cost has to be added to the cost of
flying at height x(t), which is smaller at the target height. Let us assume

l(x, u) = µ(x − h)2 + ν + σu2

|{z}
| {z }
fuel consumption at height x rising consumption

with
ẋ = u
with no constraints (u ∈ R). Euler-Lagrange equation (8.4) becomes

2σu̇ + 2µ(x − h) = 0

and then
µ
ẍ + (x − h) = 0
σ
Let us change variable y = (x − h) to get
µ
ÿ + y=0
σ
with conditions y(0) = −h and y(T ) = 0. The general solution is
õ õ
y(t) = αe σ t + βe− σ t ,

where α and β are constants to be determined based on the conditions

õ õ õ õ
αe σ 0 + βe− σ 0 = α + β = −h αe σ T + βe− σ T = 0

whose the solution is left to the reader.

Note that √ µ T → ∞,
√ µ this approach has some problems if it is formulated on an infinite horizon
because e σ tends to infinity. Conversely, there is nothing wrong in the fact that e σ t diverges,
T

since we are considering a finite horizon.

8.3 Dynamic programming

We now briefly present Bellman’s approach. Consider the problem of minimising (or maximising)
the following index over an infinite horizon:
Z ∞
J= l(x(t), u(t)) dt (8.14)
0

with the bound u(t) ∈ U, which is a set of assigned constraints, and

ẋ(t) = f (x(t), u(t))

128 Chapter 8. Optimal control: basic notions

We assume that 0 is an equilibrium

0 = f (0, 0)
ant that the cost is positive definite: l(0, 0) = 0, and l positive elsewhere. One way to proceed is to
define a cost-to-go function

ψ(x0 )  optimal value of the problem with initial conditions x0

This function must have the following properties. Let t be the initial instant and let x(t + h) be an
intermediate state of the optimal trajectory. Then it should be
 


 


 
t+h


 Z 


ψ(x(t)) = min  + ψ(x(t +
 
l(x(t), u(t)) dt h))
 

u(t)∈U 


 t | {z } 





 | {z } residual optimal cost


 partial cost 

The minimum depends on the assigned constraints. Since ψ(x(t)) is independent of u, we can
include it inside the brackets. Dividing by h, we have

ψ(x(t + h)) − ψ(x(t)) 1 t+h

( Z )
min + l(x(t), u(t)) dt = 0
u(t)∈U h h t

Assume that the function ψ is differentiable. Then, for h → 0+ , the first term gives the derivative
and the second term gives l(x(t), u(t)) due to the mean value theorem
n o
min ψ̇(x(t)) + l(x(t), u(t)) = 0
u(t)∈U

This has to be true for any t. Now we have ψ̇(x) = ∇ψ(x) ẋ = ∇ψ(x) f (x, u), which is the Lyapunov
derivative of ψ, so we obtain the equation

min {∇ψ(x) f (x, u) + l(x, u)} = 0, (8.15)

u∈U
ψ(0) = 0

named Bellman equation of dynamic programming. If the function ψ(x) is known, then the control
can be derived as the element of u = u(x) that minimises the above expression. Unfortunately, for
this equation it is not possible to determine an analytic solution in most cases. Numerical methods
have to be adopted (which are computationally very heavy).

Example 8.3. Consider the take-off problem in the y variable

l(y, u) = µy2 + ν + σu2

with
ẏ = u
The Bellman equation is ( )
d
min ψ(y)u + σu + µy = 0
2 2
u∈U dy

and the minimiser ū is achieved by setting the derivative to 0

1 ′
ū = − ψ (y)
2σ
8.3. Dynamic programming 129

Replacing ū in the equation we get

1 ′ 2
− ψ (y) + µy2 = 0
4σ
√
Hence ψ′ (y) = 2 µσy and by integration (note that ψ(0) = 0)
√
ψ(y) = µσy2

The optimal control is

µ
r
u=− y (8.16)
σ
q
µ
Note that the control (8.16) is linear. The closed loop system is ẏ = − σ y, hence it is stable. This
is NOT the same achieved with Pontryagin’s method. Why?

8.3.1 Linear quadratic control

In the particular case of linear systems and quadratic cost, the solution of the problem can be easily
determined. This is a remarkable property and this control is very popular and efficient.
Consider the problem
1 ∞ ⊤
Z 
J = x (t)Qx(t) + u⊤ (t)Ru(t) dt (8.17)
2 0
with (8.18)
ẋ(t) = Ax(t) + Bu(t)
(

x(0) = x0

Matrix Q is assumed to be positive semi-definite (of course it may be also positive definite), while
R is assumed to be symmetric positive definite. We assume that U = Rm , namely there are no
constraints. Equation (8.15) becomes
( )
1 ⊤
min ∇ψ(x)(Ax + Bu) + (x Qx + u Ru) = 0
⊤
(8.19)
u(t)∈U 2
We try a solution in the form ψ(x) = x⊤ Px/2, with P positive definite, and we replace ∇ψ(x) = x⊤ P
( )
1
min x⊤ P(Ax + Bu) + (x⊤ Qx + u⊤ Ru) = 0 (8.20)
u(t)∈U 2
For any fixed x, the minimum with respect to u is obtained by considering the gradient (with respect
to u, namely assuming x constant) and setting it equal to zero

x⊤ PB + u⊤ R = 0

This means
u = −R−1 B⊤ Px,
which is a linear control. By replacing this expression in the equation we have
1
x⊤ PAx − x⊤ PBR−1 B⊤ Px + (x⊤ Qx + x⊤ PBR−1 RR−1 B⊤ Px) = 0
2
Since x⊤ PAx = (x⊤ PAx + x⊤ A⊤ Px)/2, we obtain
1 ⊤ ⊤
x [A P + PA − PBR−1 B⊤ P + Q]x = 0,
2
130 Chapter 8. Optimal control: basic notions

which has to be valid for each x. This brings us to the Riccati equation

A⊤ P + PA − PBR−1 B⊤ P + Q = 0 (8.21)

We look for a positive definite solution P of this equation. Indeed, only positive definite solutions
are meaningful, since the cost has to be positive. Note that in general there is more than one solution.
To see this, it is sufficient to consider the scalar case

b2 2
2pa − p +q=0
r
(the products commute in this case). This second order equation has two solutions and we are
interested in the positive one. In general, the Riccati equation has a single positive definite solution.
We conclude with the following important result.

Theorem 8.1. Assume that (A, B) is stabilisable and Q is positive definite. Then (8.21) admits a
unique positive definite solution P. The optimal control is

u = −R−1 B⊤ Px = Kopt x (8.22)

and the optimal cost with initial condition x0 is

Jopt = x0⊤ Px0 (8.23)

We do not give a technical proof of the theorem. Just observe that, if Q is positive definite, then
the integrand function in the cost is positive, unless x0 = 0, hence x0⊤ Px0 > 0, so P must be positive
definite.
We can easily see that, if Q is positive definite, then the closed-loop system is asymptotically
stable. Write (8.21) as

(A⊤ P − PBR−1 B⊤ P) + (PA − PBR−1 B⊤ P) + Q + PBR−1 B⊤ P = 0

The term (A − BR−1 B⊤ P) = (A + BKopt ) = ACL is the closed-loop matrix. Then we obtain
⊤
ACL P + PACL + Q + PBR−1 B⊤ P = 0 (8.24)
| {z }
=Q̂

Being Q positive definite and PBR−1 B⊤ P positive semi-definite, also Q̂ = Q + PBR−1 B⊤ P is

positive definite. This means that (8.24) is a Lyapunov equation, hence the closed-loop system is
asymptotically stable.

Remark 8.1. It is not necessary to have Q positive definite. For instance, if we have a performance
output z = Hx, the state term is
z⊤ z = x⊤ H ⊤ Hx = x⊤ Qx
and in this case Q is positive semi-definite. Still, matrix Q̂ = Q + PBR−1 B⊤ P and matrix P can
be positive definite, and in this case the closed-loop is asymptotically stable. We need to solve the
equation and check that P and Q̂ are positive definite. A trick to avoid problems is to perturb the
cost as H ⊤ H → H ⊤ H + ϵI, which is positive definite. If ϵ is small, we are virtually not changing
the problem.

Example 8.4. Consider

" # " # " #
0 1 0 h i 1 0
A= , B= , H= 1 0 , Q=
0 0 1 0 0
8.3. Dynamic programming 131

and R = [1]. The state variables are: x1 the position, x2 the speed and u the acceleration
(controlled) of a cart. The cost is the integral of the square of the position plus the square of the
acceleration. The positive definite solution of the Riccati equation is
" √ #
2 √1
P=
1 2
Q is not positive definite, however
" √ #
Q̂ = Q + PBR B P =
−1 ⊤ √2 2
2 2
is positive definite and the control is stabilising. The closed-loop poles are
1 1
λ12 = − √ ± j √
2 2
h i
Conversely, if we take H = 0 1 , we get P = Q and the control is not stabilising. This is not
surprising. We are penalising the speed and the control. So, if the initial condition is x1 , 0 and
x2 = 0, the optimal control is u = 0 (do nothing and stay in position x1 ), with cost 0, because both
speed and acceleration are 0.
The optimal control has infinite gain margin. If we modify the control as
u = −κR−1 B⊤ Px
with κ ≥ 1 arbitrarily large, the system remains stable. This can be easily checked, because we can
derive a Lyapunov equation similar to (8.24)
(A − κBR−1 B⊤ P)⊤ P + P(A − κBR−1 B⊤ P) + Q + (2κ − 1)PBR−1 B⊤ P(κ − 1)PBR−1 B⊤ P = 0
In some cases, increasing the gain κ improves robustness (see Chapter 7.9).
An interesting observation is the following. Consider a performance output z(t) = Hx(t) and
the cost
1 ∞ ⊤ 1 ∞
Z  Z 
J= x (t)H Hx(t) + u (t)u(t) dt =
⊤ ⊤
∥z(t)∥2 + ∥u(t)∥2 dt
2 0 2 0
This is a particular case with R = I and Q = H H. If we consider the “augmented output“
⊤
" # " #
u(t) K
= x(t)
z(t) H
we get the corresponding system
ẋ(t) = [A + BK]x(t) + I∆(t) (8.25)
u(t) = K x(t) (8.26)
z(t) = Hx(t) (8.27)
with input ∆. Denoting by W(t) the matrix of impulse responses, the LQ control is the one that
minimises
1 ∞ 1 ∞X 2
Z Z
Jtr = tr[W(t) W(t)]dt =
⊤
W (t)
2 0 2 0 ij ij

where tr[S ] is the trace of the matrix S .1 Note that

X
tr[M ⊤ M] = tr[MM ⊤ ] = Mi2j
ij

The Riccati equation to be solved in this particular case is

A⊤ P + PA − PBB⊤ P + H ⊤ H = 0 (8.28)
1 P
The trace is the sum of the diagonal elements, tr[S ]  S ii .
132 Chapter 8. Optimal control: basic notions

8.3.2 Optimal observer: the Kalman-Bucy filter

Consider a system in the presence of noise

ẋ(t) = Ax(t) + Bu(t) + Ev(t)

y(t) = Cx(t) − w(t)

where v(t) and w(t) are unknown noises or disturbances. If we consider the standard observer

˙ = (A − LC) x̂(t) + Bu(t) + Ly(t),

x̂(t)

defining the error as e = x̂ − x, we obtain the error equation

ė(t) = [A − LC]e(t) − Lw(t) − Ev(t) (8.29)

η(t) = Ie(t) (8.30)

If we consider the dual system of (8.29)-(8.30)

 A " + C ⊤ (−L)  A " + B∗ K

 ⊤ ⊤ I   ∗ ∗ I 
" #∗
A − LC [−L E]
 
= ⊤ = ∗
# #
(−L) K
   
I 0  0   0 
H⊤ H∗

we get a system of the form (8.25)-(8.27), so its impulse response matrix is the transpose of the
impulse response matrix of (8.29)-(8.30). It can be easily seen that tr[M ⊤ M] = tr[MM ⊤ ]. So, to
minimise
1 ∞
Z
Jtr = tr[W(t)W(t)⊤ ]dt,
2 0

we can exploit duality and replace K with −L⊤ , H with −E ⊤ and B with C ⊤ . By transposing (8.28),
we obtain the dual Riccati equation

PA⊤ + AP − PC ⊤CP + EE ⊤ = 0 (8.31)

and the optimal filter gain

L = PC ⊤ .

Duality between optimal control and optimal filtering is a fundamental achievement in control
theory. It can be shown that, by properly combining an optimal observer and an optimal state
feedback control, we get an optimal output feedback control.

8.4 Connection filter-optimal control

In this section we examine the results we achieve by combining the optimal control with the
Kalman filter. We start with a technical computation concerning the performance loss we achieve
by replacing the optimal control by a generic stabilizing control u. For convenience we take R = I,
8.5. Model predictive control 133

Q = H ⊤ H and x(0) = x0 . Then

Z ∞ h i
J = x⊤ H ⊤ Hx + u⊤ u dt
Z0 ∞ " h i⊤ h i d #
= x H Hx + u + B Px − B Px u + B Px − B Px + x Px dt + x0⊤ Px0
⊤ ⊤ ⊤ ⊤ ⊤ ⊤ ⊤
0 dt
Z ∞ h i⊤ h i
= u + B⊤ Px u + B⊤ Px dt + x0⊤ Px0
Z0 ∞  h i⊤ 
+ x⊤ H ⊤ Hx − 2 u + B⊤ Px B⊤ Px + x⊤ PBB⊤ Px + 2x⊤ PAx + 2x⊤ PBu dt
Z0 ∞ h i⊤ h i Z ∞ h i
= u + B Px u + B Px dt + x0 Px0 +
⊤ ⊤ ⊤
x⊤ H ⊤ H − PBB⊤ P + PA + A⊤ P xdt
0 0 | {z }
=0
Z ∞ h i⊤ h i
= u + B⊤ Px u + B⊤ Px dt + x0⊤ Px0
0

As expected any control is worse than uopt = −B⊤ Px, for which the optimal performance is x0⊤ Px0
is ensured.
To evaluate the performance of the combination Kalman filter and optimal control we consider
u = −B⊤ P x̂, the feedback of the estimated state to get

Z ∞h i⊤ h i Z ∞h i⊤ h i
J = u + B Px
⊤
u + B Px dt +
⊤
x0⊤ Px0 = B⊤ P(x − x̂) B⊤ P(x − x̂) dt
Z0 ∞ 0

= ∥B⊤ Pe∥2 dt + x0⊤ Px0

0
where e = x − x̂ is the estimation error. On the other hand the Kalman filters minimizes
Z ∞
∥e∥2 dt
0
hence the combination is optimal.
If we consider, instead of a generic initial condition, the impulse response, we can replace, for
m = 1, E = x0 Z ∞
J= ∥B⊤ Pe∥2 dt + E ⊤ PE
0
For multiple input systems, we just need to take the trace
Z ∞
J= ∥B⊤ Pe∥2 dt + tr(E ⊤ PE)
0

8.5 Model predictive control

Model predictive control is a popular technique in which the control is computed by solving an
optimization problem on-line. Consider the discrete–time problem with initial time k0
∞
X
min g(x(k), u(k))
k=k0
s.t.
x(k + 1) = f (x(k), u(k))
x(k) ∈ X, u(k) ∈ U,
x(k0 ) assigned
134 Chapter 8. Optimal control: basic notions

where we assume that (0, 0) is the target equilibrium point. Sets X and U are constraint sets. In
most cases they are of the form
X = x : xi− ≤ x ≤ xi+ U = u : u−i ≤ u ≤ u+i
n o n o

The idea is the following. First consider problem with a finite horizon T
0 +T
kX
min g(x(k), u(k)) + h(x(k0 + T ))
k=k0
s.t.
x(k + 1) = f (x(k), u(k))
x(k) ∈ X, u(k) ∈ U,
x(0) assigned
with T integer large enough. The term h(x(k0 + T )) is a weight on the final state. It can be replaced
by a constraint such as x(k0 + T ) = 0. Assume that this problem can be efficiently solved on–line
(real–time). Limiting ourself in solving it at the beginning k = 0, we would get an open–loop
solution. To derive a feedback solution we can apply the following procedure.
1. At k0 = 0, given x(k0 ) compute the optimal sequence u∗ (k0 ), u∗ (k0 + 1), . . . u∗ (k0 + T − 1);
2. Apply only the first input u∗ (k0 );
3. Measure the new state x(k0 + 1) = f (x(k0 ), u∗ (k0 )) and consider it as the new initial state;
4. Set k0 := k0 + 1 and GOTO 1 (recompute the sequence);
This technique, known as model predictive control (or receding horizon control) is very popular. Its
implementation requires the solution of an optimization problem at each step. This is reasonable in
most applications in view of the current computer performances. The solution time must be smaller
than the sampling time. Several commercial solvers are available to implement the technique.
One important case is that of linear systems with linear constraints and quadratic cost
+T −1
k0X
min J= x(k)⊤ Qx(k) + u(k)⊤ Ru(k) + x(k0 + T )⊤ S x(k0 + T )
k=k0
s.t.
x(k + 1) = Ax(k) + Bu(k))
y(k) = Hx(k) + Mu(k))
y−i ≤ y ≤ y+i
x(0) assigned
where Q, R and S are assumed positive definite matrices. Matrix S weights the final state. An
alternative possibility to this weight is the additional constraint x(T ) = 0
The linear constraints ensure that by defining
 x(k0 + 1)   Q 0 . . . 0 0 . . . . . . 0 
   
 x(k + 2)  0 . . . . . . . . . . . . 0 
 0 Q 
0
..  .. .. .. .. .. .. .. .. 
  
.  . . . . . . . . 
 
 
 x(k + T )   0 . . . 0 S 0 . . . . . . 0 

0
ξ =   , Ψ = 
 0 . . . . . . 0
u(k0 ) R 0 . . . 0 
 
  
 u(k + 1)   0 . . . . . . 0

0 R . . . 0 

0
..  .. .. .. .. .. .. .. .. 
  
.  . . . . . . . . 
 
 
u(k0 + T − 1) 0 ... ... 0 0 ... 0 R
   
8.5. Model predictive control 135

we can write the optimization problem as

min J = ξ⊤ Ψξ
s.t.
Φξ = ϕ(x(k0 ))
ξ− ≤ ξ ≤ ξ+
x(k0 ) assigned
where Φ is a (large) matrix with nT rows and (n + m)T columns, ϕ(x(k0 )) is a nT –vector function of
the initial condition. This is a linear–quadratic optimization problem. This is a convex optimization
problem for which efficient solvers are available.
The next issue is how can we enforce stability of the scheme. Essentially, the following
property holds. Assume that (A, B) is reachable and that ξ− and ξ+ are positive and negative vectors
respectively (so x = 0 and u = 0 are feasible vectors). Assume that the optimization problem has a
feasible solution for the initial condition x(0). Note that under constraints this is not always ensured.
For instance
x(k + 1) = 2x(k) + u(k), |u(k)| ≤ 1
if we take x(0) = 2, there is no constrained control driving the state to 0.
The following general principle holds: The closed–loop stability is ensured if the horizon T is
large enough for all initial states which belong to a convex set D including the origin in its interior.
There are several results which provide the length of the horizon T to have stability. We
prove closed-loop stability with the additional assumption: the following constraint is added to the
optimization
x(k0 + T ) = 0 (8.32)
this means that the final state is imposed to be 0.
Theorem 8.2. The model–predictive scheme with the additional constraint (8.32) is stabilizing.
Proof Consider the optimal cost with initial condition x0
V(x0 ) = optimal constrained cost J with initial condition x0
This is a Lyapunov function for the closed–loop system. Consider the optimal sequences
u(k0 ), u(k0 + 1), . . . u(k0 + T − 1), x(k0 + 1), x(k0 + 2), . . . x(k0 + T ) = 0
(the last equality is the constraint). If we apply the first input u(k0 ) we get the state new x(k0 + 1).
This is the new initial state for the following optimization. Consider the new sequences from the
new initial state x(k0 + 1)
u(k0 +1), u(k0 +1), . . . u(k0 +T −2), 0, x(k0 +2), x(k0 +3), . . . x(k0 +T +1) = 0, x(k0 +T +1) = 0
in which the last two states are at 0, and the last control is at zero. This is an admissible new
sequence (it satisfies the constraints) because all elements are also in the previous sequence or 0.
The cost J˜ of such a new sequence is smaller than the previous sequence because the former state
x(k0 + 1) and input u(k0 ), which have positive costs, do not appear since they are replaced by the
null state x(k0 + T + 1) = 0 and by the null input u(k0 + T − 1) = 0 at the end. All the other states
and inputs of the new sequence are the shifted value of the previous sequence.
The new sequence is not optimized. If we optimize we find the optimal cost which is (of course)
˜ hence
not greater than J,
V(x(k0 + 1)) ≤ J˜ < V(x0 )
This means that the cost–to–go V(x0 ), namely the optimal cost is a Lyapunov function and this
proves asymptotic stability. □
Chapter 9

Examples of dynamical systems

In this chapter, we present models and examples of dynamical systems. These examples are used
as application benchmarks in the course.

9.1 DC electrical machine

Ra La
Lf
Va

Rf ϕ
ω
Cr
Vf

Figure 9.1: General model of a DC motor.

Consider Figure 9.1, representing the diagram of a DC (direct current) electrical machine. This
device can be used in two different modes.
• Generator mode: mechanical power is provided and electrical power is produced.
• Motor mode: electrical power is provided and mechanical power is produced.
The current flows in the armature circuit of the rotor, connected to an external supplier (or an
electrical load in the case of generator mode) through a collector. In view of the rotation, the
rotor coils move with respect to the magnetic field. This creates a force, hence a torque, and an
electromotive force. These simultaneous effects are the basic principle of the electromechanical
energy conversion.
The equations of the machine are the following. The field circuit, which has the goal of
generating the flow in the machine, has equation
di f (t)
Lf = −R f i f (t) + v f (t), (9.1)
dt

137
138 Chapter 9. Examples of dynamical systems

where L f is the inductance, i f is the field circuit current, R f is the resistance, i f the current and v f
the field generator voltage. In small machines, the field circuit is replaced by a permanent magnet
that generates a constant flow, hence this equation is not present. The excitation circuit equation is

dia (t)
La = −Ra ia (t) − k̂ ie (t) ia (t) + va (t), (9.2)
dt
where La is the armature inductance, Ra is the resistance, ia is the armature current and va the
applied voltage (or the potential difference on the load). The term k̂ ie (t) ia (t) is the electromotive
force. There are two mechanical equations. The torque equation is

J ω̇(t) = Cm (t) − f ω(t) − Cr (t)

φ̇(t) = ω(t) (9.3)

In the first equation J is the inertia, ω the rotational speed, f is a viscous coefficient, Cr (t) is a
torque due to the load. The term Cm is the torque produced by the machine. The second equation is
the definition of rotational speed, which is the derivative of the machine angle.
These equations are nonlinear. We can assume, as input and state variables, the vectors
 
   ie (t) 
 ve (t)   i (t) 
u(t) =  va (t)  ∈ R3 x(t) =  a  ∈ R4 (9.4)
 
   ω(t) 
Cr (t)
φ(t)

As far as the output of the system is concerned, this depends the problem (and also on the applied
sensors). Typically, if the goal is to design a control system for the machine, we can take as an
output vector
φ(t)
" #
y(t) = (9.5)
ω(t)

Figure 9.2: Series and parallel connections.

The two electrical circuits of the machine can be connected in different ways. There are three
main possibilities.

• Parallel connection (Figure 9.2, left): ve = va , hence the two inputs are reduced to one.

• Series connection (Figure 9.2, right): ie (t) = ia (t); hence, two state variables are replaced by
one.
9.1. DC electrical machine 139

Va

Vf = cost

Figure 9.3: Independent connection.

• Independent connection (Figure 9.3): typically we set ve (t) = ve constant. In this case the
field equation is removed and we assume ie =constant.
In the independent configuration, the equation of the excitation circuit is decoupled from the other,
as follows:
die Re ve
= ie (t) + (9.6)
dt Le Le
ve
and yields the steady-state value ie = . Therefore, assuming that the transient for ie is over, and
Re
ie is at its steady-state value, this current can be considered as a constant. The equation for ie is not
necessary anymore. Let us then write k̂ie = k. We get
dia (t)
La = −Ra ia (t) − kω(t) + va
dt
J ω̇(t) = Cm (t) − f ω(t) − Cr
φ̇(t) = ω(t)
Let us multiply the first equation by ia and the second by ω(t) to derive the power balance
d 1 dia 2
!
La + Ra i2a + kωia = v a ia
dt 2 dt |{z} |{z} |{z}
| {z } resistor dissipated power converted power supplied power
magnetic power
!
d 1 2
Jω + f ω2 + Cr ω = Cm ω
dt 2 |{z} |{z} |{z}
| {z } mechanical dissipated power used power converted power
kinetic power
Considering the converted power as it appears in the two equations, we have Cm ω = kωia , hence
the following expression for the torque:
C M = kia .
The equations are
dia (t) Ra k va (t)
= ia (t) − ω(t) +
dt La La La
k f Cr (t)
ω̇(t) = ia (t) − ω(t) −
J J J
φ̇(t) = ω(t) (9.7)
140 Chapter 9. Examples of dynamical systems

with the same outputs previously chosen. It is possible to write the system in matrix form as follows.
State, input and output vectors are
 
 ia (t) 
φ(t)
" # " #
va (t)
x(t) =  ω(t)  , u(t) = , y(t) = (9.8)

 

Cr (t) ω(t)
φ(t)
 

Then, denoting by α = Ra
La , β= k
La , γ = kJ , δ = Jf , ϵ = 1
La , µ = 1J ,

 ε 0  "

       
 x (t)   −α −β 0  x1 (t) 
d  1  
#
 u1 (t)

 x2 (t)  =  γ −δ 0

 x2 (t)  +  0 −µ  u (t)
  
dt 

  2
x3 (t) 0 1 0 x3 (t) 0 0
 
" # " #  x1 (t) 
y1 (t) 0 0 1
=
 
 x2 (t)  (9.9)
y2 (t) 0 1 0 
 
x3 (t)
To find the modes of the system, we look for the eigenvalues of matrix A, hence we write the
characteristic polynomial
 s + α β
 
0 
det(sI − A) = det  −γ s + δ 0  = s[(s + α)(s + δ) + βγ]
 

−1 s

0
A root of the characteristic polynomial is s = 0. The other two eigenvalues, for machines of this
type, are typically real and negative. The modes of the system are therefore the type eλ1 t , eλ2 t , 1
where the latest mode (due to s = 0) is always present for any parameter values. This depends on
the fact that a machine that starts from an initial non-zero angle and 0 speed and current remains in
the same position. The transfer function is:
" #
n11 (s) n12 (s)
n21 (s) n22 (s)
W(s) = (9.10)
s[(s + α)(s + δ) + βγ]
where the numerator entries can be computed according the formula
" #
sI − A −B j
ni j (s) = det (9.11)
Ci Di j
where Ci is the i-th row of matrix C, and B j is the j-th column of matrix B. For example, considering
the first input (u2 ≡ 0) and the first output y1 (t), we can compute y1 (s) = W11 (s) u1 (s), where:

 s + α +β 0 −ε 
 
 −γ s + δ 0 0 
n11 (s) = det   = εγ (9.12)
 0 −1 s 0 

0 0 1 0
The computation of the other elements ni j is analogous.

9.2 Robot arm

Consider Figure 9.4, representing a robot arm. The equations of such a system, with three degrees
of freedom, are not easy to write. In general, however, we can say that equations of the following
form can be derived
M(q(t))q̈(t) + H(q(t), q̇(t))q̇(t) + K(q(t)) = τ(t), (9.13)
9.2. Robot arm 141

ϕ
ψ

Figure 9.4: Scheme of a robot arm.

where the components of the vectors q(t), q̇(t), q̈(t) are, respectively, angles, angular velocities and
angular accelerations. In general robotic systems, q is a vector of free coordinates.
Assuming that M(q(t)) is a non-singular matrix (therefore, it is invertible), we can choose the
following control law:

τ(t) = H(q(t), q̇(t))q̇(t) + K(q(t)) + M(q(t))u(t), (9.14)

where u is a new control signal to be specified later. Note that, in order to implement such a control,
we need sensors that measure angles and angular velocities. Replacing τ in the initial equations, we
get
M(q(t))q̈(t) = M(q(t))u(t) ⇒ M(q(t))[q̈(t) − u(t)] = 0. (9.15)
Since M(q(t)) is non-singular by assumption, it must be

q̈(t) = u(t) (9.16)

The resulting system (included in the dotted rectangle in Figure 9.5) is linear and can be controlled
by means of one of the (several) available techniques. This process is known as cancellation of
the nonlinearities. Based on this, linear feedback allows us to effectively control the robot arm.
Adopting the linearising feedback provides m decoupled equations (m is the number of degrees of
freedom). Then we may decide to control all of these coordinates qi independently. In general, the
state-space representation is:
" #
q(t)
q(t) ∈ R ,
m
u(t) ∈ R , x(t) =
m
∈ R2m
q̇(t)
" # " #" # " #
d q(t) 0 I q(t) 0
= + u(t)
dt q̇(t) 0 0 q̇(t) I

where I ∈ Rm is the identity matrix. Typically, the sensors for this system are encoders (measuring
angles) hence
h i " q(t) #
y(t) = I 0
q̇(t)
If also speed sensors are available, the output is the state y(t) = x(t), that is
" #" #
I 0 q(t)
y(t) = .
0 I q̇(t)

We can actually avoid the speed sensors, because the speed can be determined from the angles by
means of an observer.
142 Chapter 9. Examples of dynamical systems

τ q
ROBOT
q

Linearization

u r
Linear
v
Control

Figure 9.5: Control of a robot arm: r and v are position and speed references.

There is a major issue here. The cancellation (9.14) is not exact, because it is based on a model:

τ(t) = H̃(q(t), q̇(t))q̇(t) + K̃(q(t)) + M̃(q(t))u(t) (9.17)

Therefore, there is an error term in the resulting equation

q̈(t) = u(t) + Ω (9.18)

caused by a non exact cancellation, where

Ω = M −1 [( M̃ − M)u + (H̃ − H)q̇ + (K̃ − K)]

is an uncertain term. Hence, the control must be robust against this uncertainty. In Section 7.9, we
consider a simple case in which Ω = ∆x.

9.3 Mortgage payment

Even economic phenomena can be studied by the theory of dynamical systems. For example, a
bank account can be described by the following equation:

x(k + 1) = x(k) + r(k) + i x(k), (9.19)

where x(k) represents the account balance, r(k) the money transfer, and i the interest. This is a
discrete-time system, i.e., k time units represent k days (or months). The underlying hypothesis is
that payments or withdrawals r(k) are accounted daily (or monthly). We have

x(k + 1) = (1 + i)x(k) + r(k) ⇒ x(k + 1) = αx(k) + r(k), with α = 1 + i, (9.20)

9.4. Positive linear systems and Markov chains 143

which is a simple linear discrete-time system. According to the formula derived in the initial chapter
on discrete-time systems, if we consider r(k) as the system input, the evolution of the bank account
balance is:
k−1
X
x(k) = αk x(0) + αk−h−1 r(h). (9.21)
h=0

9.3.1 Mortgage installment computation

Now suppose that, in the account, we must consider the negative contribution of a mortgage of
value C, which has to be extinguished in N time units (starting with with −C), by paying a constant
installment r. Which is the amount of the constant installment, to be paid at each time k, such that
the loan is extinguished at time N?
We can simply put x(0) = −C as the initial condition and consider that, at the instant k = N, we
must have x(N) = 0 by contract:
N−1
X
0 = αN (−C) + αN−h−1 r (9.22)
h=0

n
X αn+1 − 1
Since the sum is the classic geometric series αi = , we get the installment expression
i=0
α−1

αN (α − 1) (1 + i)N
r= C = iC (9.23)
αN − 1 (1 + i)N − 1
An interesting observation comes out if we write the installment according to the evolution at a
generic time
r = −i x(k) + x(k + 1) − x(k) .
 
(9.24)
When k is small, x(k) is large (negative) and the term −i x(k) > 0 prevails. The situation reverses
near the end of the loan (k close to N). One can thus conclude that, in the case of constant
installment, we return more interest than capital at the beginning, and vice-versa at the end.

9.4 Positive linear systems and Markov chains

A linear discrete-time system is positive if it is represented by the equation

x(k + 1) = Ax(k), Ai j ≥ 0

Nonnegative matrices have remarkable properties, such as positivity of the associated system:

x(0) ≥ 0 ⇒ x(k) ≥ 0, ∀k > 0.

(the notation x ≥ 0 for a vector is to be intended component-wise xi ≥ 0). A further property is the
Perron-Frobenius theorem.

Theorem 9.1. The eigenvalue with the largest modulus (dominating eigenvalue, also denoted as
Frobenius eigenvalue) of a nonnegative matrix is nonnegative and real. In other words, if we order
the eigenvalues by magnitude,
λ1 = |λ1 | ≥ |λ2 | ≥ · · · ≥ |λn |
Moreover, if λ1 is strictly dominating, i.e., λ1 = |λ1 | > |λ2 | . . . then the corresponding eigenvector
(Frobenius eigenvector) has positive components.
144 Chapter 9. Examples of dynamical systems

Example 9.1. A simplified population evolution is the following. There are three classes of age.
Youth x1 , middle age x2 , and old age x3 .1 At each time step, each population unit passes to the
older stage. In the middle age the population is assumed to reproduce and give birth, in the average,
to α new young individuals. In the old age the population is assumed to reproduce and give birth,
in the average, to β new young individuals. Then

 0 α β 
 
A =  1 0 0 
 
 
0 1 0

It can be seen that for α + β > 1 the population diverges (λ1 > 1) while for α + β < 1 the population
extinguishes (λ1 < 1).
Assume, for instance, that α + β > 1. Then the free response is

x(k) = Ak x(0) = t1 s⊤
1 x(0)λ1 + t2 s2 x(0)λ2 + t3 s3 x(0)λ3 ≈ t1 s1 x(0)λ1
k ⊤ k ⊤ k ⊤ k

1 x(0) = γ1
for large k, because the dominant real mode diverges faster than the others. Then, since s⊤
is a scalar,
1 x(0))λ1 = t1 γ1 λ1
x(k) ≈ t1 (s⊤ k k

and we see that the asymptotic distribution of the population is given by the Frobenius eigenvector,
which is positive. For α = 1 and β = 1, the eigenvalues are 1.32472 and −0.66236 ± 0.56228 j.
The complex eigenvalues have modulus 0.86884, hence their modes converge. The Frobenius
eigenvector is t1 = [0.43016 0.32472 0.24512]⊤ so the asymptotic distribution is proportional
to these components: the components have been normalised so that their sum is 1, therefore we will
have 43016% of the population in the first class and so on.
The reader can imagine a more sophisticated model with division in n age classes, each with a
different contribution to birth.

9.4.1 Markov chains

Markov chains are a special class of positive linear system, in which the state vector components
represent probability values. Given a family of independent events

{E1 , E2 , . . . , En }

that are a partition of the certain event, the vector x has components

xi (k) = Pr[Ek ] (probability of event Ek occurring)

with
n
X
xi (k) = 1, xi (k) ≥ 0. (9.25)
i=1
We write the previous conditions as

1̄⊤ xi (k) = 1, x(k) ≥ 0 (9.26)

where 1̄⊤ = [1 1 . . . 1], and the vector inequality is intended componentwise.

Assume that the event probabilities are evolving in time, according to a linear equation. The as-
sumption is that, at each time, the probability distribution depends only on the previous distribution
1
After old age, the population is assumed to reach a retired happy-leaving status and is no longer considered in the
problem.
9.4. Positive linear systems and Markov chains 145

and that the transition probabilities are time-invariant. Assuming a discrete-time evolution, we get
a model of the form
x(k + 1) = Px(k). (9.27)
Since constraints (9.25) have to be satisfied all the times, we need that
n
X
Pi j ≥ 0, Pi j = 1, ∀ j
i=1

namely the matrix has to be nonnegative and for each column the elements must sum up to 1. This
is a necessary and sufficient condition for us to say that x(k) represents a probability:

1̄⊤ x(0) = 1 and x(0) ≥ 0, ⇒ 1̄⊤ x(k) = 1 and x(k) ≥ 0. (9.28)

The entries of a discrete-time Markov matrix P represent transition probabilities

Pi j = transition probability from state j to state i.

The dominant eigenvalue of a Markov chain is 1. Indeed from (9.28), the system is marginally
stable, so its eigenvalues are in the unit disk (border included). Also, the columns sum up to 1

1̄P = 1̄ = (1)1̄,

so 1 is an eigenvalue, and since it is real and dominant it is the Frobenius eigenvalue. Assume
now that 1 is strictly dominant: 1 > |λ2 | ≥ |λ3 | . . . . The Frobenius eigenvector t1 is very important
because it expresses the asymptotic probability distribution. Indeed, we can normalise the left
eigenvector to s1 = 1̄ and we have, by definition of probability, that the initial condition satisfies

1 x(0) = 1
s⊤

hence
x(k) = t1 s⊤
1 x(0)λ1 + t2 s2 x(0)λ2 + · · · + tn sn x(0)λn → t1 s1 x(0) = t1
k ⊤ k ⊤ k ⊤
(9.29)
| {z }
→0

1 x(k) = 1, necessarily, at the limit, s1 t1 = 1. Hence we have the following.

Since s⊤ ⊤

Proposition 9.1. The asymptotic probability distribution is given by the Frobenius eigenvector
normalised to 1.
In general, if the eigenvalue 1 is not strictly dominant, then there can be infinite asymptotic
distributions, for instance PId below, or no asymptotic distribution, for instance PBd (Buridan’s
donkey phenomenon) below:
" # " #
1 0 0 1
PId = PBd =
0 1 1 0

Example 9.2. (Buffer of a processing unit.) Consider the following processing unit (machine) in
which objects to be processed are waiting in a buffer. The buffer states are supposed to be
status description
1 0 parts, buffer empty
2 1 part, first slot occupied
3 2 parts, two slots occupied
4 3 parts, three slots occupied
5 4 part, buffer full
146 Chapter 9. Examples of dynamical systems

Assume that each part is processed in one time unit T = 1. We assume that probabilities are
assigned to the numbers of part arrivals:
probability description
P0 0 parts arrival
P1 1 part arrival
P2 2 parts arrival
Pk = 0 k ≥ 3 parts arrival
This corresponds to the graph in Figure 9.6. The Markov matrix P can be derived as follows
• p11 is the probability that the buffer is empty and remains empty. This happens if zero or one
parts arrive: p11 = P0 + P1 .

• p12 = P2 is the probability that, if the buffer is empty, the next time there will be one part.

• column j is associated with the state in which there are j − 1 parts. If 0 parts arrive the
buffer decreases by 1, if 1 part arrives the buffer remains in this state and if 2 parts arrive
the buffer increases by 1.

• p55 = P1 + P2 is the probability that the full buffer remains full, namely if it receives 1 or 2
parts, in the latter case we assume that one of the two parts is rejected.

buffer filling state

Figure 9.6: Buffer: graph of states and transitions.

The transition probabilities are reported in the following matrix governing the system

 x1 (k + 1)   P0 + P1 P0 0 0

    
0   x1 (k) 
 x2 (k + 1)   P2 P1 P0 0 0   x2 (k) 
     
 x (k + 1)  =  0 P2 P1 P0 0   x (k) 
 3     3 
 x4 (k + 1)   0 0 P2 P1 P0   x4 (k) 
x5 (k + 1) 0 0 0 P2 P1 + P2 x5 (k)
    

Given the probabilities P0 , P1 and P2 , the steady-state buffer distribution is determined by comput-
ing t¯, the nonnegative eigenvector associated with λ = 1. We can examine the special cases
• P0 = 1, P1 = 0, P2 = 0: the buffer becomes empty;

• P0 = 0, P1 = 0, P2 = 1: the buffer becomes full congested;

• P0 = 0, P1 = 1, P2 = 0: the asymptotic vector does not exist, because λ = 1 is not simple.

Figure 9.7 presents some other cases.
9.5. Vibrating systems 147

Figure 9.7: Buffer simulation for different values of P0 , P1 , P2 .

9.5 Vibrating systems

Consider Figure 9.8, which shows a structure (building) under seismic action. This system is
governed by the following equations

m1 ϑ̈1 (t) = −k1 ϑ1 (t) − k12 (ϑ1 (t) − ϑ2 (t)) − m1 a(t)




m2 ϑ̈2 (t) = −k12 (ϑ2 (t) − ϑ1 (t)) − k23 (ϑ2 (t) − ϑ3 (t)) − m2 a(t)

(9.30)



 m ϑ̈ (t) = −k (ϑ (t) − ϑ (t)) − m a(t)

3 3 23 3 2 3

A ground reference has been chosen. This reference is non-inertial. The variables ϑ1 , ϑ2 , ϑ3
represent the horizontal relative displacements of the floors. The coefficients m1 , m2 , m3 are the
masses of the floors while the coefficients k1 , k2 , k3 are elastic coefficients. The term k1 ϑ1 represents

the elastic force applied on the first floor from the ground, while the terms ±ki j ϑi − ϑ j (t) are
elastic forces between floors. The term mi a(t) is a virtual force due to ground acceleration (the
reference frame acceleration).
148 Chapter 9. Examples of dynamical systems

Figure 9.8: Building under seismic action.

This system can be written in the following matrix form:

 −(k1 + k12 )

     
 m1 0 0  k12 0   −m1 
 0 m2 0  ϑ̈(t) =  k12 −(k12 + k23 ) k23  ϑ(t) +  −m2  a(t) (9.31)
     
0 0 m3 0 k23 −k23 −m3

Note that no friction has been considered and we will be back on this later.
In general, a vibrating system can be written in the following form

M ϑ̈(t) = −Kϑ(t) + Ru(t) (9.32)

To explain these equations let us write them as follows: after pre-multiplication by ϑ̇⊤ (t) (the row
vector of the speeds), we have
!
d 1 ⊤ 1 ⊤
ϑ̇ (t)M ϑ̈(t) + ϑ̇ (t)Kϑ(t) =
⊤ ⊤
ϑ̇ (t)M ϑ̇(t) + ϑ (t)Kϑ(t) = ϑ̇⊤ (t)Ru(t)
dt 2 2 | {z }
| {z } supplied power
energy=kinetic+elastic

The previous equation means that the derivative of the total energy is equal to the supplied power.
Matrix M is always positive definite (indeed, the kinetic energy is Ec = 12 ϑ̇T (t)M ϑ̇(t) > 0
if the system is in motion) and the matrix K is semidefinite (because the potential energy E P =
2 ϑ (t)Kϑ(t) ≥ 0). In the particular case of the building under seismic action, matrix K is positive
1 T

definite.
To have a state representation we pre-multiply equation (9.32) by M −1 to get:

ϑ̈(t) = −M −1 Kϑ(t) + M −1 Ru(t) (9.33)

h iT
We can choose as state vector x(t) = ϑ(t) ϑ̇(t) . Then

ϑ(t) ϑ(t)
" # " #" # " #
d 0 I 0
= + u(t) (9.34)
dt ϑ̇(t) −M −1 K 0 ϑ̇(t) M −1 R

Denoting by S = M −1 K, matrix A is given by:

" #
0 I
A= (9.35)
−S 0
9.5. Vibrating systems 149

Let us study the modes of the system. The eigenvalues and eigenvectors satisfy
" #" # " #
0 I x1 x1
Ax = λx ⇒ =λ (9.36)
−S 0 x2 x2
From the first equation, we have x2 = λx1 . This tells us that the generic eigenvector associated with
h iT
matrix A it is of the type x = xT1 λxT1 . Using the second equation
−S x1 = λx2 = λ(λx1 ) ⇒ S x1 = −λ2 x1 (9.37)
Let us now consider the expression of S = and let T =
M −1 K M 1/2 .
Then, by pre-multiplying by
T = M , we have
1/2
h i h i
M −1/2 K M −1/2 M 1/2 x1 = −λ2 M 1/2 x1 ⇒ M −1/2 K M −1/2 y  Ŝ y = −λ2 y
Matrix Ŝ = M −1/2 K M −1/2 is positive definite and its eigenvalues are real and positive. Moreover
Ŝ = M −1/2 K M −1/2 = T −1 S T , hence its eigenvalues are those of S .
Denote the m eigenvalues of S as ω21 , ω22 , . . . , ω2m , then the n = 2m eigenvalues associated with
A are
ω2k = −λ2k ⇒ λk = ± jωk (9.38)
This means that vibrating systems without damping have in general oscillating modes e± jωt , namely,
sin(ωk t) and cos(ωk t) (9.39)
which are undamped oscillations. The coefficients ωk are named proper pulsations or resonance
pulsations. The numbers fk = ωk /(2π) are the proper frequencies.
Proposition 9.2. In an undamped vibrating system, the modes are of the form (9.39), where the
proper pulsations are ωk and ω2k are the eigenvalues of M −1 K.
Consider for instance the third floor when u(t) = 0. Its displacement ϑ3 (t) has the following
general expression:
ϑ3 (t) = α1 cos(ω1 t + φ1 ) + α2 cos(ω2 t + φ2 ) + α3 cos(ω3 t + φ3 )
where the terms αi and ϕi depend on the initial conditions of the system.
To explain the significance of the eigenvectors, consider initial conditions with ϑ̇(0) = 0 and
ϑ(0) = ϑ1 , where ϑ1 is an eigenvector of the matrix. The equation that governs the free response of
the system is
ϑ̈(t) = −S ϑ(t) ϑ(0) = ϑ1
⇒ −ω21 ϑ1 cos(ω1 t) = −S ϑ1 cos(ω1 t)
⇒ S ϑ1 = ω21 ϑ1
(just compute the second derivative of ϑ1 cos(ω1 t) to check the expression and take into account
that ϑ1 is an eigenvector of S ). This means that, if we initialise the building floors in positions equal
to the components of ϑ1 , with ϑ̇(0) = 0, the system shows only the oscillation mode corresponding
to an harmonic component with pulsation ω1 . Clearly any elastic motion (guitar string, vibrating
shells, acoustic instruments, vocal sound) is the sum of all harmonic components.
Finally, assume that there is friction to be taken into account. A typical model is
M ϑ̈(t) = −Kϑ(t) − H ϑ̇(t)Ru(t), (9.40)
where H is a positive semidefinite matrix including the friction coefficients. In this case the
eigenvalues are not imaginary, but have a negative real part. The energy balance is now
!
d 1 ⊤ 1 ⊤
ϑ̇ (t)M ϑ̇(t) + ϑ (t)Kϑ(t) + ϑ̇⊤ (t)H ϑ̇(t) = ϑ̇⊤ (t)Ru(t)
dt 2 2 | {z } | {z }
| {z } dissipated power supplied power
energy=kinetic+elastic
150 Chapter 9. Examples of dynamical systems

9.6 Linear electrical circuit

I1 IL

R1 R2
u(t) vC
C L

Figure 9.9: Example of a linear circuit.

An important class of linear systems is represented by linear electrical circuits. An electrical

circuit is represented by linear equations if it includes only resistive, capacitive and inductive
components, along with voltage and current generators. The dipole laws associated with these
components are the following.

• Resistors: v(t) = Ri(t)

• Inductors: Li̇(t) = v(t)

• Capacitors: i(t) = C v̇(t)

An electrical circuit is the connection of the previous components and of voltage and current
generators. The equations can be written by using Kirchoff laws. Typically, the inputs of these
systems are provided by the generators. As state variables, we can take

• the capacitor voltages;

• the inductors currents.

Consider, for example, Figure 9.9. Applying Kirchhoff loop law to the two loops, we obtain the
following two equations:

v = R1 i1 + vC
vC = R2 iL + Li̇L (9.41)

Conversely, applying Kirchhoff node law to the central node, we have the current balance

i1 = iL + iC = iL + C v̇C , (9.42)

where we have assumed the orientation in such a way that i1 is incoming while iL and iC are
outgoing. Replacing i1 with this expression, we get two first order differential equations

v̇C = − R11C vC − C1 iL + R11C v

(
(9.43)
i̇L = L1 vC − RL2 iL

Denoting by u(t) = v(t), x1 (t) = vC (t), x2 (t) = iL (t), we obtain the matrix representation

− R11C − C1
" # " #" # " 1 #
d x1 (t) x1 (t)
= 1 R2 + R1C u(t) (9.44)
dt x2 (t) L − L
x 2 (t) 0
9.6. Linear electrical circuit 151

The choice of the output of the system depends on the problem. For example, if we want to describe
this circuit as an admittance, the output is i1 (t):
!
1 1 1 1 1
y = iL + C v̇C = iL + C − vC − iL + v = − vC + v (9.45)
R1C C R1C R1 R1
Note that y depends on the derivative v̇c of vc , and this would lead to a non-proper system. The
equation can be modified by replacing v̇c with its expression (from the state equation) in order to
have a dependence on state variables only, and not on their derivatives. The output is then
h 1 i " x1 (t) # h 1 i
y(t) = − R1 0 + R1 v(t) (9.46)
x2 (t)
This system is proper, but not strictly proper.
We stress that some circuits might have some singularities. This is the case in which there are
cut-sets of inductors (three-phases systems) in which I1 + I2 + . . . Im = 0 or circuits of capacitors
in which v1 + v2 + . . . vm = 0. Moreover, for some choices of inputs and outputs, a circuit can be
non-proper. For instance, in a capacitor, if we consider the voltage as an input and the current as an
output, we have
d
ic (t) = vc (t) with transfer function ic (s) = [s]vc (s),
dt
which is not proper.
Going back to the example, let us consider the characteristic polynomial of matrix A:
! !
1 R2 1 R2
det(sI − A) = s +
2
+ s+ +1 (9.47)
R1C L LC R1
Note that the term that multiplies s is the trace of −A, while the term that multiplies s0 is the
determinant of A. The roots of this polynomial have negative real part, since it is a second order
polynomial with positive coefficients. This is an expected result, because the circuit is dissipative.
The study of the circuit can also be done using the Laplace transform. Assuming x(0) = 0 for
simplicity, the dipole laws described earlier become:
• v(s) = Ri(s) (resistors);
• v(s) = sLi(s) (inductors);
• i(s) = sCv(s) (capacitors).
Note that, in the special case where we assume s = jω, we are representing the dipoles in the
sinusoidal regime, according to the theory of frequency analysis for electrical circuits.
These relationships are linear in the Laplace transform domain and can be used as generalised
impedances. Then we can apply standard properties of series-parallel connections between compo-
nents. In the case of the circuit under test, the passive network can be considered as an impedance
having this expression:
1 1 r1 LCs2 + (CR1 R2 + L)s + (R1 + R2 )
Z(s) = R1 + ∥ (R2 + sL) = R1 + = (9.48)
sC sC + R2 +sL
1 LCS 2 + R2Cs + 1

It is easily verifiable that the impedance is the inverse of transfer function W(s) = C(sI − A)−1B =
Z −1 (s), as expected.
The representation in terms of state description may not be useful for circuits of the dimension
considered here, because such calculations can be performed quickly using the Laplace transform.
However, for more complex circuits, the state representation can be more useful, because it can be
easily implemented on a computer.
152 Chapter 9. Examples of dynamical systems

9.7 Operational amplifiers

y=a(u+−u− )
u− − u− +
+ + y +
u u

Figure 9.10: Operational amplifier.

Operational amplifiers are components whose symbol and equivalent circuit are shown in Figure
9.10. The factor a is a very large number, ideally tending to infinity. This electronic component is
practically only used in feedback as, for example, in Figure 9.11. The Kirchoff equation for the “−’

Z2

Z1
−
+ y

Figure 9.11: Operational amplifier in inverting feedback configuration.

node is, in terms of Laplace transform (omitting the dependence on s)

y − u− u − u−
+ = i− ,
Z2 Z1
the current entering the “-” terminal. Hence
u− u−
!
y u 1 y y
+ = + +i =
−
+ + i− ≈ 0,
Z2 Z1 Z2 Z1 a Z2 Z1
where we have used the fact that the amplification is roughly infinite, a ≈ ∞, and the input
impedance is infinite, hence i− ≈ 0. Then the approximate law is
y(s) u(s)
+ = 0, (9.49)
Z2 (s) Z1 (s)
where we have restored the dependence on s.
y
In the case of two resistances Z1 = R1 and Z2 = R2 , we have R2 + u
R1 = 0, from which:

R2
y(t) = − u(t). (9.50)
R1
Then the operational amplifier allows us to have an amplifier with tunable gain. The problem of
having a negative sign can be solved by properly connecting two amplifiers (in series).
9.8. Two-tank system 153

1
Consider, for example, Z1 = R and Z2 = , i.e., adopting a resistor and a capacitor. We get
sC
1
y(s) = − u(s), (9.51)
sCR
which, up to a multiplicative negative factor, is the transfer function of an integrator block. Similarly,
1
with Z1 = and Z2 = R, we obtain the transfer function of a differentiating block.
sC
Another interesting configuration for an operational amplifier is shown in Figure 9.12.

u1 R1 R0
u2 R2
−
um Rm + y

Figure 9.12: Feedback amplifier: adder.

The equation at node u− is:

u1 − u− u2 − u− um − u− y − u−
+ + ... + + = i− (9.52)
R1 R2 Rm R0
Noting that u− = y/a ≈ 0 and i− ≈ 0, we have
m
X R0
y=− ui (9.53)
i=1
Ri

The circuit is a “summing” device that provides as an output a linear combination of the inputs.
Again, the minus sign is not really a problem, one can use other devices to invert the sign.
Since we are able to build amplifiers, integrators and summing circuits, we are able to build any
transfer function.
There are many different amplifier configurations to realise transfer functions, which are
typically optimised so as to use the smallest number of components.

9.8 Two-tank system

Consider Figure 9.13, which represents a system consisting of two tanks connected by a pipe. There
is also an outgoing pipe from the lower tanks to a basin. A pump collects the water from the basin
and injects it into the upper tank. The balance equations describing the volumetric flow rate and
volume variations are:

v̇1 = q0 − q1
v̇2 = q1 − q2
v1 = (h1 − h10 )S 1
v2 = (h2 − h20 )S 2
154 Chapter 9. Examples of dynamical systems

Figure 9.13: Two-tank system.

where q0 , q1 , q2 are the input flow in the upper tank, the flow between first and second tank, and
the flow from the second tank to the basin, respectively. Based on fluid dynamics, we can also
consider the the following equations, concerning the dependence between the heights (potential
energy) and the flows.

h1 − h2 = α2 q21
h2 = β2 q22

From these, we obtain:

1p
q1 = h1 − h2
α
1p
q2 = h2
β

Note that the inertial effect of the water in the pipes is not considered, because it can be ex-
perimentally verified that it is negligible. The derivatives of the volume of water in the tanks
are
v̇1 = S 1 ḣ1 v̇2 = S 2 ḣ2

where S is the tank section surface. The resulting state equations are

1 p

ḣ1 (t) = u − h1 (t) − h2 (t)


αS 1




 1 p 1 p
 ḣ2 (t) = αS


 h1 (t) − h2 (t) − h2 (t)
βS

2 2
9.8. Two-tank system 155

where u = q0 /S 1 is the input, while h1 and h2 are the state variables. This is a nonlinear system. To
study the static problem, write the steady-state conditions ḣ1 (t) = 0 and ḣ2 (t) = 0, which implies a
flow balance q0 = q1 = q2
q0 1 √
 0=

S 1 −√αS 1 h1 (t) − h2 (t)

 0= 1 1 √
αS 2 h1 (t) − h2 (t) − βS 2 h2 (t)


After simple computations, it turns out that

h1 = S 12 (α2 + β2 )u2
h2 = S 12 β2 u2

where u = q0 S 1 is the constant input flow. Note that h1 > h2 because α and β are positive constants.
In fact, the proposed model is valid for h1 (t) > h2 (t). If we want to consider the possibility of
having (for example, during a transient) h2 (t) > h1 (t), then the model has to be changed as follows
p p
h1 (t) − h2 (t) =⇒ sign(h1 (t) − h2 (t)) |h1 (t) − h2 (t)|

so that the equilibrium pairs (h1 , h2 ) are a straight line in the plane h1 − h2 .
There are some problems with this system, which comes from a simplified model. First, the
levels are measured by floaters, connected to variable inductors. The measurement of h1 (t), returned
by the first floater, is highly affected by noise. Second, the pump is not continuously controlled, but
it is quantised. The input flow is controlled by two identical switching valves. Therefore there are
three possible input values:
u ∈ {0, ū, 2ū}
(if both valves are closed, one is open and the other closed, they are both open). In this case, to
achieve an arbitrary average input flow, one can use the pulse-width modulation PWM technique
shown in Figure 9.14.

αT T 2T

Figure 9.14: Example of segmentation.

Let us consider the problem of achieving a desired value udes , between 0 and U. The technique
consists in fixing a sampling time T , and applying the control

U for kT ≤ t < kT + αT
(
uquant =
0̄ for kT + αT ≤ t < (k + 1)T

with
udes
α=
U
156 Chapter 9. Examples of dynamical systems

The average value is the desired value udes . This technique, therefore, allows us to obtain a quantised
input with the same mean value of the desired continuous signal. To analyse the error, write this
input as
uquant = udes + w(t)
where, if udes is constant, w(t) is a periodic error of period T . This control works well, as long as
the dominating frequency ω = 2π/T of w(t) is considerably greater than the system bandwidth. In
the real equipment, we have ω ≈ 3, which is greater then the bandwidth.

9.9 Magnetic levitator

Consider the magnetic levitator shown in Figure 9.15. This system is described by a mechanical
equation of the form
mÿ(t) = mg − f (t)
The orientation is downward: increasing y means lowering the sphere. The term f (t) is the lifting
force, where f is a a function of the current and the distance and is approximately

i2 (t)
f (t) = k .
y2 (t)

Then we get
k i2
ÿ(t) = g − .
m y2
If we introduce also the electric equation of the magnet, driven by voltage V, the equations
describing the system become:

di(t)
L = −Ri(t) + V(t)
dt
i2 (t)
mÿ(t) = mg − k
y2 (t)

If we introduce x1 = y, x2 = ẏ and x3 = i as state variables, we get the system

ẋ1 (t) = x2 (t)




 2
k x3 (t)

=

ẋ (t) g −


 2 m x2 (t)
 1
 ẋ3 (t) = − R x3 (t) + 1 u(t)


L L

9.10 Cart–pendulum system

Consider the cart–pendulum system represented in Fig. 9.16 The variables are: y, the distance of
the cart from the reference and θ the angle of the pendulum. The cart equation is

M ÿ(t) = u − φẏ(t)

where M is the mass of the cart, φ is the friction coefficient and u(t) is the applied force. We neglect
the reaction of the pole on the cart which is much heavier. The pole equation is

mr2 θ̈(t) = mgr sin(θ(t)) + mr cos(θ(t))ÿ(t)

9.10. Cart–pendulum system 157

Figure 9.15: The magnetic levitator.

where the last term is due to the fact that the reference on the cart is not inertial. We replace ÿ(t)
from the first equation to get
g 1
θ̈(t) = r sin(θ(t)) + cos(θ(t))[u − φẏ(t)]
r Mr
If we apply the linearization at θ = 0 we get sin(θ) ≈ θ and cos(θ) ≈ 1., hence
g 1
θ̈(t) = θ(t) + [u − φẏ(t)]
r Mr
Denoting by x1 = θ, x2 = θ̇, x3 = y and x4 = ẏ, we get ẋ(t) = Ax(t) + Bu(t), where
   
 0 1 0 0   0 
 α 0 0 −µφ 
  µ 
A =   , B =  
 0 0 0 1   0 
ν

0 0 0 −νφ
√
where µ = 1/M, ν = 1/(Mr) and α = g/r. This system is unstable: its poles are 0, −νφ and ± α.
It is reachable and it can be stabilized.
Real experiments show that pole assignment design is not suitable for the real problem due to
the lack of robustness. Indeed the term φ is uncertain and time–varying. To analyze the situation,
write
φ = φ0 + ∆
where φ0 is the nominal value of φ and w is uncertain for which we assume the following bound
|∆| ≤ δ
158 Chapter 9. Examples of dynamical systems

m
θ
r
M
u

Figure 9.16: The cart–pendulum

Write the model as

ẋ(t) = A0 x(t) + BH∆ + Bu(t)
with  
 0 1 0 0 
 α 0 0 −µφ0  h i
A0 =   , and H= 0 0 0 1
 0 0 0 1 

0 0 0 −νφ0
We consider now an optimal control approach to ensure robustness as follows.

• Fix Q positive definite and R = 1 and solve the Riccati equation

A⊤ P + PA − PBB⊤ P + Q

and consider the optimal control

uopt = −B⊤ Px

• Apply the control u = uopt + ur , where ur is a robustifying term that will be specified soon, to
get the system
ẋ(t) = [A0 − B⊤ P]x(t) + BH∆(t) + Bur (t)
namely
ẋ(t) = Āx(t) + BH∆(t) + Bur (t)
Note that the Lyapunov equation holds

Ā⊤ P + PĀ = −[Q + PBB⊤ P] = −Q̄

• Take
ur (t) = −γB⊤ Px
where γ is selected as explained in the Lyapunov redesign section.

It turns out the the optimal control stabilizes the cart-pendulum system.
Appendix A

Elements of Mathematics

In this section we will briefly summarise some basic concepts of mathematics that are required
for the study of dynamical systems. In particular, the concepts of linear algebra will be reported,
since it is a fundamental language for the formalisation of important concepts. Once again, this
manuscript is not a text book, but a guideline: this appendix is essentially listing the preliminary
notions that are necessary to understand the course (which should have been thoroughly studied
and learned in previous courses).
Mathematics in engineering has three fundamental roles:

• it is a language to describe models;

• it is a language to express laws and properties;
• it provides computational tools.

In this course we need to keep in mind all of these aspects.

A.1 Vector spaces

Linear algebra, including vector space algebra and matrix theory, is essential in system and control
theory. Without linear algebra it is impossible to formulate and to solve fundamental problems in
these fields.
A vector space X defined over a field C is a set in which the operations of multiplication by a
scalar and of sum are defined and have (among others) the following properties:

1. if x ∈ X and λ ∈ C, then λx ∈ X;

2. if x1 , x2 ∈ X, then x1 + x2 ∈ X.

We will consider vector spaces defined over the field R. As is known, there are some axioms
that must be satisfied.1

Example A.1. The following sets are vector spaces.

• The points (geometric vectors) in the plane or in the space.

• The points on a 2D-plane in the 3D-space including the origin.
• The set of polynomials of degree ≤ n.
• The set of n-tuple (α1 , . . . , αn ).
1
We briefly remind these axioms. For u, v ∈ X and α, β ∈ C: 1) u + (v + w) = (u + v) + w; 2) u + v = v + u; 3) (zero
element) ∃ 0 ∈ X: v + 0 = v for all v ∈ X; 4) (inverse element) ∀ v ∈ X ∃ − v ∈ X: v + (−v) = 0; 5) α(β v) = (αβ) v; 6)
1 v = v, where 1 is the multiplicative identity in C, 7) α(u + v) = αu + αv.

159
160 Chapter A. Elements of Mathematics

• The set of solutions of a linear homogeneous differential equation aÿ(t) + bẏ(t) + cy(t) = 0.
• The set of polynomials of degree n.
• The set of polynomials of degree n for which p(0) = 0.

The following sets are not vector spaces.

• The points on a 2D-plane in the 3D-space not including the origin.

• The set of solutions of a linear non-homogeneous differential equation aÿ(t) + bẏ(t) + cy(t) = 1.
• The set of polynomials of degree n for which p(0) = 1.

Definition A.1. (Generator set.) A set of vectors x1 , x2 , . . . , xm ∈ X is said to be a generator set if

any element x ∈ X can be written as a linear combination of these vectors, that is,

x = λ1 x1 + λ2 x2 + · · · + λm xm

for some coefficients λi ∈ R.

Definition A.2. (Set of linearly independent vectors.) Vectors x1 , x2 , . . . , xn ∈ X are said linearly
independent if the condition
λ1 x1 + λ2 x2 + · · · + λm xm = 0

implies that λi = 0 for each i (in simple words, if the linear combination with all coefficients equal
to zero is the only one that gives the 0 vector).

Definition A.3. (Basis.) A set of vectors x1 , x2 , . . . , xn ∈ X is called basis if it is a set of generators

that are linearly independent.

Theorem A.1. Given a basis, x1 , x2 , . . . , xn ∈ X, each vector x can be uniquely written as

x = λ1 x1 + λ2 x2 + · · · + λn xn ,

namely, the coefficients λi are uniquely determined by x.

Given a vector space X, its basis is not unique. The following fundamental property holds.

Theorem A.2. All bases of X have the same number of elements.

This number is invariant: we can replace a basis with another one having the same number of
elements. This has a considerable importance.

Definition A.4. The cardinality (the number of elements) of any basis of X is called dimension of
the vector space X:
Dim(X).

Example A.2. (Polynomials of degree 4.) Consider the set of polynomials of degree 4, having the
form:
p(s) = a4 s4 + a3 s3 + a2 s2 + a1 s1 + a0 .

The 0 polynomial is p(s) ≡ 0, with all zero coefficients. The polynomials s and s3 + 1 are linearly
independent. The polynomials s4 , s3 , s2 , s1 , 1, s3 + 1 are a set of generators, the set s4 , s3 , s2 , s1 ,
1 is a basis. We can also take (s + 1)4 , (s + 1)3 , (s + 1)2 , (s + 1)1 , 1, which is a different basis, but
has the same number of elements. The reader should prove the above statements as an exercise.
A.1. Vector spaces 161

Example A.3. (Euclidean space.) Consider the space of vectors in R3 , with 3 real components, of
the type x = [α, β, γ]⊤ . This is a vector space of dimension 3. A possible basis is
     
 1   0   0 
 0  ,  1  ,  0  ,
     
0 0 1

which is called canonical basis.

The set X = {0}, including the 0 vector only, is a vector space, and it is named trivial space. It
has dimension 0.

Definition A.5. Subspace A subset X1 of X which is also a vector space is called a subspace.

Example A.4. (Subspaces.) If X is the space of polynomials of degree 4, then the polynomials of
degree 2 form a subspace. In the 3D-space, any plane including the origin is a subspace. Precisely,
in the space of vectors (x1 , x2 , x3 ) with coordinates x1 , x2 and x3 , the subset {(x1 , x2 , x3 ) : 2x1 −
x2 + x3 = 0} is a subspace.

Given two subspaces X1 and X2 , the following operations are defined and provide subspaces.

• Intersection: \
X1 X2 = {x : x ∈ X1 , and x ∈ X2 }

• Sum:
X1 + X2 = {x = x1 + x2 , x1 ∈ X1 , and x2 ∈ X2 }

Definition A.6. (Direct sum.) The sum X of two subspaces is said direct if any element x ∈ X can
be written as x = x1 + x2 , where x1 ∈ X1 and x2 ∈ X2 are uniquely determined.

It can be shown that the sum is direct if and only if

\
X1 X2 = {0}.

When the sum is direct we use the notation

M
X = X1 X2 .

Given a basis v1 , v2 , . . . , vn , each vector of the vector space can be written as

v = x1 v1 + x2 v2 + · · · + xn vn ,

where the numbers x1 , x2 , . . . , xn are called the components of v. For a fixed basis, they are
uniquely determined. Therefore, if the basis is fixed, there is a correspondence between the vectors
v and n-tuples of numbers (x1 , . . . , xn ):
 
 x1 
 x 
v ↔  2  ∈ Rn
 : 
xn

where Rn is, by definition, the set of all n-tuples. For example, in the 4-degree polynomial space,
with basis s4 , s3 , s2 , s, 1, the polynomial x4 + 5x + 3 is associated with the tuple (1, 0, 0, 5, 3).
Therefore, without loss of generality we can consider the space Rn of n-tuples. When necessary,
we will consider Cn , the space of complex n-tuples.
162 Chapter A. Elements of Mathematics

A.2 Matrices
Given two spaces X and Y, we say that a map (or operator, or application) L : X → Y is linear if
for all xa , xb ∈ X and α, β ∈ R, (or C) we have

L(αxa + βxb ) = αL(xa ) + βL(xb ).

Examples of linear maps are: from the 4th degree polynomial space to the 3rd degree polynomial
space, the derivative; between 3D spaces, a rigid rotation (with respect to an axis that includes the
origin).
We have seen that, for a fixed basis, any space of dimension n is isomorphic to Rn . Hence,
we can focus on operators from the space Rn to the space Rm . A linear map from Rn to Rm is
represented by a matrix A of size m × n. Let y ∈ Rm and x ∈ Rn . Then, we have

y = Ax,

where
y1   a11 a12 . . . a1n x1
     
   
y2   21 a22 . . .
   a a2n   x2 
..  =  .. .. .. .. ..  .
   
.   . . . . .
  
   
am1 am2 . . .
     
ym amn xn
Hence, the coefficients yi are computed according to the rule
n
X
yi = ai j x j .
j=1

Example A.5. The derivative as an operator from the 3rd degree polynomial space to the 2nd
degree polynomial space, with basis 1, s, s2 , s3 and 1, s, s2 respectively, is represented by the
matrix  
 0 1 0 0 
D =  0 0 2 0  .
 
 
0 0 0 3

A B
Y Z X
Figure A.1: Composition of linear maps.

The composition of two linear operators represented by matrices A and B

Rn |{z}
→ Rm |{z}
→ Rp
A B

is a linear operator. Given x ∈ Rn , y ∈ Rm and z ∈ R p , if y = Ax and z = By, then

z = By = B(Ax) = (BA)x = Cx.

If B has size p × m and A has size m × n, matrix C has size p × n and its generic element is given by
m
X
[C]i j = ci j = bik ak j , i = 1, 2, . . . , p, j = 1, 2, . . . , n.
k=1
A.2. Matrices 163

Definition A.7. (Image.) The image (or range) of a m × n matrix is the set

Ra(A) = {y : y = Ax, x ∈ Rn } ⊆ Rm .

Such a set is a subspace of Rm . Given a matrix M, its columns form a set of generators for its
image, though not necessarily a basis.

Definition A.8. (Kernel.) The kernel (or nullspace) of a m × n matrix is the set

Ker(A) = {x : Ax = 0} ⊆ Rn .

Such a set is a subspace of Rn .

Example A.6. Consider the matrix

 
 2 1 1 
0 1 −1
 
 
 3 1 2 
M =  
 1 1 0 

1 1 0
 
 
2 1 1

Its columns are generators for the space Ra(M). We notice that, if we subtract the second column
from the first one, we get the third column. Therefore there are two linearly independent columns
only (we choose for instance the first two), which are a basis of Ra(M) (a subspace of the 4D space
R4 ). The kernel of matrix M is given by all the vectors of the form
 
 1 
x = α  −1  , α ∈ R,
 

−1


and is a subspace of R3 .

Example A.7. Let us compute the kernel of the following matrix:

 
 1 2 0 1 
M =  1 −1 1 1 
 
 
2 1 1 2

We have to find all the solutions x of Mx = 0, the null space. To get a basis of such a subspace, we
can subtract the first row from the second and the first row multiplied by two from the third
 
   x1 
 1 2 0 1   
  x2 
0 −3 1 0   x3  = 0

 
0 −3 1 0  
x4

and then subtract the second row from the third

 
 x 
 1 2 0 1   1 

  x2 
 0 −3 1 0   x  = 0

0 0 0 0  3 
 
x4
164 Chapter A. Elements of Mathematics

The solution is unchanged. The third row is neglected, since it is zero. We parameterise x3 = α and
x4 = β and we derive x2 = α/3 and x1 = −β − 2x2 = −β − 2α/3. Finally we have that any vector in
the kernel can be written as
 β − 23 α 
 2 
 − 3 
   
 1 
 1 α   1 
 = α  3  + β  0  = αv1 + βv2
 
xK =  3
 α   1   0 
    
−β 0 −1
The two vectors v1 and v2 form a basis for the kernel of matrix M.
Definition A.9. (Transposed matrix.) Given the m × n matrix A, its transposed matrix A⊤ is the
n × m matrix whose elements are
[A⊤ ]i j = a ji .
The transpose of a product is the product of the transposed matrices in reverse order:
(A B C . . . )⊤ = (. . . C ⊤ B⊤ A⊤ ).

A.2.1 Determinant
For a square n × n matrix, the determinant is a function defined as follows. Given the fundamental
tuple of integers (1, 2, . . . , n) and the generic permutation
p = (p1 , p2 , . . . , pn )
we say that p is of even class (class = 2) if the number of element swaps to get the fundamental is
even, of odd class (class = 1) in the opposite case.
Let P be the set of all permutations of the fundamental and cl(p) ∈ {1, 2} the class of a
permutation p. Then the determinant (according to the Leibniz formula) is defined as
X
det(A) = (−1)cl(p) a1p1 a2p2 . . . anpn .
p∈P

It turns out that X

det(A) = det(A⊤ ) = (−1)cl(p) a p1 1 a p2 2 . . . a pn n .
p∈P
The following formulae (Laplace formulae) can be used to compute the determinant of a matrix,
by rows or by columns. Denote by Âi j the complement matrix to entry (i, j), namely, the square
submatrix obtained from A by deleting ith row and and the jth column. We have that
Xn
det(A) = (−1)i+ j ai j det(Âi j ), for each i
j=1
Xn
det(A) = (−1)i+ j ai j det(Âi j ), for each j
i=1
A square matrix having zero (non-zero) determinant is called singular (non-singular).
Fundamental properties of the determinant are the following (A, B are n × n matrices):
• det(AB) = det(A) det(B);
• det(I) = 1;
• Let Ãλ be the matrix achieved from A by multiplying a row or a column by λ: then, det(Ãλ ) =
λ det(A);
• det(µA) = µn det(A);
• det(A) , 0 if and only if the row (columns) are a basis of Rn .
A.2. Matrices 165

A.2.2 Rank
The columns or the rows of a matrix are vectors in Rn and Rm . The following fundamental theorem
holds.

Theorem A.3. Given a generic matrix A ∈ Rm×n , the maximum number m̂ of linearly independent
rows is equal to the maximum number n̂ of linearly independent columns.

Definition A.10. Rank. The number n̂ = m̂ = r̂ is called rank of matrix A and is denoted by

Rank(A).

It turns out that Rank(A) is equal to the dimension of the largest square submatrix of A having
nonzero determinant.
If the matrix m × n has rank equal to the maximum possible (which is the minimum of m and n,
number of rows and columns) is said to be a full rank matrix, or to have full rank. In particular, a
square matrix has full rank if and only if its determinant is non-zero.
The following relations are useful when determining the rank of a matrix or when computing
its image or its kernel.

Theorem A.4. For a m × n matrix A, the following relations apply:

n = Rank(A) + Dim(Ker(A))
m = Rank(A) + Dim(Ker(A⊤ )).

A.2.3 Linear systems of equations

Given a m × n matrix A and a vector b ∈ Rm , consider the system of equations

Ax = b.

Theorem A.5. Given the system Ax = b, a necessary and sufficient condition for the existence of a
solution is that
Rank([A|b]) = Rank(A),
namely, the rank does not increase if b is added as a new column to A.

If a solution exists, then the set of all solutions is

x̄ + x̃,

where x̄ is any solution and x̃ ∈ Ker(A) is an element of the kernel.

In the case of a square matrix A (n × n) and a vector b ∈ Rn , consider the system of equations

Ax = b.

Since A is square, the following properties are equivalent:

• the system admits a unique solution for all b;

• A has rank n;

• A has a trivial kernel (the kernel is {0});

• det(A) , 0.
166 Chapter A. Elements of Mathematics

If det(A) = 0, there can be either no solutions or infinite solutions, depending on b. The system
Ax = 0 admits non-zero solutions if and only if det(A) = 0.

Example A.8. Consider the following matrix and vector:

" # " #
1 1 b1
A= b=
1 1 b2

Then a solution exists if and only if b1 = b2 . If (x1 , x2 ) is a solution, then (x1 + α, x2 − α) is a

solution as well.

Definition A.11. (Inverse matrix.) Given a square matrix A such that det(A) , 0, the matrix
B = A−1 is its inverse matrix if for every x

y = Ax, ⇒ x = By.

The inverse matrix has then the property

AB = BA = I

where I is the identity matrix. The following formula holds to compute the elements Bi j of B = A−1 :

det(Â ji )
Bi j = (−1)i+ j .
det A

where  ji is the complement of ai j . Note that i and j are exchanged in A ji in the formula!!
The inverse of a product is the product of the inverses in reverse order:

(A B C . . . )−1 = (. . . C −1 B−1 A−1 ).

Finally, we have
1
det(A−1 ) = .
det(A)

A.2.4 Partitioned matrices

In many occasions we have to deal with matrices of matrices, namely matrices whose elements
are themselves matrices or vectors. For example, a matrix A can be thought as a “row” of column
vectors
A = [ ā1 ā2 . . . ām ]
or as a column of row vectors  
 b̂1 
b̂2
 
B =   .
 
..
 . 
 
b̂n
More in general, a matrix can be of the type

 A11 A12 . . . A1n

 

 21 A22 . . .
 A A2n 
A =  . .. .. ..

 .. . . .


Am1 Am2 . . .
 
Amn
A.2. Matrices 167

where Ai j are submatrices. Of a particular interest is the product rule: given two partitioned matrices

 A11 A12 . . . A1n  B11 B12 . . . B1m

   
 
 21 A22 . . .  21 B22 . . .
 A A2n   B B2m 
A =  . .. .. ..  , B =  .. .. .. ..
 
 .. . . .  . . . .

 
Am1 Am2 . . . B p1 B p2 . . .
   
Amn B pm

under the assumption that the number of rows in each submatrix Ak j matches the number of columns
in the submatrix Bik , the matrix product is

 C11 C12 . . . C1n

 

 21 C22 . . .
 C C2n 
C = BA =  . .. .. ..

 .. . . .


C p1 C p2 . . .
 
C pn

where the same product rule holds

m
X
Ci j = Bik Ak j , i = 1, 2, . . . , p, j = 1, 2, . . . , n.
k=1

A.2.5 Elementary row and column tranformations

Row and column operations are fundamental to determine rank, kernel, image space of matrices, as
well as solutions of linear systems of equations.
Given a n × m matrix, we call elementary row or column transformation2 one of the following
operations:

• multiplication of one row (column) by a non-zero scalar;

• permutation of two rows (columns);

• adding one row (column) to another row (column) multiplied by a scalar.

Row operations have the following properties

• they leave the rank unchanged;

• they leave the kernel unchanged;

• they leave the set of solutions unchanged if the matrix M = [A b] represents a system of
linear equations Ax = b.

Column operations have the following properties

• they leave the rank unchanged;

• they leave the image space unchanged.

A m × n matrix M is an upper-staircase matrix if row i has νi non-zero elements in the first

positions and
if νk < n, then νk+1 > νk ,
2
This type of transformations is not the similarity transformation presented in Subsection A.2.6.
168 Chapter A. Elements of Mathematics

while if νk = n, then νh = n, for h ≥ k. For instance

 
 1 2 0 1 1 
 0 2 1 0 1 
 
 0 0 0 2 1 
 
0 0 0 0 0

is an upper-staircase matrix. A m × n matrix M is row-reduced if

• it is an upper staircase matrix;

• the first non-zero element is equal to a 1, namely Mi,νi +1 = 1, unless the row is zero;

• in each column where a pivot (unitary) element exists, all other elements are 0.

For instance the following matrix is row-reduced.

 
 1 0 3 0 1 
 0 1 1 0 1 
 
 0 0 0 1 1 
 
0 0 0 0 0

Each m × n matrix can be row-reduced by means of elementary row operations. The procedure
works as follows.

Procedure A.1.

1. Set i := 1, j := 1.

2. If, in the jth column, there exists a non-zero element in position k ≥ i, (including the ith
element) take it in position i (by row permutation). Otherwise consider the next column:
j := j + 1.

3. Multiply row i by 1/Mi j . Now the element i, j has become a pivot (it is equal to 1).

4. Add row i to all other rows k , i after multiplication by −Mk j , to set to zero all elements of
column j (excluding the pivot Mi j = 1).

5. Set i := i + 1, j := j + 1 and go to Step 2 to repeat the procedure until either i = m or all

elements Mk,h are 0 for k ≥ i and h ≥ j.

Example A.9. Given the matrix M below on the left, set i = 1 and j = 1. M11 = 1 becomes pivot.
perform the following operations: Row(2) := Row(2) − Row(1), Row(3) := Row(3) − 2Row(1),
achieving the matrix M ′ on the right,

 1 1 −1 1 0 −1  1 1 −1 1 0 −1 
   
 

 
 1 1 2 −1 0 1   0 0 3 −2 0 2 
M =   ⇒ M ′ = 
 0 0 −1 0 1 2   0 0 −1 0 1 2 

 
2 2 0 0 1 2 0 0 2 −2 1 4

Set i = 2 and j = 2. The element M22 ′ is 0 as all the other elements of the second column.

Then consider the third column, j = 3. The element M23 ′ is non-zero. Make it unitary (pivot)

by division of the row Row(2) := Row(2)/M23 , achieving the matrix M ′′ below on the left. Then
′
A.2. Matrices 169

operate: Row(1) := Row(1) − Row(2) · M13 ′′ , Row(3) := Row(3) − Row(2) · M ′′ , Row(4) :=

33
Row(4) − Row(2) · M43 , achieving the matrix M ′′′ below on the right.
′′

 1 1 −1 1 0 −1  1/3 0 −1/3 

   
  1 1 0

 0 0 1 −2/3 0 2/3  ⇒ M ′′′ =  0 0
 1 −2/3 0 2/3 
M ′′ = 

 0 0 −1 0 1 2   0 0 0 −2/3 1 8/3 

  
0 0 2 −2 1 4 0 0 0 −2/3 1 8/3

In the third column M23′′′ = 1 is non-zero (pivot). Let i = 3 and j = 4. The element M ′′′ , 0. Operate:
34
Row(3) := Row(3)/M34 ′′′ , so the element becomes 1 (pivot), achieving the matrix M ′′′′ below on the

left. Operate the following: Row(1) := Row(1) − Row(3) · M14 ′′′′ , Row(2) := Row(2) − Row(3) · M ′′′′ ,
24
Row(4) := Row(4) − Row(3) · M44 ′′′′ , achieving the matrix M ′′′′′ below on the right.

0 −1/3 
   
 1 1 0 1/3
  1 1 0 0 1/2 1 

 0 0 1 −2/3 0 2/3   0 0 1 0 −1 −2 
M ′′′′ =   ⇒ M ′′′′′ = 
 0 0 0 1 −3/2 −8/2   0 0 0 1 −3/2 −8/3 

 
0 0 0 −2/3 1 8/3 0 0 0 0 0 0

Setting i = 4 and j = 5, M45′′′′′ = 0 and there are only zero elements corresponding to greater or

equal indices: STOP.

The matrix is row-reduced. The underlined elements are pivot. The number of nonzero rows is the
rank, in this case equal to 3.

The procedure is useful to find the kernel and the solution of a system of linear equations.

Example A.10. Consider the system Ax = b, with

 
x1
 1 1 −1   −1 
     
1 0  
 x2
 1 1 2 −1 0     1 
Ax =  
 x3  = 
  2  = b

 0 0 −1 0 1  

 x4   
2 2 0 0 1  2
x5


Consider the complete system matrix

 1 1 −1 1 0 −1 
 

 1 1 2 −1 0 1 
[A|b] = 
 0 0 −1 0 1 2 


2 2 0 0 1 2

A necessary and sufficient condition for the existence of solutions is

rank(A) = rank([A|b])

equivalent to
rank(Â) = rank([Â|b̂])
where [Â|b̂] is the transformed matrix after row-reduction. Checking the existence of solutions is
easy. Indeed we arrive to a form of the type
" #
Â1 b̂1
[Â|b̂] =
0̂ b̂2
170 Chapter A. Elements of Mathematics

and the rank equality is equivalent to b̂2 = 0, which is then a necessary and sufficient condition for
the existence of solutions. In the example we get
 
 x1 x2 x3 x4 x5 b̂ 
1 1 0 0 1/2 1
 
 

 0 0 1 0 −1 −2 


 0 0 0 1 −3/2 −4 

0 0 0 0 0 [0]

and the 0 evidenced in brackets means that a solution exists.

How can we find all solutions of Ax = b? We remind that, if x̄ is any solution (A x̄ = b), the
solution set is
S ol(A, b) = {x = x̄ + x̃, where x̃ ∈ Ker(A)}.

• A particular solution x̄ is obtained by setting to 0 all elements xi which do not correspond to

columns with pivots and computing the remaining, corresponding to pivots, which are equal
to the corresponding entry of the last column.

• The kernel basis is the solution we get after setting to 0 the last column (the known terms).
This is achieved as follows. We consider the columns without pivots: each of them is
associated with a vector of the kernel basis. For each non-pivot column, we set to 1 the
corresponding component xi and to 0 all the elements corresponding to other non-pivot
columns. We then simply derive the elements corresponding to pivot columns.

In the specific case, the pivots are in the positions 1, 3 and 4. The non-pivot positions are 2 and 5.
To get a particular solution we put x2 = 0 and x5 = 0, then x1 = 1, x3 = −2 and x4 = −4,
namely h i
x̄⊤ = 1 0 − 2 −4 0

A kernel basis is achieved by setting to 0 the last column (zeroing the elements of column b̂). The
first vector of the basis is achieved by setting x2 = 1 and x5 = 0 and computing the other terms
h i
v⊤
1 = 1 −1 0 0 0

The second vector, achieved by setting x2 = 0 and x5 = 1, is

h i
2 = −1/2
v⊤ 0 1 3/2 1 .

The two vectors v1 and v2 form a basis for the 2-dimensional kernel.

In the case of square non-singular matrices, row reduction is useful to compute the inverse as
well. Let P be a square invertible matrix. Its inverse is achieved as follows.

Procedure A.2. Inverse computation

1. Form the matrix

M = [P|I]
where I is the identity of the proper size.

2. Reducing the matrix by row we get

M̂ = [I|Q]
(since P is invertible, there will be no zero rows).
A.2. Matrices 171

3. Then P−1 = Q.

Example A.11. Let

 
 1 0 1 1 0 0 
M = [P|I] =  −1 1 0 0 1 0
 
 
1 1 1 0 0 1
After row-reduction we get
 
 1 0 0 −1 −1 1 
[I|Q] =  0 1 0 −1 0 1
 
 
0 0 1 2 1 −1

where Q = P−1 .

The procedure can be applied to columns. Matrix M is column-reduced if M ⊤ is row-reduced.

A basis of the image space is given by the non-zero columns of the column-reduced matrix.

Remark A.1. If the row and column operations so far described are applied to square matrices,
they do not preserve eigenvalues, eigenvectors and characteristic polynomial. The similarity
transformations presented in Section A.2.6 have this property.

A.2.6 Basis transformation

In representing the space Rn , we implicitly refer to the canonical basis
#⊤
0 ... 0...
"
0 1 0
ek = |{z}
k−th position

(ek has all 0 elements except the kth, which is equal to 1). Indeed we have

x1
 
 
 x2  X n
x =  ..  = x je j.
 
 .  j=1
 
xn

Now suppose we want to write the vector x as the convex combination of vectors forming a new
basis, t1 , t2 , . . . , tn , grouped in the matrix

T = [t1 t2 . . . tn ].

We must have that

x̂1
 
 
n
X h i  x̂2 
x= t j x̂ j = t1 t2 . . . ..  = T x̂,

tn 
j=1
 . 
 
x̂n
where x̂ j are the components with respect to the new basis. These components, collected in the
vector x̂, can be determined as follows:

x = T x̂ ⇒ x̂ = T −1 x. (A.1)
172 Chapter A. Elements of Mathematics

Given a linear map y = Ax, the basis changes x = T x̂ and y = S ŷ lead to the transformation
ŷ = S −1 AT x̂
We consider mainly3 applications of a space to itself, for which the transformation is represented by
a square matrix A. In this case the new basis should be the same (x = T x̂ and y = T ŷ). The linear
application is then transformed as follows:
ŷ = T −1 AT x̂ = Â x̂. (A.2)
In this case the transformation is called similarity transformation. A similarity transformation has
fundamental property, we will show later. In particular we have the following. Any power Ak of Â,
with k positive integer, is transformed as A
Âk = T −1 −1
| AT T AT{z . . . T AT
−1
} = T A T.
−1 k

k times

If A is invertible, then the property applies also for negative k. This property allows us to define
analytic functions of matrices.

A.2.7 Eigenvalues and eigenvectors

Definition A.12. (Eigenvalues and Eigenvectors.) Given a square matrix A, a vector x , 0 and a
complex scalar λ such that
Ax = λx,
are called respectively eigenvector and eigenvalue of A.
The previous equation can be written in the form
(A − λI)x = 0.
For this equation to admit a nontrivial solution it is necessary and sufficient that the matrix (A − λI)
is singular (equivalently, does not have full rank):
det(λI − A) = 0.
This is a polynomial function of λ,
det(λI − A) = p(λ) = λn + an−1 λn−1 + · · · + a1 λ1 + a0 ,
called characteristic polynomial of A. It is a monic polynomial, namely, the coefficient of the
highest degree is equal to 1. The eigenvalues are therefore the roots (possibly complex) of this
polynomial. Note that complex roots can be present even if A, hence p(λ), are real.
The eigenvector x , 0 that satisfies the relation is also called right eigenvector. A vector z , 0
such that
z⊤ (A − λI) = 0
is called left eigenvector. If we transpose the relationship, we have
(A⊤ − λI)z = 0,
hence left eigenvectors are the right eigenvectors of the transpose. Note that the eigenvalues of A⊤
and A are the same, because det(λI − A) = det((λI − A)⊤ ) = det(λI − A⊤ ).
The set of the eigenvalues of A, which is the set of the roots of p(λ), is called spectrum of A
and is denoted by
σ(A) = {λ ∈ C : p(λ) = 0}.
3
Indeed always, with the exception of malicious exercises given in the tests.
A.2. Matrices 173

Example A.12. Let " #

−1 2
A=
−1 −4
Then
det(λI − A) = λ2 + 5λ + 6
The eigenvalues are λ1 = −2 and λ2 = −3. The corresponding eigenvectors are the solution of

+1 2 +2 2
" #" # " #" #
x1 x1
(A − λ1 )x = = 0 and (A − λ2 )x = =0
−1 −2 x2 −1 −1 x2

Then possible eigenvectors (there are infinite of them) are

" # " #
2 −1
t1 = and t2 = .
−1 1

Example A.13. The matrix " #

0 2
A=
−2 0
is real but its eigenvalues are ± j2, hence complex (imaginary). The eigenvectors are [1 ± j]⊤ .
Note that the components of the two vectors are conjugate.

Remark A.2. The eigenvectors are defined up to multiplication by a scalar. If x , 0 is an

eigenvector, αx , 0 is an eigenvector as well.

Remark A.3. If A is real, its eigenvalues have the conjugate property: λ ∈ σ(A) implies that the
conjugate λ∗ ∈ σ(A). This is trivial if λ is real. If, for instance −1 + 2 j ∈ σ(A), then −1 − 2 j ∈ σ(A).
Finding a complex eigenvector is harder, but once the eigenvector v of λ is known, the eigenvector
associated with λ∗ is v∗ and comes for free.

Assume, for the moment, that the matrix A admits n distinct eigenvalues, namely that the roots
of p(λ) are distinct
σ(A) = { λ1 , λ2 , . . . , λn }.
In this case the corresponding eigenvectors t j are linearly independent. Call T the matrix whose
columns are the eigenvectors h i
T= t1 t2 . . . tn .
Then the following relation holds:

λ1 0 . . . 0
 
 
h i h i  0 λ2 . . . 0 
t1 t2 . . . = t1 t2 . . . .. .. . . ..  .

A tn tn 
 . . . . 
0 0 ... λn
 

If we define S  T −1 we have the following relations

AT = TΛ
S A = ΛS
A = T ΛS
Λ = S AT.
174 Chapter A. Elements of Mathematics

Note that partitioning S by rows

 ⊤ 
 s1 
 s⊤ 
S =  2 
 : 

s⊤
n
the relation S A = ΛS tells us that the rows of S are left eigenvectors.
The case in which matrix A has no distinct eigenvalues is much more complicated. It can be
shown that there is a matrix T ∈ Rn×n such that A = T JT −1 is a block-diagonal matrix of the form
J = diag(J1 , . . . , Jr ),
where the k-th block Jk , having size µk × µk , is of the form
 λk 1 0 0 . . .
 

 0 λ 1 0 ... 
 k 
.
Jk =  0 0 λk . . 0  .
 
 . .. .. . .
 ..

. . . 1 
0 0 0 . . . λk
 

Matrix J is said to be in Jordan form, or a Jordan matrix.

So in general we may transform A as
...
 
 J1 0 0 0
S1
  
 0 J2 0 0 ...
   
  S2 
A = [T 1 T 2 . . . T r ]  0 0 J3 0
 ...  
..

(A.3)
 .. .. .. . . .. .
  
 . . . . .
  

 Sr
0 0 0 ... Jr
where T k ∈ Rn×µk and S k⊤ ∈ Rµk ×n . The Jordan form is unique up to permutations of the blocks.
The number of blocks and their dimensions are a characteristic of the matrix and cannot be changed.
For each eigenvalue λ, three basic parameters can be defined.

Algebraic Multiplicity: multiplicity of λ as a root of the characteristic polynomial.

Geometric Multiplicity: the number of blocks associated with λ in the Jordan form.
Ascent: the size of the largest block associated with λ.

Remark A.4. For simple eigenvalues all of these numbers are equal to 1.
The ascent of an eigenvalue is particularly important. There is a simple way to calculate it
without passing through the Jordan form. Define the following numbers:
ρ1 = rank(λI − A)1
ρ2 = rank(λI − A)2
:
ρg = rank(λI − A)g

It can be shown that ρi ≥ ρi+1 . The computation is carried out until equality occurs:
ρ1 > ρ2 > . . . ρg = ρg+1 .
The first index for which the equality holds is the ascent g of λ (g = mini : ρi = ρi+1 ).
A.2. Matrices 175

Example A.14. Let

   
 −2 1 0   −2 −1 −1 
A1 =  −1 −1 1  , A2 =  −1 −2 −1
   
 
−1

0 0 −1 −1 −2

Matrix A1 has the eigenvalue −1 with algebraic multiplicity 3. Computing the rank of the powers
(A1 + 1I)k , we get ρ1 = 2, ρ2 = 1, ρ3 = 0 and ρ4 = 0. The ascent of the eigenvalue −1 is therefore 3.
Matrix A2 has a simple eigenvalue λ = −4 (hence of ascent 1) and another eigenvalue λ = −1. For
the latter eigenvalue, ρ1 = 1 and ρ2 = 1, hence the ascent is 1.

The following properties concerning the eigenvalues are fundamental.

Proposition A.1. A similitude transformation preserves the characteristic polynomial, hence the
eigenvalues. In fact, if  = T −1 AT , then

det(sI − Â) = det(sT −1 T − T −1 AT ) = det[T −1 (sI − A)T ] =

= det(T −1 ) det(sI − A) det(T ) = det(sI − A).

The eigenvectors are transformed as any other vector.

A.2.8 Norms and inner product

Given a vector space X, for x ∈ X we can define a function ∥x∥, called norm, having the following
properties:

• ∥x∥ ≥ 0, and ∥x∥ = 0 if and only if x = 0;

• ∥λx∥ = |λ|∥x∥;

• ∥x + y∥ ≤ ∥x∥ + ∥y∥.

The norm represents the “size” of x. A famous norm is the geometric length, or 2-norm, or
Euclidean norm v
t n
X
∥x∥  xi2 .
i=1

Other norms are the 1-norm

n
X
∥x∥1  |xi |
i=1

and the ∞-norm

∥x∥∞  max |xi |.
i

The Euclidean and the 1-norm are special cases of the p-norm
v
t n
X
∥x∥  p |xi | p .
i=1

There are many other norms. In general ∥F x∥ is a norm of x ∈ Rn , if F has rank n. All norms in Rn
are equivalent. Precisely, given two norms ∥ · ∥1 and ∥ · ∥2 , there exist positive constants α and β
such that
α∥x∥1 ≤ ∥x∥2 ≤ β∥x∥1 .
176 Chapter A. Elements of Mathematics

We will mainly use the Euclidean norm.

Given a vector x in Cn we define the Hermitian conjugate xH as the vector obtained by
transposing x and conjugating the elements.
#H
2+ j
" h i
= 2 − j 3 + j4
3 − j4

Clearly for real vectors in Rn the Hermitian conjugate is the transposed: xH = x⊤ . Given two
vectors x, y in Cn , we define the scalar product as

(x, y)  xH y

For vectors in Rn ,
(x, y)  x⊤ y
As a particular case, when x = y we have that
n
X
(x, x) = ∥x∥2 = xi2 .
i=1

This is the square of the length, or Euclidean norm, of the vector.

Two vectors x and y are orthogonal if their scalar product is zero:

(x, y) = 0.

In three dimensions, the inner product is equal to the product of the lengths and the cosine of the
angle θ formed by the vectors
(x, y) = ∥x∥ ∥y∥ cos(θ).
This enables us to define “angles” between vectors in dimension greater than 3 as
(x, y)
cos(θ) = .
∥x∥ ∥y∥
To give a further interpretation, consider a unit vector v with ∥v∥ = 1 (note that, given a vector
x , 0, it is always possible to normalise it and get a unit vector v = ∥x∥
x
). Then the quantity (y, v)
represents the component y along the direction v.

Definition A.13. Given a subspace X of Rn , the set of all orthogonal vectors is a subspace and it is
called orthogonal complement
X ⊥ = {y : (x, y) = 0}

Definition A.14. A square n × n matrix Q = [q1 q2 . . . qn ] is said orthogonal if its columns are
non-zero and pairwise orthogonal: q⊤ i q j = 0 if i , j. It is said orthonormal if its columns are
pairwise orthogonal and of unit length:
(
0 if i , j
qi q j =
⊤
(A.4)
1 if i = j

The columns of an orthonormal matrix form an orthonormal basis. Orthonormal bases are very
convenient in computation, because the new components of a vector x are just computed as Q⊤ x.
Indeed, an orthonormal matrix has an inverse equal to the transpose

Q−1 = Q⊤
A.2. Matrices 177

because Q⊤ Q has components [Q⊤ Q]i j as in (A.4), hence is the identity.

Transformations via an orthonormal matrix preserve the scalar product and the Euclidean norm:

(Qx, Qy) = x⊤ Q⊤ Qy = x⊤ y = (x, y).

Given a matrix M, we can define an induced norm, which is the maximum stretching of the
vector on which the matrix operates:
∥Mx∥
∥M∥  sup ∥Mx∥ = sup
∥x∥=1 ∥x∥,0 ∥x∥

The induced norm depends of the chosen vector norm. If we take the Euclidean norm,
p
∥M∥ = max σ(M ⊤ M)

is the maximum square root of the eigenvalues of M ⊤ M, which are all real and nonnegative, as we
will shown soon.

A.2.9 Symmetric matrices

h i
We remind that, given a matrix A ∈ Cm×n with elements ai j , its transpose is defined as the
h i
matrix A⊤ of dimension n × m with elements a ji . For complex matrices we define the hermitian
h i
conjugate as the matrix AH of dimension n × m with elements a∗ji (transpose and conjugate).
Example A.15. For instance we have

 2 − j 3 + j4
 
#H
2+ j 1 + 3j
"
1

=  1 2 + 2j
 
3 − j4 2 − 2 j 4j  
1 − 3 j −4 j

We can easily verify the following properties:

• (AB)⊤ = B⊤ A⊤ ;

• (AB)H = BH AH .
A matrix is said Hermitian if P = PH . In the case of real matrices an Hermitian matrix
P ∈ Rn×n is called symmetric: P = P⊤ . We can prove the following.
Theorem A.6. An Hermitian matrix (symmetric if real) has real eigenvalues.
Proof Consider the expressions

Px = λx
∗ ∗
P x = λ∗ x ∗

pre-multiply the first by xH , and transpose the second, thus obtaining

xH Px = λxH x
x∗ T P∗ T = x ∗ T λ∗ ⇒ xH PH = xH λ∗

Post-multiply the second equation by x, to get

xH Px = λxH x
x H P H x = λ∗ x H x
178 Chapter A. Elements of Mathematics

Since P is Hermitian, i.e., P = PH , these two expressions are the same:

λxH x = λ∗ xH x

Since xH x = ni=1 xi∗ xi = ni=1 |xi |2 is always greater than zero because x , 0, then it must
P P
necessarily happen that λ = λ∗ , that is, the eigenvalues of the matrix P are real. □

It can also be shown the following.

Theorem A.7. For a symmetric matrix, the eigenvectors qi and q j associated with two distinct
eigenvalues are orthogonal:
q⊤i q j = q j qi = 0.
⊤

In general (even if there are repeated eigenvalues) a symmetric matrix admits n eigenvectors that
form an orthonornal basis.

Indeed, we can derive eigenvectors that are mutually orthogonal and that can be normalised
to find a basis of orthonormal eigenvectors Q = [ q1 q2 . . . qn ]. In this case, (A.4) holds and then
Q⊤ Q = I: Q is orthonormal.
The previous properties imply that, for a symmetric matrix P, the diagonalising similarity
transformation is orthonormal

PQ = Q⊤ Λ
Q⊤ P = ΛQ⊤
P = QΛQ⊤
Λ = Q⊤ AQ.

A.2.10 Semidefinite and definite matrices

A symmetric matrix P is said positive semidefinite (or negative semidefinite) if:

x⊤ Px ≥ 0 (≤ 0) ∀ x.

A symmetric matrix P is said positive definite (or negative definite) if:

x⊤ Px > 0 (< 0) ∀ x , 0.

In all other cases, when x⊤ Px can assume both negative and positive signs, we call P indefinite.
Note that P is a negative (semi)definite matrix if and only if −P is positive (semi)definite.
If P is positive definite, in two dimensions the level surfaces of the function V(x) = x⊤ Px are
ellipses, in three dimension are ellipsoids. The graph of V(x) = x⊤ Px with x two dimensional is
shown in Figure A.2 and its level curves are represented in Figure A.3.
A fundamental property of these matrices is the following.

Theorem A.8. Denote as λ1 , . . . , λn the real eigenvalues of the symmetric matrix P.

• P is positive (negative) semidefinite if and only if, for all i, λi ≥ 0 (λi ≤ 0);

• P is positive (negative) definite if and only if, for all i, λi > 0 (λi < 0).
A.2. Matrices 179

Figure A.2: Graph of the quadratic function V(x).

Figure A.3: Level curves of the quadratic function V(x).

180 Chapter A. Elements of Mathematics

Proof Matrix P is diagonalisable by similarity transformation according to the formula P =

QΛQ⊤ , where Q is orthonormal. Then, denoting by z = Qx,
X
x⊤ Px = x⊤ Q⊤ ΛQx = z⊤ Λz = λi z2i
i

This function is positive if and only if λi > 0 for all i (and so on).

An additional method to check whether a matrix P is positive definite is the Sylvester criterion.
Denote by Pk the principal sub-matrix formed by considering the first k rows and k columns.
Proposition A.2. P is positive definite if and only if det Pk > 0 ∀ k = 1, 2, . . . , n.
Example A.16. Matrix P, with
 
 3 1 0 " #
3 1

P =  1 2 1  = P3 , P2 = 1 , P1 = [3],
 
 2
0 1 1
is positive definite because the three first leading determinants determinants det(P1 ), det(P2 ) and
det(P3 ) = det(P) are positive (check it).

A.3 Polynomials and power series

A polynomial in the complex variable s is a function of the form
p(s) = p0 + p1 s + p2 s2 + · · · + pn sn .
The largest power n is the degree of the polynomial. A polynomial of degree n has always n roots
in the complex field: λ1 , λ2 , . . . , λn (which are not necessarily distinct). Then the polynomial can
be written as
p(s) = (s − λ1 )(s − λ2 ) . . . (s − λn ).
If the coefficients are real, the roots are in general complex numbers; however, the set of the roots
is symmetric with respect to the real axis: if z is a root, then also z∗ is a root.
We call power series a polynomial of “infinite” degree:
∞
X
f (s) = f0 + f1 s + f2 s + · · · + fh s + · · · =
2 h
fh sh .
h=0

A function f (s) of this type is defined in a certain domain of convergence, having the form
D = {s : |s| < ρ},
where ρ ≥ 0 is called convergence radius. It depends on the coefficients { fh }. For instance
1 + s + s2 + s3 + s4 . . .
has convergence radius ρ = 1. The series
s s2 s3 s4
1+ + + + + ...
1! 2! 3! 4!
has convergence radius ρ = ∞. These two series, where convergent, are equal to 1−s 1
and e s
respectively. A polynomial is a particular series for which all coefficients are 0 after a finite n and
its convergence radius is infinite.
Functions that can be expressed as power series are called analytic and have several significant
properties (most of which are not reported here, for brevity). The following principle applies.
A.3. Polynomials and power series 181

Theorem A.9. (Identity principle of power series.) Let l be a continuous curve in the complex
plane having one extremum at the origin and containing other points different from 0. Then, given
two series f (s) = ∞ h=0 fh s and g(s) = h=0 gh s , whose convergence disks include l, we have
P h P∞ h

that f (s) = g(s) for each point of l if and only if gh = fh for each h ≥ 0.
It is obvious that gh = fh implies that f (s) = g(s) for each point of l. The fundamental point of
the theorem is that, if f (s) = g(s) on the curve, then f and g have the same coefficients, hence the
same convergence disk, hence they are the same function.
By means of power series, we can define functions of matrices. Let f (s) be an analytic function
and M a square matrix. Assume that the set of the eigenvalues of M is contained in the convergence
disk of f .4 Then we can define the function of M associated with f (s) as
∞
X
f (M)  fh M h .
h=0

Note that M 0 = I as a postulate. As an explanation, consider the case of M diagonalisable so that

M h = T Λh T −1 ,

with Λ = diag{λ1 , λ2 , . . . , λn } diagonal. Then

∞
∞ 
X X 
f (M) = fh T Λ T
h −1
= T  fh Λ  T −1
h

h=0 h=0

where
∞  ∞ ∞ ∞

X  
X X X 
Λ h 
= λ h
, λ h
, . . . , λ h
= diag{ f (λ1 ), f (λ2 ), . . . , f (λn )}.
 

 f h  diag 
 f h 1 f h 2 f h n
 
h=0 h=0 h=0 h=0

Then it becomes apparent why we need to assume that the eigenvalues are in the convergence disk.
Example A.17. We can define
M M2 M3 M4
eM  I + + + + + ...
1! 2! 3! 4!
M2 M4
cos(M)  I − + + ...
2! 4!
and
M M3 M5
−
sin(M)  + + ...
1! 3! 5!
Polynomials are particular power series. Then we can define the polynomial of a matrix. Given
p(s) = p0 + p1 s + p2 s2 + · · · + pn sn we just define

p(M) = p0 I + p1 M + p2 M 2 + . . . pn M n .

The following property is important

Proposition A.3. (Cayley-Hamilton identity.) Let A be a square matrix and

det(sI − A) = p(s) = a0 + a1 s + a2 s2 + · · · + sn

its characteristic polynomial. Then, the corresponding matrix polynomial calculated in A is zero:

p(A) = a0 I + a1 A + a2 A2 + · · · + An = 0.
4
This hypothesis can be removed thanks to the analytic extension technique.
182 Chapter A. Elements of Mathematics

The proof is simple in the case of A diagonalisable, A = T ΛT −1 , because, as we have seen,

P(A) = T diag{p(λ1 ), p(λ2 ), . . . , p(λn )}T −1 = 0
since p(λi ) = 0 if λi is an eigenvalue.
Cayley-Hamilton identity implies the following fact.
Proposition A.4. (Dependence of the powers of A.) Each power Ak of A, with k ≥ 0, is a linear
combination of the first n powers
I , A , A2 , . . . , An−1 .
This property is trivial for the first powers 0, 1, . . . , n − 1. To prove it for the following powers,
just write Cayley-Hamilton identity as
An = −an−1 An−1 − · · · − a2 A2 − a1 A − a0 I.
This means that An is a linear combination of the first n powers. By induction, we assume that this
is true for a given k ≥ n
Ak = −αn−1 An−1 − · · · − α2 A2 − α1 A − α0 I,
and then we multiply by A getting
Ak+1 = −αn−1 An − · · · − α2 A3 − α1 A2 − α0 A =
= −αn−1 [−αn−1 An−1 − · · · − α2 A2 − α1 A − α0 I]
− · · · − α2 A3 − α1 A2 − α0 A =
= −α∗n−1 An−1 − · · · − α∗2 A2 − α∗1 A1 − α∗0 I
(where α∗j are easily computable), hence the property is true for k + 1.
Remark A.5. If A is invertible, the property is also true for negative powers, Ak with k < 0.

A.4 The impulse response

The concept of impulse is easy to grasp with intuition, but it does not have a simple mathematical
description. Intuitively, an impulse is a phenomenon with high intensity and a very short duration.
To represent it mathematically, we can consider a function δϵ (t) defined as follows
0 if t < − 2ϵ



δϵ (t) =  if − 2ϵ ≤ t ≤ 2ϵ
 1

 0ϵ


if t > ϵ 2
For any ϵ, the support of this function (namely, the interval where the function is non-zero), is
precisely [−ϵ/2, ϵ/2] and becomes increasingly small for ϵ → 0, while the amplitude of the function
in this interval, 1/ϵ, becomes increasingly large. Note that the integral of this function on the real
axis is always equal to 1. Let t0 be a point inside the interval [a, b] and let ϵ be a value such that the
support of function δϵ (t − t0 ) is included in [a, b] (t0 − ϵ/2 and t0 + ϵ/2 must both belong to [a, b]).
Intuitively, the “impulse” in t0 , δ(t − t0 ), is the “limit” for ϵ → 0 of the function δϵ (t − t0 ). Note
that δ(t − t0 ) is not a function, but a distribution. A formal treatment is omitted for simplicity: the
following is an intuitive explanation. Given f , a continuous function, consider the integral
Z b Z t0 + ϵ
2
f (t) δϵ (t − t0 )dt = f (t) δϵ (t − t0 )dt =
a t0 − 2ϵ
Z t0 + 2ϵ
1 1
= f (t) dt = f (τ)ϵ = f (τ),
t0 − 2ϵ ϵ ϵ
A.4. The impulse response 183

t0 t

Figure A.4: The impulse can be seen as the limit of a sequence of functions.

where τ is an internal point of the interval [t0 − 2ϵ , t0 + 2ϵ ], which exists for the mean value theorem.
Being f continuous, when ϵ → 0 we have
Z b
f (t) δϵ (t − t0 )dt → f (t0 ).
a

Then the limit function δ(t − t0 ) is defined as the “function” (in fact, it is a distribution) having the
property that, for every continuous function f , defined on [a, b] having t0 as an interior point, we
have that Z b
f (t) δ(t − t0 )dt = f (t0 ). (A.5)
a
We can apply this concept to the case of the forced response of a single-input-single-output linear
system, given by the convolution
Z ⊤
y(t) = W(t − σ)u(σ)dσ.
0

If we take as an input the impulse function δ(σ − t0 ) for σ ≥ 0 we have that

Z ⊤
yimp (t) = W(t − σ)δ(σ − t0 )dσ = W(t − t0 ),
0

for t > t0 (while yimp (t) = 0 for t < t0 ). Finally, note that, in order to consider the solution due to an
impulse at instant 0, we must consider the convolution from 0− to include the origin in the interior:
Z ⊤
y(t) = W(t − σ)u(σ)dσ.
0−
184 Chapter A. Elements of Mathematics

A.5 Laplace transforms

The Laplace transform is of great utility for the study of linear and time-invariant systems. We
summarise here its fundamental properties.
Given a function f (t) with positive support (namely, which is zero for t < 0), we define the
Laplace transform (if it exists) as the function
Z ∞
F(s) = L[ f (t)] = f (t)e−st dt, (A.6)
0

where s is a complex variable. If the function F is well defined, we say that f is Laplace-
transformable. The Laplace transform gives a one-to-one correspondence: F(s) = G(s) if and only
if f (t) = g(t).
With a certain abuse of notation (but with great convenience of representation), it is often
accepted to represent the function and its Laplace transform with the same letter
Z ∞
f (s) = f (t)e−st dt.
0

The expression (A.6) is defined in a suitable domain of s, of the type

Dom f = {s : ℜ(s) > c}.

In particular, the smallest value c f of c for which convergence is ensured, namely for which
Z ∞
f (s) = | f (t)|e−st dt
0

converges, is called abscissa of convergence (relatively to f ). Then Dom f = {s : ℜ(s) > c f } is a

suitable domain.
We assume now that the functions f and g have positive support and are continuous and
differentiable for t > 0. We consider the common domain of convergence Dom = Dom f Domg .
T
We denote by f (s) = L[ f (t)] and g(s) = L[g(t)], respectively. The following fundamental properties
holds.

Linearity
L[α f (t) + βg(t)] = α f (s) + βg(s).

Transform of the derivative

L[ f ′ (t)] = s f (s) − f (0).

Final limit theorem Assuming that the limits below exist and are both finite we have that

lim f (t) = lim s f (s).

t→∞ s→0

Initial limit theorem Assuming that the limits below exist and are both finite we have that

lim f (t) = lim s f (s).

t→0 s→∞

Convolution theorem
"Z ⊤ # "Z ⊤ #
L g(t − σ) f (σ)dσ = L g(σ) f (t − σ)dσ = f (s)g(s).
0 0
A.5. Laplace transforms 185

Shifting theorem
L[ f (t − τ)] = e−sτ F(s).

Multiplication by t
d
L[t f (t)] = − F(s).
ds
Multiplication by eξt
L[eξt f (t)] = F(s − ξ).

A.5.1 Transforms of rational functions

In this subsection we consider functions of the type tk eξt cos(ωt) and tk eξt sin(ωt) and all special
cases (ξ = 0, ω = 0, k = 0). Let us start from the simplest case
1
L eλt =
h i
.
s−λ
The property above applies in the case of real and complex λ.
Consider the transform of L[cos(ωt)] and L[sin(ωt)]. The simplest way to compute it (simul-
taneously) is to consider the function e jωt = cos(ωt) + j sin(ωt). By linearity, real and imaginary
parts will provide us the transforms of the functions cos(ωt) and sin(ωt) . We get
h i 1 s + jω 1
L e jωt = = =
s − jω s + jω s − jω
s + jω s ω
= = +j 2
s2 + ω2 |s2 {z+ ω}
2
| {zω}
s + 2

L[cos(ωt)] L[cos(ωt)]

Then
s
L[cos(ωt)] =
s2 + ω2
ω
L[sin(ωt)] = .
s + ω2
2

Using the same fact and setting λ = ξ + jω

eλt = e(ξ+ jω)t = eξt (cos(ωt) + j sin(ωt))

since
h i 1
L e(ξ+ jω)t =
s − ξ + jω
we get, taking real and imaginary parts separately,
s−ξ
L[eξt cos(ωt)] =
(s − ξ)2 + ω2
ω
L[eξt sin(ωt)] = .
(s − ξ)2 + ω2

Now consider the Laplace transform of the function teλt . This is obtained by considering the
theorem of multiplication by t:
d 1 1
L[teλt ] = − = .
ds s − λ (s − λ)2
186 Chapter A. Elements of Mathematics

From the trivial fact that tk eλt = t(tk−1 eλt ) we can recursively demonstrate that
k!
L[tk eλt ] = .
(s − λ)k+1
The transform of the impulse δ(t − τ) is important:
Z ∞
L[δ(t − τ)] = δ(t − τ)e−st dt = e−sτ
0

In particular for τ = 0 this means that the impulse function in the origin is just

L[δ(t)] = 1.

The Heaviside step function in τ is

t≤τ
(
0 for
gr(t − τ) =
1 for t>τ

and its transform can be obtained as follows:

Z t=∞ Z σ=∞
L[gr(t − τ)] = gr(t − τ)e−st dt = gr(σ)e−s(σ+τ) dσ =
t=0 σ=−τ
Z σ=∞
e−sτ
= e −sτ
e dσ =
−sσ
.
σ=0 s
In particular, the transform of the step in t = 0 is
1
L[gr(t)] = .
s
Finally note that the forced response of a linear system has transform
"Z t #
L W(t − σ)u(σ)dσ = W(s)u(s).
0

The transforms which have been considered so far are rational. There is an interesting case of
non-rational transform. Consider the T -delay operator

y(t) = DLT (u)  u(tT )

This operator is linear. Note that if u has positive support, then u(tT ) = 0 as t < T . In terms of
transform we get
y(s) = e−T s u(s)
The proof is simple and left to the reader.

A.5.2 Rational functions and their inverse transform

We have seen in the previous section some of the main transforms. In this section, we deal with
the inverse transform problem. In particular, we consider the inverse transform of proper rational
functions5 of the type
n0 + n1 s + n2 s2 + · · · + nν sν
f (s) = .
d0 + d1 s + d2 s2 + · · · + sν
5
A rational function is proper if the degree of the denominator is greater than or equal to the degree of the numerator.
A.5. Laplace transforms 187

Note that the coefficient of sν at the denominator is dν = 1: this is not restrictive because, if
dν , 1, we can always divide both numerator and denominator by dν (which is non-zero because
we consider proper functions). A proper rational function f (s) can be always be transformed into
the sum of a constant and a function strictly proper rational function6

n0 + n1 s + n2 s2 + · · · + nν sν nν (d0 + d1 s + d2 s2 + · · · + sν )
f (s) = = +
d0 + d1 s + d2 s2 + · · · + sν d0 + d1 s + d2 s2 + · · · + sν
(n0 − nν d0 ) + (n1 − nν d1 )s + · · · + (nν−1 − nν dν−1 )sν−1
+
d0 + d1 s + d2 s2 + · · · + sν
ñ0 + ñ1 s + ñ2 s2 + · · · + ñν−1 sν−1
= nν + = nν + f˜(s)
d0 + d1 s + d2 s2 + · · · + sν

where f˜(s) is strictly proper. Note that

L−1 [ f (s)] = nν δ(t) + L−1 [ f˜(s)],

Then we just need to determine the inverse transform of the strictly proper part.
Consider the strictly proper rational function f (s) . If f has distinct poles (roots of the
denominator) λ1 , λ2 , . . . , λν , it can always be written in the form
ν
X ri
f (s) = , (A.7)
i=1
s − λi

where the coefficients ri are called residuals and are computable as the limit

ri = lim (s − λi ) f (s),
s→λi

as we can see from (A.7). If we factorise the denominator writing f as

n(s)
f (s) =
(s − λ1 )(s − λ2 ) . . . (s − λν )

the formula reduces to

n(s)
ri = Q .
j,i (s − λ j ) s=λi

As an alternative, we can solve the problem by means of a system of linear equations. Take the
expression (A.7) and consider the common denominator
Pν Pν
j,i (s − λ j ) ri Ψi (s)
Q
n(s) i=1 ri
f (s) = = = i=1
d(s) (s − λ1 )(s − λ2 ) . . . (s − λν ) d(s)

The identity
ν
X
n(s) = ri Ψi (s),
i=1

must hold. Both sides are polynomials of degree ν in s. By applying the identity principle
for polynomials, the coefficients ri are determined by a linear system (obtained by equating the
coefficients of the terms with the same degree).
6
A rational function is strictly proper if the degree of the numerator is smaller than that of the denominator.
188 Chapter A. Elements of Mathematics

Example A.18. Consider the function

s+4 r1 r2 r3
f (s) = = + +
(s + 1)(s + 2)(s + 3) s + 1 s + 2 s + 3
We have that
s+4 3
r1 = =
(s + 2)(s + 3) s=−1 2
s+4
r2 = = −2
(s + 1)(s + 3) s=−2
s+4 1
r3 = =
(s + 1)(s + 2) s=−2 2
If we proceed with the linear system method, we have that

(s + 4) = r1 (s + 2)(s + 3) + r2 (s + 1)(s + 3) + r3 (s + 1)(s + 2)

= r1 (s2 + 5s + 6) + r2 (s4 + 4s + 3) + r3 (s2 + 3s + 2)
= (r1 + r2 + r3 )s2 + (5r 1 + 4r2 + 3r3 )s + 6r1 + 3r2 + 2r3

from which we derive the linear system of equations

r1 + r2 + r3 = 0
5r 1 + 4r2 + 3r3 = 1
6r1 + 3r2 + 2r3 = 4

which (as expected) provides the same values for r1 , r2 , r3 .

This procedure can work but “gives troubles” if some poles λi are complex. If so, then we can
decide to work in a real setting. We factorise the denominator as follows
m
Y q
Y
d(s) = (s − λi ) (s2 − 2ξi s + ξi2 + ω2i )
i=1 i=1
m
Y q
Y
= (s − λi ) ((s − ξi )2 + ω2i )
i=1 i=1

where the factors (s − λi ) are associated with the m real roots, while the factors ((s − ξi )2 + ω2i ) are
associated with the q pairs of complex conjugate roots ξi + jωi (m + 2q = ν). Then we have that
m q
X ri X ai s + bi
f (s) = + .
i=1
(s − λi ) i=1
(s − ξi )2 + ω2i

The coefficients ri can be determined by means of the limit formula. To compute all of the
coefficients ri , ai and bi , we can write the common denominator and calculate the coefficients
through a linear system.
Example A.19. Consider the rational function
2s3 + 1
f (s) = .
(s + 1)(s + 2)((s + 1)2 + 4)
This function can be written as
n(s) r1 r2 a1 s + b1
f (s) = = + + 2 .
d(s) s + 1 s + 2 (s + 2s + 5)
A.5. Laplace transforms 189

After adopting the common denominator, equating the numerators provides

2s3 + 1 = r1 (s + 2)(s2 + 2s + 5) + r2 (s + 1)(s2 + 2s + 5) + (a1 s + b1 )(s + 1)(s + 2)

Through the principle of identity, we can determine the coefficients r1 , r2 , a1 and b1 of the polynomial
by solving the linear system

r1 + r2 + a1 = 2
4r1 + 3r2 + 3a1 + b1 = 0
9r1 + 7r2 + 2a1 + 3b1 = 0
10r1 + 5r2 + 2b1 = 1

The solution is
1
r1 = −
4
r2 = 3
3
a1 = −
4
23
b1 = − .
4
Let us briefly consider now the case of rational functions with multiple poles. The most general
case of decomposition is
m X gi
X ri j
,
i=1 j=1
(s − λi ) j

where gi is the multiplicity of the poles λi . The ri j ’s are obtainable through a system as in the
following example.

Example A.20.

s2 + s + 1
f (s) = (A.8)
(s + 1)2 (s + 2)3
r11 r12 r21 r22 r23
= + + + + (A.9)
s + 1 (s + 1) 2 s + 2 (s + 2)2 (s + 2)3

With the usual technique of the common denominator, we can equate the numerators and get

s2 + s + 1 = r11 (s + 1)(s + 2)3 + r12 (s + 2)3 + r21 (s + 1)2 (s + 2)2 +

+ r22 (s + 1)2 (s + 2) + r23 (s + 1)2

from which we can derive the coefficients r11 , r12 , r21 , r22 and r23 . Equating the coefficients of the
corresponding terms, we get the linear system

r11 + r21 = 0
7r11 + r12 + 6r21 + r22 = 0
18r11 + 6r12 + 13r21 + 4r22 + r23 = 1
20r11 + 12r12 + 12r21 + 5r22 + 2r23 = 1
8r11 + 8r12 + 4r21 + 2r22 + r23 = 1
190 Chapter A. Elements of Mathematics

The solution is

r11 = −4
r12 = 1
r21 = 4
r22 = 3
r23 = 3.

Once we have calculated the decomposition, the problem of the inverse transform can be solved
term by term (in view of linearity). For the terms whose denominator is of the first degree
 r 
L −1
= r eλt ,
s−λ
while for terms with a second degree denominator we can write
as + b a(s − ξ) (b + aξ)
= +
(s − ξ) + ω
2 2 (s − ξ) + ω
2 2 (s − ξ)2 + ω2
whereby
as + b b + aξ ξt
" #
L −1
= a eξt cos(ωt) + e sin(ωt).
(s − ξ)2 + ω2 ω
In the case of multiple eigenvalues, we have terms of degree k + 1. For these we have
" #
r r k λt
L −1
= t e
(s − λ) k+1 k!
where λ is either real or complex.

A.6 Zeta transform

The Zeta transform is the counterpart of the Laplace transform for the analysis of discrete-time
systems. Given the close analogy, we give only a brief sketch. Given the sequence (or function in
discrete-time) f (k), defined for k ≥ 0, the Z-transform of f is the function of the complex variable z
defined as follows
∞
X 1
Z[ f ]  f (k) k = f (z),
k=0
z

namely, it is the corresponding series of powers of 1z . The convergence domain of this function is a
set of the type
D = {z : |z| > ρ f }
(which is the complement of the disk of radius ρ f ), where the radius ρ f depends on the considered
function f .
The Zeta transform has properties that are quite similar to those of the Laplace transform. We
remember only the following properties.
Linearity Z[α f + βg] = αZ[ f ] + βZ[g].

Transform of the anticipated f Z[ f (k + 1)] = zZ[ f ] − z f (0).

Convolution transform
Xk−1
Z[ f (k − h)g(k)] = Z[ f ]Z[g].
h=0
A.7. Convex and least square optimization 191

The utility of the Zeta transform in the study of discrete-time systems is entirely comparable to that
of the Laplace transform in the study of continuous-time systems.

Example A.21. (Exercise.) The sequence λk has Zeta transform

∞
X λk z
= .
k=0
z k z − λ

To find the transforms of cos(ωk) and sin(ωk), just consider that

e jωk = cos(ωk) + j sin(ωk) = [e jω ]k

has transform
z z
= = ...
z − e jω z − (cos(ω) + j sin(ω))
and then split real and imaginary parts . . .

A.7 Convex and least square optimization

Consider the constrained optimization problem

min l(θ) (A.10)

s.t.
θ∈Θ (A.11)

where l(θ) is an objective function of a vector θ ∈ Rn defined on a domain θ ∈ Θ. The set Θ is in

general represented by equalities and inequalities.
Finding the solution of this type of problems is important in many contexts including control
theory. The solution is not in general easy to find. The main problem it that (A.12) (A.11) my have
many local minima so we need to find all of these and take the smallest. A remarkable exception is
the one in which the problem is convex.
A set Θ is convex if assuming that θa and θb belong to Θ, then all their convex combinations
belong to Θ
θ = αθa + βθb ∈ Θ, ∀α + β = 1, α, β ≥ 0
A functional l(θ) defined on a convex set Θ is convex if

l(αθa + βθb ) ≤ αl(θa ) + βl(θb ), ∀α + β = 1, α, β ≥ 0

If Θ and l are convex the optimization problem is said convex.

Proposition A.5. In a convex optimization problem, any local minimum is a global minimum.

The previous result, is very important and its consequence is that a convex optimization problem
is easier than the general optimization problems.

Example A.22. The following problem

min h⊤ θ
s.t. Mθ = r
Nθ ≤ s
192 Chapter A. Elements of Mathematics

is a linear programming problem and can be solved for systems of dimension 1000 or more with no
difficulties. The following problem

min θ⊤ Pθ
s.t. Mθ = r
Nθ ≤ s

with P semi–positive definite, is a quadratic programming problem and can be also efficiently
solved.
Special cases of quadratic problems are the minimum euclidean norm problems with linear
constraints. Consider the linear systems
Ax = b
There are three cases.
1. Matrix A is square invertible, then the solution is unique.

2. There are infinite solutions. In this case one wish to find the one with smallest Euclidean
norm.

3. There are no solutions. In this case one wish to find the vector for which the error e = Ax − b
has the smallest Euclidean norm.
The previous are called generalized solutions.
For brevity assume A full rank. The first solution can be found as

1
min ∥x∥2
2
s.t. Ax = b

The Lagrangian is
1 2
∥x∥ + λ⊤ [Ax − b]
2
Differentiating with respect to x we get

x ⊤ + λ⊤ A = 0

namely x = −A⊤ λ. We replace this in Ax = b to get −AA⊤ λ = b or λ = −[AA⊤ ]−1 b. Then the
minimum norm solution is
x∗ = −A⊤ [AA⊤ ]−1 b
In the second case we wish to minimize

min ∥Ax − b∥2

Write the function as

f (x) = [Ax − b]⊤ [Ax − b] = x⊤ A⊤ Ax − b⊤ Ax − x⊤ A⊤ b + b⊤ b = x⊤ A⊤ Ax − 2b⊤ Ax + b⊤ b

Differentiating with respect to x we get

2x⊤ A⊤ A − 2b⊤ A = 0

transposing A⊤ Ax − A⊤ b = 0, then
x̂ = [A⊤ A]−1 A⊤ b
A.7. Convex and least square optimization 193

A.7.1 Application to system identification

Assume that a frequency test is performed on a system with transfer function F with inputs
cos(ωk t), k = 1, 2, . . . , m
measuring the steady state output

|F( jωk )| cos(ωk t + ϕk )

From magnitude and phase, experimentally determined, we can determine
F( jωk ) = αk + jβk
If F(s) has to be determined, we parameterize it with its coefficient. As an example consider
cs + d
F(s) =
+ as + b s2
where a, b, c, d have to be determined. We can write at each frequency experiment
c jω + d
= α + jβ
−ω2 + a jω + b
namely
[α + jβ][−ω2 + a jω + b] = c jω + d
We can split real an imaginary part
−αω2 + αb − aβω = d and − βω2 + ωαa + βb = ωc
So we get two equations for the unknown coefficients
 
#  a  "
−βω α αω2
" #
0 −1  b 
 =
ωα β −ω 0  c  βω2
 
d
Repeating the measure m times we get
 −β1 ω1 α1 0 −1 α1 ω21
   
  
 ω α β1 −ω1 0   β1 ω21 
 1 1   
 −β2 ω2 α2 0 −1     α2 ω22 
 ω2 α2 β2 −ω2 a
β2 ω22
     
0   
b  
 −β3 ω3 α3 α3 ω23
   
0 −1    ,  
 ω α c  
β β3 ω23
 
−ω

3 3 3 3 0    
d
  
 : : : : 
  : 

 −βm ωm αm αm ω2m

0 −1   
  
ωm αm βm −ωm βm ω2m

0
This linear system is of the form
Φθ = Γ
and it has no solution in general. So it can be solved by minimizing the error norm ∥Φθ − Γ∥.
A different technique for identification is based on time–domain measures. A discrete time
input u(k) is applied to the systems and the output y(k) is measured. Consider a model of the form
n
X n−1
X
y(k) = ah y(k − h) + bh u(k − h), k = 2, 3, 4 . . .
h=1 h=1
This model is called ARMA model (AutoRegressive-Moving-Average). The identification of its
coefficients can be performed via the least square method presented above.