Module 11: Introduction To Optimal Control: Lecture Note 1
Module 11: Introduction To Optimal Control: Lecture Note 1
Module 11: Introduction To Optimal Control: Lecture Note 1
The optimal control theory relies on design techniques that maximize or minimize a given
performance index which is a measure of the effectiveness of the controller.
A functional is a mapping or transformation that depends on one or more functions and the
values of the functionals are numbers. Examples of functionals are performance indices which
will be introduced later.
In the following section we would discuss the Euler-Lagrange equation for discrete time sys-
tems.
where F (k, x(k), x(k + 1), u(k)) is a differentiable scalar function and x(k) ∈ Rn , u(k) ∈ Rm .
I. Kar 1
Digital Control Module 11 Lecture 1
The above can be the state equation of the system, as well as other equality or inequality con-
straints.
Design techniques for optimal control theory mostly rely on the calculus of variation, according
to which, the problem of minimizing one function while it is subject to equality constraints is
solved by adjoining the constraint to the function to be minimized.
Let λ(k + 1) ∈ Rn×1 be defined as the Lagrange multiplier. Adjoining J with the constraint
equation,
N
X −1
Ja = F (k, x(k), x(k + 1), u(k))+ < λ(k + 1), [x(k + 1) − f (k, x(k), u(k))] >
k=0
Calculus of variation says that the minimization of J with constraint is equivalent to the mini-
mization of Ja without any constraint.
Let x∗ (k), x∗ (k + 1), u∗ (k)) and λ∗ (k + 1) represent the vectors corresponding to optimal
trajectories. Thus one can write
where η(k), µ(k), ν(k) are arbitrary vectors and ǫ, δ, γ are small constants.
N
X −1
Ja = F (k, x∗ (k) + ǫη(k), x∗ (k + 1) + ǫη(k + 1), u∗ (k) + δµ(k)) +
k=0
< λ∗ (k + 1) + γν(k + 1), [x∗ (k + 1) + ǫη(k + 1) − f (k, x∗ (k) + ǫη(k), u∗ (k) + δµ(k))] >
I. Kar 2
Digital Control Module 11 Lecture 1
Expanding Fa using Taylor series around x∗ (k), x∗ (k + 1), u∗ (k)) and λ∗ (k + 1), we get
Fa (k, x(k), x(k + 1), u(k), λ(k + 1)) = Fa (k, x∗ (k), x∗ (k + 1), u∗ (k), λ∗ (k + 1)) +
∂Fa∗ (k) ∂Fa∗ (k) ∂Fa∗ (k)
ǫη(k), + ǫη(k + 1), ∗ + δµ(k), +
∂x∗ (k) ∂x (k + 1) ∂u∗ (k)
∂Fa∗ (k)
γν(k + 1), + higher order terms
∂λ∗ (k + 1)
where
Fa∗ (k) = Fa (k, x∗ (k), x∗ (k + 1), u∗ (k), λ∗ (k + 1))
The necessary condition for Ja to be minimum is
∂Ja
= 0
∂ǫ ǫ=δ=γ=0
∂Ja
= 0
∂δ ǫ=δ=γ=0
∂Ja
= 0
∂γ ǫ=δ=γ=0
N −1
X ∂F ∗ (k) ∂Fa∗ (k)
η(k), a∗ + η(k + 1), ∗ = 0 (1)
k=0
∂x (k) ∂x (k + 1)
N −1
X ∂Fa∗ (k)
µ(k), = 0 (2)
k=0
∂u∗ (k)
N −1
X ∂Fa∗ (k)
ν(k + 1), = 0 (3)
k=0
∂λ∗ (k + 1)
N −1 N
X ∂F ∗ (k) X ∂Fa∗ (k − 1)
η(k), a∗ = − η(k),
k=0
∂x (k) k=1
∂x∗ (k)
N −1
X ∂Fa∗ (k − 1)
= − η(k), ∗ (k)
+
∂x
k=0
∂Fa∗ (k − 1) ∂Fa∗ (k − 1)
η(k), − η(k),
∂x∗ (k)
k=0 ∂x∗ (k)
k=N
I. Kar 3
Digital Control Module 11 Lecture 1
where
Fa∗ (k − 1) = Fa (k − 1, x∗ (k − 1), x∗ (k), u∗ (k − 1), λ∗ (k))
According to the fundamental lemma of calculus of variation, equation (4) is satisfied for any
η(k) only when the two components of the equation are individually zero. Thus,
Equation (5) is known as the discrete Euler-Lagrange equation and equation (6) is called the
transversality condition which is nothing but the boundary condition needed to solve equation
(5).
Discrete Euler-Lagrange equation is the necessary condition that must be satisfied for Ja to
be an extremal.
With reference to the additional conditions (2) and (3), for arbitrary µ(k) and ν(k + 1),
∂Fa∗ (k)
= 0, j = 1, 2, · · · m (7)
∂u∗j (k)
∂Fa∗ (k)
= 0, i = 1, 2, · · · n (8)
∂λ∗i (k + 1)
which means that the state equation should satisfy the optimal trajectory. Equation (7) gives
the optimal control u∗ (k) in terms of λ∗ (k + 1)
In a variety of the design problems, the initial state x(0) is given. Thus η(0) = 0 since x(0) is
fixed. Hence the tranversality condition reduces to
∂Fa∗ (k − 1)
η(k), = 0
∂x∗ (k)
k=N
I. Kar 4
Digital Control Module 11 Lecture 1
Again, a number of optimal control problems are classified according to the final conditions.
If x(N ) is given and fixed, the problem is known as fixed-endpoint design. On the other
hand, if x(N ) is free, the problem is called a free endpoint design.
For fixed endpoint (x(N ) = fixed, η(N ) = 0) problems, no transversality condition is required
to solve.
∂Fa∗ (k − 1)
= 0
∂x∗ (k) k=N
For more details, one can consult Digital Control Systems by B. C. Kuo.
I. Kar 5