Iterative Learning Control (ILC) : Bo Bernhardsson and Karl Johan Åström

Iterative Learning Control (ILC)
Bo Bernhardsson and Karl Johan Åström
Department of Automatic Control LTH,

Lund University
Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

ILC
ILC - the main idea

Time Domain ILC approaches
Stability Analysis
Example: The Milk Race
Frequency Domain ILC
Example: Marine Vibrator
Material:

ILC and Repetitive Control
Picture from the master thesis project by Domenico Scalamogna, 2000-2001

ILC and Repetitive Control
Suitable when: Repetitive task + Repetetive disturbances

Use knowledge from previous iteration to improve next iteration
Examples
CD-disc radial control + eccentricity
Industrial robot laser cutting + backlash and friction
Signal transmitter + amplifier nonlinearities
Different versions (time-interval [0, T ], n= iteration index):

Repetitive control xn+1 (0) = xn (T )
ILC xn (0) = x0 , where x0 is the same for all n

About time scales
ILC
ym ILC (batch)
Model
Controller parameters Adjustment Adaptation

mechanism
(slow)
uref uc
+ Controller Plant
y
’Ordinary’
u
feedback (fast)
NOTE: ILC works on whole signal sequences and modifies uref after
’off-line’ calculations.

ILC
Consider the system
y(t) = T u(t)
e(t) = r(t) − y(t)
unew (t) = u(t) + L(q)e(t)
If T is linear and there are no disturbances then L = T −1 gives

enew (t) ≡ 0
Is it a good strategy? Why/why not?

ILC update law
Process at iteration k :
yk (t) = Tr (q)r(t) + Tu (q)uk (t)

ek (t) = r(t) − yk (t)
A common update law for ILC is
uk (t) = Q(q)[uk−1 (t) + L(q)ek−1 (t)]
where Q and L are linear filters (need not to be causal!).

Example
ω02
G(s) = (s+1)(s2 +2ζ0 ω0 s+ω02 )
, sample rate h = 0.1.
Step Response
1.4 0
-5
1.2
-10
1
-15
Amplitude
0.8
-20
0.6 -25
-30
0.4
-35
0.2
-40
0
0 2 4 6 8 10 12 -45
Time (seconds) -2
10
-1
10 10
0 1
10
Demonstration

Example
L(q) = kq n with k = 1 and n = 6.
Q = 2nd order zero-phase low pass filter (filtfilt) with bandwidth
wf = 20
Singular Values
1.4
yd 10
y
u
1.2
-10
Singular Values (dB)

0.8
0.6 -20
0.4
-30
0.2
-40
-50
-2 -1 0 1 2
10 10 10 10 10
-0.2 Frequency (rad/s)
0 1 2 3 4 5 6 7 8 9 10
Left: After 10 iterations, Right: Gain of (I − LTu ) and Q(I − LTu )

g = 1/(s+1)*w0^2/(s^2+2*w0*z0*s+w0^2);
gd = c2d(g,h);
u1 = (t>4.5);
y1=lsim(gd,u1,t);
yr = max(0,min(1,-5:h:5))’;
...
delta = 6; k=1;
wf=20; lp = wf/(s+wf); lpd = c2d(lp,h);
[b,a]=tfdata(lpd,’v’);
for iter = 1:itermax-1
u(:,iter+1)=filtfilt(b,a,u(:,iter)+...
k*[e(delta+1:end,iter);zeros(delta,1)]);
y(:,iter+1) = lsim(gd,u(:,iter+1),t);
e(:,iter+1) = yr - y(:,iter+1);
end
sigma(1-k*z^delta*gd); hold on
sigma(lpd*lpd*(1-k*z^delta*gd))

Error Analysis
With G := Q(I − LTu ), H := QL and r̃ = (I − Tr )r we have
uk = Guk−1 + H r̃
If G is contraction then the iteration converges and
e∞ = (I − Tu (I − G)−1 H)r̃
If Tu is right invertible one can prove that
e∞ = Tu (I − G)−1 (I − Q)Tu−1 r̃
Q = I gives e∞ = 0. But smaller Q gives better robustness.

Main Convergence Result
Theorem (Norrlöf, 1999)

Given a SISO LTI system and ILC algorithm

uk (t) = Q(q)[uk−1 (t) + L(q)ek−1 (t)]
convergence will be achieved if
| 1 − L(eiωts ) · Tu (eiωts ) |<| Q−1 (eiωts ) |
for all ω ∈ [−π, π] where ts is the sampling time.

Divergence
What if
|1 − L(jω)T (jω)| > |Q−1 (jω)|
for some frequencies?
|Q(1 − LT )|
Can do some iterations, enough to reduce the worst parts. (Low

frequency errors will decay rapidly, some others will grow ...)
Generalized Convergence Result
Theorem (Ardakani, Khong, Bernhardsson, 2015)

Given a SISO LTI system and ILC algorithm

uk (t) = Q(q)[uk−1 (t) + L(q)ek−1 (t)]
convergence on finite time intervals [0, T ] will for any T be achieved if

there is r > 0 so
| 1 − L(reiωts ) · Tu (reiωts ) |<| Q−1 (reiωts ) |
for all ω ∈ [−π, π] where ts is the sampling time.

Causal and anti-causal filters
L and Q are allowed to be non-causal !
Causal representation (stable if |a| < 1):
y(t + 1) = ay(t) + u(t)
u y
1
q−a
y(t) = u(t − 1) − au(t − 2) . . .
Anti-causal representation (stable if |a| > 1):

1 1
y(t) = y(t + 1) − u(t)
a a
u y
−1/a
1−q/a
y(t) = − a1 u(t) − 1
a2 u(t + 1) . . .

Causal and anti-causal filters
A general transfer function can be split into two parts
H(q) = H+ (q) + H− (q)

| {z } | {z }
causal anti−causal
To implement Q(q) one often uses a noncausal zero-phase low pass

filter.
B(q) B(q −1 )
The command filtfilt gives a filter of the form A(q) A(q −1 )

ILC Design
ILC design approaches
Heuristic design
Model Based design
Optimization Based design

Heuristic design
Heuristic design procedure [MN]:

1 Choose the Q filter as a low pass with cut-off frequency such that
the band-width of the learning algorithm is sufficient.
2 Let L(q) = κ · q δ .Choose κ and δ such that the stability criterion
above is fulfilled. Often it suffices to choose δ as the time delay
and κ : 0 < κ ≤ 1 to get a stable ILC system.
Why it works: With a well tuned controller Tu ≈ 1 up to the cut-off

frequency

Model based ILC design
Model based design procedure:

1 Build a model of the relations between the ILC input and the
resulting correction on the output (i.e. find a model Tbc of Tc ).
2 Choose a filter Hb (q) such that it represents the desired
convergence rate for each frequency. Normally this means a
high-pass filter
3 Calculate L by
L(q) = Tbc−1 (q)(1 − Hb (q)).
4 Choose the Q filter as a low pass with cut-off frequency such that
the band-width of the resulting ILC is high enough and desired
robustness is achieved.

Optimization based design procedure
Impulse response matrix model
yk = Tr r + Tu uk
where Tr and Tu are quadratic matrices of size equal to the number

of time points.
Given the cost:
Jk+1 = eTk+1 We ek+1 + uTk+1 Wu uk+1 + λ(uk+1 − uk )T (uk+1 − uk )
by minimizing it w.r.t. uk+1 we derive the following algorithm.

1 Build a model of the relations between the ILC input and the
b c of Tc . The
resulting correction on the output (i.e. find a model T
b
matrix Tc is simply the lower triangular Toeplitz matrix created
from the impulse response of Tbc (q))
2 Choose the weight matrices as We = I and Wu = ρI with
ρ > 0 , choose also λ > 0
3 Q and L are calculated according to
bTT
Q = ((ρ + λ) · I + T b −1 bT b
c c ) (λ · I + Tc Tc )
bTT
L = (λ · I + T b −1 b T
c c ) Tc
4 Use the ILC updating equation

uk+1 = Q(uk + Lek ) with u0 = 0.

Example - The Milk Race [Grundelius]
fill
seal
fold
direction
The motion is performed stepwise, step time T . Want:

To shorten the motion time T
To control the slosh inside (s(t) ≤ smax )
To reduce the residual slosh after the motion
Determine best acceleration profile u(t) that minimizes T .

During the motion the slosh is measured by a laser sensor.
No direct feedback is applied from these measurements, but they are
used in the ILC after each motion.
liquid

The acceleration profile was calculated iteratively by an ILC algorithm
(SLOSH ILC). Optimization based ILC respecting the slosh
constraints.
uk
SLOSH PROCESS
SLOSH ALGORITHM
slosh
Assumes reliable, accurate acceleration uk can be applied to the

package all iterations (i.e. a perfect robot).

In practice we need two ILC schemes:
uk u∗
k
ROBOT SLOSH PROCESS
SLOSH ALGORITHM
slosh
At each iteration the robot tries to track the acceleration uk but it

applies to the container a different acceleration û∗k . Solution: ILC
applied to the Robot joints (ROBOT ILC) in order to improve the
tracking of uk .

Outer and Inner Loops
1. Calculate the initial acceleration u0 by
solving the Minimum Energy Problem
and calculate the slosh reference r(t)
2. Consider the acceleration reference uk (t)
2.1. Execute an iteration of ROBOT ILC
and measure the acceleration
performed u∗k (t)
2.2. If u∗k (t) doesn’t approximate well
uk (t) go to step 2.1.
3. Execute an iteration of SLOSH ILC by
reproducing on the robot the acceleration
u∗k (t) performed in the last iteration of ROBOT ILC.
4. Calculate the new acceleration uk (t)
using the SLOSH ILC algorithm. Let k ← k + 1
5. If the slosh behavior needs to be
improved go to step 2.
Joint Scheme:
position ref. JOINT

response
velocity
The Heuristic ILC algorithm is applied in ROBOT ILC. It was applied to

joint 2 and joint 3 of the robot. ILC Scheme applied:
position ILC
position ref.
DERIVATIVE
JOINT
response
LOW-PASS
velocity

Outer loop iteration 0 on joint 2 (T = 0.46 s.)
ITERATION 0
80
des. J2 position
des. J2 velocity
measured J2 pos. measured J2 vel.
20 ILC pos. control
60
40
15
20
10 0
-20
5
-40
-60
0
0 0.5 1 1.5 2 2.5 3

-80
0 0.5 1 1.5 2 2.5 3
kROB = 0
ITERATION 3
80
des. J2 position
des. J2 velocity
measured J2 pos. measured J2 vel.
20 ILC pos. control
60
40
15
20
10 0
-20
5
-40
-60
0
0 0.5 1 1.5 2 2.5 3

-80
0 0.5 1 1.5 2 2.5 3
kROB = 3

Slosh behavior (T = 0.46 sec.)
4 4
3 3
2 2
1 1
0 0
-1 -1
-2 -2
-3
0 0.5 1 1.5
-3
0 0.5 1 1.5
kSLOSH = 0
4 4
3 3
2 2
1 1
0 0
-1 -1
-2 -2
-3
0 0.5 1 1.5
-3
0 0.5 1 1.5
kSLOSH = 1

4 4
3 3
2 2
1 1
0 0
-1 -1
-2 -2
-3
0 0.5 1 1.5
-3
0 0.5 1 1.5
kSLOSH = 2
4 4
3 3
2 2
1 1
0 0
-1 -1
-2 -2
-3
0 0.5 1 1.5
-3
0 0.5 1 1.5
kSLOSH = 3

Acceleration profiles (T = 0.46 s.)
-2
-4
-6
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
Initial acceleration (dotted) and acceleration after three iterations of

SLOSH ILC (full)
Don’t push it too much...
Example: Robot tracking

Choose a wanted reference motion which should be possible to
achieve without high energy input signals. The robot can’t change
direction instantly. Respect the limitations.
If you are measuring angles on the motor side, you may very well
improve the tracking on the motor side, but it may degrade the motion
on the arm side !!
Typically enough with a limited number of iterations, say 10 or so.
Errors might increase with too many iterations if models are bad.

Frequency based ILC
Similar as above, but in frequency domain
uk+1 (f ) = Q2 (f )uk (f ) + Q(f )G−1 (f )(R(f ) − Y (f ))
Filters chosen as
(
0.1 − 0.5 for frequencies we want the ILC to be active
Q(f ) =
0 otherwise
Q2 (f ) ≡ 1
Note: Can use (non-causal) filter L = G−1 without much effort.

Example - Marine Vibrators
How to do seismic surveying:
Generate a HUGE acoustic signal

Pick up echoes using a HUGE (kilometers) sensor array
Do some signal processing (correlation analysis)

Output from seismic survey
Higher frequencies -> Great resolution near surface structure

Lower frequency -> Better characterization of structure at depth

Acoustic Sources
Air guns have traditionally dominated the market
Higher peak pressures than most other man-made sources, except
explosives
New novel constructions have the potential for reduced "acoustic
footprints"

Design Challenges with Marine Vibrators
Want
High output power

High efficiency (for used frequencies)
Exact acoustic signals (linearity, repeatability)
Instead of airguns: Electro-mechanical constructions with well

designed useful mechanical resonances
Problems: Backlash, friction, saturation effects, ...

The Control Problem
Input (2):
Current to coils affecting each side
Measurement sensors (2):
Accelerometer(s) on shells of vibrator
Experiments show that imperfections generate very repeatable errors
Good candidate for iterative learning control (ILC)
Very satisfactory results with ILC

System Identification 2x2 MIMO
Many resonances. Very high system order.

Decided to do ILC in the frequency domain

ILC algorithm, FFT-based
uk+1 (f ) = Q2 (f )uk (f ) + Q(f )G−1 (f )(R(f ) − Y (f ))
Wanted reference chosen as
R(f ) = F(chirp)
Filters chosen as
(
0.1 − 0.5 for frequencies we want the ILC to be active
Q(f ) =
0 otherwise
Q2 (f ) ≡ 1
Note: G−1 matrix inverse in the 2 × 2 case

Before ILC
Input signal before ILC
Output signal before ILC

After ILC
Input signal after ILC
Output signal after ILC

Spectrograms after ILC - double shell sensor
Output spectra on the shells (ILC active in [30,650] Hz)
>40dB suppression
Note: Reference = constant amplitude chirp

Spectrograms after ILC - double shell sensor
Spectrum on the accelerometers on the two sides
Both sides move according to wanted reference

40dB suppression

Convergence - double shell sensors
Show
movies/ilc.avi
movies/doubleshell.avi
movies/skal131028spectograms.avi

Iterative Learning Control (ILC) : Bo Bernhardsson and Karl Johan Åström

Uploaded by

Copyright:

Available Formats

Iterative Learning Control (ILC) : Bo Bernhardsson and Karl Johan Åström

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Iterative Learning Control (ILC) : Bo Bernhardsson and Karl Johan Åström

Uploaded by

Copyright:

Available Formats

Iterative Learning Control (ILC)

Bo Bernhardsson and Karl Johan Åström

Department of Automatic Control LTH,

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

ILC - the main idea

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Picture from the master thesis project by Domenico Scalamogna, 2000-2001

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Suitable when: Repetitive task + Repetetive disturbances

Different versions (time-interval [0, T ], n= iteration index):

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Controller parameters Adjustment Adaptation

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Consider the system

If T is linear and there are no disturbances then L = T −1 gives

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

yk (t) = Tr (q)r(t) + Tu (q)uk (t)

A common update law for ILC is

uk (t) = Q(q)[uk−1 (t) + L(q)ek−1 (t)]

where Q and L are linear filters (need not to be causal!).

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Singular Values (dB)

Left: After 10 iterations, Right: Gain of (I − LTu ) and Q(I − LTu )

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

With G := Q(I − LTu ), H := QL and r̃ = (I − Tr )r we have

If G is contraction then the iteration converges and

If Tu is right invertible one can prove that

Q = I gives e∞ = 0. But smaller Q gives better robustness.

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Theorem (Norrlöf, 1999)

yk (t) = Tr (q)r(t) + Tu (q)uk (t)

convergence will be achieved if

| 1 − L(eiωts ) · Tu (eiωts ) |<| Q−1 (eiωts ) |

for all ω ∈ [−π, π] where ts is the sampling time.

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Can do some iterations, enough to reduce the worst parts. (Low

Theorem (Ardakani, Khong, Bernhardsson, 2015)

yk (t) = Tr (q)r(t) + Tu (q)uk (t)

convergence on finite time intervals [0, T ] will for any T be achieved if

| 1 − L(reiωts ) · Tu (reiωts ) |<| Q−1 (reiωts ) |

for all ω ∈ [−π, π] where ts is the sampling time.

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

y(t + 1) = ay(t) + u(t)

Anti-causal representation (stable if |a| > 1):

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

A general transfer function can be split into two parts

H(q) = H+ (q) + H− (q)

To implement Q(q) one often uses a noncausal zero-phase low pass

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

ILC design approaches

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Heuristic design procedure [MN]:

Why it works: With a well tuned controller Tu ≈ 1 up to the cut-off

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)

Model based design procedure:

Bo Bernhardsson and Karl Johan Åström Iterative Learning Control (ILC)