C1 L3

Numerical Differentiation
Chapter 1 Lecture 3
At the end of this session students should be able to

estimate the derivative using forward, backward and
centered finite divided difference.
The Taylor Series expansion (Forward Taylor Series expansion) of the
function f at xi 1 is given by
f " ( xi ) 2
f ( n ) ( xi ) n
f ( xi 1 ) f ( xi ) f ' ( xi )h
h ...
h Rn
2!
n!
f ( n 1) ( ) n 1
h
where Rn
is a remainder term.
(n 1)!
Truncate the series after the first derivative term:
f ( xi 1 ) f ( xi ) f ' ( xi )h R1
This equation can be solved for
f ' ( xi )
f ( xi 1 ) f ( xi ) R1
h
h
f ( xi 1 ) f ( xi ) R1
f ' ( xi )
h
h
First-order
approximation
Truncation error
The first part of the equation is the relationship used to approximate

the derivative and the second part is used to estimate the truncation
error associated with this approximation of the derivative.
f (x)
f ( xi 1 )
f ' ( xi )
f ( xi )
h
xi
xi 1
f ( xi 1 ) f ( xi ) R1
f ' ( xi )
h
h
Here
f " ( ) 2
h
2
R1 f " ( )

h O ( h)
h
2
R1
Thus, the estimate of the derivative has a truncation error of order

h.
If we halve the step size, i.e.,
h 12 h , we would expect to halve
the error of the derivative, i.e.,
R1
f " ( )
h
h
4
Forward Difference Approx. of the First Derivative
The equation
f ( xi 1 ) f ( xi ) R1
h
h
f ( xi 1 ) f ( xi )
O ( h)
h
f ' ( xi )
is called a Forward finite difference because it utilizes

data at xi and xi 1 to estimate the derivative, where
f " ( xi )
f (3) ( xi ) 2 f ( 4 ) ( xi ) 3
O (h)
h
h
h ........
2!
3!
4!
Backward Difference Approx. of the First Derivative

f ' ( xi )
f (x)
f ( xi )
f ( xi 1 )
xi 1
Let
h xi xi 1
xi
so that
f ( xi 1 ) f ( xi h)
f " ( xi ) 2 f ( 3) ( xi ) 3
f ( xi ) f ' ( xi )h
h
h .......
2!
3!
Backward Difference Approx. of the First Derivative
Truncating this equation after the first derivative
f ( xi 1 ) f ( xi ) f ' ( xi )h R1
and rearranging yields

where
f ' ( xi )
f ( xi ) f ( xi 1 )
O ( h)
h
f " ( xi )
f (3) ( xi ) 2
O ( h)
h
h .......
2!
3!
This equation is called a Backward Finite Difference because it

utilizes data at xi 1 and xi to estimate the derivative.
Centered Difference Approx. of the First Derivative
A third way to approx. the first derivative is to subtract

the backward from the forward Taylor series expansion
as follows:
Forward :
f " ( xi ) 2 f (3) ( xi ) 3
f ( xi 1 ) f ( xi ) f ' ( xi )h
h
h ......
2!
3!
Backward :
f " ( xi ) 2 f ( 3) ( xi ) 3
f ( xi 1 ) f ( xi ) f ' ( xi )h
h
h .......
2!
3!
Forward Backward:
f ( 3) ( xi ) 3
f ( xi 1 ) f ( xi 1 ) 2 f ' ( xi ) h
h ......
3!
f (3) ( xi ) 2
2h f ' ( xi )
h ......
3!
f ( xi 1 ) f ( xi 1 )
f (3) ( xi ) 2
f ' ( xi )
h .......
2h
3!
f ( xi 1 ) f ( xi 1 ) f ( 3) ( xi ) 2
f ' ( xi )
h .......
2h
3!
f ( xi 1 ) f ( xi 1 )
O(h 2 )
2h
Centered Difference Approx. of the

First Derivative
where
f ( 3) ( xi ) 2 f (5) ( xi ) 4
O(h )
h
h ......
3!
5!
2
This equation is shown in the diagram below:
f (x)
f ( xi 1 )
f ' ( xi )
f ( xi )
f ( xi 1 )
xi 1
xi
xi 1
This equation is called a Centered Finite Difference

representation of the first derivative. Note that the
2
truncation error is of the order of h whereas the
forward and backward approximation are of the order of
h. Consequently, the centered difference is a more
accurate representation of the derivative. Because it
would approximately halve the truncation error, whereas
for centered difference, the error would be quartered.
Example 1
Use forward and backward difference approx. and

centered difference approx. to estimate the first
derivative of the function
f ( x) 0.1x 4 0.15 x 3 0.5 x 2 0.25 x 1.2

at x=0.5 using a step size h=0.5. Repeat the
computation using h=0.25. Compute your results with
the true value of the derivative.
Solution
f ( x) 0.1x 4 0.15 x 3 0.5 x 2 0.25 x 1.2
The derivative of f(x) is
f ' ( x) 0.4 x 3 0.45 x 2 1.0 x 0.25

The true value is f ' (0.5) 0.9125
For the step size h=0.5, we have
xi 1 0
f ( xi 1 ) f (0) 1.2
xi 0.5
f ( xi ) f (0.5) 0.925
xi 1 1.0
f ( xi 1 ) f (1.0) 0.2
Solution
The backward difference approx. is

f ( xi ) f ( xi 1 )
h
0.925 1.2
f ' (0.5)
0.55
0.5
0.9125 0.55
t
100% 39.7%
0.9125
f ' ( xi )
Solution
The forward difference approx. is

f ( xi 1 ) f ( xi )
h
0.2 0.925
f ' (0.5)
1.45
0.5
0.9125 1.45
t
100% 58.9%
0.9125
f ' ( xi )
Solution
The centered difference approx. is

f ( xi 1 ) f ( xi 1 )
2h
0.2 1.2
f ' (0.5)
1.0
2(0.5)
f ' ( xi )
0.9125 1.0
100% 9.6%
0.9125
Solution
For the step size h=0.25, we have
xi 1 0.25
f ( xi 1 ) f (0.25) 1.10351563
xi 0.5
f ( xi ) f (0.5) 0.925
xi 1 0.75
f ( xi 1 ) f (0.75) 0.63632813
The backward difference approx. is

f ( xi ) f ( xi 1 )
h
0.925 1.10351563
f ' (0.5)
0.714
0.25
0.9125 0.714
t
100% 21.75%
0.9125
f ' ( xi )
Solution
The forward difference approx. is

f ( xi 1 ) f ( xi )
h
0.63632813 0.925
f ' (0.5)
1.155
0.25
0.9125 1.155
t
100% 26.5%
0.9125
f ' ( xi )
Solution
The centered difference approx. is

f ( xi 1 ) f ( xi 1 )
2h
0.63632813 1.10351563
f ' (0.5)
0.934
2(0.5)
f ' ( xi )
0.9125 0.934
100% 2.4%
0.9125
Solution
h=0.5
Backward
Forward
h=0.25
t 39.7%
t 21.75%
t 58.9%
t 26.5%
t 9.6%
t 2.4%
Centered
For both step sizes, the centered difference approx. is more accurate
than forward and backward differences. Also as predicted by the
Taylor series analysis, halving the step size approximately halves the
error of backward and forward difference and quarters the error of
the centered difference.
Finite Difference Approximation of Higher Derivatives
Taylor series expansion can be used to derive numerical estimates of

higher derivatives. To do this, we write a forward Taylor series
f ( xi 2 )
expansion for f ( xi ) in terms of
:
f " ( xi )
f (3) ( xi )
2
f ( xi 2 ) f ( xi ) f ' ( xi )( 2h)
( 2 h)
(2h) 3 ....
2!
3!
(1)
From the forward Taylor series expansion
f " ( xi ) 2 f ( 3) ( xi ) 3
f ( xi 1 ) f ( xi ) f ' ( xi )h
h
h ......
2!
3!
(2)
Finite Difference Approximation of

Higher Derivatives
Eq.(1) - 2xEq.(2) gives
f ( xi 2 ) 2 f ( xi 1 )
f ( xi ) f " ( xi )h 2 f (3) ( xi )h 3 127 f ( 4 ) ( xi )h 4 ......

which can be solved for
f " ( xi )
where
f ( xi 2 ) 2 f ( xi 1 ) f ( xi )
O(h)
2
h
O(h) f (3) ( xi )h 127 f ( 4 ) ( xi )h 2 ......
This relationship is called the Second forward finite difference.

Higher Derivatives
Similar manipulations can be employed to derive a

Second backward finite difference as
f " ( xi )
where
f ( xi ) 2 f ( xi 1 ) f ( xi 2 )
O ( h)
2
h
O(h) f (3) ( xi )h 127 f ( 4) ( xi ) h 2 ......

Higher Derivatives
The Second centered finite difference is
f " ( xi )
where
f ( xi 1 ) 2 f ( xi ) f ( xi 1 )
O ( h)
h2
1
O( h) 121 f ( 4 ) ( xi )h 2 360
f ( 6 ) ( xi )h 4 ......
Alternatively, the Second Centered finite difference can

be expressed as
f ( xi 1 ) f ( xi ) f ( xi ) f ( xi 1 )
h
h
f " ( xi )
h
Example 6
Use a centered difference approximation to estimate the

second derivative of the function
f ( x) 25 x 3 6 x 2 7 x 88
at x=2 using step sizes of h=0.25 and 0.125. Compare
your estimates with the true value of the second
derivative. Interpret your results on the basis of the
remainder term of the Taylor series expansion.
Solution
f ( x) 25 x 3 6 x 2 7 x 88
f ' ( x) 75 x 2 12 x 7
f " ( x) 150 x 12
The true value of the second derivative at x=2 is
f " (2) 150(2) 12 288
Solution
xi 1 1.75
f ( xi 1 ) f (1.75) 39.85938
xi 2
f ( xi ) f (2) 102
xi 1 2.25
f ( xi 1 ) f (2.25) 182.1406
h=0.25
f ( xi 1 ) 2 f ( xi ) f ( xi 1 )
h2
f (2.25) 2 f (2) f (1.75)
f " ( 2)
288
2
(0.25)
f " ( xi )
Solution
h=0.125
xi 1 1.875
f ( xi 1 ) f (1.875) 68.82617
xi 2
f ( xi ) f (2) 102
xi 1 2.125
f ( xi 1 ) f (2.125) 139.6738
f (2.125) 2 f (2) f (1.875)

288
(0.125) 2
Both results are exact because the errors are a function
of 4th and higher derivatives which are zero for a 3rd
order polynomial function.
f " (2)

C1 L3

Uploaded by

C1 L3

Uploaded by

Numerical Differentiation

At the end of this session students should be able to

The first part of the equation is the relationship used to approximate

Thus, the estimate of the derivative has a truncation error of order

Forward Difference Approx. of the First Derivative

is called a Forward finite difference because it utilizes

Backward Difference Approx. of the First Derivative

Backward Difference Approx. of the First Derivative

Truncating this equation after the first derivative

and rearranging yields

This equation is called a Backward Finite Difference because it

Centered Difference Approx. of the First Derivative

A third way to approx. the first derivative is to subtract

Centered Difference Approx. of the First Derivative

Centered Difference Approx. of the

This equation is shown in the diagram below:

Centered Difference Approx. of the First Derivative

This equation is called a Centered Finite Difference

Use forward and backward difference approx. and

f ( x) 0.1x 4 0.15 x 3 0.5 x 2 0.25 x 1.2

f ' ( x) 0.4 x 3 0.45 x 2 1.0 x 0.25

The backward difference approx. is

The forward difference approx. is

The centered difference approx. is

For the step size h=0.25, we have

The backward difference approx. is

The forward difference approx. is

The centered difference approx. is

Finite Difference Approximation of Higher Derivatives

Taylor series expansion can be used to derive numerical estimates of

Finite Difference Approximation of

f ( xi ) f " ( xi )h 2 f (3) ( xi )h 3 127 f ( 4 ) ( xi )h 4 ......

O(h) f (3) ( xi )h 127 f ( 4 ) ( xi )h 2 ......

This relationship is called the Second forward finite difference.

Finite Difference Approximation of

Similar manipulations can be employed to derive a

O(h) f (3) ( xi )h 127 f ( 4) ( xi ) h 2 ......

Finite Difference Approximation of

The Second centered finite difference is

Alternatively, the Second Centered finite difference can

Use a centered difference approximation to estimate the

f " (2) 150(2) 12 288

f (2.125) 2 f (2) f (1.875)

You might also like