NUMERICAL METHODS WITH

APPLICATIONS
(MEC500)

Dr. Siti Mariam binti Abdul Rahman

Faculty of Mechanical Engineering
Office: T1-A14-01C
e-mail: [email protected]
Outcomes of Chapter 2
 To determine the root of equation using Open Method:
◦ One point iteration (Fixed point) Method
◦ Newton Raphson Method
◦ Secant Method
◦ Modified Newton-Raphson Method (Multiple roots)
Learning outcomes
 Recognizing the difference between bracketing and open
methods for root location.
 Understanding the fixed-point iteration method and how you
can evaluate its convergence characteristics.
 Knowing how to solve a roots problem with the Newton-
Raphson method and appreciating the concept of quadratic
convergence.
 Knowing how to implement both the secant and the modified
secant methods.
Introduction
 Bracketing methods:
◦ root is bracketed by the lower and upper limits of an
interval.
◦ it is convergent –move closer to the true value as iteration
progress.

 Open methods:
◦ require only a single starting value or two starting values
that do not necessarily bracket a root.
◦ based on formula to project from xi to xi+1in iterative
manner.
◦ may diverge as the computation progresses, but when they
do converge, they usually do so much faster than
bracketing methods.
Introduction

Diverging open method

Bracketing method
Converging open method
Fixed-Point Iteration
 Open method employs a formula to predict the roots.
 Simple fixed-point also called one-point iteration or
successive substitution
 Developing formula for simple fixed-point iteration:

 This is done by rearranging the function f(x) = 0 so that x is

on the left side of the equation,
x = g(x)  the formula for new estimate root xi+1 using initial
guess xi.
 Express x = g(x) in iterative formula to predict a new value of
x.
xi+1=g(xi)  Formula for simple fixed-point iteration
 Fixed-Point Iteration
 For example
f (x)  x 3  x 2  x  4  0
 x  4  x2  x3
 x  4  x  x3
f (x)  ln(x)  0
 x  ln(x)  x
f (x)  1  x  xex  0  ?
 The approximate error is given by
xi 1  xi
 a  100%
xi 1
Example 1
Fixed-Point Iteration
x
Use simple fixed-point iteration to locate the root of f (x)  e x

Solution:
Rewrite as x = g(x) by isolating x 
x  ex
xi 1  ex i  Simple fixed-point iteration formula

Start with an intial guess (here, 0)

 Continue until some tolerance is reached (root = 0.56714329)

Open method may converge OR diverge…
so need to check the possibility of convergence!
Example 1
Fixed-Point Iteration
Solution:

i xi |εa|% |εt|% |εt|i/|εt|i-1

0 0.0000 100.0000
1 1.0000 100.0000 76.322 0.763
2 0.3679 171.828 35.135 0.460
3 0.6922 46.854 22.050 0.628

10 0.5649 1.11 0.399 0.566

|εt|% is roughly proportional to the error from
previous iteration (~50–60%) –‘linear
convergence’ is a characteristic of fixed point
iteration!
Remark
 The method does not work when
◦ |g’(xi)| > 1 at all points in the interval [a,b]. It diverge from
xr either in staircase or in oscillation.
 The method is applicable only when
◦ |g’(xi)| < 1 at all points in the interval [a,b].

 How to overcome
◦ Choose different initial guess
◦ Modify the function g(x) into another form
◦ Choose another
method
Fixed-Point Iteration: Convergence
 Convergence:
◦ move closer to the true value as
computation progress
 Checking for possibility of convergence:
 1.Express x = g(x) as a pair of equations
◦ y1 = f(x1)= x & y2 = f(x2) = g(x)
◦ y1 & y2: component equations
 2.Plot y1& y2 separately.
◦ The root is at the intersection of the
component equations.
◦ This method can be used to check for
convergence of the iteration
◦ In general there are 4 possible shapes
of plot of y2= g(x)
 Fixed-Point Iteration: Convergence
 Initial guess = x0 & xi+1=g(xi)
 Open method will converge if
g (x)  1
 [Note: Slope of f(x1) = y1 = 1]
a. Convergent, 0≤g’(x)<1
b.
Convergent, -1<g’(x)≤0
c. Divergent, g’(x)>1
d. Divergent, g’(x)<-1
 Newton-Raphson
 The most widely used of all root-finding formula
 Based on forming the tangent line to the f(x) curve at some guess
x, then following the tangent line to where it crosses the x-axis.

 Derivation of formula:
 From the geometry of the graph, first
derivative is equivalent of the slope:

f (xi )
xi 1  xi 
f (xi )
 xi = initial guess
 xi+1 = improve estimate
  [Alternatively, the formula can be developed From Taylor series]
Newton-Raphson
1. Evaluate f’(x) symbolically
2. Use an initial guess of the root, x i, to estimate the new value of
the root, as
f (xi )
xi 1  xi 
f (xi )
3. Find the absolute relative
approximate error, as
 xi 1  xi
a  100%
xi 1
4. Repeat STEP 2 & 3 until one of the following is reached:
◦ a  s
 ◦ allowable number of iterations


 Example 2
Newton Raphson
x
Use Newton-Raphson method to estimate f (x)  e x,
employing initial guess x0 = 0.
f (x)  ex  x
f (x)   ex  1 

f (xi ) ex i  xi
Thus, xi 1  xi   xi 
f (xi ) ( ex i  1)

i xi |εt|%

0 0.000000000 100
 1 0.500000000 11.8
Decrease faster
2 0.566311003 0.147
3 0.567143165 0.000022
4 0.567143290 <10-8
 Remarks
 A convenience method for function whose derivative can be
evaluated analytically
 May not be convenience for function whose derivative cannot
be evaluated analytically
 Termination criterion:
xi 1  xi
a  s and  a  100%
xi 1
Advantages:
 ◦ Converges fast (if converge): The error of the (i+1)-th
iteration
 is roughly proportional to the square of the error
of the (i)-th iteration -this is called quadratic convergence
◦ Require only one guess
Remarks
Disadvantages:
◦ Divergence at inflection points
◦ Oscillations near local maximum and minimum
◦ Root jumping
◦ Division by zero

 No general convergence criteria for NR

◦ Some functions show slow or poor convergence
◦ Convergence depends on the nature of the curve & the
initial guess
◦ The only remedy is to have good initial guess (i.e.
sufficiently close to the root)
◦ Good guess can be obtained from the knowledge of the
physical problem or from the plot of the graph
Newton-Raphson
Near inflection point (f’(x) = 0) Root jumping

Oscillations Divisions by zero, f’(x) = 0

 Secant Method
 A slight variation of Newton’s method for functions whose derivatives are
difficult to evaluate.
 For these cases the derivative can be approximated
by a backward finite divided difference.

f (xi 1 )  f (xi )
f (xi ) 
xi 1  xi
 Substitution of this approximation for the derivative
to the Newton-Raphson method equation gives:

 f (xi )(xi 1  xi )
xi 1  xi 
f (xi 1)  f (xi )
 This method requires two initial estimates of x but does not require an
analytical expression of the derivative.
 The secant method has the same properties as Newton’s method.

Convergence is not guaranteed for all xo, f(x).

 Derivative is approximated  convergence (IF converge) is slower than NR
 Secant Method
1. Calculate the next estimate of the root from two initial guesses
using
f (xi )(xi 1  xi )
xi 1  xi 
f (xi 1)  f (xi )
2. Find the absolute relative
approximate error, as
 xi 1  xi
a  100%
xi 1

3. Repeat STEP 2 & 3 until one of the following is reached:

◦ a  s

◦ allowable number of iterations


 Example 3
Secant Method
x
Use the secant method to estimate the root of f (x)  e  x.
Start with initial estimates of x−1 = 0 and x0 = 1.0.
Recall that the true root is 0.56714329.
First iteration: 
x1  0 f (x1)  1.0000
x0  1 f (x0 )  0.63212
0.63212(0  1)
x1  1  0.61270 t  8.0%
1 (0.63212)
Second iteration:
x0  1 f (x0 )  0.63212
 x1  0.61270 f (x1)  0.07081
0.07081(1 0.61270)
x2  0.61270   0.56384 t  0.58%
0.63212  (0.07081)
Example 3
Secant Method
Third iteration
x1  0.61270 f (x1 )  0.07081
x2  0.56384 f (x2 )  0.00518
0.00518(0.61270  0.56384)
x3  0.56384   0.56717 t  0.0048%
0.07081 (0.00518)
Secant Method
Advantages;
◦ Converge fast, if converge
◦ Requires two initial guesses that do not need to bracket the
root

Drawbacks;
◦ Division by zero
◦ Root jumping
Secant vs. False Position Method

False Position Method

f (xu )(xl  xu )
xr  xu 
f (xl )  f (xu )

 Secant Method

f (xi )(xi 1  xi )
xi 1  xi 
f (xi 1 )  f (xi )


Comparison of several methods
 Modified Secant Method
 Secant method use 2 values to estimate the derivative:
f (xi )(xi 1  xi )
xi 1  xi 
f (xi 1)  f (xi )
 Modified Secant method use a small perturbation of the independent
variable to estimate f’(x).
 f (xi  xi )  f (xi ) Where = a small perturbation fraction
f (xi ) 
xi (eg x0= 1.0 , δ = 0.01)

 Substitution of this approximation for the derivative to the Newton-

Raphson method equation gives:
 xi f (xi )
xi 1  xi 
f (xi  xi )  f (xi )
Modified Secant Method
 The choice of value of δ is not automatic
◦ too small  increase round-off error
◦ too big  technique becomes inefficient & divergent
 Nice alternative for cases where evaluating derivative is difficult and
developing two initial guesses is inconvenient
Example 4
Modified Secant Method
Use the modified secant method to estimate the root of
f (x)  ex  x . Use a value of 0.01 for δ and start with x0 = 1.0.
Recall that the true root is 0.56714329.

First iteration:
x0  1 f (x0 )  0.63212
x0  x0  1.01 f (x0  x0 )  0.64578
0.01(0.63212)
x1  1  0.537263 t  5.3%
0.64578  (0.63212)
Second iteration:
x1  0.537263 f (x1 )  0.047083
x1  x1  0.542635 f (x1  x1 )  0.038579
0.005373(0.047083)
x2  0.537263   0.56701 t  0.0236%
0.038579  0.047083
 Example 4
Modified Secant Method
Third iteration
x2  0.56701 f (x2 )  0.000209
x2  x2  0.57268 f (x2  x2 )  0.00867
0.00567(0.000209)
x3  0.56701  0.567143 t  2.365  10 5%
0.00867  0.000209
 Multiple Root –Problem & Solution
 Multiple roots correspond to a point where the function is
tangent to x-axis  more than one root
f (x)  x 2  10x  25  (x  5)(x  5)  Double roots = 5's
f (x)  (x  3)(x  1)(x  1)  Double roots = 1's
f (x)  (x  3)(x  1)(x  1)(x  1)  Triple roots = 1's



 f(x) is tangent to x-axis & doesn’t cross  EVEN number of

Multiple roots
 Multiple roots pose some difficulties for many of the
numerical methods discussed earlier.
◦ The fact that the function does not change sign at even
multiple roots precludes the use of the bracketing methods
◦ Another possible problem is related to the fact that not
only f(x) but also f’(x) goes to zero at the root. This poses
problems for both the Newton-Raphson and secant
methods.

 To overcome these issues, a modified Newton-Raphson

method is used.
Modified Newton Raphson
f (xi )
 Newton Rahpson: xi 1  xi  [1]
f (xi )
f (xi )
 Set u(xi )  [2]
f (xi )
u(xi )
 Express NR as xi 1  xi  [3]
u(xi )
 Differentiate u(x) to find u’(x) and substitute in [3]
 Thus modified Newton-Raphson for multiple roots is:

f (xi ) f (xi ) Modified Newton-
xi 1  xi  Raphson Formula
[ f (xi )]2  f (xi ) f (xi )


 Example 5
Modified Newton-Raphson
Use both standard and modified Newton-Raphson methods to
evaluate the multiple roots of the following function with an
initial guess, x0= 0.
NOTE: True multiple roots are 1
f (x)  x 3  5x 2  7x  3
Standard NR
f (x)  3x  10x  7
2
i xi εt (%)
Modified NR 0 0.0000000 100
i xi εt (%)
1 0.4285714 57
 0 0.000000 100
2 0.6857143 31
1 1.105263 11
3 0.8328654 17
2 1.003082 0.31
4 0.9133290 8.7
3 1.000002 0.00024
5 0.9557833 4.4
6 0.9776551 2.2
 Example 6
Modified Newton-Raphson
 Use both standard and modified Newton-Raphson methods
to evaluate the single root of the following function with an
initial guess, x0= 4.
 NOTE: True single root is 3
f (x)  x 3  5x 2  7x  3
f (x)  3x 2  10x  7
Standard NR Modified NR
i xi εt (%) i xi εt (%)
 0 4.000000 33.00 0 4.000000 33.0
1 3.400000 13.00 1 2.636364 12.0
2 3.100000 3.30 2 2.820225 6.0
3 3.008696 0.29 3 2.961728 1.3
4 3.000075 0.0025 4 2.998479 0.051
Exercise 1: Question
gm  gcd 
Determine the real root of f (cd )  tanhwith t  v(t)
cd  m 
m = 80kg and v(4) = 36 m/s


(a) Graphically by taking x from 0 to 1 with increment of 0.1.
(b) Using bisection to determine the root to εs = 10%. Employ
initial guesses of xl = 0.1 and xu = 0.2.
(c) Perform the same computation as in (b) but use the false
position method and εs = 2 %.
 Exercise 1: Solution
Substituting the given values
9.81(80)  9.81cd 
f (cd )  tanh 4  36
cd  80 

(a) Graphically ?
(b) Bisection, first iteration:
 0.1  0.2
xr   0.15
2
f (0.1)  f (0.15)  8.60291(0.204516)  0.175944

Therefore, the root is in the first interval and the upper guess is
redefined as x = 0.15.
u
 Exercise 1: Solution
The second iteration is
0.1  0.15
xr   0.125
2
0.125  0.15
a  100%  20%
0.125
f (0.1)  f (0.125)  8.60291(0.318407)  0.273923
Therefore, the root is in the second interval and the lower guess
 is redefined as xu = 0.125. The iterations are displayed in the
 following table:
i xl xu xr εa

1 0.1 0.2 0.15 -

2 0.1 0.15 0.125 20%
3 0.125 0.15 0.1375 9.09%
 Example 1: Solution
c) False Position, first iteration:
1.19738(0.1 0.2)
xr  0.2   0.141809
0.860291 (1.19738)
f (0.1)(0.141809)  0.860291(0.0352109)  0.03029

Therefore, the root is in the first interval and the upper guess is redefined
as xu = 0.141809. The second iteration is:
 0.0352109(0.1 0.141809)
xr  0.141809   0.140165
0.860291 (0.0352109)
0.140165  0.141809
a  100%  1.17%
0.140165

Therefore, after only two iterations we obtain a root estimate of 0.140165

with an approximate error of 1.17% which is below the stopping criterion of
 2%.
Example 2: Question
Use simple fixed-point iteration to locate the root of
f (x)  sin( x)  x
Use an initial guess of xi = 0.5 and iterate until εa ≤ 2%.


Solution:

The function can be set up for fixed-point iteration by solving

it for x:
xi 1  sin( x i )
Using an initial guess of xi = 0.5.


Exercise 2: Solution
First iteration yields:
x1  sin( 0.5)  0.649637
0.649637  0.5
a  100%  23.034%
0.649637

Second iteration:
x1  sin( 0.649637)  0.721524
0.721524  0.649637
a  100%  9.9632%
0.721524
Exercise 2: Solution
The solutions are:

i xi |εa|% εt % εt,j/εt,j-1

0 0.500000 23.034 0.26865

1 0.649637 9.9632 0.11901 0.44300
2 0.721524 3.9123 0.04712 0.39596
3 0.750901 1.4691 0.00655 0.37660
 Exercise 3: Question
Determine the real root of f (x)  x 3  6x. 2Use
 11x
aninitial
6.1 guess of xi =
3.5 and iterate until εa ≤ 2%.

a) 
Using the Newton-Raphson method (three iterations, xi = 3.5).
b) Using the secant method (three iterations, xi–1 = 2.5 and xi = 3.5).
c) Using the modified secant method (three iterations, xi = 3.5, δ =
Solution:
0.01).

The function can be set up for Newton-Raphson by solving it

for x:
xi 1  xi 3  6xi 2  11xi  6.1
Using an initial guess of xi = 3.5.


 Exercise 3: Solution
a) Newton-Raphson Method to solve:
f (x)  x 3  6x 2  11x  6.1
f (xi ) xi 3  6xi 2  11xi  6.1
xi 1  xi   xi 
f (xi ) 3xi 2  12xi  11

i xi |εa|%

 0 3.500000 -
1 3.191304 9.673
2 3.068699 3.995
3 3.047317 0.702
 Exercise 3: Solution
b) Secant Method to solve:
f (x)  x 3  6x 2  11x  x  6.1
f (xi )(xi 1  xi )
xi 1  xi 
f (xi 1 )  f (xi )

x1  2.5
First iteration: and x0  3.5

xi 1  2.711111 with  a  29.098%


 x0  3.5 and x1  2.711111
Second iteration
xi 1  2.871091 with  a  5.572%


 Exercise 3: Solution
Third iteration:
x1  2.711111 and x2  2.871091
xi 1  3.221923 with  a  10.889%


i xi |εa|%

0 3.500000 -
1 2.711111 29.098
2 2.871091 5.572
3 3.221923 10.889
 Exercise 3: Solution
b) Modified Secant Method to solve:
f (x)  x 3  6x 2  11x  x  6.1
xi f (xi )
xi 1  xi 
f (xi  xi )  f (xi )

x0  3.5
First iteration: and x0  x0  3.535

x1  3.199597 with  a  9.389%


 x1  3.199597 and x1  x1  3.271725
Second iteration:
x2  3.075324 with  a  4.041%


 Exercise 3: Solution
Third iteration:
x2  3.075324 and x2  x2  3.143675
x3  3.048818 with  a  0.869%

 i xi |εa|%

0 3.500000 -
1 3.199597 9.389
2 3.075324 4.041
3 3.048818 0.869
What about modified Newton-Raphson?
f (xi ) f (xi )
xi 1  xi 
[ f (xi )]2  f (xi ) f (xi )
Roots of Equations
ROOTS
ROOTS OF
OF
EQUATIONS
EQUATIONS

GRAPHICAL
GRAPHICAL BRACKETING
BRACKETING OPEN
OPEN METHODS
METHODS
METHODS
METHODS METHODS
METHODS

ONE-POINT
ONE-POINT
BISECTION
BISECTION METHOD
METHOD ITERATION
ITERATION (FIXED
(FIXED
POINT)
POINT) METHOD
METHOD

FALSE-POSITION
FALSE-POSITION NEWTON-RAPHSON
NEWTON-RAPHSON
METHOD
METHOD METHODD
METHODD

SECANT
SECANT METHOD
METHOD

MODIFIED
MODIFIED NEWTON-
NEWTON-
RAPHSON
RAPHSON METHOD
METHOD
(MULTIPLE
(MULTIPLE ROOTS)
ROOTS)
 Next week
Linear algebraic equation - Gauss elimination