Numerical Methods (CENG 2084) Lecture Note

CHAPTER - IV
SOLUTIONS OF SYSTEMS OF NON LINEAR EQUATIONS

The problem of finding the solution of a set of non linear equations is much more difficult than
for linear equations. Moreover, there are no good and general methods for solving systems of
more than one non- linear equations.
Consider non- linear problem, where we want to solve simultaneously the system
f 1 ( x1 , x 2 ,..., x n ) = 0
f 2 ( x1 , x 2 ,..., x n ) = 0
(4.1)
......
f n ( x1 , x 2 ,..., x n ) = 0

The most commonly used methods of solution are the iterative methods and Newton – Raphson
method.
i. Iterative method
Fixed -Point iteration for Functions of Several Variables

Where each function fi of Eqn (4.1) can be thought of as mapping a vector x = (x1,x2,x3, , ,xn)t of
the n-dimensional space Rn into the real line R. This system of n nonlinear equations in n
unknowns can also be represented by defining a function F mapping Rn into Rn as

If vector notation is used to represent the variables x1,x2 xn, system (4.1) assumes the form

F(x) =0. (4.2)

The functions f1,f2,... , fn are the coordinate functions of F.

The following procedure can be used for solving non- linear problem by iterative method.
i. the ith equation is solved for xi, the system is changed into the fixed-point problem
ii. Using the initial guess x 2 , x3 … solve for x1

iii. Using the values of x1 from step ii and x3 ,x4, … solve for x 2

iv. Using the value of x1 from step ii and that of x 2 from step iii solve for x3
v. Repeat this procedure until the required accuracy is achieved. /interval is smaller than
some specified tolerance/

___________________________________________________________________________________________________________________
AAiT ADDIS ABABA INSTITUTE OF TECHNOLOGY by: Yisihak G. 1
 Numerical Methods (CENG 2084) Lecture Note
Example 1: solve the system of non linear equation by iterative method

If the ith equation is solved for xi, the system is changed into the fixed-point problem

To approximate the fixed point p, we choose x(0) = (0.1, 0.1, -0.1)t. The sequence of vectors
generated by

converges to the unique solution of (4.3). The results in Table 4.1 were generated until; the error
/X (k) - X (k-1) / < 10-5
Table 4.1
k X1(k) X2(k) X3(k) /X (k) - X (k-1)/
0 0.1 0.1 -0.1
1 0.49998333 0.00944115 -0.52310127 0.423
2 0.49999593 0.00002557 -0.52336331 9.4 x 10-3
3 0.5000000 0.00001234 -0.52359814 2.3 x 10-4
4 0.5000000 0.00000003 -0.52359847 1.2 x 10-5
5 0.5000000 0.00000002 -0.52359877 3.1 x 10-7
___________________________________________________________________________________________________________________
AAiT ADDIS ABABA INSTITUTE OF TECHNOLOGY by: Yisihak G. 2
 Numerical Methods (CENG 2084) Lecture Note

One way to accelerate convergence of the fixed-point iteration is to use the latest estimates
x1(k), … , xi-1(k) instead of x1(k-1), … , xi-1(k-1) to compute xi(k), as in the Gauss-Seidel method for
linear systems. The component equations then become

So the convergence was indeed accelerated for this problem by using the Gauss-Seidel method.
However, this method does not always accelerate the convergence.

Table 4.2
k X1(k) X2(k) X3(k) /X (k) - X (k-1)/
0 0.1 0.1 -0.10000000
1 0.49998333 0.02222979 -0.52304613 0.423
2 0.49997747 0.00002815 -0.52359807 2.2 x 10-2
3 0.5000000 0.00000004 -0.52359877 2.8 x 10-5
4 0.5000000 0.00000000 -0.52359877 3.8 x 10-8

ii. Newton – Raphson method

The problem in Example 1 of the previous section is transformed into a convergent fixed- point
problem by algebraically solving the three equations for the three variables x1, x2, and x3. It is,
however, rather unusual for this technique to be successful. In this section, we consider an
algorithmic procedure to perform the transformation in a more general situation.

One of the methods available for solving such a system /non linear system of equations/ is based
on a multi-dimensional version of Newton–Raphson method. The method consists of linearizing
the non-linear systems at each iteration step and solving system of linear equations.

To construct the algorithm that led to an appropriate fixed-point method in the one- dimensional
case, we found a function ϕ with the property that g(x) = x-ϕ(x)f(x) gives quadratic convergence
to the fixed point p of the function g(x). From this condition Newton's method evolved by
choosing ϕ(x)= l/f'(x), assuming that f'(x)≠0. Using a similar approach in the n-dimensional case
involves a matrix

___________________________________________________________________________________________________________________
AAiT ADDIS ABABA INSTITUTE OF TECHNOLOGY by: Yisihak G. 3
 Numerical Methods (CENG 2084) Lecture Note

where each of the entries aij(x) is a function from Rn into R. This requires that A(x) be
found so that G(x) =X-A(x) -1F(x) gives quadratic convergence to the solution of F(x) = 0,
assuming that A(x) is nonsingular at the fixed point p of G.

An appropriate choice for A (x) is, consequently, A (x) = J (x) since this satisfies condition
The function G is defined by
-1 (0)
G(x)=X-J(x) F(x), and the functional iteration procedure evolves from selecting x and
generating, for k > 1.
X (k) = G(x (k-1)) = x (k-1)-J(x (k-1)-1F(x (k-1))

This is called Newton's method for non-linear systems, and it is generally expected to give
quadratic convergence, provided that a sufficiently accurate starting value is known and J(p)-1
exists. The matrix J(x) is called the Jacobian matrix and has a number of applications in
analysis.

___________________________________________________________________________________________________________________
AAiT ADDIS ABABA INSTITUTE OF TECHNOLOGY by: Yisihak G. 4
 Numerical Methods (CENG 2084) Lecture Note

The disadvantages in Newton’s method are:-

Example 2: Solve the nonlinear system by Newton’s method

Newton’s method will be used to obtain this approximation using the initial approximation is
x(0)=(0.1,0.1,-0.1) and

Thus, at the kth step, the linear system J(x(k-1))y(k-1) = -F(x(k-1)) must be solved.

___________________________________________________________________________________________________________________
AAiT ADDIS ABABA INSTITUTE OF TECHNOLOGY by: Yisihak G. 5
 Numerical Methods (CENG 2084) Lecture Note

Taking x1(0)=0.1, x2(0)=0.1and x3(0)=-0.1 as initial approximation, equation (*) reduces to

3 9.99x 10-4 -9.99x 10-4 1.19995

Solving this linear system of equation, results in y1=0.39987, y2=- 0.0805331 and y3=-0.42121.
Use this results, the next approximation for the unknown variables can be calculated as

X1(1) = x1(0)+ y1=0.1+0.39987=0.49987

X2(1) = x2(0)+ y2=0.1-0.0805331=0.019466 and

X3(1) = x3(0)+ y3=-0.1-0.42121= -0.521520

Now substituting the calculated values of X1(1), X2(1) and X3(1) in eqn (*),we construct a new linear
system and solve it to obtain the new correction factor y1, y2, y3 and then calculate the new
approximations X1(2), X2(2) and X3(2). This process is continued until convergent is achieved.

The results using this iterative procedure are shown in table 4.3

Table 4.3
k X1(k) X2(k) X3(k) /X (k) - X (k-1) /
0 0.1 0.1 -0.1
1 0.49987 0.01946686 -0.52152047 0.422
2 0.50004593 0.00158859 -0.52355711 1.79 x 10-2
3 0.50000034 0.00001244 -0.52359845 1.58 x 10-3
4 0.50000000 0.00000000 -0.52359877 1.24 x 10-5
5 0.50000000 0.00000000 -0.52359877 0

The example illustrates that Newton's method can converge very rapidly once an approximation
is obtained that is near the true solution.
___________________________________________________________________________________________________________________
AAiT ADDIS ABABA INSTITUTE OF TECHNOLOGY by: Yisihak G. 6