06 Jacobi Gauss Seidel
06 Jacobi Gauss Seidel
06 Jacobi Gauss Seidel
methods
1. Jacobi Method
2. Gauss-Siedel Method
1
Iterative Methods
Iterative methods can be expressed in the
general form: x(k) =F(x(k-1))
where s s.t. F(s)=s is called a Fixed Point
Non Stationary:
x(k+1) =x(k)+akp(k)
where computation involves information that
change at each iteration
3
Iterative – Stationary
Jacobi
In the i-th equation solve for the value of xi while assuming the
other entries of x remain fixed:
bi mij x j bi mij x j
( k 1)
N
mij x j bi xi j i
j i
(k )
xi
j 1 mii mii
mij x j bi xi j i
j i j i
(k )
xi
j 1 mii mii
( k 1) j i j i
wxi (1 w) xi
(k ) (k ) (k )
xi xi
mii
x ( k ) D wL ( wU (1 w) D ) x k 1 w( D wL ) 1 b
1
x11
1
(b1 a12 x20 a1n xn0 ) 1 i 1 n
a11
k 1
xi bi
aii
aij x aij x
k
j
k
j
1
j 1 j i 1
x12 (b2 a21 x10 a23 x30 a2 n xn0 )
a22
1
x1n (bn an1 x10 an 2 x20 ann1 xn01 )
ann
7
Gauss-Seidel (GS) iteration
a11x1 a12 x2 a1n xn b1 x10
Use the latest a21x1 a22 x2 a2 n xn b2 0
x2
update x
0
0
an1 x1 an 2 x2 ann xn bn xn
x11
1
(b1 a12 x20 a1n xn0 ) 1 i 1 n
a11
k 1
xi bi
aii
a ij x k 1
j aij x k
j
1 j 1 j i 1
x2
1
(b2 a21 x1 a23 x3 a2 n xn )
1 0 0
a22
1
x1n (bn an1 x11 an 2 x12 ann1 x1n 1 )
ann
8
Gauss-Seidel Method
An iterative method.
Basic Procedure:
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after each iteration
to check if error is within a pre-specified tolerance.
9
Gauss-Seidel Method
Why?
The Gauss-Seidel Method allows the user to control round-off error.
10
Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:
If: the diagonal elements are
a11x1 a12 x2 a13 x3 ... a1n xn b1 non-zero
a21x1 a22 x2 a23 x3 ... a2n xn b2 Rewrite each equation solving
. .
. . for the corresponding unknown
. .
ex:
an1x1 an 2 x2 an3 x3 ... ann xn bn
First equation, solve for x1
Second equation, solve for x2
11
Gauss-Seidel Method
Algorithm
Rewriting each equation
c1 a12 x2 a13 x3 a1n xn
x1 From Equation 1
a11
c2 a21 x1 a23 x3 a2 n xn
x2 From equation 2
a22
cn 1 an 1,1 x1 an 1, 2 x2 an 1,n 2 xn 2 an 1,n xn From equation n-1
xn 1
an 1,n 1
cn an1 x1 an 2 x2 an ,n 1 xn 1
xn From equation n
ann
12
Gauss-Seidel Method
Algorithm
General Form of each equation
n
a
n
c1 a1 j x j cn 1 n 1, j xj
j 1 j 1
j n 1
x1
j 1 xn 1
a11 an 1,n 1
n
c n a nj x j
n
c2 a2 j x j
j 1 j 1
j n
x2
j 2
xn
a 22 a nn
13
Gauss-Seidel Method
Algorithm
General Form for any row ‘i’
n
ci aij x j
j 1
j i
xi , i 1,2,, n.
aii
14
Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X] Use rewritten equations to solve for
each value of xi.
Important: Remember to use the
x1 most recent value of xi. Which
x means to apply values calculated to
2 the calculations remaining in the
current iteration.
xn -1
xn
15
Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error
xinew xiold
a i new
100
xi
So when has the answer been found?
16
Suppose that for conciseness we limit ourselves to a
33 set of equations.
j 1 j 1
b a x a x
x1j 1 12 2 13 3
(11.1a)
a11
b2 a21 x1j a23 x3j 1
x
2
j
(11.1b)
a22
b a x j
a x j
x3j 3 31 1 32 2
(11.1c)
a33
where j and j -1 are the present and previous iterations.
To start the solution process, initial guesses must be
made for the x’s. A simple approach is to assume that
they are all zero.
17
Convergence can be checked
using the criterion that for i,
xij xij 1
a ,i j
100% s
xi
18
Graphical depiction of the difference between (a) the Gauss-Seidel and (b) the
Jacobi iterative methods for solving simultaneous linear algebraic equations.
19
Jacobi Iterative Technique
20
Convert the set Ax = b in the form of x =
Tx + c.
1 1 3
x1 x2 x3
10 5 5
1 1 3 25
x2 x1 x3 x4
11 11 11 11
1 1 1 11
x3 x1 x2 x4
5 10 10 10
3 1 15
x4 x2 x3
8 8 8
21
Start with an initial approximation of:
0, x2 0, x3 0 and x4 0.
( 0) ( 0) ( 0) ( 0)
x 1
1 (0) 1 (0) 3
x3
(1)
x1 x2
10 5 5
1 (0) 1 (0) 3 (0) 25
x3 x4
(1)
x2 x1
11 11 11 11
1 (0) 1 (0) 1 (0) 11
x1
(1)
x3 x2 x4
5 10 10 10
3 (0) 1 (0) 15
x2 x3
(1)
x4
8 8 8
22
1 1 3
(0)
(1)
x1 (0)
10 5 5
1 1 3 25
(0) (0)
(1)
x2 (0)
11 11 11 11
1 1 1 11
(0) (0) (0)
(1)
x3
5 10 10 10
3 1 15
(0) (0)
(1)
x4
8 8 8
0.6000, x 2 2.2727,
(1) (1)
x1
24
1 ( k 1 ) 1 ( k 1 ) 3
x1 x3
(k)
x2
10 5 5
1 ( k 1 ) 1 ( k 1 ) 3 ( k 1 ) 25
x3 x4
(k)
x2 x1
11 11 11 11
1 ( k 1 ) 1 ( k 1 ) 1 ( k 1 ) 11
x1
(k)
x3 x2 x4
5 10 10 10
3 ( k 1 ) 1 ( k 1 ) 15
x2 x3
(k)
x4
8 8 8
25
Results of Jacobi Iteration:
k 0 1 2 3
(k ) 0.0000 0.6000 1.0473 0.9326
x1
(k ) 0.0000 2.2727 1.7159 2.0530
x2
(k ) 0.0000 -1.1000 -0.8052 -1.0493
x3
(k ) 0.0000 1.8750 0.8852 1.1309
x4
26
Gauss-Seidel Iterative Technique
27
1 ( k 1 ) 1 ( k 1 ) 3
x3
(k)
x1 x2
10 5 5
1 ( k 1 ) 1 ( k 1 ) 3 ( k 1 ) 25
x3 x4
(k )
x2 x1
11 11 11 11
1 ( k 1 ) 1 ( k 1 ) 1 ( k 1 ) 11
x1
(k )
x3 x2 x4
5 10 10 10
3 ( k 1 ) 1 ( k 1 ) 15
x2 x3
(k )
x4
8 8 8
28
1 ( k 1 ) 1 ( k 1 ) 3
x1 x3
(k)
x2
10 5 5
1 (k) 1 ( k 1 ) 3 ( k 1 ) 25
x3 x4
(k)
x2 x1
11 11 11 11
1 (k) 1 (k) 1 ( k 1 ) 11
x1
(k)
x3 x2 x4
5 10 10 10
3 (k) 1 (k) 15
x2 x3
(k)
x4
8 8 8
29
Results of Gauss-Seidel Iteration:
(Blue numbers are for Jacobi iterations.)
k 0 1 2 3
(k ) 0.0000 0.6000 1.0300 1.0065
x1
0.6000 1.0473 0.9326
(k ) 0.0000 2.3272 2.0370 2.0036
x2 2.2727 1.7159 2.0530
(k ) 0.0000 -0.9873 -1.0140 -1.0025
x3 -1.1000 -0.8052 -1.0493
(k ) 0.0000 0.8789 0.9844 0.9983
x4 1.8750 0.8852 1.1309
30
The solution is: x1= 1, x2 = 2, x3 = -1, x4 = 1
31
EXAMPLE Gauss-Seidel Method
Problem Statement. Use the Gauss-Seidel method to
obtain the solution for
3x1 0.1x2 0.2 x3 7.85
0.1x1 7 x2 0.3x3 19.3
0.3x1 0.2 x2 10 x3 71.4
32
7.85 0.1x2 0.2 x3
x1 (E11.1.1)
3
19.3 0.1x1 0.3x3
x2 (E11.1.2)
7
71.4 0.3x1 0.2 x2
x3 (E11.1.3)
10
By assuming that x2 and x3 are zero
7.85 0.10 0.20
x1 2.616667
3
This value, along with the assumed value of x3 =0,
can be substituted into Eq.(E11.1.2) to calculate
19.3 0.12.616667 0.30
x2 2.794524
7
33
The first iteration is completed by substituting the
calculated values for x1 and x2 into Eq.(E11.1.3) to yield
71.4 0.32.616667 0.2 2.794524
x3 7.005610
10
For the second iteration, the same process is repeated to
compute
7.85 0.1 2.794524 0.27.005610
x1 2.990557
3
19.3 0.12.990557 0.37.005610
x2 2.499625
7
71.4 0.32.990557 0.2 2.499625
x3 7.000291
10
34
The method is, therefore, converging on the true
solution. Additional iterations could be applied to
improve the answers. Consequently, we can estimate
the error. For example , for x1
2.990557 2.616667
a ,1 100% 12.5%
2.990557
35
Example: Unbalanced three phase load
Three-phase loads are common in AC systems. When the system
is balanced the analysis can be simplified to a single equivalent circuit
model. However, when it is unbalanced the only practical solution
involves the solution of simultaneous linear equations. In a model the
following equations need to be solved.
0.7460 0.4516 0.0100 0.0080 0.0100 0.0080 I ar 120
0.4516 0.7460
0.0080 0.0100 0.0080 0.0100 I ai 0.000
0.0100 0.0080 0.7787 0.5205 0.0100 0.0080 I br 60.00
0.0080 0.0100 0.5205 0.7787 0.0080 0.0100 I bi 103.9
0.0100 0.0080 0.0100 0.0080 0.8080 0.6040 I cr 60.00
0.0080 0.0100 0.0080 0.0100 0.6040 0.8080 ci 103.9
I
Find the values of Iar , Iai , Ibr , Ibi , Icr , and Ici using the Gauss-Seidel
method.
36
Example: Unbalanced three phase load
Rewrite each equation to solve for each of the unknowns
120.00 0.4516I ai 0.0100 I br 0.0080I bi 0.0100 I cr 0.0080I ci
I ar
0.7460
0.000 0.4516 I ar 0.0080 I br 0.0100 I bi 0.0080 I cr 0.0100 I ci
I ai
0.7460
60.00 0.0100 I ar 0.0080I ai 0.5205I bi 0.0100 I cr 0.0080I ci
I br
0.7787
103.9 0.0080 I ar 0.0100 I ai 0.5205I br 0.0080 I cr 0.0100 I ci
I bi
0.7787
60.00 0.0100 I ar 0.0080I ai0.0100 I br 0.0080I bi 0.6040I ci
I cr
0.8080
I ar 20
I 20
ai
Initial Guess: I br 20
I bi 20
I cr 20
I ci 20
38
Example: Unbalanced three phase load
Substituting the guess values into the first equation
120 0.4516I ai 0.0100 I br 0.0080I bi 0.0100 I cr 0.0080I ci
I ar
0.7460
172.86
Substituting the new value of Iar and the remaining guess values into
the second equation
39
Example: Unbalanced three phase load
Substituting the new values Iar , Iai , and the remaining guess values into
the third equation
60.00 0.0100 I ar 0.0080I ai 0.5205I bi 0.0100 I cr 0.0080I ci
I br
0.7787
67.039
Substituting the new values Iar , Iai , Ibr , and the remaining guess values into
the fourth equation
103.9 0.0080 I ar 0.0100 I ai 0.5205I br 0.0080 I cr 0.0100 I ci
I bi
0.7787
89.499
40
Example: Unbalanced three phase load
Substituting the new values Iar , Iai , Ibr , Ibi , and the remaining guess
values into the fifth equation
Substituting the new values Iar , Iai , Ibr , Ibi , Icr , and the remaining guess
value into the sixth equation
41
Example: Unbalanced three phase load
At the end of the first iteration, the solution matrix is:
I ar 172.86
I 105.61
ai
I br 67.039
I
bi 89.499
I cr 62.548
ci
I 176 .71
42
Example: Unbalanced three phase load
Calculating the absolute relative approximate errors
172.86 20 62.548 20
a 1 100 88.430% a 5 100 131.98%
172.86 62.548
105.61 20 176.71 20
a 2 100 118.94% a 6 100 88.682%
105.61 176.71
67.039 20
a 3 100 129.83% The maximum error after
67.039
the first iteration is:
89.499 20 131.98%
a 100 122.35%
4
89.499 Another iteration is needed!
43
Example: Unbalanced three phase load
I ar 172.86
I 105.61
Starting with the values obtained in iteration #1 ai
I br 67.039
I
bi 89.499
I cr 62.548
I ci 176.71
44
Example: Unbalanced three phase load
Substituting the new value of Iar and the remaining values from
Iteration 1 into the second equation
Substituting the new values Iar , Iai , and the remaining values from
Iteration 1 into the third equation
45
Example: Unbalanced three phase load
Substituting the new values Iar , Iai , Ibr , and the remaining values from
Iteration 1 into the fourth equation
103.9 0.0080 I ar 0.0100 I ai 0.5205I br 0.0080 I cr 0.0100 I ci
I bi
0.7787
44.299
Substituting the new values Iar , Iai , Ibr , Ibi , and the remaining values
From Iteration 1 into the fifth equation
46
Example: Unbalanced three phase load
Substituting the new values Iar , Iai , Ibr , Ibi , Icr , and the remaining
value from Iteration 1 into the sixth equation
47
Example: Unbalanced three phase load
Calculating the absolute relative approximate errors for
the second iteration
57.259 (62.548)
a 5 100 209.24%
99.600 172.86 57.259
a 1 100 73.552%
99.600
87.441 176.71
a 6 100 102.09%
60.073 (105.61) 87.441
a 2 100 75.796%
60.073
The maximum error after
a 3
136.35 (67.039)
100 50.762% the second iteration is:
136.35
209.24%
44.299 (89.499)
a 4 100 102.03% More iterations are needed!
44.299
48
Example: Unbalanced three phase load
Repeating more iterations, the following values are obtained
Iteration Iar Iai Ibr Ibi Icr Ici
1 172.86 −105.61 −67.039 −89.499 −62.548 176.71
2 99.600 −60.073 −136.15 −44.299 57.259 87.441
3 126.01 −76.015 −108.90 −62.667 −10.478 137.97
4 117.25 −70.707 −119.62 −55.432 27.658 109.45
5 119.87 −72.301 −115.62 −58.141 6.2513 125.49
6 119.28 −71.936 −116.98 −57.216 18.241 116.53
Iteration a 1 % a 2 % a 3 % a 4 % a 5 % a 6 %
1 88.430 118.94 129.83 122.35 131.98 88.682
2 73.552 75.796 50.762 102.03 209.24 102.09
3 20.960 20.972 25.027 29.311 646.45 36.623
4 7.4738 7.5067 8.9631 13.053 137.89 26.001
5 2.1840 2.2048 3.4633 4.6595 342.43 12.742
6 0.49408 0.50789 1.1629 1.6170 65.729 7.6884
49
Example: Unbalanced three phase load
I ar 119.28
I 71.936
After six iterations, ai The maximum error after
the solution matrix is I br 116.98 the sixth iteration is:
I
bi 57.216 65.729%
I cr 18.241
I ci 116.53
The absolute relative approximate error is still high, but allowing for
more iterations, the error quickly begins to converge to zero.
What could have been done differently to allow for a faster
convergence?
50
Example: Unbalanced three phase load
Repeating more iterations, the following values are obtained
Iteration a 1 % a 2 % a 3 % a 4 % a 5 % a 6 %
51
Example: Unbalanced three phase load
I ar 119.33
I 71.973
After 33 iterations, the solution matrix is ai
I br 116.66
I
bi 57.432
I cr 13.940
I ci 119.74
52
Gauss-Seidel Method: Pitfall
Even though done correctly, the answer may not converge to the
correct answer.
This is a pitfall of the Gauss-Siedel method: not all systems of
equations will converge.
Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.
54
Gauss-Seidel Method: Example 2
Given the system of equations The coefficient matrix is:
12 x1 3x2- 5x3 1 12 3 5
x1 5 x2 3x3 28 A 1 5 3
3x1 7 x2 13x3 76 3 7 13
55
Gauss-Seidel Method: Example 2
Checking if the coefficient matrix is diagonally dominant
a11 12 12 a12 a13 3 5 8
12 3 5
A 1 5 3 a22 5 5 a21 a23 1 3 4
3 7 13
a33 13 13 a31 a32 3 7 10
The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method
56
Gauss-Seidel Method: Example 2
Rewriting each equation With an initial guess of
12 3 5 a1 1 x1 1
1 5 3 a 28 x 0
2 2
3 7 13 a3 76 x3 1
1 3 x 2 5 x3 1 30 51
x1 x1 0.50000
12 12
28 x1 3 x3 28 0.5 31
x2 x2 4.9000
5 5
76 3 x1 7 x 2 76 30.50000 74.9000
x3 x3 3.0923
13 13
57
Gauss-Seidel Method: Example 2
The absolute relative approximate error
0.50000 1.0000
a 1 100 100.00%
0.50000
4.9000 0
a 2
100 100.00%
4.9000
3.0923 1.0000
a 3
100 67.662%
3.0923
The maximum absolute relative error after the first iteration is 100%
58
Gauss-Seidel Method: Example 2
After Iteration #1
x1 0.5000
x 4.9000
2
x3 3.0923
76 30.14679 74.900
x3 3.8118
13
59
Gauss-Seidel Method: Example 2
Iteration #2 absolute relative approximate error
0.14679 0.50000
a 1 100 240.61%
0.14679
3.7153 4.9000
a 2
100 31.889%
3.7153
3.8118 3.0923
a 3
100 18.874%
3.8118
The maximum absolute relative error after the first iteration is 240.61%
This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?
60
Gauss-Seidel Method: Example 2
Repeating more iterations, the following values are obtained
Iteration a1 a 1 % a2 a 2 % a3 a 3 %
x1 0.99919 x1 1
The solution obtained x 3.0001 is close to the exact solution of x 3 .
2 2
x3 4.0001 x3 4
61
Gauss-Seidel Method: Example 3
Given the system of equations
62
Gauss-Seidel Method: Example 3
Conducting six iterations, the following values are obtained
Iteration a1 a 1 % A2 a 2 % a3 a 3 %
1 21.000 95.238 0.80000 100.00 50.680 98.027
2 −196.15 110.71 14.421 94.453 −462.30 110.96
3 −1995.0 109.83 −116.02 112.43 4718.1 109.80
4 −20149 109.90 1204.6 109.63 −47636 109.90
5 2.0364×105 109.89 −12140 109.92 4.8144×105 109.89
6 −2.0579×105 109.89 1.2272×105 109.89 −4.8653×106 109.89
63
Gauss-Seidel Method
The Gauss-Seidel Method can still be used
3 7 13
The coefficient matrix is not
diagonally dominant
A 1 5 3
12 3 5
But this is the same set of 12 3 5
equations used in example #2,
which did converge.
A 1 5 3
3 7 13
64
Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equations
x1 x2 x3 3
2 x1 3x2 4 x3 9
x1 7 x2 x3 9
65