Gaussian Elimination: Major: All Engineering Majors Author(s) : Autar Kaw
Gaussian Elimination: Major: All Engineering Majors Author(s) : Autar Kaw
Gaussian Elimination: Major: All Engineering Majors Author(s) : Autar Kaw
http://numericalmethods.eng.usf.e
du
Transforming Numerical Methods Education for STEM
Undergraduates
http://numericalmethods.eng.usf.
edu
Two steps
1. Forward Elimination
2. Back Substitution
Forward Elimination
The goal of forward elimination is to
transform the coefficient matrix into an
upper triangular matrix
25 5 1 x1 106.8
64 8 1 x 177.2
144 12 1 x3 279.2
5
1 x1 106.8
25
0 4.8 1.56 x 96.21
0
0.7 x3 0.735
0
Forward Elimination
A set of n equations and n unknowns
.
.
.
Forward Elimination
Step 1
For Equation 2, divide Equation 1a11by
multiply bya21 .
a21
a ( a11 x1 a12 x2 a13 x3 ... a1n xn b1 )
11
a21
a21
a21
a21 x1
a12 x2 ...
a1n xn
b1
a11
a11
a11
and
Forward Elimination
Subtract the result from Equation 2.
a21 x1 a22 x2 a23 x3 ... a2 n xn b2
a21
a21
a21
a21 x1 a a12 x2 ... a a1n xn a b1
11
11
11
_________________________________________________
a21
a21
a21
a22
a12 x2 ... a2 n
a1n xn b2
b1
a11
a11
a11
o
r
a x ... a x b
'
22 2
'
2n n
'
2
Forward Elimination
Repeat this procedure for the remaining
equations to reduce the set of equations
as
a11 x1 a12 x2 a13 x3 ... a1n xn b1
'
'
a22
x2 a23
x3 ... a2' n xn b2'
'
'
a32
x2 a33
x3 ... a3' n xn b3'
.
.
.
.
.
.
.
.
.
'
an' 2 x2 an' 3 x3 ... ann
xn bn'
End of Step 1
Forward Elimination
Step 2
Repeat the same procedure for the 3rd
term of Equation 3.
a11 x1 a12 x2 a13 x3 ... a1n xn b1
'
'
a22
x2 a23
x3 ... a2' n xn b2'
"
a33
x3 ... a3" n xn b3"
.
.
.
.
.
.
"
an" 3 x3 ... ann
xn bn"
End of Step 2
Forward Elimination
At the end of (n-1) Forward Elimination steps, the
system of equations will look like
a x ... a x b
"
33 3
"
3n n
.
.
.
n 1
"
3
.
.
.
n 1
ann xn bn
End of Step (n-1)
a11
0
0
a12
'
a22
0
a1n x1 b1
'
'
a 2 n x2
b2
"
"
a3n x3 b3
(n 1 )
(n-1 )
0 ann xn bn
a13
'
a23
a
"
33
Back Substitution
Solve each equation starting from the last
equation
5
1
25
0 4.8 1.56
0
0.7
0
x1 106.8
x 96.21
2
x3 0.735
Example of a system of 3
equations
"
a33
x3 ... an" xn b3"
.
.
.
n 1
.
.
.
n 1
ann xn bn
Back Substitution
Start with the last equation because it has only one unknown
( n 1)
n
( n 1)
nn
b
xn
a
Back Substitution
( n 1)
n
( n 1)
nn
b
xn
a
xi
aii
i 1
xi
bi
aij i 1 x j
j i 1
i 1
ii
for i n 1,...,1
for i n 1,...,1
THE END
http://numericalmethods.eng.usf.edu
Additional Resources
For all resources on this topic such as digital
audiovisual lectures, primers, textbook chapters,
multiple-choice tests, worksheets in MATLAB,
MATHEMATICA, MathCad and MAPLE, blogs, related
physical problems, please visit
http://numericalmethods.eng.usf.edu/topics/gaussia
n_elimination.html
http://numericalmethods.eng.usf
.edu
Example 1
The upward velocity of a rocket is given at three
different times
Table 1 Velocity vs. time data.
Time, t s
5
Velocity,v m/s
177.2
12
279.2
106.8
v t a t a t a ,
5 t 12.
Example 1 Cont.
Assume
v t a1t 2 a2t a3 , 5 t 12.
Results in a matrix template of the form:
t1
2
t
2
t32
t2
t12
t3
1
1
a1
v1
a v
2
2
a3
v3
25 5 1 a1
106.8
64 8 1 a 177.2
144 12 1 a3
279.2
Example 1 Cont.
5 1 a1
5 1 106.8
25
106.8
25
64
8 1 a 2 177.2 64
8 1 177.2
144 12 1 a 3
279.2
144 12 1 279.2
1. Forward Elimination
2. Back Substitution
Forward Elimination
144 12 1 279.2
25
Divide Equation 1 by 25
and
64
2.56
25
multiply it by 64,
64
Subtract the result from 64
Equation 2
0
.
8
12.8
1
2.56
4.8 1.56
177.2
273.408
96.208
5
1
106.8
25
0 4.8 1.56 96.208
144 12
1
279.2
25
Divide Equation 1 by 25
and
144
5.76
25
multiply it by 144,
.
12
1
Subtract the result from144
144
28.8 5.76
Equation 3
0 16.8 4.76
.
279.2
615.168
335.968
5
1
106.8
25
Substitute new equation
4
.
8
1
.
56
96
.
208
for Equation 3
5
4.8
106.8
1.56 96.208
1
Divide Equation 2 by
4.8
16.8multiply it by
and
3.5
16.8,
4 .8
0
0
0
16.8 4.76
335.968
0 .7
0.76
5
1
106.8
25
0 4.8 1.56 96.208
0.7
0.76
Back Substitution
Back Substitution
5
1
106.8
5
1
25
25
0 4.8 1.56 96.2 0 4.8 1.56
0
0
0
0.7 0.7
0
0.7
Solving for a3
0.7a3 0.76
0.76
a3
0 .7
a3 1.08571
a1 106.8
a 96.208
2
a3 0.76
0
0
0.7 a3 0.76
Solving for a2
4.8a2 1.56a3 96.208
96.208 1.56a3
a2
4.8
96.208 1.56 1.08571
a2
4.8
a2 19.6905
0
0
0.7 a3 0.76
Solving for a1
25a1 5a2 a3 106.8
106.8 5a2 a3
a1
25
106.8 5 19.6905 1.08571
25
0.290472
144 12 1 a3
279.2
a1 0.290472
a 19.6905
2
a3 1.08571
Example 1 Cont.
a1 0.290472
a 19.6905
2
a3 1.08571
Solution
The solution vector is
The polynomial that passes through the three data points is then:
v t a1t 2 a2t a3
129.686 m/s.
THE END
http://numericalmethods.eng.usf.edu
http://numericalmethods.eng.usf.edu
Pitfall#1. Division by
zero
10 x2 7 x3 3
6 x1 2 x2 3x3 11
5 x1 x2 5 x3 9
0 10 7
6 2
3
5 1 5
x1 3
x 11
2
x3 9
12 10
6 5
5
24 1 5
x1 15
x 14
2
x3 28
7 x1 15
0
6.5 x2 6.5
12 21 19 x3 2
12
0
10
1
5
x 1.751
2
x3 9
Exact Solution
x1
x
2
x3
1
1
1
5
x 1.751
2
x3 9
x1 0.9625
x 1.05
2
x3 0.999995
1
5
x 1.751
2
x3 9
x1 0.625
x 1.5
2
x3 0.99995
Avoiding Pitfalls
Increase the number of significant
digits
Decreases round-off error
Does not avoid division by zero
Avoiding Pitfalls
Gaussian Elimination with Partial
Pivoting
Avoids division by zero
Reduces round off error
THE END
http://numericalmethods.eng.usf.edu
http://numericalmethods.eng.usf
.edu
Avoiding Pitfalls
Increase the number of significant
digits
Avoiding Pitfalls
Gaussian Elimination with Partial
Pivoting
Avoids division by zero
Reduces round off error
a13
'
a23
'
32
'
33
a11
0
'
an 2
'
'
an 3 an 4
a1n x1 b1
'
'
a 2 n x2
b2
'
'
a3n x3 b3
'
'
ann xn bn
14
5.1 3.7
7
4
9
6
12
23
1
1
6
0 17 12
11
6 x1 5
2 x2 6
11 x3 8
8 x4 9
43 x5 3
4
12 1 11 x3 8
0
x
0
9
23
6
8
9
4
0 7 6
1
2 x5 6
Switched Rows
Gaussian Elimination
with Partial Pivoting
A method to solve simultaneous
linear equations of the form [A]
[X]=[C]
Two steps
1. Forward Elimination
2. Back Substitution
Forward Elimination
Same as nave Gauss elimination
method except that we switch rows
before each of the (n-1) steps of
forward elimination.
a12
a13
'
22
'
32
'
23
'
33
'
n2
'
n3
'
n4
a1n x1 b1
'
'
a 2 n x2
b2
'
'
a3n x3 b3
'
'
ann xn bn
a13
a
'
23
"
33
a1n
'
a2 n
a
(n 1 )
ann
"
3n
x1
x
2
b1
b'
2
x3 b
xn bn(n-1 )
"
3
"
a33
x3 ... an" xn b3"
.
.
.
n 1
.
.
.
n 1
ann xn bn
Back Substitution
( n 1)
n
( n 1)
nn
b
xn
a
i 1
xi
bi
i 1
aij x j
j i 1
i 1
ii
for i n 1,...,1
THE END
http://numericalmethods.eng.usf.edu
http://numericalmethods.eng.usf
.edu
Example 2
5 1 a1
25
106.8
64
8 1 a 2 177.2
144 12 1 a 3
279.2
Example 2 Cont.
25 5 1 a1
106.8
25 5 1 106.8
64 8 1 a 177.2
64
8
1
177
.
2
2
144 12 1 a 3
279.2
144 12 1 279.2
1. Forward Elimination
2. Back Substitution
Forward Elimination
25 , 64 , 144
Largest absolute value is 144 and exists in row
3.
Switch row 1 and row 3.
25
64
1 106.8
144 12 1 279.2
1 177.2 64 8 1 177.2
144 12 1 279.2
25 5 1 106.8
5
8
64
25
8
5
1 177.2
1 106.8
.
144 12 1 279.2 0.4444 63.99 5.333 0.4444 124.1
8
1 177.2
Subtract the result from 64
63.99 5.333 0.4444 124.1
Equation 2
.
1
279.2
Substitute new equation 144 12
0 2.667 0.5556 53.10
for Equation 2
25
5
1
106.8
25
106.8
.
144 12 1 279.2 0.1736 25.00 2.083 0.1736 48.47
25
25
0
1 106.8
2.917
0.8264 58.33
12
1
279.2
144
0 2.667 0.5556 53.10
0
2.667 , 2.917
Largest absolute value is 2.917 and exists in
row 3.
Switch row 2 and row 3.
12
1
279.2
12
1
279.2
144
144
0 2.667 0.5556 53.10 0 2.917 0.8264 58.33
0
0
0
0.2 0.23
12
1
279.2
144
0 2.917 0.8264 58.33
0.2
0.23
Back Substitution
Back Substitution
12
1
279.2
1
144
144 12
0 2.917 0.8264 58.33 0 2.917 0.8264
0
0
0
0.2 0.23
0
0.2
Solving for a3
0.2a3 0.23
0.23
a3
0 .2
1.15
a1
279.2
a 58.33
2
a3
0.23
0
0.2
0
a1 279.2
a 58.33
2
a3 0.23
Solving for a2
2.917 a2 0.8264a3 58.33
58.33 0.8264a3
a2
2.917
58.33 0.8264 1.15
2.917
19.67
0
0.2
0
a1 279.2
a 58.33
2
a3 0.23
Solving for a1
144
0.2917
144 12 1 a3
279.2
a1
0.2917
a 19.67
2
a3
1.15
http://numericalmethods.eng.usf
.edu
10 x1 7 x2 7
3 x1 2.099 x2 6 x3 3.901
5 x1 x2 5 x3 6
In matrix form
7 0
10
3 2.099 6
5
1 5
x1
7
x
3.901
2 =
x3
6
5
1 5 x3 6
7
0 x1 7
10
0 0.001 6 x 6.001
0
2.5
5 x3 2.5
0
2.5
5 x3 2.5
7
0 x1 7
10
0
x 2.5
2
.
5
5
0 0.001 6 x3 6.001
0
10 7
0 2.5
0
0 6.002
x1 7
x 2.5
2
x3 6.002
10 7
0 2.5
0 x1
7
5 x 2 2.5
6.002
6.002 x3
6.002
x3
1
6.002
2.5 5 x3
x2
1
2.5
7 7 x 2 0 x3
x1
0
10
X calculated
x1 0
x2 1
x3 1
X exact
x1 0
x 2 1
x3 1
THE END
http://numericalmethods.eng.usf.edu
Determinant of a Square
Matrix
Using Nave Gauss Elimination
Example
http://numericalmethods.eng.usf.edu
Theorem of Determinants
If a multiple of one row of [A]nxn is added or
subtracted to another row of [A]nxn to result
in [B]nxn then det(A)=det(B)
Theorem of Determinants
The determinant of an upper triangular matrix [A] nxn is
given by
aii
i 1
Forward Elimination of a
Square Matrix
Using forward elimination to transform [A]nxn to an
upper triangular matrix, [U]nxn.
A nn U nn
det A det U
Example
Using nave Gaussian elimination find
the determinant of the following
square matrix.
25 5 1
64 8 1
144 12 1
Forward Elimination
144 12 1
25
Divide Equation 1 by 25
and
64
2.56
25
multiply it by 64,
64
64
0
5
1
25
0 4.8 1.56
144 12
1
8
12.8
1
2.56
4.8 1.56
and
5.76
144 12
1 multiply it by 144,
25
.
25 5 1 5.76 144
28.8 5.76
.
144 12
144 28.8
0 16.8
1
5.76
4.76
5
1
25
0 4.8 1.56
0 16.8 4.76
0 16.8 4.76
Divide Equation 2 by
4.8
16.8multiply it by
and
3 .5
4 .8
16.8,
0
0
0
16.8 4.76
16.8 5.46
0
0.7
5
1
25
0 4.8 1.56
0.7
5
1
25 5 1
25
64 8 1 0 4.8 1.56
0
0.7
144 12 1
0
.
25 4.8 0.7
84.00
Summary
-Forward Elimination
-Back Substitution
-Pitfalls
-Improvements
-Partial Pivoting
-Determinant of a Matrix
Additional Resources
For all resources on this topic such as digital
audiovisual lectures, primers, textbook chapters,
multiple-choice tests, worksheets in MATLAB,
MATHEMATICA, MathCad and MAPLE, blogs, related
physical problems, please visit
http://numericalmethods.eng.usf.edu/topics/gaussia
n_elimination.html
THE END
http://numericalmethods.eng.usf.edu