Algorithms Course notes-2018 v.

2
Nii Quaynor, Ike Mensah, Wilberforce

Course description
Concepts and techniques fundamental to the science of programming are
developed. Methods of enquiry of algorithms of design, theory and
implementation in programs and architectures. Topics include analysis of
best, average and worst case of time and space complexity. Proof of
correctness of algorithms and use of invariants in making proofs of
iterative and recursive algorithms. Introduction to numerical, recursion,
sequential and linked allocation, sorting, searching, encryption, grammar
and graphs algorithms. The African calculating instrument is also explored.

The Notes

Algorithms
Informally, an Algorithm is a well defined computational procedure that
takes some values as inputs and produces some values as output. Thus a
sequence of computational steps that transform input into output

There are three main methods of enquiry, design, theory and

implementation

Design: creation of algorithms, programs and architectures

Abstraction as a way of treating complex operations as “primitive” while
writing algorithms in terms meaningful in the problem

Recursion as a tool for controlling algorithms

Theory: mathematical modeling and analysis of algorithms, programs and

architectures

Mathematics to predict execution time of algorithms

Mathematics to verify correctness of algorithms

Implementation: empirical analysis

Use scientific methods to study algorithms, programs and architectures

1
Rigorous notion of experiments

Techniques for collecting and analysis of data on execution times of

programs

Program is software implementation of algorithms

Architecture is hardware/ucode implementation of algorithms

Problems
Must be general enough to appear over and over in slightly different forms
(“instances”)

What distinguishes problem instance is called “inputs” or “parameters”

There’s a “characteristic function” to tell whether a potential answer is right

or wrong

Algorithms
Process for solving a problem

It must be unambiguous (describe every step of process in enough detail

that a machine can carry out algorithm)

It must always solve the problem

Defined as a finite, ordered sequence of unambiguous steps that lead to a

solution to a problem

Algorithms and complexity

Complexity captures the rate of growth of time and space ie resources
required to solve larger and larger instances of problem

We associate an integer n (called size) as measure of quantity of input

Examples
Size of matrix multiplication might be the largest dimension of the two
matrices

Size of a graph might be number edges

2
The time needed by an algorithm is expressed as a f(size of problem)
called time complexity of an algorithm

The limiting behavior as n -> infinity is called the asymptotic time

complexity

Example: suppose process input size n in time g(n)=c*(n**2) for some

constant c, then time complexity is of order of n**2

More precisely, a function g(n) is said to be O(f(n)), if there exist a constant

c such that g(n)<=c*f(n), for all finite non negative values of n

In practice we look at large terms in T(n), time to execute

The time complexity may take different functions

Examples: n, n*log(n), n**2, n**3, 2**n, log(n), sqrt (n), n!,….

To get a feel for how time complexity affects amount of work done
consider a machine that one unit of work takes 1 msec ie 1/1000 secs and
ask what is the maximum problem size can solve in 1 second for each of
these time complexities: n, n*log(n), n**2, n**3 and 2**n

Use formula (complexity*(1/1000)) = 1 sec and create table like this

Application Time complexity Max problem size in 1 sec

A1 n 1000

A2 n*log(n) 140

A3 n**2 31

A4 n**3 10

A5 2**n 9

A1
n*(1/1000) = 1

Therefore n = 1000

3
A2
n*log (n)*(1/1000) = 1
n*log(n) = 1000

A3
(n**2)*(1/1000) = 1

n**2 = 1000

n = sqrt (1000)

n = 31

A4
(n**3)*(1/1000) = 1

n = cube root (1000)

n = 10

A5
(2**n)*(1/1000) = 1

2**n = 1000

Log(2**n) = Log(1000)

n = Log (1000) = 9

Problem: {determine if an element is in a list}

Design
Objects from problem statement include list a[1..n] and element a[i]. The
element looking for x, have or not found it.

That is:
found, x, i, n, a[1..n]

Procedure is to look at each element and decide if equal to item x

4
x=3; n=40; i=1
found=false

while ((i <= n) & ( not found)) {

If (x==a[i]) {

found=true

i=i+1

printf(“%d”, found)

Line Statement Cost Number of times

executed

1 Initialization C1 1

2 While ((i <= n) & ( ! C2 n+1

found) ) {

3 If (x==a[i]) { C3 n

4 found=true} C4 0

5 i++} C5 1

5
Theory - complexity

Note that the guard ie the test condition of an iteration statement is

executed an extra time to determine if the end conditions satisfied

Time complexity
T = c1 + c2(n+1) + c3n + c5n + c6
T = (c2 + c3 + c5)n + (c1 + c2 + c6)

Similar to y = ax + b

As can see the algorithm behaves linear ie O(n)

Minimum
The minimum number of comparisons is when i=1
The number of comparisons in general is equal to i+1 for i=1 it’s equal to 2

Maximum
The maximum number of comparisons is when i=n and therefore equal to
(n+1)

Average
To determine the average for a size n taking into account that the
algorithm terminated when an item x is found requires more calculations

When found=1 we terminate on the next comparison in the while loop

It is equally likely to find x in any position i for element a[i]
Hence can be found with number of comparisons:

i = 1+1, 1+2, 1+3,……,(1+n)

Therefore number of comparisons C = sum(i=1,,n) of (i+1)

Equal to sum(i)+sum(1)
Equal to (n*(n+1))/2 + n

There are n places to find x at ie. at i=1,2,3,,,,n

6
The average A = C/n

Equal to (n+1)/2 + n

Equal to n/2 + 1.5

The average A is O(n)

Theory - correctness

Theorem: the algorithm terminates

Proof:
The algorithm starts with i less than n. Initialized with i=1 and n=40

The algorithm terminates when i > n

Every iteration increment i by 1

When execute loop often enough i > n

Therefore the algorithm terminates

Problem: sort n elements in increasing order using bubble sort

algorithm

We define the sorting problem

Input: sequence of n numbers <a1,a2,…,an>
Output: a permutation (reordering) <a1’,a2’,…,an’> of the input sequence
such that

7
a1’ <= a2’ <= a3’ … <= an’

Theory - complexity

Line Statement Cost #times

1 for i = 1 to (n-1) { C1 n

2 for j=I+1 to n { C2 Sum:1..(n-1) of ti

3 if (a[i] > a[j]) { C3 Sum:1..(n-1) of (ti-1)

4 Swap(i,j) C4 Sum:1..(n-1) of (ti-1)

Assume a function swap(i,j) exist to swap elements a[i] and a[j] such as

function swap(int i, int j) {

int temp;
temp=a[i];
a[i]=a[j];
a[j]=temp;
}

Let ti = #times line 2 is executed

T(n)= c1*n + c2*(Sum:1..(n-1) of ti) + c3*(Sum:1..(n-1) of (ti-1)) + c4*(Sum:1..

(n-1) of (ti-1))

Best case is same as worst case

When ti = i
i Action #times(ti)

1 j=2..n n-1

2 j=3..n n-2

n-1 j=n..n 1

As we observe from the table, the ti = i

8
We know,
Sum:1..(n-1) of i is equal to ((n-1)*n)/2
And
Sum:1..(n-1) of (i-1) is equal to Sum:1..(n-1) of i minus Sum:1..(n-1) of 1
Sum:1..(n-1) of 1 is equal to (n-1)

Hence, Sum:1..(n-1) of (i-1) Is equal to ((n-1)*n)/2 - (n-1)

Equal to ((n-1)*(n-2))/2

Therefore, T(n) = c1*n + c2*(((n-1)*n)/2) + c3*(((n-1)*(n-2))/2) +

c4*(((n-1)*(n-2))/2)

Equal to (c2/2 + c3/2 + c4/2)* n**2 + (c1 - c2/2 -3*c3 - 3*c4)*n + (c3 + c4)

Similar to y = ax**2 + b*x + c, a quadratic function hence time complexity

is order of O(n**2)

Problem: sort n elements into this increasing order using insertion

sort

We define the sorting problem

Input: sequence of n numbers <a1,a2,…,an>
Output: a permutation (reordering) <a1’,a2’,…,an’> of the input sequence
such that
a1’ <= a2’ <= a3’ … <= an’

The numbers to sort are known as keys. The input comes in form of an
array

Insertion sort is efficient algorithm for sorting small size inputs

9
Theory-complexity

Line Statement Cost #times

1 for j=2 to a.length { C1 n

2 key = a[j] C2 n-1

3 //insert a[j] into sorted 0 n-1

a[1..(j-1)]

4 i = (j-1) c4 n-1

5 while i >0 & a[i]> key { c5 sum:j=2..n of tj

6 a[i+1] = a[i] c6 sum:j=2..n of (tj-1)

7 i = i - 1} c7 sum:j=2..n of (tj-1)

8 a[i+1] = key} c8 n-1

Let tj = number of times line 5 (while) is executed for that value j

Tests execute one more time than the body of loops (1,5)

T(n) = c1*n + c2*(n-1) + c4*(n-1) + c5*(sum:j=2..n of tj) + c6*(sum:j=2..n of

(tj-1)) + c7*(sum:j=2..n of (tj-1)) + c8*(n-1)

Best case
List is already sorted
For each j=2, 3, .., n find that a[i]<= key and line 5 fail when i has initial
value of (j-1)

Therefore tj=1 for j=2,3,..,n

With tj=1 then (tj - 1) = 0 and simplifies the T(n) as

T(n) = c1*n + c2*(n-1) + c4*(n-1) + c5*(n-1) + c8*(n-1)

T(n) = (c1 + c2 + c4 + c5 + c8)*n + (c2 + c4 + c5 + c8)

Similar to ax + b, linear O(n)

10
Worst case
The array is in reverse order
Have to compare each element a[j] to all elements in subarray a[1..(j-1)]

So, tj = j for j = 2,3,..,n

Note tj is one more

Sum:j=2..n of j, is equal to n(n+1)/2 - 1

And
Sum:j=2..(j-1), is equal to n(n-1)/2

T(n) = c1*n + c2*(n-1) + c4*(n-1) + c5*(sum:j=2..n of tj) + c6*(sum:j=2..n of

(tj-1)) + c7*(sum:j=2..n of (tj-1) + c8*(n-1)

T(n) = c1*n + c2*(n-1) + c4*n-1) + c5*(n(n+1)/2 - 1) + c6*(n(n-1)/2) +

c7*(n(n-1)/2) + c8*(n-1)

T(n) = ((c5 + c6 + c7)/2)*n**2 + (c1+c2+c4+c5/2-c6/2 - c7/2 + c8)*n -

(c2+c4+c5+c8)

Similar to a*x**2 + b*x + c, a quadratic function and O(n**2)

Correctness

What it means for an algorithm to work or considered “correct”

An algorithm is said to be correct if, for every input instance, it halts with
the correct output ie a correct algorithm solves the given computational
problem and an incorrect algorithm may not halt at all

An algorithm is thus correct, Iff the algorithm ends with all post conditions
true, whenever started with all preconditions true

{P} ->Algorithm->{Q}

To decide correctness most do

Testing (empirical, exhaustive)
Prove using logic

Proof show algorithm works for all possible inputs and works directly with
the abstract algorithm (idea)
11
They can’t tell anything about implementation

Proof means an argument that convinces audiences beyond doubt that

some claim or “theorem” is true

A Theorem
“The sum of two odd numbers is always even”

Testing
3+5=8
5+5=10,,,etc
This helps but not convincing

Intuition
(Think about why true)

Odd number is 1+even

So adding 2 odd numbers is like adding 2 even numbers + 2 extra “1”s

Clearer and not completely convincing- assume adding 2 even numbers

produce an even number

Rigorous Argument
Odd number is (2*i + 1)

Therefore,
2*i + 1 + 2*j + 1

Equals

2*i + 2*j + 2 = 2*(i + j + 1)

Which is an even number

Structure of Proofs

Precondition
A statement of assumptions

12
Theorem
Statement to be proven
Show that the post conditions hold at end of algorithm
Proof proper ( Argument as to why theorem is true)

Proof methods
Modus Ponens

Start with known facts or precondition and make series of deductions until
obtain desired goal

Each deduction reason as follows:

“I know that P is true,

and P implies Q
So, therefore, Q must be true”

P => Q
P
———
Q

P => Q is equivalent to
If P then Q

(Implications)

P is called the “Antecedent” and Q is called the “Consequent”

Be careful with this construct as may be easily distorted as example

“If something is a crocodile, then it eats meat”

“My dog eats meat”

“Therefore my dog is a crocodile “

This is false

Correctness of iterative algorithms

Design loops from invariants

13
Prove post conditions when finish

Proving a loop correct requires:

1 prove that the loop actually terminates
2 prove that loop invariants of algorithms hold on every iteration
3 prove post conditions when loop exist

Proving Termination
Show that every iteration makes finite progress to satisfying exit conditions
(minimum, non zero progress)

Example,

for (i t I=0; i < length; i ++) {

a[i] = 33;

Loop exits whenever i >= length

Since length is constant, and i < length initially
Increasing i by one often enough makes i exceed length

Correctness of Insertion Sort

A trace for A = <5,2,4,6,1,3>

Loop invariants and correctness

At the beginning of each iteration of the for loop indexed by j, the subarray
a[1..(j-1)] constitutes the currently sorted hand, and the remaining
elements a[(j+1)..n] correspond to the pile of cards still on the table

In fact, elements a[1..(j-1)] are the elements originally in positions 1..(j-1)

but in sorted order

We formally state the invariant as follows:

14
At the start of each iteration of the for loop of line 1-8, the subarray a[1..
(j-1)] consists of the elements originally in a[1..(j-1)] but in sorted order

We must show 3 things about the lop invariant:

Initialization: it is true prior to first iteration of the loop

Maintenance: if it is true before an iteration of the loop, it remains true

before the next iteration

Termination: when the loop terminates, the invariant gives a useful

property that helps show that the algorithm is correct

For insertion sort,

Initialization: the loop invariant holds before the first loop iteration, when
j=2. The subarray a[1..(j-1)] therefore consists of single element a[1], which
is the original a[1]

The subarray is sorted trivially

Therefore, the loop invariant holds initially

Maintenance: showing that each iteration maintain the loop invariant,

The body of loop works by moving a[j-1], a[j-2], a[j-3],, etc by one position
to the right until the proper position for a[j]. Lines 4-7. At which point it
inserts the value of a[j]. Line 8

The subarray a[1..j] consist of the elements originally in a[1..j] and in sorted

Thus incrementing j for the next iteration of the for loop then preserves the
loop invariant

Termination: the for loop terminates when j > a.length = n

Each loop iteration increases j by 1

At termination we must have j = n + 1 at the time

Substituting n + 1 for j in the wording of the loop invariant,

15
We have the subarray a[1..n] consisting of elements originally in a[1..n] and
in sorted order

As the subarray a[1..n] is the entire array, we conclude the array is sorted.
Hence algorithm is correct

Binary Search

Suppose have sorted array

Want to search it for particular value x

Narrow focus to a section of array that must contain x if x is in the array, ie

x must come before every element greater than x, and after every element
less than x

The section of focus of the array is known as “section of interest “

Iteratively home in on x with a loop that shrinks the “section of interest”

while maintaining loop invariant “that x is in the section of interest “

——————————————
|<—-x lies in—->|
——————————————

^ ^
| |
low high

Initialize low and high to include whole array

Repeat the following until either (low > high) or (you find x
If x < item at 1/2 way between positions low and high
then set high equal to position before the half waypoint
Otherwise,
If x > item at 1/2 between positions low and high
then set low equal to position after half way point

16
Boolean binarySearch (int x) {

#precondition: array is sorted in increasing order

low = 0;
high = n-1;

while ( low <= high & a[(low+high)/2] != x ) {

If ( x< a[(low+high)/2]) {

high = (low + high)/2 -1

} else {

low = (low + high)/2 + 1

}
If (low > high) {

return false

} else {

return true

Complexity of binary search

The number of comparisons to determine if x is in a sorted list is same as

number of times we half the array

That is, number of times 2 divides into n

n/2/2/2…2
17
x times

2**x = n

log (2**x) = log (n)

That is x = log (n)

Termination of Binary Search algorithm

while (low <= high && a[(low+high)/2] != x) {

if (x < a[(low+high)/2]) {
high = (low+high)/2 - 1

} else {

low = (low+high)/2 + 1
}

Theorem: the loop of Binary Search must exit

Proof:
the loop exits if low > high, even if x is not in array

Show that every iteration makes progress to low becoming > high

Every iteration either change high or change low

Change high
(L + H)/2 - 1

L = H => 2*H/2 - 1 => H - 1

Change low
(L + H)/2 + 1

H = L => 2*L/2 + 1 => L + 1

18
Every iteration either
reduces high by 1
Or
Increases low by 1

So, low must eventually exceed high and terminate

Correctness: proof by case

Separation of cases
For example: some assumption or case true and when the assumption is
false

p => q
p
———
q

p: assumptions —proof—> q: proven result

That is general proof technique

19
Proof by case

Theorem: p= !( !(p) )
Proof:
Consider 2 cases: p is true and p is not true

Case1: assume p is true. Then !p is false and !(!p) must be !(false) which is
true
So when p is true !(!p) is also true

Case2: assume p is false. Then !p is true and !(!p) must be !(true) which is
false
So when p is false, !(!p) is also false

Therefore p = !( !(p) )

Here are a few useful logical truth tables

P1 P2 P1 and P2

F F F

F T F

T F F

T T T

P1 P2 P1 or P2

F F F

F T T

T F T

T T T

p q p => q (implication)

F F T

20
 F T T

T F F

T T T

Halting Problem

Is it possible to write an algorithm that can examine another algorithm with

a possible input and state whether or not the second algorithm halts

No not possible (uses proof by contradiction)

Theorem: no algorithm can examine an arbitrary algorithm, together with

possible input and determine whether or not that algorithm will halt

Proof: proof by contradiction. Assume terminate( ) exists

boolean Algorithm terminate( candidate, input) {

if candidate (input) will terminate then {

return true
} else {
return false
}

Now we define a second algorithm magic as follows:

Algorithm magic (input) {

If terminate (input, input) then {

loop forever

} else {

return immediately

21
}

Consider a call to magic with itself

magic (magic)
|
| invokes
|
terminate (magic, magic)

Proof by cases

Case1: terminate ( magic, magic) returns true

That is, says magic will terminate

Then magic enters its “then clause” and loops forever (diverges)
Thus terminate must be wrong (a contradiction)

Case2: terminate (magic, magic) returns false

That is, says magic will not terminate

Then magic enters its else clause and immediately terminate
Thus terminate must be wrong ( a contradiction)

In either case “magic terminates if and only if it does not terminate”

Making proofs of recursive algorithms

Proving recursive algorithms correct is different from methods used earlier

We had seen Modus Ponens

p => q
p
——-
q

We use induction:
(Generalize individual observations)
22
1 BASE CASE: find small examples for which it is true

2 INDUCTION STEP: if statement true for small numbers then assume true
for larger numbers
Use assumption to show the theorem for a larger number

Example of proof by induction

Theorem: sum of n natural numbers sum:i=1..n of i = n*(n+1)/2

Base n=1
Sum:i=1..1 of i = 1 and 1(1+1)/2 = 1

Step
If holds for n=k-1
Then holds for n=k

If n=k-1,
sum:i=1..k-1 of i must equal by assumption (k-1)((k-1)+1)/2
Equals (k-1)k/2

We show for n=k true next

Sum:i=1..k of i is equal to sum:i=1..k-1 of i + k

Equals (k-1)k/2 + k
Equals (k-1)k/2 + 2k/2
Equals (k-1+2)k/2
Equals k(k+1)/2

Therefore sum of n natural numbers sum:i=1..n of i = n*(n+1)/2

Application to recursion

Here is a typical recursive algorithm

int factorial (int x) {

23
 if ( x > 0 ) then { return ( x * factorial(x-1)) }
else { return 1}

Notice that the function definition includes a call to itself inside the then
clause

If we were to call the function print factorial(3), it would trace like this

factorial(3) => 3*factorial(2) => 3*2*factorial(1) => 3*2*1

Proof of termination
The exit condition is when x is not greater than 0, ie

Exit when x <= 0

Since start with factorial (x) with x>0, and every new recursive call to
factorial reduces parameter x by 1

Eventually the recursion will execute the else clause when x=0 and e it
terminating the execution

Theorem: algorithm factorial computes n!

Proof: proof by induction
24
Base case factorial (2) = 2*1

Induction step; assume true for n-1

That is, factorial (n-1) is equal to (n-1)!

Then factorial (n) = n*factorial(n-1)

Equals n*(n-1)
Equals n!

Another simple recursive algorithm is Fibonacci sequence

1, 1, 2, 3, 5, 8, 13,….

Fn = Fn-1 + Fn-2. for all n >= 2, with F0 = 1 and F1 = 1

int Fibonacci (int m) {

If (m=0) then { return 1}

else {
if (m=1) then {return 1}
else {

return Fibonacci (m-1) + Fibonacci (m-2)

}
}

Selection sort

Assuming we have a function swap(m,c), we may implement a variat of a

sort in which items are swapped only once in each pass of n-1, but only
after have scanned the rest of items and determined which index has the
smallest element in the remaining unsorted list

25
Selection_sort() {

array data [size];

for (int c=0; c< (size-1); c++ ) {

Int m = c;//location with minimum in the unsorted list and

//c is key to compare for swap at end

for (int s=c+1; s < size; s++) {

if ( data[s] < data[m]) then {

m = s; //change where you found current

//smallest number in unsorted

//can swap now

Swap(m, c);
}

Sieve of Eratosthenes

You may generate small primes with a simple algorithm like this. Assume
have a Boolean array a, which contain true or false if it’s not a multiple of a
smaller number or false

Primes {

int n;
boolean [ ] a;//

// initialization

for i=2 to n-1 {

26
 a[i] = true;//assume all indexes are prime
}

//marking
for i=2 to n-1 {

if (a[i] != false) {

for (j=i; j*i; j++) {

{a[i*j] = false;//cancel multiples
}

//prepare output printing largest 100

for (int i = 2; i< n; I++) {

if (i<(n-100) ) then {
if a[i] { print i}
}

Integration of functions

A simple method of calculating an area under a curve may be realized as

follows

27
Area {

x(0) = A;
y(0) = f(A);
n = 20;//number of steps
h = ( B - A)/n;

T = 0;//total area added so far

for i = 1 to n {
y(i) = f( A + i*h);
T = T + ( y(i-1) + y(i) )*h/2;
y(i-1) = y(i);
}

Print T;
}

28
Single key encryption procedure
Suppose have a table of alphabets and a number code associated. In this
character table is made up of alphabets A to Z and a space mark
separator (27 characters)

Alphabet Code

Space 0

A 1

B 2

C 3

29
 D 4

E 5

F 6

G 7

H 8

I 9

J 10

K 11

L 12

M 13

N 14

O 15

P 16

Q 17

R 18

S 19

T 20

U 21

V 22

W 23

X 24

Y 25

Z 26

In single encryption it is assumed that the single secret key is exchanged

elsewhere eg at a bar

You are able to encrypt and decrypt Communications thereafter through

an insecure medium. The procedures would use three arrays I[], K [] and E
[]. Information to communicate is in I while the key is K and the mangled
cypher is in E
30
Routines
Assuming a 100 byte Communication each time,

encrypt {
// inputs are I[i], K[i]
for i = 0 to 100 {
t = I[i] + K[i];
if t >= 27 then t = t - 27;
E[i] = t;
}
}
31
decrypt {
// inputs are E[i], K[i]
for i = 0 to 100 {
t = 27 - K[i] + E[i]
if t >= 27 then t = t - 27;
I[i] = t;
}

Simultaneous linear equations

The idea of Gaussian elimination

The problem of finding the numerical solution of a set of simultaneous
equations

a1,1*x1 + a1,2*x2 + … + a1,n*xn = b1

a2,1*x1 + a2,2*x2 + … + a2,n*xn = b2
……………………………………………
an,1*x1 + an,2*x2 + … + an,n*xn = bn

occurs frequently in electrical engineering and other applications

The system of equations may also be written using sigma for summing
elements

Sum:j=1..n of (a1,j* xj ) = b1)

Sum:j=1..n of (a2,j* xj ) = b2)
……………………………….
Sum:j=1..n of (a1,j* xj ) = bn)

Or simply as

Sum:j=1..n of (ai,j * xj ) = bi for i=1,2,..,n

For example,

(1) 3x + 6y + 9z = 39
(2) 2x + 5y - 2z = 3

32
(3) x + 3y - z = 2

In this case the Ax = b representation has A as follows

3 6 9

2 5 -2

1 3 -1

And b, the right hand side as

39

If divide (1) by 3

1 2 3

2 5 -2

1 3 -1

And the new b, right hand side is

13

Subtract 2 times (1’) from (2’) and (1’) from (3’)

We get (1’’), (2’’), (3’’) the A coefficients are

1 2 3

0 1 -8

0 1 -4

33
And the b,

13

-23

-11

Dividing (2’’) by 1 and subtract (2’’) from (3’’) we get (1’’’), (2’’’), (3’’’) in A and
b respectively

1 2 3

0 1 -8

0 0 4

And b,
13

-23

12

If divide (3’’’) by 4 we get final set as A and b respectively

1 2 3

0 1 -8

0 0 1

And. As follows

13

-23

With this can do back substitution to obtain the results

z=3
y = -23+ 24 = 1

34
x = 13 -2 -9 = 2

The divisors were 3, 1, 4

Thus the determinant has the value 3x1x4 = 12

The design employs three variables i, j, k and arrays A and b to calculate x

The algorithm is as follows

//
// main.c
// Linear
//
// Created by nii n. quaynor on 9/25/14.
// Copyright (c) 2014 GDC. All rights reserved.
//

#include <stdio.h>

int main(int argc, const char * argv[])

{

/*create coefficients*/
float a[3][3] = {
{3,6,9},
{2,5,-2},

35
 {1,3,-1}
};

/*create right hand side*/

float b[3]={39,3,2};

/*define result array x()*/

float x[3];

int n=3; /*number of variables*/

int i,j,k;
float d,q;

for (i=0;i<=n-1;i++)
{
/*pivot*/
d=a[i][i];
a[i][i]=a[i][i]/d;

for (k=i+1;k<=n-1;k++)
{
a[i][k]=a[i][k]/d;
}
b[i]=b[i]/d;

/*elimination*/
for (j=i+1;j<=n-1;j++)
{
/*left hand side*/
q=a[j][i];
for (k=i;k<=n-1;k++)
{
a[j][k]=a[j][k]-q*a[i][k];
}

/*right hand side*/

b[j]=b[j]-q*b[i];
}

printf("%s %2.1f\n","a[0,0]=",a[0][0]);
printf("%s %2.1f\n","a[0,1]=",a[0][1]);
printf("%s %2.1f\n\n","a[0,2]=",a[0][2]);

printf("%s %2.1f\n","a[1,0]=",a[1][0]);
printf("%s %2.1f\n","a[1,1]=",a[1][1]);
printf("%s %2.1f\n\n","a[1,2]=",a[1][2]);

36
 printf("%s %2.1f\n","a[2,0]=",a[2][0]);
printf("%s %2.1f\n","a[2,1]=",a[2][1]);
printf("%s %2.1f\n\n","a[2,2]=",a[2][2]);

printf("%s %2.1f\n","b[0]=",b[0]);
printf("%s %2.1f\n","b[1]=",b[1]);
printf("%s %2.1f\n\n","b[2]=",b[2]);

/*back substitution*/
x[n-1]=b[n-1];

for (i=n-2;i>=0;i--)
{
printf("%s %d\n", "i=",i);

x[i]=b[i];

printf("%s %f\n\n","x[i]=",x[i]);

for (k=i+1;k<=n-1;k++)
{

printf("%s %d\n","k=",k);

x[i]=x[i]-a[i][k]*x[k];

printf("%s %f\n\n","x[i]=",x[i]);
}
}

printf("%s %2.1f\n","x[0]=",x[0]);
printf("%s %2.1f\n","x[1]=",x[1]);
printf("%s %2.1f\n","x[2]=",x[2]);

// insert code here...

//printf("Hello, World!\n");
return 0;
}

Performance analysis and conditionals

Consider this algorithm

37
A1
If test then A2
A3

Assume each action A1, A2, A3 requires same time unit

This algorithm will run either 2 or 3 time units depending on the value of
test

Conditionals make it impossible to say exactly how long the algorithm will
run

To know the likely probabilities of value of text is difficult

Bounding execution times

Worst case
Upper bound: the algorithm requires no more than 3 units of time

Best case
Lower bound: the algorithm requires at least 2 units of time

Random error cannot be eliminated before it happens

A “variable” does not have a single “true” value

We use average to approximate “true” value where there’ll be higher and
lower values

Estimating “true” values

We have 5 measurements and outcomes 2,1,2,3,3 where n=5

Average is (2+1+2+3+3)/5 equal to 2.2

Error in average
Due to variations among measurements
More measurements provide more improvement and average more likely
to be close to “true” value

Standard deviation

S = sqrt ((sum:i=1..n of (xi - xavg)**2)/(n-1))

38
(x1 - xavg)**2 = (2 - 2.2)**2 = (-0.2)**2 = 0.04
(x2 - xavg)**2 = 1.44
(x3 - xavg)**2 = 0.04
(x4 - xavg)**2 = 0.64
(x5 - xavg)**2 = 0.64

S = sqrt ( 2.8/4) = 0.84

Means measurements differ from average by 0.84 secs

Standard error E = S/sqrt(n) => average is within 0.36 of “true” time

Produce a table
Example: sorting times

Experiment \Size 1000 2000 4000

Avg

Standard Error

Grammars
Machines and languages have a relationship. The simplest language is the
regular languages whose equivalent for automatons is finite state
machines

Although there are other languages such as context free, context sensitive
and Type 0 languages we will focus our attention on regular languages
implemented as finite state machines and algorithms of interest

39
A grammar G = (N, T, P, S) where N are a set of non terminals, T is a set of
terminal symbols, P is a set of production rules and a starting symbol S

A piece of typical English language may be described by a grammar such

as below

N = { <sentence >, <noun phrase >, <verb phrase>, <adjective>, <noun>,

<verb>, <adverb> }

T = { the, little, fish, swam, quickly }

P = { <sentence > -> <noun phrase ><verb phrase >,(1)

<noun phrase > -> <adjective ><noun phrase >,(2)
<noun phrase > -> <adjective ><noun>,(3)
<verb phrase > -> <verb><adverb>,(4)
<adjective > -> the | little (5)
<noun> -> fish (6)
<verb> -> swam (7)
<adverb> -> quickly } (8)

S = {<sentence >}

A syntax tree is shown

40
The derivation from <sentence > is as follows

<sentence >
=1=> <noun phrase ><verb phrase >
=2=> <adjective ><noun phrase ><verb phrase >
=5=> the <noun phrase ><verb phrase >
=3=> the <adjective ><noun><verb phrase >
=5=> the little <noun><verb phrase >
=6=> the little fish <verb phrase >
=4=> the little fish <verb><adverb>
=7=> the little fish swam <adverb>
=8=> the little fish swam quickly

The above grammar is familiar yet is much more complicated than the
grammars we study here

The regular grammars have more constrained production rules

The rules are of the form

A -> aB
A -> a

Where N = { A, B } and T = {a,..}

As we saw in the English language example to define the language a

grammar generates we need a relations operator =G=> and =G=*>
between strings in V* where V = Union ( N, T)

Thus =G=> relates 2 strings when the second is obtained from the first by
application of a single production

We define a language generated by G, L(G), to be { w | w is in T* and S

=G=*> w }

That is string consists solely of terminals and derived from starting symbol
S
41
Finite state machines
These are used in design of electronics computers and communications
equipment but also used to control complex coordination among software
agents

In this formulation a machine M is defined as M = (K, sigma, delta, q0, F)

where

K = set of states
Sigma is a set of inputs
F = set of final states

Delta = a set of state transitions of form delta (state, input) = next state

A sentence x is said to be accepted by machine M if delta(q0,x) = p for

some p in F

The set of all x accepted by machine M is designated T(M), ie

T(M) = { x | delta(q0,x) is in F }

For such a machine we can derive the grammar G from M as follows

Let M = (K, sigma, delta, q0, F)

Define G = ( N, T, P, S) as
G = ( K, sigma, P, q0)

And follow the procedure to define

production rules for each delta of
machine M

1 B ->aC is in P if delta(B, a)=C

2 B -> a is in P if delta(B, a)=C and C
is in F

An example might be

Sigma = {0, 1}
K = { q0, q1, q2, q3}
42
F = { q0 }

Delta(q0,0) = q2
Delta(q0,1) = q1

Delta(q1,0) = q3
Delta(q1,1) = q0

Delta(q2,0) = q0
Delta(q2,1) = q3

Delta(q3,0) = q1
Delta(q3,1) = q2

Here is a diagram for this

43
The productions in the resulting grammar P:
P = {q0->0q2
q0->1q1

q1->0q3
q1->1q0
q1->1

q2->0q0
q2->0
q2->1q3

q3->0q1
q3->1q2}

Notice that all the rules are of form A->bC or A->b where A and C are non
terminals and b is a terminal symbol. It’s a grammar for a regular language
as expected

Sequential storage allocation

Several algorithms rely on organized storage to function. Here are

important storage management algorithms. In sequential allocation the
storage elements are contiguous. We may readily represent these with
arrays

Sequential Stack

A stack has a top and two operations. A push() to place some item at the
top of the stack and a pop() to remove at item. Use array stack[0..n] to
represent stack

push(item) {
if top+1 <= n then {

44
 top=top+1
stack[top] = item

} else {

print “stack overflow”

pop (item) {

if top>=0 then {

item=stack[top]
top=top-1

} else {

print “stack is empty”

}
45
Sequential queue (circular buffer)

A queue has two ends, a front F and a rear R. New items are added at the
rear R and items are removed from the front F

This is a systems programming object constructed from linear array of

memory. A mechanism of implementing producer consumer relationships

Insert(item) {

If (@1 != F) then {

R = R@1
Buffer[R] = item

} else {
Print “buffer full”
46
 }

Remove (item) {

If F != R then {

F = F@1
item=Buffer[F]
} else {
Print “buffer empty”
}

Modulo arithmetic
int @(x,y) {

T= x+y
While T>= n {
T=T-n
}
return T
}

47
Linked allocation
Quite often in systems programming you do not have contiguous storage
to allocate as resources. Instead may have available several smaller pieces
which together far exceeds the space requested. In linked allocation,
storage is organized as having two parts: an information part and a
reference pointer to where to find the next part. Hence a pointer refers to
an object with info and link

We will implement with two arrays info[] and Link[] hence a pointer PTR
refers to both Info[PTR] and Link[PTR]

Linked stack
The stack is only accessible at the top. To work a pointer TOP, points to
the top of the stack. From this first element the links all point to item below

48
them. Thus from the TOP one may traverse the stack to the end usually
marked with a sentinel

The sentinel in examples is -1

This stack may serve as an available storage manager by simply

maintaining a linked stack

We may initialize the two arrays Info[] and Link[] to form a stack as follows

Initialization
Info[0] = 0
Link[0] = -1

for i = 1 to N {

Info[i] = 0
Link[i] = i-1

}
TOP = N

The arrays will look like this and TOP = N

Address INFO LINK

0 0 -1

1 0 0

2 0 1

3 0 2

N-2

N-1 0 N-2

N 0 N-1

We may dump the arrays for examination

49
Dump () {
for i = 0 to N {
Print Info[i]
Print Link[i]
}

push(PTR) {

Link[PTR] = TOP
TOP = PTR
PTR = -1

50
Note the grounding of PTR to make sure PTR no longer refers to the
structure

pop(PTR) {

if TOP ! -ve {

PTR = TOP //get item and Link

TOP = Link[TOP] //remove from stack
Link[PTR] = -1 // ground link
} else {
Print “empty stack”
}

51
Linked queue
In the event that need to build a queue with linked node you may reference
below
The states of a linked queue are analyzed. Empty queue is represented by
both F and R equal to ground -1. From empty state with addition of a node
result in F and R equal to PTR and PTR equal -1

And similarly, when the last item in queue is removed it returns F and R
equal to ground -1

52
The routines Remove() and Insert() are as depicted here

Remove(ptr) {

If F>= 0 then {

ptr = F
F = Link[F]

If F=-1 then R=-1 //last item

Link[ptr] = -1

53
 } else {
Print “empty queue”
}

Insert (ptr) {

Link[ptr] = -1
If R >= 0 then {

Link[R] = ptr
R = ptr
} else {

F=R=ptr

ptr = -1

54
}

Doubly linked lists

In situations where we wish to traverse linked structure in two directions

you may chose to implement doubly linked lists. Can implement this with
three arrays, LLink, Info, RLink. That means any pointer PTR has
LLink[PTR], Info[PTR] and RLink[PTR]

One would observe that the first item in the structure and pointed to by
Front has its left pointer shown L to -1 (ground) invalid address. Recall
addresses are array indices

You can use this to build an even more complex data structures such as
an array of Front like pointers. That is identified as a Hash table of entries
that you may traverse forward and backwards. Colliding hashes are
grouped into an Array H[0..(n-1)]. Hash may be any function but normally
want a hash that spreads uniformly to increase performance of n finding
items

An illustration of this is available here

55
Linked circular lists

In thee event that have interest to conserve link space may implement a
linked circular list using exclusive OR to put more information in the link
but also ask for additional information to determine object

P Q (+)

0 0 0

0 1 1

1 0 1

1 1 0

In this case we put the exclusive OR of Left and Right items into a pointer

In that case the behavior is

56
If know L and a pointer PTR then can derive R

And likewise,
If know R and a pointer PTR then can derive L

This enables traversing the link in any direction. If it is circular can traverse
circularly

Routines

If know L and PTR then

L (+) Link (PTR)

is equal to L (+) (L (+) R )
is equal to ( L (+) L) (+) R
is equal to (0000..000)(+)R
Is equal to R

If know R and PTR then

Link[PTR] (+) R
Is equal to (L (+) R) (+) R
is equal to L (+) (R (+) R)
is equal to L (+) (0000…000)
57
is equal to L

Graph Algorithms
We have two basic ways to represent graphs as adjacency lists or as
adjacency matrix

Either representation applies to both directed and undirected graphs

Adjacency lists are more compact to represent sparse graphs, that is

those for which |E| is much smaller than |V**2|

That is |E| << |V**2|

The graph may be represented in an adjacency matrix by table below

Nodes 1 2 3 4 5

1 0 1 0 0 1

2 1 0 1 1 1

3 0 1 0 1 0

4 0 1 1 0 1

5 1 1 0 1 0

58
The adjacency list for the graph would be implemented as a hash array of
linked stacks

59
60