Automata Theory and Computability
Automata Theory and Computability
Automata Theory and Computability
Module – 1: Syllabus:-
A Language Hierarchy(ch-3)
Computation(ch-4)
What is computability?
Tractable/Intractable Problems:
Alphabet - definition:
Defn: An alphabet is a non-empty, finite set of characters/symbols
Use to denote an alphabet set
Examples
= { a, b }
= { 0, 1, 2 }
= { a, b, c,…z, A, B, … Z }
= { #, $, *, @, & }
String definition: A string is a finite sequence, possibly empty, of characters
drawn from some alphabet .
is the empty string
* is the set of all possible strings over an alphabet .
Examples of strings:
= {a, b}
Strings derived from are…..
….. , a, b, aa, ab, ba, bb, aaa, aab, aba, ..
= {0, 1}
Strings derived from are…..
….. , 0, 1, 00, 01, 10, 11, 000, 001, 010, ..
= {a}
Strings derived from are…..
…… , a, aa, aaa, aaaa, aaaaa, aaaaa,….
Functions on Strings
Length – to find the length of a string Operator used | |
Concatenation – to join two or more strings. Operator - s||t, or nothing i.e. st
Replication – strings raised to some power. Operator - a3
Reversal – reverse a string
Operator - (w)R
Examples of Length of a string
• ||=0
• |101| = 3
• |VTU_Edusat| = 10
Examples of Concatenation of a string
• x = good, y = student
• Concatenation operation x||y or xy
• xy = goodstudent
Examples of Replication of a string
• a3 = aaa
• (good)3 = goodgoodgood
• a0 b3 = bbb = bbb
Examples of Reversal of a string
• (abc)R = cba
• x= ab, y=cd, (xy)R = dcba
• xR yR =badc
Relation on Strings
• Substring:
• aaa is substring of aaa and also aaabbccc
• Proper substring:
Defn: A string s is a proper substring of a string t iff s is a substring of t and s
≠t
Examples:
S = good then proper substrings are ..
..... , g, go, goo only
Prefix and Suffix functions
• A string s is a prefix of t iff ꓱx ϵ *(t = sx)
• , a, ab,abb are prefixes of string abb
• Proper prefix:
• , a, ab, are proper prefixes of string abb
• A string s is a suffix of t iff ꓱx ϵ *(t = xs)
• , b, bb, abb are suffixes of string abb
• Proper suffix:
• , b,bb,are proper suffixes of string abb
Languages:
Defn: A language is (finite or infinite) set of strings over a finite alphabet Σ
Example if Σ = { a } following languages can be derived
• Language L1= {a, aaa, aaaaa, aaaaaaa,.......}
• Language L2= {, aa, aaaa, aaaaaa,.......}
• Language L3= {a, aaaaa, aaaaaaaaa,.......}
• Language L4= {a, aaa, a7, a9 , a13 , ....}
Note: number of languages that can be derived even from singe alphabet set
is INFINITE
Techniques for defining Languages by enumeration/defining property
Examples: (by enumeration)
• Let L = {w ϵ {a, b}* : all string begin with a}
• L={a, ab, aab, abbbb, ...}
• Strings not in L are:
• {b, ba, , bbbbb, baaaaaa, …..}
• Let L = {w ϵ {a}* : |w| is even}
• L={, aa, aaaa, aaaaaa, aaaaaaaa, ..}
• Strings not in L are:
• {a, aaa, aaaaa, aaaaaaa, …..} //odd no of a’s
Examples: ( defining property)
• Let L = {w ϵ {a, b}* : all string ending in a}
• L={a, aba, aaba, bbbba, ...}
• Strings not in L are:
• {b, bb, , bbbbb, aaaaaab, …..}
• Let L = {w ϵ {a}* : |w| mod 3 =1}
• L={a, a4, a7, a10, ....}
• Strings not in L are:
• {, a2,a3, a5, a6, a8, a9, ....}, …..}
Functions on Languages.
Languages are sets. Therefore, all set operations like Union, Intersection,
Difference, and Complement can be applied.
• Example if Σ = { a }
• L1 = {, a2,a4, a6, a8, a10, a12, ....} //even no of a’s
L2 = {a1,a3, a5, a7, a9, a11, .........} //add no of a’s
Set Operations on Languages
• L1 = {, a2,a4, a6, a8, a10, a12, ....} //even no of a’s
• L2 = {a1,a3, a5, a7, a9, a11, .........} //add no of a’s
• L1 U L2 = Σ* or { a }* // union operation
• L1 ∩ L2 = Ф or {} // intersection operation
• L1 - L2 = L1 // difference operation
• L2 – L1 = L2 // difference operation
• ~(L1 - L2) = L2 // complement operation
• ~(L2 – L1) = L1 // complement operation
Concatenation of Languages
• L1 = {aa, ab}
• L2 = {xx, yy}
• L1L2={aaxx, aayy, abxx, abyy}
Some important results
• L1 = { } =Ф
• L2 = {xx, yy}
• L1L2={}
• In general for L
L Ф = ФL = Ф
Some important results
• L1 = {}
• L2 = {xx, yy}
• L1L2=L2
• In general for all L
• L {} = L {} = L
• (L1L2)L3= L1(L2L3) // associative
• L1 = { an | n >= 0}
• L2 = { bn | n >= 0}
• L1L2= {an bm | n,m >=0} = a*b* // note n & m
• Kleene star operation
• L*={ set of all strings that can be formed by concatenating zero or
more strings from L}
• a* = {, a, aa, aaa, aaaa, aaaaa, ....infinite}
What is L+ ?
• L+ = LL* //assuming L does not have Ꜫ
• L+ = L* - {}
Example
a* = {, a, aa, aaa, aaaa, aaaaa, ....infinite}
a+ = a* - {}
Assigning Meaning to the strings of a Language
Following codes of C/Java have the same meaning.
-- int x=4; x++;
-- int x=4; ++x;
-- int x=4; x=x+1;
-- int x=4; x=x-(-1)
chapter-5
Finite State Machines(FSM)
Defn: A FSM(DFSM) , M is a quintuple:
(K, ∑, δ, s, A)
• K is a finite set of states,
• ∑ is the input alphabet,
• s ϵ K is the start state
• A subset of K is the set of accepting states and
• δ is the transition function it maps from:
k x ∑ to k
Problem – 2:
Write a DFSM to accept the language
L = { w ϵ {a, b}* | |w| is even length}
Step 1: Write strings accepted by L i.e.
L = { ɛ, aa, bb, ab, ba, aaaa, bbbb, bbaa, baba,..}
(note : ɛ is even, because its length is 0, which is even)
~L= { a, b, aaa, bbb, aba, bab, bba, aab, aabbb,..}
Step 2: since min string are {ɛ, aa}, 2 states are required.
Step 3: Write Transition Diagram.
Problem – 2
Problem – 3
Write a DFSM to accept the language
L = { w ϵ {a, b}* | ab is a substring of w}
Step 1: Write strings accepted by L i.e.
L = { ab, abab, aaab, abaaa, abbbb, bbababab, babb, bbab, baba,..}
~L= { a, b, aa, bb, bbb, bba, bba, aaa, bbbbb,..}
Step 2: since min string is { ab}, 3 states are required.
Step 3:Write Transition Diagram.
Problem – 4
Write a DFSM to accept the language
L = { w ϵ {a, b}* | every w ends in b }
L = { b, ab, abab, aaab, abaab, abbbb, bbababab, babb, bbab, babb,..}
~L= { a, aa, ba, bba, baa, baba, aaa, bbbba,..}
Step 2: since min string are { b}, 2 states are required.
Step 3: Write Transition Diagram.
Difficulties with FSMs
Write a DFSM to accept the language
L = { ab, ba, abab, aaba, abaab, abbba, bbababab, baba, bbab, baba,..}
Step 2: since min string are {ab, ba}, we are not able to guess no of states.
Note : this is a difficult problem, we end up in spending lot of time to find the
solution
Step 2: since min string are not there, we are not able to guess no of states.
Note : this is a difficult problem, we end up in spending lot of time to find the
solution
(K, ∑, ∆, s, A)
(K x (∑ U {Ꜫ})) to K
Example of NFSMs
L = { ab, ba, abab, aaba, abaab, abbba, bbababab, baba, bbab, baba,..}
Step 2: since min string are {ab, ba}, we are not able to guess no of states.
Note : this is a difficult problem, we end up in spending lot of time to find the
solution
baba,..}
Step 2: since min string are not there, we are not able to guess no of states.
Note : this is a difficult problem, we end up in spending lot of time to find the
solution
Solution is as follows:
Consider the following problem for which we know the
NFSM. And apply the procedure to convert it to FSM
Procedure:
s’=s={1} // note the set notation
Compute δ'
Active states ={{1}}, consider {1}
δ‘({1},a)= {1,2} .. add
δ‘({1},b)= {1,3} .. add
Active states ={{1},{1,2},{1,3}}, consider {1,2}
δ‘({1,2},a)= {1,2},a)U Ф ={1,2} ..exists
δ‘({1,2},b)= {1,3}U{4})={1,3,4} add
Lecture – 5 : chapter 5
Procedure to convert NFSM to DFSM:
The problem which we are attempting to convert has Ꜫ
transition.
We need to calculate eps for each state using the algorithm
as follows:
Eps(q: state) // algorithm
1. result = {q} and some
2. while there exists p ϵ result r not ϵ of result and
some transition(p, Ꜫ, r) ϵ transition function do:
insert r into result.
3. return result.
Note: It means connect all states that can be reached on Ꜫ
Example-1 for calculation of eps:
Example-2 for calculation of eps:
Mealy machine(transducer):
Defn: A Mealy machine, M is a six tuple:
(K, ∑, O, δ, s, A), where
• K is a finite set of states,
• ∑ is the input alphabet,
• O is the output alphabet,
• s ϵ K is the start state
• A subset of K is the set of accepting states and
• δ is the transition function it maps from
(K x ∑ ) to (K x O*)
Note: output is associated with each input.
Example of Mealy machine:
Transition table:
Computation(chapter 4):
In this chapter, effort is made to:
Define problems as languages to be decided
Define programs as state machines whose input is a string
and whose output is accept or reject
• Key ideas :
1. Decision procedures
2. Non determinism
3. Function on languages.
Decision Procedures
Defn: A decision problem is one for which we must make a
yes/no decision.
A decision procedure is an algorithm to solve a decision
problem.
It a program whose result is a Boolean value.
In order to return a Boolean value, a decision procedure
must be guarantee to halt on all inputs
Decision procedures are to answer question such as:
-- Is string s in Language L?
-- Given a machine, does it accept any string?
-- Given two machines, do they accept the same strings?
-- Given a machine, is it the smallest m/c that does its job?
Three imp things about Procedures:
1.Does there exist a decision procedure(algorithm)
2.If any decision procedures exist, find one
3. If exists, find the most efficient one, and how efficient it
is?
Decision procedures are programs, and they must have two
correctness properties:
1. Program must be guaranteed to halt.
2. The answer must be correct.
Example – 1:
Checking for even numbers:
even(x: integer)=
If (x/2)*2=x then return True
else return False.
If x=3 then x/2=1 and 1*2 != 3 therefore “false”
If x=8 then x/2=4 and 4*2 = 8 therefore “true”
Example – 2:
Checking for Prime numbers:
prime(x: positive integer) =
For i = 2 to ceiling(sqrt(x)) do:
If (x/i)*i = x then return False
return True
Assume x = 7 then ceiling(sqrt(x))= ceiling(2.65) = 3
i = 2; 7/2*2 != 7 next iteration
i = 3; 7/3*2 != 7 next iteration(no more iterations)
Returns True
Example-3:
Checking for Programs that halt on a particular input.
haltOnw(p: program, w: string)=
1. Simulate the execution of p on w
2. If the simulation halts return True
else return False.
note: 1. this is not a procedure, because it never returns
False.
2. No decision procedure exists for this.
Determinism and non determinism:
Consider a program:
Choose(action 1;;
action 2;;
....
action n;; )
Observation on choose:
Returns some successful value, if there is one
If there is no successful value, the choose will:
- Halt and return False if all the actions halt and return
False
- Fail to halt if any of the actions fails to halt.
(note that this has a potential to return successful value,
it may be taking more time)
Deterministic and non Deterministic:
If a program does not use choose then it is deterministic.
If a program includes choose then it is non deterministic.
Functions on Languages and Programs:
The function chop:
chop(L): is all the odd length strings in L with their middle
character chopped out.
The function firstchars:
firstchars(L): determines the first characters by looking at all
strings in L
Examples of chop(L):
Examples of firstchars(L)
Closure of Languages:
Langauage Hierarchy:
Hierarchy of Languages and corresponding automata
Regular languages: FSMs
Context-free languages: PDAs
D(decidable) Languages: Turing machine
SD(semi decidable) languages: Turing machine
Importance of classification:
The factors are:
1.Computational efficiency: As function of input length
FSMs - Linear with respect to input string
PDAs - cube of the length of input string
TM - exponentially with respect to input
String
2.Decidability: Answer to the questions
FSM - accepts some string?
FSM - is it minimal?
FSMs- are two FSMs identical?
PDAs- only some of the above can be
answered
TM - none of the above can be answered
3.Clarity: tools that enable analysis- exist?
FSM - yes
PDA - yes
TM - none
.............................. End of Module – 1 .................
Operating Systems:
Internals and Design Principles, 6/E
Wil iam Stal ings
Chapter 6
Concurrency: Deadlock and
Starvation
Patricia Roy
Manatee Community College, Venice, FL
©2008, Prentice Hall
ATC-Module-2- Dr.Girijamma H A
Chapter-6
Regular Expressions
Regular Expression (RE)
1. ø is a RE.
2. ε is a RE.
3. Every element in ∑ is a RE.
4. Given two REs α and β,αβ is a RE.
5. Given two REs α and β, α U β is a RE.
6. Given a RE α, α* is a RE.
7. Given a RE α, α+ is a RE.
8. Given a RE α, (α) is a RE.
1
ATC-Module-2- Dr.Girijamma H A
= (L(a) U L(b))*L(b)
=({a} U {b})*{b}
= {a,b}*{b}
(a U b)*b is the set of all strings over the alphabet {a, b} that end in b.
L = {abb,aabb,babb,ababb-------}
RE = (a U b)*abb
2
ATC-Module-2- Dr.Girijamma H A
L = {a,aaa,ababa,bbaaaaba------}
RE = b*(ab*ab*)* a b* or b*ab*(ab*ab*)*
3
ATC-Module-2- Dr.Girijamma H A
1. Kleene star
2. Concatenation
3. Union
Kleene's Theorem
Theorem 1:
Any language that can be defined by a regular expression can be accepted by some
finite state machine.
Theorem 2:
Any language that can be accepted by a finite state machine can be defined by
some regular expressions.
Note: These two theorems are proved further.
Figure (1)
4
ATC-Module-2- Dr.Girijamma H A
Figure (2)
3. If α is ε,we construct simple FSM shown in Figure(3).
Figure (3)
4. Let β and γ be regular expressions.
If L(β) is regular,then FSM M1 = (K1, ∑ , δ1, s1, A1).
If L(γ) is regular,then FSM M2 = (K2, ∑ , δ2, s2, A2).
If α is the RE β U γ, FSM M3=(K3, ∑ , δ3, s3, A3) and
L(M3)=L(α)=L(β) U L(γ)
M3 = ({S3} U K1 U K2, ∑ , δ3, s3, A1 U A2), where
δ3 = δ1 U δ2 U { ((S3, ε), S1),((S3, ε),S2)}.
α=βUγ
5
ATC-Module-2- Dr.Girijamma H A
α = βγ
6. If α is the regular expression β*, FSM M2 = (K2, ∑, δ2 s2, A2) such that
L (M2) = L (α)) = L (β )*.
M2 = ({S2} U K1, ∑, δ2,S2,{S2} U A1), where
δ2 = δ1 U {((S2, ε ),S1)} U {((q, ε ),S1):q ϵ A1}.
α = β*
An FSM for b
6
ATC-Module-2- Dr.Girijamma H A
An FSM for a
An FSM for ab
7
ATC-Module-2- Dr.Girijamma H A
8
ATC-Module-2- Dr.Girijamma H A
9
ATC-Module-2- Dr.Girijamma H A
fsmtoregexheuristic(M: FSM) =
3. If the start state of M is has incoming transitions into it, create a new start
state s.
4. If there is more than one accepting state of M or one accepting state with
outgoing transitions from it, create a new accepting state.
6. Until only the start state and the accepting state remain do:
Let M be:
Step 1:Create a new start state and a new accepting state and link them to M
11
ATC-Module-2- Dr.Girijamma H A
1-2-1:ab U aaa*b
1-2-5:a
RE = (ab U aaa*b)*(a U ε)
12
ATC-Module-2- Dr.Girijamma H A
Proof : By Construction
L(M) = L(α)
If any of the transitions are missing, add them without changing L(M) by labeling
all of the new transitions with the RE ø.
13
ATC-Module-2- Dr.Girijamma H A
Select a state rip and remove it and modify the transitions as shown below.
Consider any states p and q.once we remove rip,how can M get from p to q?
Let R(p,q) be RE that labels the transition in M from P to Q.Then the new machine
M' will be removing rip,so R'(p,q)
= R(1,3) U R(1,2)R(2,2)*R(2,3)
= ø U ab*a
= ab*a
modified machine M
1.Standardize (M:FSM)
iv. If there is more than one transition between states p and q ,collapse them to
single transition
14
ATC-Module-2- Dr.Girijamma H A
2.buildregex(M:FSM)
iii. until only the start state and the accepting state remain do:
iv. Return the RE that labels from start state to the accepting state
1-4-2 : bb
1-2: a U bb
15
ATC-Module-2- Dr.Girijamma H A
1-3: (a U bb)b*a
RE = (a U bb)b*a
16
ATC-Module-2- Dr.Girijamma H A
p-q-p: 01
p-r-p: 10
RE = (01 U 10)*
17
ATC-Module-2- Dr.Girijamma H A
18
ATC-Module-2- Dr.Girijamma H A
19
ATC-Module-2- Dr.Girijamma H A
20
ATC-Module-2- Dr.Girijamma H A
Building DFSM
• K can be defined by RE
Algorithm- buildkeywordFSM
• To build dfsm that accepts any string with atleast one of the specified
keywords
Buildkeyword(K:Set of keywords)
21
ATC-Module-2- Dr.Girijamma H A
22
ATC-Module-2- Dr.Girijamma H A
• More generally string processing, where the data need not be textual.
RE = -? ([0-9]+(\.[0-9]*)? | \.[0-9]+)
• (α)? means the RE α can occur 0 or 1 time.
((a-z) U (A-Z))
23
ATC-Module-2- Dr.Girijamma H A
• α* means that the pattern may occur any number of times(including zero).
• α{n,m} means that the pattern must occur atleast n times but not more than
m times
• So RE of a legal password is :
• RE for an ip address is :
RE = ((0-9){1,3}(\.(0-9){1,3}){3})
Examples: 121.123.123.123
118.102.248.226
10.1.23.45
• Union is Commutative
αUβ=βUα
24
ATC-Module-2- Dr.Girijamma H A
• Union is Associative
(α U β) U ү = α U (β U ү)
αUΦ=ΦUα=α
• union is idempotent
αUα=α
• Concatenation is associative
(αβ)ү = α(βү)
αε = εα = α
αΦ = Φα = Φ
• Φ* = ε
• ε* = ε
• (α*)* = α*
• α*α* = α*
25
ATC-Module-2- Dr.Girijamma H A
• If α* ⊆ β* then α*β* = β*
• (α U β)* = (α*β*)*
= a* U aa //(α*)* = α*
= a* // L(aa) ⊆ L(a*)
= b* // α*α* = α*
= b* //L(ε U b) ⊆ L(b*)
26
ATC-Module-2- Dr.Girijamma H A
Chapter-7
Regular Grammars
and terminals.
XY
Legal Rules
Sa
Sε
TaS
SaSa
STT
aSaT
ST
27
ATC-Module-2- Dr.Girijamma H A
• Start symbol of any grammar G will be the symbol on the left-hand side of
the first rule in RG
DFSM accepting L
Sε
SaT
SbT
TaS
TbS
28
ATC-Module-2- Dr.Girijamma H A
S => aT
=> abT
=> abaS
=> ababS
=> abab
THEOREM
Statement:
The class of languages that can be defined with regular grammars is exactly the
regular languages.
L (M) = L (G):
Algorithm-Grammar to FSM
29
ATC-Module-2- Dr.Girijamma H A
to # labeled w.
accepting.
from D to D labeled i.
Example 2:GrammarFSM
RE = (a U b)*aaaa
Regular Grammar G
SaS
SbS
SaB
BaC
CaD
Da
30
ATC-Module-2- Dr.Girijamma H A
31
ATC-Module-2- Dr.Girijamma H A
32
ATC-Module-2- Dr.Girijamma H A
33
ATC-Module-2- Dr.Girijamma H A
RE = (a U bb)b*a
Grammar
AaB
AbD
BbB
BaC
DbB
Cε
A => aB
=> abB
=> abaC
=> aba
A => bB
=> bbB
=> bbaC
=> bba
number of b's}
34
ATC-Module-2- Dr.Girijamma H A
Grammar
AaB
AbC
BaA
BbD
CbA
CaD
DbB
DaC
Cε
A => aB
=> abD
=> abaC
=> ababA
=> ababbC
=> ababb
35
ATC-Module-2- Dr.Girijamma H A
w ends in a}.
SbS
SaT
T ε
TaS
TbX
XaS
XbX
36
ATC-Module-2- Dr.Girijamma H A
• But regular grammars are often used in practice as FSMs and REs are easier
to work.
• But as we move further there will no longer exist a technique like regular
expressions.
Chapter-8
Statement:
Proof:
37
ATC-Module-2- Dr.Girijamma H A
languages:
regular languages.
Proof:
is regular.
the R.E: s1 U s2 U …U sn
• So it too is regular
Regular expressions are most useful when the elements of L match one or
more patterns.
FSMs are most useful when the elements of L share some simple structural
properties.
38
ATC-Module-2- Dr.Girijamma H A
Examples:
current US president}.
Fn = 22n + 1 , n >= 0.
• All of them are prime. It appears likely that no other Fermat numbers are
prime. If that is true,then L6
39
ATC-Module-2- Dr.Girijamma H A
• Union
• Concatenation
• Kleene star
• Complement
• Intersection
• Difference
• Reverse
• Letter substitution
40
ATC-Module-2- Dr.Girijamma H A
Theorem:
Proof:
Steps:
M2=(K, ∑,δ,s,K-A)
Example:
RE = (0 U 1)*01
Theorem:
Proof:
• Note that
• We have already shown that the regular languages are closed under both
complement and union.
• Example:
L = L1 ∩ L2, where
43
ATC-Module-2- Dr.Girijamma H A
L = {w Є {a,b}* : w contains an even number of a’s and an odd number of b’s and
all a’s come in runs of three }.
Theorem:
Proof:
Theorem:
Proof:
Example:
By construction.
• Initially, let M′ be M.
44
ATC-Module-2- Dr.Girijamma H A
• Example 1
sub(a) = 0, sub(b) = 11
• Example 2
h(0120) = h(0)h(1)h(2)h(0)
= aabbaa
45
ATC-Module-2- Dr.Girijamma H A
h(01*2) = h(0)(h(1))*h(2)
= a(ab)*ba
Proof:
• Each time it reads an input character, it visits some state. So ,in processing a
string of length n, M creates a total of n+1 state visits.
• If n+1 > | K |, then, by the pigeonhole principle, some state must get more
than one visit.
• So, if n>= | K |,then M must visit at least one state more than once.
|xy| <= k,
y ≠ ε,and
Proof:
Let k be |K|
• We can carve w up and assign the name y to the first substring to drive M
through a loop.
• Then x is the part of w that precedes y and z is the part of w that follows y.
• We show that each of the last three conditions must then hold:
• |xy| <= k
• y≠ε
• ∀q >= 0 (xyqz ϵ L)
be in L.
1. Assume L is regular.
6. Our assumption is wrong and hence the given language is not regular.
47
ATC-Module-2- Dr.Girijamma H A
Proof by contradiction.
1 2
aaaaa…aaaaabbbb…bbbbbb
x y z
k |xy| ,
y
• Not regular.
• L consists of all strings of the form a*b* where the number of a’s is five
more than the number of b’s.
• Let w = ak+5bk.
48
ATC-Module-2- Dr.Girijamma H A
• We can pump y out once, which will generate the string ak+5-pbk, which is not
in L because the number of a’s is is less than 5 more than the number of b’s.
49
ATC-Module-2- Dr.Girijamma H A
50
Subject: Automata Theory and Computability
Sub Code: 15CS54
Module -III
Context-Free Grammars and Pushdown Automata (PDA)
Course Outcomes-(CO)
At the end of the course student will be able to:
i. Explain core concepts in Automata and Theory of Computation.
ii. Identify different Formal language Classes and their Relationships.
iii. Design Grammars and Recognizers for different formal languages.
iv. Prove or disprove theorems in automata theory using their properties.
v. Determine the decidability and intractability of Computational problems.
Syllabus of Module 3
i. Context-Free Grammars(CFG): Introduction to Rewrite Systems and Grammars
ii. CFGs and languages, designing CFGs,
iii. Simplifying CFGs,
iv. Proving that a Grammar is correct,
v. Derivation and Parse trees, Ambiguity,
vi. Normal Forms.
vii. Pushdown Automata (PDA): Definition of non-deterministic PDA,
viii. Deterministic and Non-deterministic PDAs,
ix. Non-determinism and Halting, Alternative equivalent definitions of a PDA,
x. Alternatives those are not equivalent to PDA.
Text Books:
1. Elaine Rich, Automata, Computability and Complexity, 1st Edition, Pearson
Education, 2012/2013. Text Book 1: Ch 11, 12: 11.1 to 11.8, 12.1 to 12.6 excluding
12.3.
2. K L P Mishra, N Chandrasekaran , 3rd Edition, Theory of Computer Science, PHI,
2012
Reference Books:
1. John E Hopcroft, Rajeev Motwani, Jeffery D Ullman, Introduction to Automata
Theory, Languages, and Computation, 3rd Edition, Pearson Education, 2013
2. Michael Sipser : Introduction to the Theory of Computation, 3rd edition, Cengage
learning,2013
3. John C Martin, Introduction to Languages and The Theory of Computation, 3rd
Edition,Tata McGraw –Hill Publishing Company Limited, 2013
4. Peter Linz, “An Introduction to Formal Languages and Automata”, 3rd Edition,
Narosa Publishers, 1998
5. Basavaraj S. Anami, Karibasappa K G, Formal Languages and Automata theory,
WileyIndia, 2012
1
Learning Outcomes:
At the end of the module student should be able to:
Sl.No TLO’s
1. Define context free grammars and languages
2. Design the grammar for the given context free languages.
3. Apply the simplification algorithm to simplify the given grammar
4. Prove the correctness of the grammar
5. Define leftmost derivation and rightmost derivation
6. Draw the parse tree to a string for the given grammar.
7. Define ambiguous and inherently ambiguous grammars.
8. Prove whether the given grammar is ambiguous grammar or not.
9. Define Chomsky normal form. Apply the normalization algorithm to
convert the grammar to Chomsky normal form.
10. Define Push down automata (NPDA). Design a NPDA for the given
CFG.
11. Design a DPDA for the given language.
12. Define alternative equivalent definitions of a PDA.
2
Rewrite systems can be used to define functions. We write rules that operate on an input
string to produce the required output string. Rewrite systems can be used to define languages.
The rewrite system that is used to define a language is called a grammar.
3
When to Stop
Case 1: The working string no longer contains any nonterminal symbols (including, when it
is ). In this case, we say that the working string is generated by the grammar.
Example: S ⇒ aSb ⇒ aaSbb ⇒ aabb
Case 2: There are nonterminal symbols in the working string but none of them appears on the
left-hand side of any rule in the grammar. In this case, we have a blocked or non-terminated
derivation but no generated string.
Given: S aSb ----- rule 1
S bTa ----- rule 2
S ----- rule 3
Derivation so far: S ⇒ aSb ⇒ abTab ⇒
Case 3: It is possible that neither case 1 nor case 2 is achieved.
Given: S Ba -----rule 1
B bB -----rule 2
Then all derivations proceed as: S ⇒ Ba ⇒ bBa ⇒ bbBa ⇒ bbbBa ⇒ bbbbBa ⇒ ...
The grammar generates the language Ø.
Recall Regular Grammar which has a left-hand side that is a single nonterminal and have a
right-hand side that is or a single terminal or a single terminal followed by a single
nonterminal.
X →Y
( NT) ( or T or T NT)
4
Definition Context-Free Grammar
A context-free grammar G is a quadruple, (V, , R, S), where:
• V is the rule alphabet, which contains nonterminals and terminals.
• (the set of terminals) is a subset of V,
• R (the set of rules) is a finite subset of (V - ) V*,
• S (the start symbol) is an element of V - .
Given a grammar G, define x ⇒G y to be the binary relation derives-in-one-step, defined so
that ∀ x,y V* (x ⇒G y iff x = A, y = and there exists a rule A is in RG )
Any sequence of the form w0 ⇒G w1 ⇒G w2 ⇒G . . . ⇒G wn is called a derivation in G. Let
⇒G* be the reflexive, transitive closure of ⇒G. We’ll call ⇒G* the derive relation.
A derivation will halt whenever no rules on the left hand side matches against working-string.
At every step, any rule that matches may be chosen.
5
Self-Embedding Grammar Rules
A grammar is self-embedding iff it contains at least one self-embedding rule. A rule in a
grammar G is self-embedding iff it is of the form X w1Yw2, where Y ⇒* w3Xw4 and both
w1w3 and w4w2 are in +. No regular grammar can impose such a requirement on its strings.
Example: S aSa is self-embedding
S aS is recursive but not self- embedding
S aT
T Sa is self-embedding
Example : PalEven = {ww : w {a, b}*}= The L of even length palindrome of a’s and b’s.
R
6
Example2: A CFG for C++ compound statements:
<compound stmt> { <stmt list> }
<stmt list> <stmt> <stmt list> | epsilon
<stmt> <compound stmt>
<stmt> if ( <expr> ) <stmt>
<stmt> if ( <expr> ) <stmt> else <stmt>
<stmt> while ( <expr> ) <stmt>
<stmt> do <stmt> while ( <expr> ) ;
<stmt> for ( <stmt> <expr> ; <expr> ) <stmt>
<stmt> case <expr> : <stmt>
<stmt> switch ( <expr> ) <stmt>
<stmt> break ; | continue ;
<stmt> return <expr> ; | goto <id> ;
S NP VP
NP the Nominal | a Nominal | Nominal | ProperNoun | NP PP
Nominal N | Adjs N
N cat | dogs | bear | girl | chocolate | rifle
ProperNoun Chris | Fluffy
Adjs Adj Adjs | Adj
Adj young | older | smart
VP V | V NP | VP PP
V like | likes | thinks | shots | smells
PP Prep NP
Prep with
If L has a property that every string in it has two regions & those regions must bear some
relationship to each other, then the two regions must be generated in tandem. Otherwise,
there is no way to enforce the necessary constraint.
7
Example 1: L = {anbncm : n, m ≥ 0} = L = {, ab, c, abc, abcc, aabbc, …….}
The cm portion of any string in L is completely independent of the anbn portion, so we should
generate the two portions separately and concatenate them together.
G = ({S, A, C, a, b, c}, {a, b, c}, R, S} where:
R = { S AC /* generate the two independent portions
A aAb | /* generate the anbn portion, from the outside in
C cC | } /* generate the cm portion
Example derivation in G for string abcc:
S ⇒ AC ⇒ aAbC ⇒ abC ⇒ abcC ⇒ abccC ⇒ abcc
Example 4: L = {anwwR bn: w {a, b}*} = {, ab, aaab, abbb, aabbab, aabbbbab, ..…….}
The anbn is the inner portion and wwR is the outer portion of any string in L.
G = {{S, A, a, b}, {a, b}, R, S}, where:
R = {S aSb ----- rule 1
SA ----- rule 2
A aAa ----- rule 3
A bAb ----- rule 4
A ----- rule 5 }.
Example derivation in G for string aabbab:
S ⇒ aSb ⇒ aAb ⇒ aaAab ⇒ aabAbab ⇒ aabbab
8
Example 5: Equal Numbers of a’s and b’s. = {w {a, b}*: #a(w) = #b(w)}.
L = {, ab, ba, abba, aabb, baba, bbaa, …….}
G = {{S, a, b}, {a, b}, R, S}, where:
R = { S aSb ----- rule 1
S bSa ----- rule 2
S SS ----- rule 3
S ----- rule 4 }.
Example derivation in G for string abba:
S ⇒ aSa ⇒ abSba ⇒ abba
Example 6
L = {aibj : 2i = 3j + 1} = {a2b1 , a5b3 , a8b5 …….}
G = {{S, a, b}, {a, b}, R, S}, where:
aibj 2i = 3j + 1
2 1
ab 2*2= 3*1 + 1 = 4
5 3
ab 2*5= 3*3 + 1 = 10
8 5
ab 2*8= 3*5 + 1 = 16
R={ S aaaSbb | aab }
Example derivation in G for string aaaaabbb:
S ⇒ aaaSbb ⇒ aaaaabbb
9
Example: G = ({S, A, B, C, D, a, b}, {a, b}, R, S), where
R={ S AB | AC
A aAb |
B bA
C bCa
D AB }
1) a and b terminal symbols are productive
2) A is productive( because A aAb )
3) B is productive( because B bA )
4) S & D are productive(because S AB & D AB )
5) C is unproductive
On eliminating C from both LHS and RHS the rule set R obtained is
R = { S AB A aAb | B bA D AB }
Example
G = ({S, A, B, C, D, a, b}, {a, b}, R, S), where
R = {S AB
A aAb |
B bA
D AB }
S, A, B are reachable but D is not reachable, on eliminating D from both LHS and RHS the
rule set R is
R = { S AB
A aAb |
B bA }
10
5. Proving the Correctness of a Grammar
Given some language L and a grammar G, can we prove that G is correct (ie it generates
exactly the strings in L)
To do so, we need to prove two things:
1. Prove that G generates only strings in L.
2. Prove that G generates all the strings in L.
Algorithms used for generation and recognition must be systematic. The expansion order is
important for algorithms that work with CFG. To make it easier to describe such algorithms,
we define two useful families of derivations.
a. A leftmost derivation is one in which, at each step, the leftmost nonterminal in the
working string is chosen for expansion.
b. A rightmost derivation is one in which, at each step, the rightmost nonterminal in the
working string is chosen for expansion.
Example 1 : S → AB | aaB A → a | Aa B → b
Left-most derivation for string aab is S ⇒ AB ⇒ AaB ⇒ aaB ⇒ aab
Right-most derivation for string aab is S ⇒ AB ⇒ Ab ⇒ Aab ⇒ aab
11
Left-most Derivation for the string “the smart cat smells chocolate”
S ⇒ NP VP
⇒ the Nominal VP
⇒ the Adjs N VP
⇒ the Adj N VP
⇒ the smart N VP
⇒ the smart cat VP
⇒ the smart cat V NP
⇒ the smart cat smells NP
⇒ the smart cat smells Nominal
⇒ the smart cat smells N
⇒ the smart cat smells chocolate
Right-most Derivation for the string “the smart cat smells chocolate”
S ⇒ NP VP
⇒ NP V NP
⇒ NP V Nominal
⇒ NP V N
⇒ NP V chocolate
⇒ NP smells chocolate
⇒ the Nominal smells chocolate
⇒ the Adjs N smells chocolate
⇒ the Adjs cat smells chocolate
⇒ the Adj cat smells chocolate
⇒ the smart cat smells chocolate
Parse Trees
Regular grammar: in most applications, we just want to describe the set of strings in a
language. Context-free grammar: we also want to assign meanings to the strings in a
language, for which we care about internal structure of the strings. Parse trees capture the
essential grammatical structure of a string. A program that produces such trees is called a
parser. A parse tree is an (ordered, rooted) tree that represents the syntactic structure of a
string according to some formal grammar. In a parse tree, the interior nodes are labeled by
non terminals of the grammar, while the leaf nodes are labeled by terminals of the grammar
or .
A parse tree, derived by a grammar G = (V, S, R, S), is a rooted, ordered tree in which:
1. Every leaf node is labeled with an element of ∑ ∪{ },
2. The root node is labeled S,
3. Every other node is labeled with some element of: V –∑, and
4. If m is a nonleaf node labeled X and the children of m are labeled x1, x2, …,
xn, then R contains the rule X → x1, x2, …, xn.
12
Example 1: S AB | aaB A a | Aa B b
Left-most derivation for the string aab is S ⇒ AB ⇒ AaB ⇒ aaB ⇒ aab
Parse tree obtained is
Example 3: Parse Tree -Structure in English for the string “the smart cat smells
chocolate”. It is clear from the tree that the sentence is not about cat smells or smart cat
smells.
A parse tree may correspond to multiple derivations. From the parse tree, we cannot tell
which of the following is used in derivation:
S ⇒ NP VP ⇒ the Nominal VP ⇒
S ⇒ NP VP ⇒ NP V NP ⇒
Parse trees capture the important structural facts about a derivation but throw away the details
of the nonterminal expansion order. The order has no bearing on the structure we wish to
assign to a string.
Generative Capacity
Because parse trees matter, it makes sense, given a grammar G, to distinguish between:
1. G’s weak generative capacity, defined to be the set of strings, L(G), that G generates,
and
2. G’s strong generative capacity, defined to be the set of parse trees that G generates.
When we design grammar, it will be important that we consider both their weak and their
strong generative capacities.
7. Ambiguity
Sometimes a grammar may produce more than one parse tree for some (or all ) of the strings
it generates. When this happens we say that the grammar is ambiguous. A grammar is
ambiguous iff there is at least one string in L(G) for which G produces more than one parse
tree.
13
Example 1: Bal={w { ),(}*: the parenthesis are balanced}.
G={{S,),( }, {),(},R,S} where R={ S S SS S (S) }
Left-most Derivation1 for the string (())() is S ⇒ S⇒(S)S ⇒ ((S))S ⇒ (())S ⇒ (())(S) ⇒ (())()
Left-most Derivation2 for the string (())() is S ⇒ SS ⇒SSS ⇒SS ⇒ (S)S ⇒ ((S))S ⇒ (())S
⇒ (())(S) ⇒ (())()
Since both the parse trees obtained for the same string (())() are different, the grammar is ambiguous.
Since both the parse trees obtained for the same string iytiytxex are different, the grammar is
ambiguous.
Since both the parse trees obtained for the same string aab are different, the grammar is
ambiguous.
14
Why Is Ambiguity a Problem?
With regular languages, for most applications, we do not care about assigning internal
structure to strings.
With context-free languages, we usually do care about internal structure because, given a
string w, we want to assign meaning to w. It is generally difficult, if not impossible, to assign
a unique meaning without a unique parse tree. So an ambiguous G, which fails to produce a
unique parse tree is a problem.
Consider string 2+3*5 written as id +id*id, left-most derivation for string id +id*id is
E ⇒ E*E ⇒ E+E*E ⇒ id+E*E ⇒ id+id*E ⇒ id+id*id.
Similarly the right-most derivation for string id +id*id is
E ⇒ E+E ⇒ E+E*E ⇒ E+E*id ⇒ E+id*id ⇒ id+id*id.
The parse trees obtained for both the derivations are:-
Should the evaluation of this expression return 17 or 25? Designers of practical languages
must be careful that they create languages for which they can write unambiguous grammars.
Techniques for Reducing Ambiguity
No general purpose algorithm exists to test for ambiguity in a grammar or to remove it when
it is found. But we can reduce ambiguity by eliminating
a. rules like S →
b. Rules with symmetric right-hand sides
• A grammar is ambiguous if it is both left and right recursive.
• Fix: remove right recursion
• S → SS or E→E+E
c. Rule sets that lead to ambiguous attachment of optional postfixes.
15
a. Eliminating -Rules
Let G =(V, , R, S) be a CFG. The following algorithm constructs a G such that L(G ) =
L(G)-{} and G contains no rules:
removeEps(G: CFG) =
1. Let G = G.
2. Find the set N of nullable variables in G.
3. Repeat until G contains no modifiable rules that haven’t been processed:
Given the rule PQ, where Q N, add the rule P if it is not already present
and if and if P .
4. Delete from G all rules of the form X .
5. Return G.
16
What If L?
Sometimes L(G) contains and it is important to retain it. To handle this case the algorithm
used is
atmostoneEps(G: CFG) =
1. G = removeEps(G).
2. If SG is nullable then /* i. e., L(G)
2.1 Create in G a new start symbol S*.
2.2 Add to RG the two rules:S* and S* SG.
3. Return G.
The new grammar built is better than the original one. The string (())() has only one parse
tree.
But it is still ambiguous as the string ()()() has two parse trees?
So we get: S* | S
S SS1 /* force branching only to the left
S S1 /* add rule
S1 (S) | ()
17
Unambiguous Grammar for Bal={w { ),(}*: the parenthesis are balanced}.
G={{S,),( }, {),(},R,S} where
S* n | S
S SS1 | S1
S1 (S) | ()
The parse tree obtained for the string ()()() is
18
Proving that the grammar is Unambiguous
A grammar is unambiguous iff for all strings w, at every point in a leftmost derivation or
rightmost derivation of w, only one rule in G can be applied.
S* ---(1)
S* S ---(2)
S SS1 ---(3)
S S1 ---(4)
S1 (S) ---(5)
S1 () ---(6)
S* ⇒ S ⇒SS1 ⇒SS1S1 ⇒S1S1S1 ⇒ () S1S1 ⇒ () () S1 ⇒ () () ()
Inherent Ambiguity
In many cases, for an ambiguous grammar G, it is possible to construct a new grammar G
that generate L(G) with less or no ambiguity. However, not always. Some languages have the
property that every grammar for them is ambiguous.We call such languages inherently
ambiguous.
Example: L = {aibjck: i, j , k 0, i=j or j=k}.
Every string in L has either (or both) the same number of a’s and b’s or the same number of
b’s and c’s. L = {anbncm: n, m 0} {anbmcm: n, m 0}.
One grammar for L has the rules:
S S1 | S2
S1 S1c | A /* Generate all strings in {anbncm}.
A aAb |
S2 aS2 | B /* Generate all strings in {anbmcm}.
B bBc |
Consider the string a2b2c2 .
It has two distinct derivations, one through S1 and the other through S2
S ⇒ S1 ⇒ S1c⇒ S1cc ⇒ Acc ⇒ aAbcc ⇒ aaAbbcc ⇒ aabbcc
S ⇒ S2 ⇒ aS2 ⇒ aaS2 ⇒ aaB ⇒ aabBc ⇒ aabbBcc⇒ aabbcc
Given any grammar G that generates L, there is at least one string with two derivations in G.
19
8. Normal Forms
We have seen in some grammar where RHS is , it makes grammar harder to use. Lets see
what happens if we carry the idea of getting rid of -productions a few steps farther. To
make our tasks easier we define normal forms.
Normal Forms - When the grammar rules in G satisfy certain restrictions, then G is said to be
in Normal Form.
• Normal Forms for queries & data can simplify database processing.
• Normal Forms for logical formulas can simplify automated reasoning in AI systems
and in program verification system.
• It might be easier to build a parser if we could make some assumptions about the form
of the grammar rules that the parser will use.
Normal Forms for Grammar
Among several normal forms, two of them are:-
• Chomsky Normal Form(CNF)
• Greibach Normal Form(GNF)
20
Converting to Chomsky Normal Form
Apply some transformation to G such that the language generated by G is unchanged.
1. Rule Substitution.
Example: X aYc Y b Y ZZ equivalent grammar constructed is X abc | aZZc
There exists 4-steps algorithm to convert a CFG G into a new grammar Gc such that: L(G) =
L(Gc) – {}
convertChomsky(G:CFG) =
1. G' = removeEps(G:CFG) S
2. G'' = removeUnits(G':CFG) AB
3. G''' = removeMixed(G'':CFG) A aB
4. G'v = removeLong(G''' :CFG) S ABCD
return Gc
21
removeUnits returns G'' :
S aACa | aAa | aCa | aa
A cC | a | c
B cC | c
C cC | c
22
Now, by apply removeLong returns G'v :
S TaS1 | TaS3 | TaS2 | TaTa
S1 AS2
S2 CTa
S3 ATa
A TcC | a | c
B TcC | c
C TcC | c
Ta a
Tc c
23
Example 3: Apply the normalization algorithm to convert the grammar to CNF
G: S → ABC
A → aC | D
B → bB | ε | A
C → Ac | ε | Cc
D → aa
removeEps(G:CFG) returns
G: S → ABC | AC | AB | A
A → aC | D | a
B → bB | A | b
C → Ac | Cc | c
D → aa
removeUnits(G':CFG) returns
G : S → ABC | AC | AB | aC | aa | a
A → aC | aa | a
B → bB | aC | aa | a | b
C → Ac | Cc | c
D → aa
removeMixed(G'':CFG) returns
G : S → ABC | AC | AB | Ta C | Ta Ta | a
A → Ta C | Ta Ta | a
B → Tb B | Ta C | Ta Ta | a | b
C → A Tc | C Tc | c
D → Ta Ta
Ta a
Tb b
Tc c
24
9. Pushdown Automata
An acceptor for every context-free language. A pushdown automata , or PDA, is a finite state
machine that has been augmented by a single stack.
Definition of a (NPDA) Pushdown Automaton
M = (K, S, G, Δ , s, A), where:
K is a finite set of states,
S is the input alphabet,
G is the stack alphabet,
s ∈ K is the initial state,
A ⊆ K is the set of accepting states, and
Δ is the transition relation.
Configuration
A configuration of PDA M is an element of K X S* X G*. Current state, Input that is still left
to read and, Contents of its stack.
The initial configuration of a PDA M, on input w, is (s, w, ).
If s1s2…sn is pushed onto the stack: the value after the push is s1s2…sncba
Yields-in-one-step
Yields-in-one-step written |-M relates configuration1 to configuration2 iff M can move from
configuration1 to configuration2 in one step. Let c be any element of ∑ U { }, let 1,
2 and be any elements of G*, and let w be any element of S*. Then:
(q1, cw, 1 ) |-M (q2, w, 2 ) iff ((q1, c, 1), (q2, 2)) ∈ Δ .
The relation yields, written |-M* is the reflexive, transitive closure of |-M C1 yields
configuration C2 iff C1 |-M* C2
25
Computation
A computation by M is a finite sequence of configurations C0, C1, C2,,,,,,,,,,,,,Cn for some n ≥0
such that:
• C0 is an initial configuration,
• Cn is of the form (q, , ), for some q ∈ K and some string in G*, and
• C0 |-M C1 |-M C2 |-M ,,,,,,,,,,,, |-M Cn.
Nondeterminism
If M is in some configuration (q1, s, ) it is possible that:
● Δ contains exactly one transition that matches. In that case, M makes the specified
move.
● Δ contains more than one transition that matches. In that case, M chooses one of
them.
● Δ contains no transition that matches. In that case, the computation that led to that
configuration halts.
Accepting
Let C be a computation of M on input w then C is an accepting configuration
iif C= (s, w, ) |-M* (q, , ), for some q ∈ A.
A computation accepts only if it runs out of input when it is in an accepting state and the
stack is empty.
C is a rejecting configuration iif C= (s, w, ) |-M* (q, w, ),
where C is not an accepting computation and where M has no moves that it can makes from
(q, w, ). A computation can reject only if the criteria for accepting have not been met and
there are no further moves that can be taken.
Let w be a string that is an element of S* . Then:
• M accepts w iif atleast one of its computations accepts.
• M rejects w iif all of its computations reject.
The language accepted by M, denoted L(M), is the set of all strings accepted by M. M rejects
a string w iff all paths reject it.
It is possible that, on input w, M neither accepts nor rejects. In that case, no path accepts and
some path does not reject.
Transition
Transition ((q1, c, 1), (q2, 2)) says that “If c matches the input and g1 matches the current
top of the stack, the transition from q1 to q2 can be taken. Then, c will be removed from the
input, 1 will be popped from the stack, and 2 will be pushed onto it. M cannot peek at the
top of the stack without popping
• If c = , the transition can be taken without consuming any input.
• If 1 = , the transition can be taken without checking the stack or popping anything.
Note: it’s not saying “the stack is empty”.
• If 2 = , nothing is pushed onto the stack when the transition is taken.
26
Example1: A PDA for Balanced Parentheses. Bal={w { ),(}*: the parenthesis are
balanced}
M = (K, S, G, Δ, s, A),
where:
K = {s} the states
S = {(, )} the input alphabet
= {(} the stack alphabet
A = {s} the accepting state
Δ = { ((s, (, ), (s, ( )) ----- (1)
((s, ), ( ), (s, )) ----- (2) }
Unread
Transition State Stack
input
S (()))
1 S ())) (
1 S ))) ((
2 S )) (
2 S )
The computation has reached the final state S and stack is empty, but still the string is
rejected because the input is not empty.
27
Example2 of Rejecting -- Input string = ((())
Transition State Unread input Stack
S ((())
1 S (()) (
1 S ()) ((
1 S )) (((
2 S ) ((
2 S (
(S, ((()),) |- (S, (()),( |- (S,( )),(() |- (S, )),((() |- (S, ),(() |- (S, ,()
The computation has reached the final state S and runs out of input, but still the string is
rejected because the stack is not empty.
The computation has reached the final state f, the input string is consumed and the stack is
empty. Hence the string aabb is accepted.
28
Δ = {((s, a, ), (s, a) -----(1)
((s, b, ), (s, b)) -----(2)
((s, c, ), (f, )) -----(3)
((f, a, a), (f, )) -----(4)
((f, b, b), (f, ))} -----(5)
29
10. Deterministic and Nondeterministic PDAs
Exploiting Nondeterministic
Previous examples are DPDA, where each machine followed only a single computational
path. But many useful PDAs are not deterministic, where from a single configuration there
exist multiple competing moves. As in FSMs, easiest way to envision the operation of a
NDPDA M is as a tree.
Each node in the tree corresponds to a configuration of M and each path from the root to a
leaf node may correspond to one computation that M might perform. The state, the stack and
the remaining input can be different along different paths. As a result, it will not be possible
to simulate all paths in parallel, the way we did for NDFSMs.
30
Example 2: PDA for {w {a, b}* : #a(w) = #b(w)}= Equal Numbers of a’s and b’s.
L = {, ab, ba, abba, aabb, baba, bbaa, …….}
M = (K, S, G, Δ, s, A),
where:
K = {s} the states
S = {a, b} the input alphabet
Γ = {a, b} the stack alphabet
A = {s} the accepting state
Δ ={((s, a, ), (s, a)) -----(1)
((s, b, ), (s, b)) -----(2)
((s, a, b), (s, )) -----(3)
((s, b, a), (s, ))} -----(4)
Example 3: The a Region and the b Region are Different. L = {ambn : m ≠ n; m, n > 0}
It is hard to build a machine that looks for something negative, like ≠. But we can break L
into two sublanguages: {ambn : 0 < n < m} and {ambn : 0 < m < n}
• If stack and input are empty, halt and reject
• If input is empty but stack is not (m > n) (accept)
• If stack is empty but input is not (m < n) (accept)
31
Δ = { ((1, a, ), (1, a )) -----(1)
((1, b, a ), (2, )) -----(2)
((2, b, a ), (2, )) -----(3)
((2, , a ), (3, )) -----(4)
((3, , a ), (3, )) } -----(5)
32
Two problems with this M:
1. We have no way to specify that a move can be taken only if the stack is empty.
2. We have no way to specify that the input stream is empty.
3. As a result, in most of its moves in state 2, M will have a choice of three paths to take.
Case1: A transition that should be taken only if the stack is empty competes against one or
more moves that require a match of some string on the stack.
Problem: Nondeterminism could be eliminated if it were possible to check for an empty
stack.
Solution: Using a special bottom-of-stack marker ( # )
Before doing anything, push a special character onto the stack. The stack is then logically
empty iff that special character (#) is at the top of the stack. Before M accepts a string, its
stack must be completely empty, so the special character must be popped whenever M
reaches an accepting state.
Now the transition back to state 2 no longer competes with the transition to state 4, which can
only be taken when the # is the only symbol on the stack. The machine is still
nondeterministic because the transition back to state 2 competes with the transition to state 3.
33
Case2: A transition that should be taken only if the input stream is empty competes against
one or more moves that require a match against a specific input character.
Problem: Nondeterminism could be eliminated if it were possible to check for an empty input
stream.
Solution: using a special end-of-string marker ($ )
Adding an end-of-string marker to the language to be accepted is a powerful tool for reducing
nondeterminism. Instead of building a machine to accept a language L, build one to accept
L$, where $ is a special end-of-string marker.
Now the transition back to state 2 no longer competes with the transition to state 3, since the
can be taken when the $ is read. The $ must be read on all the paths, not just the one where
we need it.
34
But the facts about CFGs and PDAs are different from the facts about RLs and FSMs.
1. There are context-free languages for which no deterministic PDA exists.
2. It is possible that a PDA may
● not halt,
● not ever finish reading its input.
However, for an arbitrary PDA M, there exists M that halts and L(M) = L(M).
There exists no algorithm to minimize a PDA. It is undecidable whether a PDA is minimal.
Problem 2 : Let M be a PDA that accepts some language L. Then, on input w, if w L then
M will halt and accept. But if w L then, while M will not accept w, it is possible that it will
not reject it either.
Example1: Let S = {a} and consider M =
For L(M) = {a}. The computation (1, a, e) |- (2, a, a) |- (3, e, e) causes M to accept a.
Example2: Consider M =
35
There are two alternatives to this:
1. PDA by Final state: Accept if, when the input has been consumed, M lands in an
accepting state, regardless of the contents of the stack.
2. PDA by Empty stack: Accept if, when the input has been consumed, the stack is
empty, regardless of the state M is in.
All of these definitions are equivalent in the sense that, if some language L is accepted by a
PDA using one definition, it can be accepted by some PDA using each of the other definition.
For example:- If some language L is accepted by a PDA by Final state then it can be accepted
by PDA by Final state and empty stack. If some language L is accepted by a PDA by Final
state and empty stack then can be accepted by PDA by Final state.
We can prove by showing algorithms that transform a PDA of one sort into and equivalent
PDA of the other sort.
Equivalence
1. Given a PDA M that accepts by accepting state and empty stack, construct a new
PDA M that accepts by accepting state alone, where L(M) = L(M).
2. Given a PDA M that accepts by accepting state alone, construct a new PDA M that
accepts by accepting state and empty stack, where L(M) = L(M).
Hence we can prove that M and M accept the same strings.
1. Accepting by Final state Alone
Define a PDA M = (K, S, G, Δ , s, A). Accepts when the input has been consumed, M lands
in an accepting state, regardless of the contents of the stack. M accepts if C= (s, w, ) |-M* (q,
, g), for some q A.
M will have a single accepting state qa. The only way for M to get to qa will be to land in an
accepting state of M when the stack is logically empty. Since there is no way to check that
the stack is empty, M will begin by pushing a bottom-of-stack marker #, onto the stack.
Whenever # is the top symbol of the stack, then stack is logically empty.
The construction proceeds as follows:
1. Initially, let M = M.
2. Create a new start state s.
Add the transition ((s, , ),(s, #)),
3. For each accepting state a in M do:
Add the transition ((a, ,#),(qa, )),
4. Make qa the only accepting state in M
Example:
36
It is easy to see that M lands in its accepting state(qa) iff M lands in some accepting state
with an empty stack. Thus M and M accept the same strings.
2. Accepting by Final state and Empty stack
The construction proceeds as follows:
1. Initially, let M = M.
2. Create a new accepting state F
3. From each accepting state a in M do:
Add the transition ((a, , ),(F, )),
4. Make F the only accepting state in M
5. For every element g of Γ,
Add the transition to M ((F, , g), (F, )).
In other words, iff M accepts, go to the only accepting state of M′ and clear the stack. Thus
M′ will accept by accepting state and empty stack iff M accepts by accepting state.
Example:-
37
Sl.No Sample Questions
1. Define context free grammars and languages.
11. Define Chomsky normal form. Apply the normalization algorithm to convert the grammar to
Chomsky normal form.
a. S → aSa S → B B → bbC
B → bb C → ε C → cC
b. S → ABC A → aC | D B → bB | ε | A
C → Ac | ε | Cc D → aa
12. Define Push down automata (NPDA). Design a NPDA for the CFG given in Question (2).
13. Design a PDA for the given language.L$, where L = {w ∈ {a, b}* : #a(w) = #b(w)}.
14. Design a PDA for the language: L={ aibjck : i+j=k ,i>=0,j>=0}
16. Design a PDA for the language: L={ aibjck : i+k=j ,i>=0,k>=0}
17. Design a PDA for the language: L={ aibjck : k+j=i ,k>=0,j>=0}
38
Module-4
Context-Free and Non-Context-Free Languages
Where Do the Context-Free Languages Fit in the Big Picture?
Showing that a Language is Context-Free
Pumping theorem for CFL
Important closure properties of CFLs
Deterministic CFLs
Algorithms and Decision Procedures for CFLs: Decidable questions
Undecidable questions
Turing Machine: Turing machine model
Representation
Language acceptability by TM
Design of TM
Techniques for TM construction.
Theorem: The regular languages are a proper subset of the context-free languages.
Proof: We first show that every regular language is context-free. We then show that there
exists at least one context-free language that is not regular.
There cannot be more CFLs than CFGs. So there are at most a countably infinite number of
context-free languages. There is not a one-to-one relationship between CFLs and CFGs, since
there are an infinite number of grammars that generate any given language. But we know that,
every regular language is context free and there is a countably infinite number of regular
languages.
Theorem: The length of the yield of any tree T with height h and
branching factor b is <= bh.
Proof:
If h is 1, then a single rule applies. So the longest yield is of length less than or equal to b.
Assume the claim is true for h=n. We show that it is true for h=n+1.
Consider any tree with h=n+1. It consists of a root, and some number of subtrees, each of height
<=n. By the induction hypothesis, the length of the yield of each of those subtrees is <= bn . So the
length of the yield must be <=b.(bn )=bn+1 =bh .
The tree rooted at [1] has height at most n+1.Thus its yield, vxy, has length less than or equal
to bn+1 ,which is k. Further, vy≠Ɛ .Since if vy were Ɛ then there would be a smaller parse tree for w
and we choose T so that h at wasn't so.
Finally, v and y can be pumped: uxz must be in L because rule2 could have been used immediately
at[1]. And, for any q≥1, uvqxyqz must be in L because rule1 could have been used q times before
finally using rule2.
Call the part before the c the leftside and the part after the c the right side. We consider all the
cases for where v and y could fall and show that in none of them are all the condition so f the theorem met:
• If either v or y overlaps region 3, set q to 0. The resulting string will no longer contain a c and so is
not in WcW.
• If both v and y occur before region 3 or they both occur after region 3, then set q to 2. One side will
be longer than the other and so the resulting string is not in WcW.
• If either v or y overlaps region1 ,then set q to 2. In order to make the right side match. Something
would have to be pumped into region 4. But any v,y pair that did that would violate the requirement
that lvxyl ≤ k.
• If either v or y overlaps region2, then set q to 2. In order to make the right side match, something
would have to be pumped into region 5. But any v,y pair that did that would violate the requirement
that lvxyl ≤ k.
• There is no way to divide w into uvxyz such that all the conditions of the Pumping Theorem
are met . So WcW is not context-free.
• If L1 and L2 are context free languages then there exists a context-free grammar G 1 = (V1 ,Σ1 , R1 ,S1 )
and G2 =(V2 ,Σ2 ,R2 ,S2 ) such that L1 =L(G1 ) and L2 =L(G2 ).
• We will build a new grammar G such that L(G)=L(G1 )UL(G2 ). G will contain all the rules of both
G1 and G2 .
• We add to G a new start symbol S and two new rules. S→S 1 and S→S2 . The two new rules allow G
to generate a string iff at least one of G1 or G2 generates it.
So, G = ( V1 U V2 U {S}, Σ1 U Σ2 , R1 U R2 U {S→ S1 ,S→S2 }, S )
• Every rule in G is of the form X→BC or X→a, where X, B, and C are elements of (V-Σ) and a ϵΣ
• So construct, from G, a new grammar G1 , Such that L(G1 )= LR .
• G1 = (VG, ΣG, R', SG) , Where R' is constructed as follows:
For every rule in G of the form X→BC, add to R' the rule X→CB
For every rule in G of the form X→ a then add to R' the rule X→ a
• If L2 is regular then there exists a DFSM M2 = (K 2 ,Σ,𝛿,S2 ,A2 ) that accepts it.
• We construct a new PDA, M3 that accepts L1 ∩ L2 . M3 will work by simulating the parallel
execution of M1 and M2 .
•For each transition ((q1 , a, 𝛃) ,(p1 , 𝛄)) in 𝚫1 and each transition ((q2 , a ) ,p2 ) in 𝛿, add 𝚫3 the
transition: (((q1 ,q2 ),a,𝛃) ,((p1 ,p2 ), 𝛄)).
•For each transition ((q1 ,ℇ,𝛃) ,(p1 , 𝛄)) in 𝚫1 and each state q2 in k2 , add to 𝚫3 the transition:
(((q1 ,q2 ),ℇ,𝛃) ,((p1 ,p2 ), 𝛄)).
For example, the PDA accepts L, but it is nondeterministic because the transition to state3 (where the
a's will be popped) can compete with both of the other transitions from state1.
With an end-of-string marker, we can build the deterministic PDA, which can only take the transition
to state3, the a-popping state. When it sees the $:
NOTE: Adding the end-of-string marker cannot convert a language that was not
context-free into one that is.
•There exist languages that are in the outer donut because they are inherently ambiguous. Two
examples are:
{aibjck |i,j,k ≥ 0 and ((i≠j)or(j≠k))}
{aibjck |i,j,k ≥ 0 and ((i=j) or (j=k))}
Regular Language is Deterministic Context-Free
Theorem: Every regular language is deterministic context-free.
Proof: The proof is by construction. {$} is regular. So, if L is regular then so is L$ (since the
regular languages are closed under concatenation).So there is a DFSM M that accepts it. Using the
construction to show that every regular language is context-free Construct, from M a PDA P that
accepts L$. P will be deterministic.
buildunambiggrammar(M:deterministicPDA)=
1. Let G=buildgrammar(convertPDAtodetnormalform(M)).
2. Let G' be the result of substituting ε for $ in each rule in which $ occurs.
3. Return G'.
NOTE: The algorithm convertPDAtodetnormalform, is described in the theorem that proves the
deterministic context-free languages are closed under complement.
This question can be answered for every context-free language and for every context-free
language L there exists a PDA M such that M accepts L. But existence of a PDA that accepts L does
not guarantee the existence of a procedure that decides it.
It turns out that there are two alternative approaches to solving this problem, both of which
work:
● Use a grammar: Using facts about every derivation that is produced by a grammar in
Chomsky normal form, we can construct an algorithm that explores a finite number of
derivation paths and finds one that derives a particular string w iff such a path exists.
● Use a PDA : While not all PDAs halt, it is possible, for any context-free language L, to
craft a PDA M that is guaranteed to halt on all inputs and that accepts all strings in L and
rejects all strings that are not in L.
1. If L is specified as a PDA, use PDA to CFG, to construct a grammar G such that L(G) =L (M).
4. If w ≠ ε then:
4.1. From G, construct G' such that L (G') = L(G)-{ε} and G' is in Chomsky normal form.
4.2. If G derives to, it does so in (2 • |w| - 1) steps. Try all derivations in G of that number
Elimination of ε-Transitions
Theorem: Given any context-free grammar G=(V,Σ,R,S), there exists a PDA M
such that L(M)=L(G)-{ε} and M contains no transitions of the form
((q1,ε,α),(q2,𝜷)). In other words, every transition reads exactly one input
character.
Proof: The proof is by a construction that begins by converting G to Greibach normal form. Now
consider again the algorithm cfgtoPDAtopdown, which builds, from any context-free grammar G, a
PDA M that, on input w, simulates G deriving w, starting from S.
M= ({p,q},Σ,V,Δ, p,{q}), where Δ contains:
1. The start-up transition ((p,ε,ε),(q,S)), which pushes the start symbol on to the stack and
goes to state q.
2. For each rule X→s1 s2 ...sn , in R, the transition ((q,ε,X),(q,s1 s2 ...sn )), which replaces X by
s1 s2 ...sn . If n=0 (i.e., the right-hand side of the rule is ε), then the transition ( (q, ε, X), (q, ε)).
3. For each character c ∈ Σ. the transition ((q, c, c), (q,ε)), which compares an expected
character from the stack against the next input character.
If G contains the rule X→cs2 ...sn , (where c ∈Σ and s2 through sn , are elements of V-Σ), it is not
necessary to push c onto the stack, only to pop it with a rule from step 3.
Instead, we collapse the push and the pop into a single transition. So we create a transition that can
be taken only if the next input character is c. In that case, the string s2 through sn is pushed onto the
stack.
Since terminal symbols are no longer pushed onto the stack. We no longer need the transitions
created in step3 of the original algorithm.
So, M=({p,q},Σ,V,Δ,p,{q}), where Δ contains:
1. The start-up transitions: For each rule S→cs2 ...sn the transition ((p,c,ε),(q,s2 ...sn )).
2. For each rule X→cs2 ...sn (where c∈Σ and s2 through sn , are elements of V-Σ), the
transition ((q,c,X),(q,s2 ...sn )).
cfgtoPDAnoeps(G:context-freegrammar)=
1. Convert G to Greibach normal form, producing G'.
2. From G' build the PDA M described above.
TURING MACHINE
The Turing machine provides an ideal theoretical model of a computer. Turing machines are useful
in several ways:
• Turing machines are also used for determining the undecidability of certain languages and
• As an automaton, the Turing machine is the most general model, It accepts type-0
languages.
• It can also be used for computing functions. It turns out to be a mathematical model of
partial recursive functions.
• Measuring the space and time complexity of problems.
Turing assumed that while computing, a person writes symbols on a one-dimensional paper (instead
of a two dimensional paper as is usually done) which can be viewed as a tape divided into cells. In
Turing machine one scans the cells one at a time and usually performs one of the three simple
operations, namely:
(i) Writing a new symbol in the cell being currently scanned,
(ii) Moving to the cell left of the present cell, and
(iii) Moving to the cell right of the present cell.
REPRESENTATION OF TURINGMACHINES
We can describe a Turing machine employing
(i) Instantaneous descriptions using move-relations.
(ii) Transition table, and
(iii) Transition diagram (Transition graph).
REPRESENTATION BY INSTANTANEOUS DESCRIPTIONS
Definition: An ID of a Turing machine M is a string 𝛼𝛽𝛾, where 𝛽 is the present state of M, the
entire input string is split as 𝛼𝛾, the first symbol of 𝛾 is the current symbol a under the R/W head and
𝛾 has all the subsequent symbols of the input string, and the string 𝛼 is the substring of the input
string formed by all the symbols to the left of a.
EXAMPLE: A snapshot of Turing machine is shown in below Fig. Obtain the instantaneous
description.
We give the definition of 𝛅 in the form of a table called the transition table If (q, a)=(𝛾,𝛼,𝛽). We
write 𝛼𝛽𝛾 under the 𝛼-column and in the q-row. So if we get 𝛼𝛽𝛾 in the table, it means that 𝛼 is
written in the current cell, 𝛽 gives the movement of the head (L or R) and 𝛾 denotes the new state
into which the Turing machine enters.
EXAMPLE:
Consider, for example, a Turing machine with five states q1 ,...,q5 where q1 is the initial state and q5 is
the (only) final state. The tape symbols are 0,1and b. The transition table given below describes 𝛅:
As (q5,1) is not defined, M halts; so the input string 011 is not accepted
The Turing machine M has the initial state q 0. The initial ID for M is q0 0m10n . On seeing 0,the
following moves take place
q1 is the initial state of COPY. The following moves take place for M1 :
After exhausting 0s, q1 encounters 1. M1 moves to state q4 . All 2's are converted back to 0's
and M1 halts in q5 . The TM M picks up the computation by starting from q5 The q0 and q6 are the
states of M. Additional states are created to check whether reach 0 in 0m gives rise to 0m at the end of
the rightmost 1 in the input string. Once this is over, M erases 10n 1 and finds 0mn in the input tape.
M can be defined by M=({q0 ,q1 ,....q12 }{0,1},{0,,2,b},𝛅,q0 ,b,{q12 }) where 𝛅 is defined by table given
below:
ADDITIONAL PROBLEMS
1. Design a Turing machine to obtain complement of a binary number.
IDEA OF CONSTRUCTION:
1) If symbol is 0 change it to 1, move read write head to RIGHT
2) If symbol is 1 change it to 0, move read write head to RIGHT
3) Symbol is b (blank) don’t change, move read write head to RIGHT, and HALT.
The construction is made by defining moves in the following manner:
(a) ql is the initial state. On scanning 1, no change in state and write 0 and move head to RIGHT.
(c) If M is in state q1 and scans blank, it enters q2 and writes b move to right.
(d) q2 is the only accepting state.
Symbolically, M=({ql,q2 },{1,0,b},{1,0,b},𝛅,ql,b,{q2 }) Where 𝛅 is defined by:
2. Design a TM that converts binary number into its 2’s complement representation.
IDEA OF CONSTRUCTION:
Read input from left to right until right end blank is scanned.
Begin scan from right to left keep symbols as it is until 1 found on input file.
If 1 found on input file, move head to left one cell without changing input.
Now until left end blank is scanned, change all 1’s to 0 and 0’s to 1.
We require the following moves:
(a) Let q1 be initial state, until blank is scanned, move head to RIGHT without changing
anything. On scanning blank, move head to RIGHT change state to q2 without changing the
content of input.
(b) If q2 is the state, until 1 is scanned, move head to LEFT without changing anything. On
reading 1, change state to q3 , move head to LEFT without changing input.
(c) If q3 is the state, until blank is scanned, move head to LEFT, if symbol is 0 change to 1,
otherwise if symbol is 1 change to 0.On finding blank change state to q4 , move head to LEFT
without Changing input.
(d) q4 is the only accepting state.
We construct a TM M as follows:
M = (Q, Σ,, 𝛅, q0 ,b, F)
Q = {q1 ,q2 ,q3 ,q4 }
F = {q4 }
Σ = {0,1}
𝚪= {0,1,b}
PRACTICE PROBLEMS
1. Design a Turing machine to replace all a’s with X and all b’s with Y.
2. Design a Turing machine to accept an bm n>m.
3. Design a Turing machine to accept an bn n<m.
4. Design a Turing machine to accept (0+1)*00(0+1)* .
5. Design a Turing machine to increment a given input.
6. Design a Turing machine to decrement a given input.
7. Design a Turing machine to subtract two unary numbers.
8. Design a Turing machine to multiply two unary numbers.
9. Design a Turing machine to accept a string 0’s followed by a 1.
10. Design a Turing machine to verify if the given binary number is an even number or not.
11. Design a Turing machine to shift the given input by one cell to left.
12. Design a Turing machine to shift the given input to the right by one cell .
13. Design a Turing machine to rotate a given input by one cell.
14. Design a Turing machine to erase the tape.
15. Design a Turing machine to accept an bn cn .
16. Design a Turing machine to accept any string of a’s & b’s with equal number of a’s & b’s.
17. Design a Turing machine to accept an b2n.
18. Design a Turing machine to accept an bk cm : where n=m+k.
19. Design a Turing machine to accept an bk cm : where m=n+k.
20. Design a Turing machine to accept an bk cm : where k=m+n.
From: Dr shreedhara K S, Professor in CSE, UBDT college of engineering, Davanagere
Church-Turing thesis-1936
• Any algorithmic procedure that can be carried out by a human or a computer, can also be
carried out by a Turing machine.
• Now it is universally accepted by computer scientists that TM is a Mathematical model of an
algorithm.
• TM has an algorithm and an algorithm has a TM. If there is an algorithm problem is
decidable, TM solves that problem
• The statement of the thesis –
“ Every function which would naturally be regarded as computable can be
computed by a Turing machine”
Implies
• Any mechanical computation can be performed by a TM
• For every computable problem there is a TM
• If there is no TM that decides P there is no algorithm that can solve problem P.
• In our general life, we have several problems and some of these have solutions, but some
have not, we simply say a problem is decidable if there is a solution otherwise undecidable.
example:
• Does Sun rises in the East? YES
• Will tomorrow be a rainy day ? ( YES/NO ? )
Decidable and Undecidable Languages
• A problem is said to be decidable if its language is recursive OR it has solution.
Example:
Decidable :
-Does FSM accept regular language?
- is the power of NFA and DFA same
Undecidable:
- For a given CFG is L(G) ambiguous?
L is Turing decidable (or just decidable) if there exists a Turing machine M that accepts all strings in L
and rejects all strings not in L. Note that by rejection means that the machine halts after a finite
number of steps and announces that the input string is not acceptabl e.
1. (Recursive)
TMs that always halt, no matter accepting or
non-accepting DECIDABLE PROBLEMS
2. (Recursively enumerable)
TMs that are guaranteed to halt only on acceptance.
If non-accepting, it may or may not halt (i.e., could loop forever).
• Undecidable problems are those that are not recursive
Recursive languages
A Language L over the alphabet∑ is called recursive if there is a TM M that accepts every word in L
and rejects every word in L’
Accept (M)=L
Reject(M)=L’
loop(M)= ø
Example: b(a+b)*
3. Prove that
PCP is a combinatorial problem formulated by Emil Post in 1946. This problem has many applications
in the field theory of formal languages. A correspondence system P is a finite set of ordered pairs of
non empty strings over some alphabet. Let A w1 , w2 , , wn B v1 , v2 , , vn
i, j , , k
wi w j wk vi v j vk
Index wj Vi
1 100 001
2 11 111
3 111 11
Let W = w2w1w3 =v2v1v3 = 11100111 we have got a solution. But we may not get solution always
for various other combinations and strings of different length. Hence PCP is undecidable.
A w1 , w2 , , wn B v1 , v2 , , vn
1, i, j,, k
w1 wi w j wk v1vi v j vk
If the index start with 1 and then any other sequence then it is called MPCP
Time complexity
Space complexity
◦ Best case
◦ Worst case
Average case
Asymptotic analysis
The big-Oh notation is used widely to characterize running times and space bounds
The big-Oh notation allows us to ignore constant factors and lower order terms and focus on
the main components of a function which affect its growth
Given functions f(n) and g(n), we say that f(n) is O(g(n)) if there are positive constants
c and n0 such that
f(n) cg(n) for n n0
Example: 2n + 10 is O(n)
◦ 2n + 10 cn
◦ (c 2) n 10
◦ n 10/(c 2)
It is true for c = 3 and n0 = 10
7n-2 is O(n)
need c > 0 and n0 1 such that 7n-2 c•n for n n0
this is true for c = 7 and n 0 = 1
f(n)= O(g(n)) iff there exist positive constants c and n0 such that f(n) ≤ cg(n) for all n ≥ n0
O-notation to give an upper bound on a function
The big-Oh notation gives an upper bound on the growth rate of a function
The statement “f(n) is O(g(n))” means that the growth rate of f(n) is no more than the
growth rate of g(n)
We can use the big-Oh notation to rank functions according to their growth rate
f n a0 a1n a2 n 2 ... ad n d
If is f(n) a polynomial of degree d, then f(n) is O(nd), i.e.,
1. Drop lower-order terms
2. Drop constant factors
Use the smallest possible class of functions
1. Say “2n is O(n)” instead of “2n is O(n2 )”
Use the simplest expression of the class
Say “3n + 5 is O(n)” instead of “3n + 5 is O(3n)”
Following are the terms usually used in algorithm analysis:
1. Constant 1
2. Logarithmic log n
3. Linear n
4. N-Log-N n log n
5. Quadratic n2
6. Cubic n3
7. Exponential 2n
Class P Problems:
P stands for deterministic polynomial time. A deterministic machine at each time executes an
instruction. Depending on instruction, it then goes to next state which is unique. Hence time
complexity of DTM is the maximum number of moves made by M is processing any input string of
length n, taken over all input of length n.
The class P consists of those problems that are solvable in polynomial time.
More specifically, they are problems that can be solved in time O(n k) for some constant k,
where n is the size of the input to the problem
The key is that n is the size of input
Def: A language L is said to be in class P if there exists a DTM M such that M is of time complexity
P(n) for some polynomial P and M accepts L.
Class NP Problems
Def: A language L is in class NP if there is a nondeterministic TM such that M is of time complexity
P(n) for some polynomial P and M accepts L.
NP is not the same as non-polynomial complexity/running time. NP does not stand for not
polynomial.
NP = Non-Deterministic polynomial time
NP means verifiable in polynomial time
Verifiable?
◦ If we are somehow given a ‘certificate’ of a solution we can verify the legitimacy in
polynomial time
Quantum Computers
Computers are physical objects, and computations are physical processes. What
computers can or cannot compute is determined by the law of physics alone, and not by
pure mathematics. Computation with coherent atomic-scale dynamics.
The behavior of a quantum computer is governed by the laws of quantum mechanics.
In 1982 Richard Feynmann, a Nobel laurite in physics suggested to build computer based on
quantum mechanics.
Quantum mechanics arose in the early 1920s, when classical physics could not explain
everything.
QM will provide tools to fill up the gulf between the small and the relatively complex
systems in physics.
Bit ( 0 or 1 ) is the fundamental concept of classical computation and information. Classical
computer built from electronic circuits containing wires and gates.
Quantum bit and quantum circuits which are analogous to bits and circuits. Two possible
states of a qubit (Dirac)are 0 1
Quantum bit is qubit described mathematically (where (alpha) is complex number)
0 0 1 1
Qubit can be in infinite number of state other than dirac |0> or |1>
i 1
1 0 i 1 1 0 0 1 1 i 1
1 i
0 0 1 1 0 0 1 1 0
2 2 2 2 2 2 2 2
Any linear operation that takes states 0 1 satisfying and maps them to be UNITARY
0 1
i.e. 0 1 1
2 2
0 0 1 1 0 0 1 1
0 1 01 1 0 10 1 1 11
0 0 00
And it is describes the state as 0 0 00 0 1 01 10 10 11 11
Quantum computer is a system built from quantum circuits, containing wires and
elementary quantum gates, to carry out manipulation of quantum information.
1. Multiple tapes: It consists of finite control with k tape heads and k tapes each
tape is infinite in both directions. On a single move depending on the state of the
finite control and symbol scanned by each of the tape head the machine can
change state Or print new symbol on each of cell scanned etc..
2. With One tape but M ultiple heads: a K head TM has fixed k number of heads
and move of TM depends on the state and the symbol scanned by each head. (
head can move left, right or stationary).
3. Multidimensional TM: It has finite control but the tape consists of a K-
dime nsional array of cells infinite in all 2 k directions. Depending on the state
and symbol scanned , the device changes the state, prints a ne w symbol, and
moves its tape head in one of the 2 k directions, either positively of negatively
along one of the k axes.
4. Non deterministic TM: In the TM for a given state and tape symbol scanned by
the tape head, the machine has a finite numbe r of choices for the next move.
Each choice consists of new state, a tape symbol to print and direction of head
motion.
< a b a b a >
Left EM Right EM
Two extra symbols < and > are used left end marker and right
end marker.
Input lies between these markers