Turing Machines (TM) : - Generalize The Class of Cfls
Turing Machines (TM) : - Generalize The Class of Cfls
Turing Machines (TM) : - Generalize The Class of Cfls
Recursive Languages
Context-Free Languages
Regular Languages
1
• Another Part of the Hierarchy:
Recursive Languages
Context-Sensitive Languages
Context-Free Languages - ε
Regular Languages - ε
2
• Recursively enumerable languages are also known as type 0 languages.
3
• TMs model the computing capability of a general purpose computer, which
informally can be described as:
– Effective procedure
• Finitely describable
• Well defined, discrete, “mechanical” steps
• Always terminates
– Computable function
• A function computable by an effective procedure
Finite
Control
• Two-way, infinite tape, broken into cells, each containing one symbol.
• Two-way, read/write tape head.
• An input string is placed on the tape, padded to the left and right infinitely with
blanks, read/write head is positioned at the left end of input string.
• Finite control, i.e., a program, containing the position of the read head, current
symbol being scanned, and the current state.
• In one move, depending on the current state and the current symbol being
scanned, the TM 1) changes state, 2) prints a symbol over the cell being
scanned, and 3) moves its’ tape head one cell left or right.
• Many modifications possible, but Church-Turing declares equivalence of all. 5
Formal Definition of a DTM
• A DTM is a seven-tuple:
M = (Q, Σ, Γ, δ, q0, B, F)
Intuitively, δ(q,s) specifies the next state, symbol to be written, and the direction of tape
head movement by M after reading symbol s while in state q.
6
• Example #1: {w | w is in {0,1}* and w ends with a 0}
0
00
10
10110
Not ε
0 1 B
->q0 (q0, 0, R) (q0, 1, R) (q1, B, L)
q1 (q2, 0, R) - -
q 2* - - -
– q0 is the start state and the “scan right” state, until hits B
– q1 is the verify 0 state
– q2 is the final state 7
• Example #2: {0n1n | n ≥ 1}
0 1 X Y B
->q0 (q1, X, R) - - (q3, Y, R)0’s finished -
q1 (q1, 0, R)ignore1 (q2, Y, L) - (q1, Y, R) ignore2 - (more 0’s)
q2 (q2, 0, L) ignore2 - (q0, X, R) (q2, Y, L) ignore1 -
q3 - - (more 1’s) - (q3, Y, R) ignore (q4, B, R)
q 4* - - - - -
0 1 X Y B
q0 (q1, X, R) - - (q3, Y, R) -
q1 (q1, 0, R) (q2, Y, L) - (q1, Y, R) -
q2 (q2, 0, L) - (q0, X, R) (q2, Y, L) -
q3 - - - (q3, Y, R) (q4, B, R)
q4 - - - - -
Logic: cross 0’s with X’s, scan right to look for corresponding 1, on finding it cross it with Y, and
scan left to find next leftmost 0, keep iterating until no more 0’s, then scan right looking for B.
– The TM matches up 0’s and 1’s
– q1 is the “scan right” state, looking for 1
– q2 is the “scan left” state, looking for X
– q3 is “scan right”, looking for B
– q4 is the final state
0 1 B
q0 (q1, B, R) (q4, B, R)
q1 (q1, 0, R) (q1, 1, R) (q2, B, L)
q2 - (q3, B, L) -
q3 (q3, 0, L) (q3, 1, L) (q0, B, R)
q 4* - - -
Logic: Keep deleting 0 and corresponding 1 from extreme ends, until none left.
– q0 deletes a leftmost 0 and let q1 scan through end of string, q0 accepts on epsilon
– q1 scans over the string and makes q2 expecting 1 on the left
– q2 deletes 1 and let q3 “scan left” looking for the start of current string
– q3 lets q0 start the next iteration
– q4 is the final state
Any bug?
Try on:
000111 00
11 001
011
11
• And his example of a correct TM for the language that goes on infinite loop outside language: {0n1n | n ≥ 0}
0 1 B
q0 (q1, B, R) (q3, 1, L) (q4, B, R)
q1 (q1, 0, R) (q1, 1, R) (q2, B, L)
q2 - (q3, B, L) -
q3 (q3, 0, L) (q3, 1, L) (q0, B, R)
q 4* - - -
Logic: This machine still works correctly for all strings in the language, but
start a string with 1 (not in the language),
and it loops on B1 for ever.
12
• Exercises: Construct a DTM for each of the following.
13
Formal Definitions for DTMs
• Let M = (Q, Σ, Г, δ, q0, B, F) be a TM.
x1x2…xi-1qxixi+1…xn
x1x2…xi-1qxixi+1…xn |— x1x2…xi-1ypxi+1…xn
x1x2…xnq |— x1x2…xnyp
15
• Definition: Let M = (Q, Σ, Г, δ, q0, B, F) be a TM, and let w be a string in Σ*. Then w is
accepted by M iff
{w | w is in Σ* and w is accepted by M}
• Notes:
– In contrast to FA and PDAs, if a TM simply passes through a final state then the
string is accepted.
– Given the above definition, no final state of a TM need to have any transitions.
Henceforth, this is our assumption.
– If x is NOT in L(M) then M may enter an infinite loop, or halt in a non-final
state.
– Some TMs halt on ALL inputs, while others may not. In either case the language
defined by TM is still well defined.
16
• Definition: Let L be a language. Then L is recursively enumerable if there exists a TM
M such that L = L(M).
Notes:
– The set of all recursive languages is a subset of the set of all recursively enumerable
languages
Recursive Languages
Context-Sensitive Languages
Context-Free Languages - ε
Regular Languages - ε
18
• Observation: Let L be an r.e. language. Then there is an infinite list M 0, M1, … of TMs
such that L = L(Mi).
• Question: Let L be a recursive language, and M0, M1, … a list of all TMs such that L =
L(Mi), and choose any i>=0. Does Mi always halt?
• Answer: Maybe, maybe not, but at least one in the list does.
• Question: Let L be a recursive enumerable language, and M0, M1, … a list of all TMs
such that L = L(Mi), and choose any i>=0. Does Mi always halt?
• Answer: Maybe, maybe not. Depending on L, none might halt or some may halt.
• Question: Let L be a r.e. language that is not recursive (L is in r.e. – r), and M 0, M1, … a
list of all TMs such that L = L(Mi), and choose any i>=0. Does Mi always halt?
L is Recursive:
at least one TM halts on L and on ∑*-L, others may or may not
L is not R.E:
no TM exists
20
• Let M be a TM.
21
• Let M be a TM that halts on all inputs:
22
• Let M be a TM.
– Question: Suppose, we know L(M) is recursive. Does that mean M always halts?
– Answer: Not necessarily. However, some TM M’ must exist such that L(M’) =
L(M) and M’ always halts.
23
• Let M be a TM, and suppose that M loops forever on some string x.
24
Modifications of the Basic TM Model
25
Closure Properties for Recursive and
Recursively Enumerable Languages
26
• TM Block Diagrams:
– If L is a recursive language, then a TM M that accepts L and always halts can be
pictorially represented by a “chip” or “box” that has one input and two outputs.
yes
w M
no
yes
w M
– Conceivably, M could be provided with an output for “no,” but this output cannot
be counted on. Consequently, we simply ignore it. 27
• Theorem 1: The recursive languages are closed with respect to complementation, i.e., if
L is a recursive language, then so is L * L
M’
yes
yes
w M no
no
• Note That:
– M’ accepts iff M does not
– M’ always halts since M always halts
• Question: How is the construction achieved? Do we simply complement the final states
in the TM? No! A string in L could end up in the complement of L.
– Suppose q5 is an accepting state in M, but q0 is not.
– If we simply complemented the final and non-final states, then q 0 would be an accepting state in
M’ but q5 would not.
– Since q0 is an accepting state, by definition all strings are accepted by M’ 28
• Theorem 2: The recursive languages are closed with respect to union, i.e., if
L1 and L2 are recursive languages, then so is L3 L1 L2
• Proof: Let M1 and M2 be TMs such that L1 = L(M1) and L2 = L(M2) and M1
and M2 always halts. Construct TM M’ as follows:
M’ yes
yes start
w M1 M2 no
no
• Note That:
– L(M’) = L(M1) L(M2)
• L(M’) is a subset of L(M1) U L(M2)
• L(M1) U L(M2) is a subset of L(M’)
– M’ always halts since M1 and M2 always halt
• Proof: Let M1 and M2 be TMs such that L1 = L(M1) and L2 = L(M2). Construct M’ as
follows:
M’ yes yes
M1
w
yes
M2
• Note That:
– L(M’) = L(M1) U L(M2)
• L(M’) is a subset of L(M1) U L(M2)
• L(M1) U L(M2) is a subset of L(M’)
– M’ halts and accepts iff M1 or M2 halts and accepts
M’ yes yes
M1
no
w
yes
M2 no
no
• Answer: You could get two outputs – one “yes” and one “no.”
– At least M1 will halt and answer accept, M2 may or may not halt.
– As before, for the sake of convenience the “no” output will be ignored.
31
• Theorem 4: If L and L are both recursively enumerable then L (and therefore L ) is
recursive.
• Proof: Let M1 and M2 be TMs such that L = L(M1) and L= L(M2). Construct M’ as
follows:
M’ yes yes
M1
w
yes
M2 no
• Note That:
– L(M’) = L
• L(M’) is a subset of L
• L is a subset of L(M’)
– M’ is TM for L
– M’ always halts since either M1 or M2 halts for any given string
– M’ shows that L is recursive
It follows from this that L (and therefore its’ complement) is recursive.
So, L is also recursive (we proved it before).
32
• Corollary of Thm 4: Let L be a subset of Σ*. Then one of the following must
be true:
– In other words, it is impossible to have both L and L r.e. but not recursive
33
• In terms of the hierarchy: (possibility #1)
L L
Recursive Languages
34
• In terms of the hierarchy: (possibility #2)
L L
Recursive Languages
35
• In terms of the hierarchy: (possibility #3)
L L
Recursive Languages
36
• In terms of the hierarchy: (Impossibility #1)
L L
Recursive Languages
37
• In terms of the hierarchy: (Impossibility #2)
Recursive Languages
38
• In terms of the hierarchy: (Impossibility #3)
Recursive Languages
39
• Note: This gives/identifies three approaches to show that a language is not
recursive.
– Show that the language’s complement is not recursive, in one of the two ways:
• Show that the language’s complement is recursively enumerable but not recursive
• Show that the language’s complement is not even recursively enumerable
40
The Halting Problem - Background
• Definition: A decision problem is a problem having a yes/no answer (that one
presumably wants to solve with a computer). Typically, there is a list of parameters on
which the problem is based.
– Given a list of numbers, is that list sorted?
– Given a number x, is x even?
– Given a C program, does that C program contain any syntax errors?
– Given a TM (or C program), does that TM contain an infinite loop?
From a practical perspective, many decision problems do not seem all that interesting.
However, from a theoretical perspective they are for the following two reasons:
– Decision problems are more convenient/easier to work with when proving complexity results.
– Non-decision counter-parts can always be created & are typically at least as difficult to solve.
• Notes:
– The following terms and phrases are analogous:
• Analogy:
Input: DFA M with input alphabet Σ and string w in Σ*.
Question: Is w in L(M)?
Is this problem (regular language) decidable? Yes! DFA always accepts or rejects.
42
• Over-All Approach:
• As We Will See:
– P3 will correspond to the language Lu
– Proving P3 (un)decidable is equivalent to proving Lu (non)recursive
43
Converting the Problem to a Language
• Let M = (Q, Σ, Γ, δ, q1, B, {qn}) be a TM, where
111code111code211code311 … 11coder111
where each codei is one transitions’ encoding, and 11’s are barriers between transitions from the table
row-major. Let this encoding of M be denoted by <M>.
44
• Less Formally:
– Every state, tape symbol, and movement symbol is encoded as a sequence of 0’s:
q1 , 0
q2 , 00
q3 000
:
0 0
1 00
B 000
L 0
R 00
– Note that 1’s are not used to represent the above, since 1 is used as a special separator symbol.
– Example:
δ(q2, 1) = (q3 , 0, R)
Is encoded as:
00100100010100 45
0 1 B
q1 (q1, 0, R) (q1, 1, R) (q2, B, L)
q2 (q3, 0, R) - -
q3 - - -
1110101010100110100101001001101000100100010110010100010100111
01100001110001
111111
46
• Definition:
– Question: Is Lt recursive?
– Answer: Yes. [Check only for format, i.e. the order and number of 0’s and 1’s,
syntax checking]
– Question: Is Lt decidable:
– Answer: Yes (same question).
47
The Universal Language
• Define the language Lu as follows:
48
• Recall:
0 1 B
q1 (q1, 0, R) (q1, 1, R) (q2, B, L)
q2 (q3, 0, R) - -
q3 - - -
1110101010100110100101001001101000100100010110010100010100111
111010101010011010010100100110100010010001011001010001010011101110
111010101010011010010100100110100010010001011001010001010011100110111
01100001110001
111111
49
• Compare P3 and Lu:
(P3):
Input: Turing machine M with input alphabet Σ and one final state, and string w in Σ*.
Question: Is w in L(M)?
50
• Define another language Ld as follows:
• [Ld_bar = {self accepting TM encodings}, everything else is Ld ]
51
• Recall:
0 1 B
q1 (q1, 0, R) (q1, 1, R) (q2, B, L)
q2 (q3, 0, R) - -
q3 - - -
11101010101001101001010010011010001000100010110010100010100111
01100001110001
Change above machine to accept strings ending with 1: the encoding will not be in L d
52
• Lemma: Ld is not recursively enumerable. [No TM for Ld!!!]
Ld = L(M) (2)
Case 1) w is in Ld (4)
By definition of Ld given in (1), either w does not encode a TM, or w does encode a TM, call it M, and w is not in L(M).
But we know that w encodes a TM (3: that’s where it came from). Therefore:
But then (2) and (5) imply that w is not in Ld contradicting (4).
w is in L(M) (7)
Since both case 1) and case 2) lead to a contradiction, no TM M can exist such that Ld = L(M). Therefore Ld is not
recursively enumerable. 53
• Note:
• Proof: If Ld were recursive, then Ld would be recursive, and therefore recursively enumerable, a
contradiction.
54
• Theorem: Lu is not recursive.
Suppose that Lu = L(M’) where M’ is a TM that always halts. Construct an algorithm (i.e., a TM that
always halts) for Ld as follows:
55
L_u is recursively enumerable
(you may ignore this slide, for now)
Input the string
Decode the TM prefix, if it doesn't have one then the string is not in Lu
Otherwise, run/simulate the encoded TM on the suffix
If it terminates and accepts then the original string is in Lu.
If a given string is in Lu, then the above algorithm will correctly determine that, halt and say yes.
If the given string is not in Lu, then there are three cases:
1) the string doesn't have a TM as a prefix. In this case the above algo correctly detects this fact, and reports the
string is not in Lu.
2) the string has a TM prefix, and the TM halts and rejects on the suffix. In this case the above algo correctly
reports the string is not in Lu.
3) the string has a TM prefix, but it goes into an infinite loop on the suffix. In this case the above algo also goes
into an infinite loop, but that’s ok since the string as a whole is not in Lu anyway, and we are just trying to show
there exists a TM for only accepting strings in Lu.
From this proof note that if the prefix TM is a DFA or PDA, then our machine will also halt in the 3 rd case above,
no matter what the suffix is.
57
• Define another language Lh:
(P2):
Input: Turing machine M with input alphabet Σ and string w in Σ*.
Question: Does M halt on w?
58
• Theorem: Lh is not recursive.
Suppose that Lh = L(M’) where M’ is a TM that always halts. Construct an algorithm (i.e., a TM that
always halts) for Lu as follows:
1. If Lh is recursive, then so is Lu
2. Lu is not recursive
[Lu and Lh both are recursively enumerable: for proof see Dr. Shoaff!]
60
Examples of non-halting program:
http://cs.fit.edu/~ryan/tju/russell.c
http://cs.fit.edu/~ryan/tju/russell.scm
http://cs.fit.edu/~ryan/tju/russell.py
61
• Define another language Lq:
Or equivalently:
Note that:
Lq = {x | x is in {0, 1}*, and either x does not encode a TM, or it does encode a TM, call it M,
and there exists a string w in {0, 1}* such that M does not terminate on w}
(P0):
Input: Program P.
Question: Does P contain an infinite loop?
Using the techniques discussed, what can we prove about Lq or its’ complement?
62
• More examples of non-recursive languages:
63
Ignore
• this
Lemma: slide
L is notdrecursively enumerable: [No TM for L !!!] d
Ld = L(M) (2)
Case 1) wj is in Ld (4)
By definition of Ld given in (1), either wj does not encode a TM, or wj does encode a TM, call it M, and wj is not in
L(M). But we know that wj encodes a TM (3: that’s where it came from). Therefore:
But then (2) and (5) imply that wj is not in Ld contradicting (4).
wj is in L(M) (7)
Since both case 1) and case 2) lead to a contradiction, no TM M can exist such that Ld = L(M). Therefore Ld is not
recursively enumerable. 64
Roger’s TM for balanced parenthesis:
( ) B
findPair (findPair2, "(", R) - (final, B, R)
findPair2 (findPair2, "(", R) (removePair, ")", L) -
removePair (fetch, "(", R) (fetch, ")", R) (goBack, B, L)
fetch (retrieve, "(", R) (retreive, ")", R) (retreive, B, R)
retreive (returnOpen, "(", L) (returnClosed, ")", L) (returnBlank, B, L)
returnOpen (writeOpen, "(", L) (writeOpen, ")", L) (writeOpen, B, L)
returnClosed (writeClosed, "(", L) (writeClosed, ")", L) (writeClosed, B, L)
returnBlank (writeBlank "(", L) (writeBlank, ")", L) (writeBlank, B, L)
writeOpen (removePair, "(", R) (removePair, "(", R) -
writeClosed (removePair, ")", R) (removePair, ")", R) -
writeBlank (removePair, B, R) (removePair, B, R) -
goBack - - (backAgain, B, L)
backAgain - - (seekFront, B, L)
seekFront (seekFront, "(", L) (seekFront, ")", L) (findPair, B, R)
final* - - -
65
On 111 111 as a TM encoding
<Quote> It was ambiguous, in my opinion, based on the definition in the Hopcroft book, i.e., the
definition in the Hopcroft book was not clear/precise enought to
account this special case. I don't have the book in front of me right now, but I think this is the example
I used in class: Consider the TM that has exactly one state, but no transitions. Perfectly valid TM, and
it would give us this encoding (111111). In that case the encoded machine would accept sigma*
because the highest numbered state would be q0, the only state, and that would be the final state under
the Hopcroft encoding. Now consider the TM that has exactly two states, but no transitions. Also a
perfectly valid TM, and it would give us the same encoding. In that case the encoded machine would
not accept anything because the final state is q1 (highest numbered state), and there is no way to get to
it. I used it only as a way to raise that issue in class, i.e., the the Hopcroft definition is a bit ambiguous
in this case.
One way to resolve the ambiguity is to require the encoding to specifically specify the final state (at the
end or something). In that case, 111111 isn't even a valid TM, since it doesn't specify the final state.
Another related question is, does a TM even have to have any states at all to be a valid TM? The
encoding would have to be able to isolate that as a unique string also. <End Quote>
Phil Bernhard
66