Finite Automata

Reading: Chapter 2

1
Finite Automaton (FA)
 Informally, a state diagram that comprehensively
captures all possible states and transitions that a
machine can take while responding to a stream or
sequence of input symbols
 Recognizer for “Regular Languages”

 Deterministic Finite Automata (DFA)

 The machine can exist in only one state at any given time
 Non-deterministic Finite Automata (NFA)
 The machine can exist in multiple states at the same time

2
Deterministic Finite Automata
- Definition
 A Deterministic Finite Automaton (DFA)
consists of:
 Q ==> a finite set of states
 ∑ ==> a finite set of input symbols (alphabet)
 q0 ==> a start state
 F ==> set of accepting states
 δ ==> a transition function, which is a mapping
between Q x ∑ ==> Q
 A DFA is defined by the 5-tuple:
 {Q, ∑ , q0,F, δ }

3
What does a DFA do on
reading an input string?
 Input: a word w in ∑*
 Question: Is w acceptable by the DFA?
 Steps:
 Start at the “start state” q0
 For every input symbol in the sequence w do
 Compute the next state from the current state, given the
current input symbol in w and the transition function
 If after all symbols in w are consumed, the current
state is one of the accepting states (F) then accept
w;
 Otherwise, reject w.
4
Regular Languages
 Let L(A) be a language recognized by a
DFA A.
 Then L(A) is called a “Regular Language”.

 Locate regular languages in the

Chomsky Hierarchy

5
The Chomsky Hierachy
• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) Context- Recursively-
free
sensitive enumerable
(PDA)
(LBA) (TM)

6
The Transition Function
 Takes two arguments: a state and an
input symbol.
 δ(q, a) = the state that the DFA goes to
when it is in state q and input a is
received.

7
Graph Representation of
DFA’s
 Nodes = states.
 Arcs represent transition function.
 Arc from state p to state q labeled by all
those input symbols that have transitions
from p to q.
 Arrow labeled “Start” to the start state.
 Final states indicated by double circles.

8
 Example: Graph of a DFA
Accepts all strings without two consecutive 1’s.

0 0,1
1 1
A B C

Start 0
Previous Previous Consecutive
string OK, String OK, 1’s have
does not ends in a been seen.
end in 1. single 1.
9
 Alternative Representation:
Transition Table
Final states
starred Columns =
0 1 input symbols
* A A B
Arrow for * B A C
start state C C C

Rows = states

10
Extended Transition Function
 We describe the effect of a string of
inputs on a DFA by extending δ to a
state and a string.
 Induction on length of string.
 Basis: δ(q, ε) = q
 Induction: δ(q,wa) = δ(δ(q,w),a)
 w is a string; a is an input symbol.

11
 Extended δ: Intuition
 Convention:
 … w, x, y, x are strings.
 a, b, c,… are single symbols.
 Extended δ is computed for state q and
inputs a1a2…an by following a path in
the transition graph, starting at q and
selecting the arcs with labels a1, a2,…,an
in turn.
12
Example: Extended Delta
0 1
A A B
B A C
C C C

δ(B,011) = δ(δ(B,01),1) = δ(δ(δ(B,0),1),1) =

δ(δ(A,1),1) = δ(B,1) = C

13
 Delta-hat
 Some people denote the extended δ with
a “hat” to distinguish it from δ itself.
 Not needed, because both agree when
the string is a single symbol.
˄ ˄
 δ(q, a) = δ(δ(q, ε), a) = δ(q, a)

Extended deltas

14
Example #1
 Build a DFA for the following language:
 L = {w | w is a binary string that contains 01 as a substring}
 Steps for building a DFA to recognize L:
 ∑ = {0,1}
 Decide on the states: Q
 Designate start state and final state(s)
 δ: Decide on the transitions:
 “Final” states == same as “accepting states”
 Other states == same as “non-accepting states”

15
 Regular expression: (0+1)*01(0+1)*

DFA for strings containing 01

• What makes this DFA deterministic? • Q = {q0,q1,q2}

1 0 0,1 • ∑ = {0,1}
• start state = q0
start 0 1
q0 q1 q2 • F = {q2}
Accepting • Transition table
state symbols
0 1
q0 q1 q0
states

• What if the language allows q1 q1 q2

empty strings? *q2 q2 q2

16
Example #2
Clamping Logic:
 A clamping circuit waits for a ”1” input, and turns on forever.
However, to avoid clamping on spurious noise, we’ll design
a DFA that waits for two consecutive 1s in a row before
clamping on.
 Build a DFA for the following language:
L = { w | w is a bit string which contains the
substring 11}
 State Design:
 q0 : start state (initially off), also means the most recent input
was not a 1
 q1: has never seen 11 but the most recent input was a 1
 q2: has seen 11 at least once
17
Example #3
 Build a DFA for the following language:
L = { w | w is a binary string that has even
number of 1s and even number of 0s}
 ?

18
Extension of transitions (δ) to
Paths (δ)
 δ (q,w) = destination state from state q
on input string w

 δ (q,wa) = δ (δ(q,w), a)

 Work out example #3 using the input

sequence w=10010, a=1:


δ (q0,wa) = ?
19
Language of a DFA
A DFA A accepts string w if there is a path
from q0 to an accepting (or final) state
that is labeled by w

 i.e., L(A) = { w | δ(q0,w)  F }

 I.e., L(A) = all strings that lead to an

accepting state from q0
20
Regular Languages
 A language L is regular if it is the
language accepted by some DFA.
 Note: the DFA must accept only the strings
in L, no others.
 Some languages are not regular.
 Intuitively, regular languages “cannot
count” to arbitrarily high integers.

21
Example: A Nonregular Language
L1 = {0n1n | n ≥ 1}
 Note: ai is conventional for i a’s.
 Thus, 04 = 0000, e.g.
 Read: “The set of strings consisting of n
0’s followed by n 1’s, such that n is at
least 1.
 Thus, L1 = {01, 0011, 000111,…}

22
 Another Example
L2 = {w | w in {(, )}* and w is balanced }
 Note: alphabet consists of the parenthesis
symbols ’(’ and ’)’.
 Balanced parens are those that can appear
in an arithmetic expression.
 E.g.: (), ()(), (()), (()()),…

23
But Many Languages are
Regular
• Regular Languages can be described in
many ways, e.g., regular expressions.
• They appear in many contexts and have
many useful properties.
• Example: the strings that represent
floating point numbers in your favorite
language is a regular language.

24
Example: A Regular Language
L3 = { w | w in {0,1}* and w, viewed as a
binary integer is divisible by 23}
 The DFA:

 23 states, named 0, 1,…,22.

 Correspond to the 23 remainders of an
integer divided by 23.
 Start and only final state is 0.

25
Transitions of the DFA for L3
 If string w represents integer i, then
assume δ(0, w) = i%23.
 Then w0 represents integer 2i, so we
want δ(i%23, 0) = (2i)%23.
 Similarly: w1 represents 2i+1, so we
want δ(i%23, 1) = (2i+1)%23.
 Example: δ(15,0) = 30%23 = 7; δ(11,1)
= 23%23 = 0. Key idea: design a DFA
by figuring out what
each state needs to
remember about the past.
26
 Another Example
L4 = { w | w in {0,1}* and w, viewed as the
reverse of a binary integer is divisible by
23}
• Example: 01110100 is in L , because its
4
reverse, 00101110 is 46 in binary.
• Hard to construct the DFA.

• But theorem says the reverse of a

regular language is also regular.

27
Non-deterministic Finite
Automata (NFA)
 A Non-deterministic Finite Automaton
(NFA)
 is of course “non-deterministic”
 Implying that the machine can exist in more
than one state at the same time
 Transitions could be non-deterministic

1 qj
qi …
• Each transition function therefore
1 maps to a set of states
qk
28
Non-deterministic Finite
Automata (NFA)
 A Non-deterministic Finite Automaton (NFA)
consists of:
 Q ==> a finite set of states
 ∑ ==> a finite set of input symbols (alphabet)
 q0 ==> a start state
 F ==> set of accepting states
 δ ==> a transition function, which is a mapping
between Q x ∑ ==> subset of Q
 An NFA is also defined by the 5-tuple:
 {Q, ∑ , q0,F, δ }

29
How to use an NFA?
 Input: a word w in ∑*
 Question: Is w acceptable by the NFA?
 Steps:
 Start at the “start state” q0
 For every input symbol in the sequence w do
 Determine all possible next states from all current states, given
the current input symbol in w and the transition function
 If after all symbols in w are consumed and if at least one of
the current states is a final state then accept w;
 Otherwise, reject w.

30
 Regular expression: (0+1)*01(0+1)*

NFA for strings containing 01

Why is this non-deterministic?

• Q = {q0,q1,q2}

0,1 0,1 •  = {0,1}

• start state = q0
start 0 1
q0 q1 q2 • F = {q2}
Final • Transition table
state symbols
0 1
What will happen if at state q1 q0 {q0,q1} {q0}
states

an input of 0 is received? q1 Φ {q2}

*q2 {q2} {q2}

31
Note: Omitting to explicitly show error states is just a matter of design convenience
(one that is generally followed for NFAs), and
i.e., this feature should not be confused with the notion of non-determinism.

What is an “error state”?

 A DFA for recognizing the key word
“while” w h i l e
q0 q1 q2 q3 q4 q5

Any other input symbol

qerr
Any symbol

 An NFA for the same purpose:

w h i l e
q0 q1 q2 q3 q4 q5

Transitions into a dead state are implicit 32

Example #2
 Build an NFA for the following language:
L = { w | w ends in 01}
 ?
 Other examples
 Keyword recognizer (e.g., if, then, else,
while, for, include, etc.)
 Strings where the first symbol is present
somewhere later on at least once

33
Extension of δ to NFA Paths
 Basis: δ (q,) = {q}

 Induction:
 Let δ (q0,w) = {p1,p2…,pk}
 δ (pi,a) = Si for i=1,2...,k

 Then, δ (q0,wa) = S1 U S2 U … U Sk

34
Language of an NFA
 An NFA accepts w if there exists at least
one path from the start state to an
accepting (or final) state that is labeled
by w
 L(N) = { w | δ(q0,w) ∩ F ≠ Φ }

35
Advantages & Caveats for NFA
 Great for modeling regular expressions
 String processing - e.g., grep, lexical analyzer

 Could a non-deterministic state machine be

implemented in practice?
 Probabilistic models could be viewed as extensions of non-
deterministic state machines
(e.g., toss of a coin, a roll of dice)
 They are not the same though

 A parallel computer could exist in multiple “states” at the same time

36
Technologies for NFAs
 Micron’s Automata Processor (introduced in 2013)
 2D array of MISD (multiple instruction single data)
fabric w/ thousands to millions of processing
elements.
 1 input symbol = fed to all states (i.e., cores)
 Non-determinism using circuits
 http://www.micronautomata.com/

37
But, DFAs and NFAs are equivalent in their power to capture langauges !!

Differences: DFA vs. NFA

 NFA
DFA
1. All transitions
Some transitions
are could
deterministic
be non-deterministic
 Each
A transition
transition
could
leads
leadtotoexactly
a subset
one
ofstate
states
2. Not each
For all symbol
state,transitions
transition on
need
all to
possible
be defined
symbols
explicitly
(alphabet)
(if undefined
should
be defined
will go to an error state – this is just a design convenience, not to be
3. confused with if“non-determinism”)
Accepts input the last state visited is in F
3.
4. Accepts
Sometimes input if onetoofconstruct
harder the last states is inofFthe number of states
because
4.
5. Generally easier than a DFA
Practical implementation to construct
is feasible
5. Practical implementations limited but emerging (e.g., Micron
automata processor)

38
 Equivalence of DFA & NFA
 Theorem:
Should be  A language L is accepted by a DFA if and only if
true for it is accepted by an NFA.
any L
 Proof:
1. If part:
 Prove by showing every NFA can be converted to an
equivalent DFA (in the next few slides…)

2. Only-if part is trivial:

 Every DFA is a special case of an NFA where each
state has exactly one transition for every input symbol.
Therefore, if L is accepted by a DFA, it is accepted by
a corresponding NFA. 39
Proof for the if-part
 If-part: A language L is accepted by a DFA if it
is accepted by an NFA
 rephrasing…
 Given any NFA N, we can construct a DFA D
such that L(N)=L(D)
 How to convert an NFA into a DFA?
 Observation: In an NFA, each transition maps to a
subset of states
 Idea: Represent:
each “subset of NFA_states”  a single “DFA_state”

Subset construction
40
NFA to DFA by subset construction
 Let N = {QN,∑,δN,q0,FN}
 Goal: Build D={QD,∑,δD,{q0},FD} s.t.
L(D)=L(N)
 Construction:
1. QD= all subsets of QN (i.e., power set)
2. FD=set of subsets S of QN s.t. S∩FN≠Φ
3. δD: for each subset S of QN and for each input symbol
a in ∑:
 δD(S,a) = U
p inδs (p,a)
N

41
 Idea: To avoid enumerating all of
power set, do
“lazy creation of states”

NFA to DFA construction: Example

 L = {w | w ends in 01} 1 0
NFA: DFA: 0 1
0,1 [q0] [q0,q1] [q0,q2]
0
0 1 1
q0 q1 q2

δD 0 1 δD 0 1
δN 0 1
Ø Ø Ø [q0] [q0,q1] [q0]
q0 {q0,q1} {q0}
[q0] {q0,q1} {q0} [q0,q1] [q0,q1] [q0,q2]
q1 Ø {q2}
[q1] Ø {q2} *[q0,q2] [q0,q1] [q0]
*q2 Ø Ø *[q2] Ø Ø
[q0,q1] {q0,q1} {q0,q2}
*[q0,q2] {q0,q1} {q0} 0. Enumerate all possible subsets
*[q1,q2] Ø {q2} 1. Determine transitions
*[q0,q1,q2] {q0,q1} {q0,q2}
2. Retain only those states
reachable from {q0}
42
 NFA to DFA: Repeating the example
using LAZY CREATION
 L = {w | w ends in 01} 1 0
NFA: DFA: 0 1
0,1 [q0] [q0,q1] [q0,q2]
0
0 1 1
q0 q1 q2

δN 0 1 δD 0 1
q0 {q0,q1} {q0} [q0] [q0,q1] [q0]
q1 Ø {q2} [q0,q1] [q0,q1] [q0,q2]
*q2 Ø Ø *[q0,q2] [q0,q1] [q0]

Main Idea:
Introduce states as you go
(on a need basis)
43
Correctness of subset construction
Theorem: If D is the DFA constructed from
NFA N by subset construction, then
L(D)=L(N)
 Proof:

 Show that δD({q0},w) ≡ δN(q0,w} , for all w

 Using induction on w’s length:
 Let w = xa
 δD({q0},xa) ≡ δD( δN(q0,x}, a ) ≡ δN(q0,w}

44
A bad case where
#states(DFA)>>#states(NFA)
 L = {w | w is a binary string s.t., the kth symbol
from its end is a 1}

 NFA has k+1 states

 But an equivalent DFA needs to have at least 2k

states

(Pigeon hole principle)

 m holes and >m pigeons
 => at least one hole has to contain two or more pigeons
45
Applications
 Text indexing
 inverted indexing
 For each unique word in the database, store all
locations that contain it using an NFA or a DFA
 Find pattern P in text T
 Example: Google querying
 Extensions of this idea:
 PATRICIA tree, suffix tree

46
A few subtle properties of
DFAs and NFAs
 The machine never really terminates.
 It is always waiting for the next input symbol or making

transitions.
 The machine decides when to consume the next symbol from
the input and when to ignore it.
 (but the machine can never skip a symbol)

 => A transition can happen even without really consuming an

input symbol (think of consuming  as a free token) – if this
happens, then it becomes an -NFA (see next few slides).
 A single transition cannot consume more than one (non-)
symbol.

47
Summary
 DFA
 Definition

 Transition diagrams & tables

 Regular language
 NFA
 Definition

 Transition diagrams & tables

 DFA vs. NFA

 NFA to DFA conversion using subset construction
 Equivalency of DFA & NFA
 Removal of redundant states and including dead states
 Pigeon hole principles
 Text searching applications

48