CSCI 3313-10: Foundation of Computing

1 Overview

Foundation of Computing

- Theory of Computing

• Automata theory

• Computability
- solvable vs. unsolvable problems

• Complexity
- computationally easy vs. hard problems

• Formal language theory

Chomsky Hierarchy

• Type-3: Regular languages (RL); Finite state automata

• Type-2: Context-free languages (CFL); Pushdown automata

• Type-1: Context-sensitive languages (CSL); Linear-bound Turing machines

• Type-0: Recursively enumerable languages (REL); Turing machines

RL ⊂ CF L ⊂ CSL ⊂ REL

1.1 Mathematical Notations and Terminologies

• sets: element, member, subset, proper subset, finite set, infinite set, empty set, union, inter-
section, complement, power set, Cartesian product (cross product)

• sequence, tuple: k-tuple: a sequence with k elements

• functions: mapping, domain, co-domain, range, one-to-one function, onto function, one-to-
one correcspondence

• relation:
- reflexive: xRx
- symmetric: xRy ⇒ yRx
- transitive: xRy ∧ yRz ⇒ xRz
- equivalence relation

1
 • graphs:

• strings, languages:
- alphabet: any non-empty finite set
- string over an alphabet: a finite sequence of symbols from the alphabet
- |w|: length of a string w (w = w1 w2 · · · wn , where w ∈ Σ, for an alphabet Σ)
- empty string: 
- reverse of w: wR
- substring
- concatenation

• logic

• theorem, proof
- by construction, induction contradiction

2 Regular Languages

2.1 Finite State Automata

Definition: A finite state automaton (FSA) is a 5-tuple (Q, Σ, δ, q0 , F ), where

1. Q is a finite set called the states.

2. Σ is a finite set called the alphabet.

3. δ : Q × Σ → Q is the transition function.

4. q0 ∈ Q is the start state.

5. F ⊆ Q is the set of accept states.

Formal definition of computation:

Let w = w1 w2 · · · wn be a string such that w ∈ Σ, and M = (Q, Σ, δ, q0 , F ) be a FSA. Then, M

accepts w if a sequence of states r0 , r1 , · · · , rn ∈ Q exists with conditions:

1. r0 = q0 ,

2. δ(ri , wi+1 ) = ri+1 for i = 0, 1, · · · , n − 1, and

3. rn ∈ F .

We say M recognizes A if A = {w | M accepts w }.

A language is called a regular language if some FSA recognizes it.

2
2.2 Designing FSA

2.3 Regular Operations

Let A and B be languages. We define regular operations as follows:

union: A ∪ B = {x | x ∈ A or x ∈ B}

concatenation: A ◦ B = {xy | x ∈ A and y ∈ B}

star: A∗ = {x1 x2 · · · xk | k ≥ 0 and each xi ∈ A}

Example:
A = {0, 1}, B = {a, b}:
A ∪ B = {0, 1, a, b}
A ◦ B = {0a, 0b, 1a, 1b}
A∗ = {, 0, 1, 00, 01, 10, 11, 000, 001, · · · , 111, 0000, · · · }

Theorem 2.1 The class of regular languages is closed under the union operation, i.e., if A1 and
A2 are regular languages, so is A1 ∪ A2 .

Proof: Let A1 and A2 be regular languages. By definition, A1 and A2 are recognized by FSA
M1 and M2 , resp. Let M1 = (Q1 , Σ1 , δ1 , q1 , F1 ) and M2 = (Q2 , Σ2 , δ2 , q2 , F2 ). We construct
M = (Q, Σ, δ, q0 , F ) from M1 and M2 such that

1. Q = Q1 × Q2 ,
i.e., Q = {(r1 , r2 ) | r1 ∈ Q1 , r2 ∈ Q2 }

2. Σ = Σ1 ∪ Σ2

3. δ((r1 , r2 ), a) = (δ(r1 , a), δ(r2 , a))

4. q0 = (q1 , q2 )

5. F = {(r1 , r2 ) | r1 ∈ F1 or r2 ∈ F2 },
i.e., F = (F1 × Q2 ) ∪ (Q1 × F2 ). (Note that F 6= F1 × F2 .)

Example:
Let L1 = {w | w has even number of 1’s} and L2 = {w | w contains 001 as a substring}. Construct
a FSA M for L1 ∪ L2 .

Theorem 2.2 The class of regular languages are closed under intersection operation.

Proof: Proof is same as above, except that F = F1 × F2 .

Example:
Let L1 = {w | w has odd number of a’s} and L2 = {w | w has one b}. Construct a FSA M for
L = L1 ∩ L2 , i.e., L = {w | w has odd number of a’s and one b. }

3
2.4 Nondeterminism

Formal definition of non-deterministic FSA (NFA):

An NFA is a 5-tuple M = (Q, Σ, δ, q0 , F ), where

1. Q is a finite set of states.

2. Σ is an alphabet.

3. δ : Q × Σ → P (Q) is the transition relation,

where Σ = Σ ∪ {} and P (Q) is the power set of Q.

4. q0 ∈ F is the initial state.

5. F ⊆ Q is the set of accept states.

2.5 Equivalence of NFA and DFA

Theorem 2.3 Every NFA has an equivalent DFA.

Proof: Let N = (Q, Σ, δ, q0 , F ) be an NFA recognizing language A. We construct a DFA M =

(Q0 , Σ, δ 0 , q00 , F 0 ) as follows.

(i) First, assume that N does not have -transition.

1. Q0 = P (Q).
2. For R ∈ P (Q), let δ 0 (R, a) = {q ∈ Q | q ∈ δ(r, a) for some r ∈ R}
(or, let δ 0 (R, a) = ∪{δ(r, a) | r ∈ R}.)
3. q00 = {q0 }.
4. F 0 = {R ∈ Q0 | R contains an accept state of N }.

(ii) Next, assume that N contains -transitions. For any R ∈ P (Q), let
E(R) = {q | q can be reached from R by traveling along 0 or more  arrow. }
Let δ 0 (R, a) = {q ∈ Q | q ∈ E(δ(r, a)) for some r ∈ R }. The rest are same as in case (i)

Example (i): Let N = (Q, Σ, δ, q0 , F ) be an NFA, where

1. Q = {q0 , q1 }

2. Σ = {0, 1}

3. δ(q0 , 0) = {q0 }; δ(q0 , 1) = {q0 , q1 };

4. initial state = q0

5. F = {q1 }

4
A DFA M = (Q0 , Σ, δ 0 , q00 , F 0 ) that is equivalent to N is then constructed as:

1. Q0 = {{q0 }, {q0 , q1 }}

2. Σ = {0, 1}

3. δ 0 ({q0 }, 0) = {q0 }; δ 0 ({q0 }, 1) = {q0 , q1 }; δ 0 ({q0 , q1 }, 0) = {q0 }; δ 0 ({q0 , q1 }, 1) = {q0 , q1 }

4. initial state = {q0 }

5. F = {{q0 , q1 }}

Example (ii): Let N = (Q, Σ, δ, q0 , F ) be an NFA, where

1. Q = {q0 , q1 , q2 , q3 , q4 }

2. Σ = {a, b}

3. δ(q0 , ) = {q1 }; δ(q0 , b) = {q2 }; δ(q1 , ) = {q2 , q3 }; δ(q1 , a) = {q0 , q4 }; δ(q2 , b) = {q4 }; δ(q3 , a) =
{q4 }; δ(q4 , ) = {q3 }

4. initial state = q0

5. F = {q4 }

Note that E(q0 ) = {q0 , q1 , q2 , q3 }, E(q1 ) = {q1 , q2 , q3 }, E(q2 ) = {q2 }, E(q3 ) = {q3 }, and E(q4 ) =
{q3 , q4 }. We then construct a DFA M = (Q0 , Σ, δ 0 , q00 , F 0 ) by following the algorithm in (i) as
follows:

1. Q0 = {p0 , p1 , p2 , p3 , p4 } where p0 = {q0 , q1 , q2 , q3 }, p1 = {q0 , q1 , q2 , q3 , q4 }, p2 = {q2 , q3 , q4 },

p3 = {q3 , q4 }, and p4 = ∅ (or a trap state).

2. Σ = {a, b}

3. δ 0 (p0 , a) = p1 ; δ 0 (p0 , b) = p2 ; δ 0 (p1 , b) = p2 ; δ 0 (p1 , a) = p1 ; δ 0 (p2 , a) = p3 ; δ 0 (p2 , b) = p3 ;

δ 0 (p3 , a) = p3 ; δ 0 (p3 , b) = p4 ; δ 0 (p4 , a) = p4 , and δ 0 (o4 , b) = p3 .

4. initial state = p0

5. F = {p1 , p2 , p3 }.

5
2.6 Closure Properties of Regular Languages

Theorem 2.4 Regular languages are closed under the following operations:

(1) union

(2) intersection

(3) concatenation

(4) star operation (or Kleene star operation)

Note: We can construct an NFA N for each case and find a DFA M equivalent to N .

2.7 Regular Expressions

- To describe regular languages

Examples: (0 ∪ 1)0∗ , (0 ∪ 1) = ({0} ∪ {1}), (0 ∪ 1)∗

Definition: We say R is a regular expression if R is

(1) a for some a ∈ Σ

(2) 

(3) ∅

(4) R1 ∪ R2 , where R1 and R2 are regular expressions.

(5) R1 ◦ R2 , where ◦ is a concatenation operation, and R1 and R2 are regular expressions.

(6) (R1 )∗ , where R1 is a regular expression.

- recursive or inductive definition

- () may be omitted.

- R+ = RR∗ or R∗ R

- R+ ∪  = R∗

- Rk = R ◦ R ◦ · · · ◦ R (i.e., R is concatenated k times.)

- L(R)

Examples: 0∗ 10∗ , Σ∗ 1Σ∗ , 1∗ (01+ )∗ , (0 ∪ )1∗ = 01∗ ∪ 1∗ , (0 ∪ )(1 ∪ ) = {01, 0, 1, }, 1∗ ◦ ∅ = ∅,

1 ◦  = 1∗ , ∅∗ = {}

6
2.8 Equivalence of Regular Expression and DFA

Recall: A language is regular if and only if a DFA recognizes it.

Theorem 2.5 A language is regular if and only if some regular expression can describe it.

Proof is based on the following two lemmas.

Lemma 2.1 If a language L is described by a regular expression R, then it is a regular language,

i.e., there is a DFA that recognizes L.

Proof. We will convert R to an NFA N (equivalently a DFA).

(1) R = a ⇒ L(R) = {a}

(2) R =  ⇒ L(R) = {}

(3) R = ∅ ⇒ L(R) = ∅

(4) R = R1 ∪ R2 ⇒

(5) R = R1 ◦ R2 ⇒

(6) R = R1∗ ⇒

Example: R = (ab ∪ a)∗ ⇒ N :

Lemma 2.2 If L is a regular language, then it can be described by a regular expression.

Proof: Reference: text, Lemma 1.60.

2.8.1 Alternate proof:

Since L is a regular language, there must be a DFA that recognizes L. We then apply the following
result.

7
 Lemma: Let M = (Q, Σ, δ, q0 , F ) be a DFA. Then there exists a regular expression E
such that L(E) = L(M ), where L(E) denotes the language represented by E.

Proof: Let Q = {q1 , · · · , qm } such that q1 is the start state of M . For 1 ≤ i, j ≤ m and 1 ≤ k ≤
m+1, we let R(i, j, k) denote the set of all strings in Σ∗ that derive M from qi to qj without passing
through any state numbered k or greater.

When k = m + 1, it follows that

R(i, j, m + 1) = {x ∈ Σ∗ | (qi , x) `∗M (qj , )}.

Therefore, L(M ) = ∪{R(1, j, m + 1) |qj ∈ F }.

The crucial point is that each set R(i, j, k) is regular, and hence so is L(M ). The proof is by
induction on k. For k = 1, we have the following.

{a ∈ Σ | δ(qi , a) = qj } if i 6= j
R(i, j, 1) =
{} ∪ {a ∈ Σ | δ(qi , a) = qj } if i = j

Each of these sets is finite, and therefore regular. For k = 1, · · · , m, provided that all the sets
R(i, j, k) have been defined, each set R(i, j, k + 1) can be defined in terms of previously defined
languages as
R(i, j, k + 1) = R(i, j, k) ∪ R(i, k, k)R(k, k, k)∗ R(k, j, k).
This equation states that to get from qi to qj without passing through a state numbered greater
than k, M may either

(i) go from qi to qj without passing through a state numbered greater than k − 1, or

(ii) go from qi to qk ; then from qk to qk repeatedly; and then from qk to qj , in each case without
passing through a state numbered greater than k − 1.

Therefore, if each language R(i, j, k) is regular, so is each language R(i, j, k + 1). This completes
the induction.

8
2.9 Non-regular Languages (Pumping Lemma)

Review ...

• Let L be an arbitrary finite set. Is L a regular language?

• Give a regular expression for the set L1 of non-negative integers.

Let Σ = {0, 1, · · · , 9}. Then, L1 = {0} ∪ {1, 2, · · · , 9} ◦ Σ∗ .

• Give a regular expression for the set L2 of non-negative integers that are divisible by 2.
Then, L2 = L1 ∩ Σ∗ ◦ {0, 2, 4, 6, 8}

• Give a regular expression for the set L3 of integers that are divisible by 3.
Then, L3 = L1 ∩ L(M ), where
M is defined as:

• Let Σ = {a, b}, and L4 ⊆ Σ∗ be the set of odd length, containing an even # of a’s.
Then, L4 = L5 ∩ L6 , where L5 is the set of all strings of odd length, i.e., L5 = Σ(ΣΣ)∗ , and
L6 is the set of all strings with an even # of a’s, i.e., L6 = b∗ (ab∗ ab∗ )∗ .

Now, consider the following...

• A1 = {0n 1n | n ≥ 1}

• A2 = {w | w has an equal number of occurrences of a’s and b’s.}

• A3 = {w | w has an equal number of occurrences of 01 and 10 as substrings.}

Lemma 2.3 (Pumping Lemma for Regular Languages)

If A is a regular language, then there is a positive integer p called the pumping length where if s is
any string in A of length at least p, then s may be divided into three substrings s = xyz for some
x, y, and z satisfying the following conditions:

(i) |y| > 0 (|x|, |z| ≥ 0)

(ii) |xy| ≤ p

(iii) for each i ≥ 0, xy i z ∈ A.

2.9.1 Non-regular languages

• {wwR | w ∈ {0, 1}∗ }.

• {ww | w ∈ {0, 1}∗ }

9
 • {an bam ban+m | n, m ≥ 1}

• {ww | w ∈ {a, b}∗ where w stands for w with each occurrence of a replaced by b, and vice
versa.}

• L = {w |w has equal number of 0’s and 1’s }

• L = {am bn | m 6= n}

Answer true or false:

(a) Every subset of a regular language is regular.

(b) Every regular language has a subset that is regular.

(c) If L is regular, then so is {xy | x ∈ L and y ∈

/ L}

(d) L = {w | w = wR } is regular.

(e) If L is regular, then LR = {wR | w ∈ L} is regular.

(f) L = {xyxR | x, y ∈ Σ∗ } is regular.

(g) If L is regular, then L1 = {w | w ∈ L and wR ∈ L} is regular.

2.9.2 more non-regular languages proved by Pumping lemma

2
1. L = {an | n ≥ 1}
n
2. L = {a2 | n ≥ 1}

3. L = {aq |q is a prime number.}

4. L = {an! | n ≥ 1}

5. L = {am bn | m > n}

6. L = {am bn | m < n}

7. L = {w ∈ {a, b}∗ | na (w) = nb (w)}

8. L = {w ∈ {a, b}∗ | na (w) 6= nb (w)}

9. L = {ap bq | p and q are prime numbers.}

2 2
10. L = {an bm | n, m ≥ 1}

11. L = {w ∈ {a, b}∗ | na (w) and nb (w) both are prime numbers}

12. L = {an! bm! | n, m ≥ 1}

10
2.9.3 additional properties of regular languages

• Given two regular languages L1 and L2 , describe an algorithm to determine if L1 = L2 .

• There exists an algorithm to determine whether a regular language is empty, finite, or infinite.

• membership

11
3 Context Free Languages and Context Free Grammars

Definition. A context-free grammar (CFG) is a 4-tuple (V, Σ, R, S) where

1. V is a finite set called the variables (or non-terminals).

2. Σ is a finite set called the terminals.

3. R is a finite set of production rules such that

R : V → (V ∪ Σ)∗ .

4. S ∈ V is a start symbol.

12
Examples of Context-Free Grammars

G0 : E → E + E | E ∗ E | id

G1 : E → T E0
E 0 → +T E 0 | 
T → FT0
T 0 → ∗F T 0 | 
F → (E) | id

G2 : E →E+T | T
T →T ∗F | F
F → (E) | id

G3 : E0 → E
E → E + T |T
T →T ∗F | F
F → (E) | id

G4 : S0 → S
S→L=R
S→R
L → ∗R
L → id
R→L

G5 : S0 → S
S → aAd | bBd | aBe | bAe
A→c
B→c

13
3.1 Context Free Grammar

1. L = {an bn | n ≥ 0}

S → aSb | 

2. L = {am bn | m > n}

S → AC
C → aCb | 
A → aA | a

3. L = {am bn | m < n}

S → CB
C → aCb | 
B → bB | b

4. L = {am bn | m 6= n}

S → AC | CB
C → aCb | 
A → aA | a
B → bB | b

5. L = {w ∈ {a, b}∗ | na (w) = nb (w)}.

S → SS | aSb | bSa | 

6. L = {w ∈ {a, b}∗ | na (w) > nb (w)}.

S0 → AS | SAS | SA
S → SS | SAS | aSb | bSa | 
A → aA | a

Proof: Note that any string generated by the above rules has more a’s than b’s. We next
proceed to show that any string w ∈ L can be generated by these rules. We first note that
any string z such that na (z) = nb (z) must be split into substrings such that z = z1 z2 · · · zl

14
 where (i) each zj has equal number of a’s and b’s, (ii) the first and the last symbols of zj are
different, and (iii) any such zj does not contain a substring that has the same number of a’s
and b’s but the first and the last symbols are same. For example, aabbab cannot be such a zj
since it contains abba, but aababb can be such a zj . It is then noted that for any w ∈ L, w
can be denoted as:
w = al0 z1 al1 z2 al2 · · · zk alk ,
where (1) each zi satisfies the above three conditions (i) - (iii); (2) for each i, 0 ≤ i ≤ k,
li ≥ 0; and (3) l0 + l1 + · · · + lk > 0. For example, w = aaababbaaaabbaaa may be decomposed
into w = aa · ab · ab · ba · a · aabb · aaa, where l0 = 2, z1 = ab, l1 = 0, z2 = ab, l2 = 0, z3 = ba,
l3 = 1, z4 = aabb, and l4 = 3.
From the start state S0 , one of the following three cases occurs: If l0 > 0, S0 ⇒ AS; else if
lk > 0, S0 ⇒ SA; otherwise, S0 ⇒ SAS. We then recursively apply S → SS or S → SAS
such that a single S generates a substring zj satisfying conditions (i)-(iii) above.
Consider the example above: w = aaababbaaaabbaaa. w is then split into a2 z1 z2 z3 a1 z4 a3 ,
and is generated as follows.

S→AS S→SS S→SS S→SAS

S0 =⇒ AS =⇒ ASS =⇒ ASSS =⇒ ASSSAS
S→SAS S→ ∗
=⇒ ASSSASAS =⇒ ASSSASA ⇒ aaz1 z2 z3 az4 aaa, which is aaababbaaaabbaaa.
Note: The following also work correctly. You can verify the correctness using the similar
arguments.

S → RaR | aRR | RRa

R → RaR | aRR | RRa | aRb | bRa | 

7. L = {w ∈ {a, b}∗ | na (w) 6= nb (w)}.

Note that L = {w ∈ {a, b}∗ | na (w) > nb (w) or na (w) < nb (w)}.

8. L = {w ∈ {a, b, c}∗ | na (w) + nb (w) = nc (w)}.

S → SS | aSc | cSa | bSc | cSb | 

9. L = {w ∈ {a, b, c}∗ | na (w) + nb (w) > nc (w)}.

S0 → T S | ST S | ST
S → SS | ST S | aSc | cSa | bSc | cSb | 
T → aT | bT | a | b

10. L = {w ∈ {a, b, c}∗ | na (w) + nb (w) 6= nc (w)}.

Note that L = {w ∈ {a, b, c}∗ | na (w) + nb (w) > nc (w) or na (w) + nb (w) < nc (w)}.

15
 11. L = {w ∈ {a, b, c}∗ | na (w) + nb (w) > 2nc (w)}.

S0 → T S | ST S | ST
S → SS | ST S | 
S → SDDC | DSDC | DDSC | DDCS
SDCD | DSCD | DCSD | DCDS
SCDD | CSDD | CDSD | CDDS
D → a|b
C → c
T → aT | bT | a | b

12. L = {w ∈ {a, b, c}∗ | na (w) + nb (w) < 2nc (w)}.

S0 → T S | ST S | ST
S → SS | ST S | 
S → SDDC | DSDC | DDSC | DDCS
SDCD | DSCD | DCSD | DCDS
SCDD | CSDD | CDSD | CDDS
D → a|b
C → c
T → cT | c

13. L = {w ∈ {a, b, c}∗ | na (w) + nb (w) 6= 2nc (w)}.

Note that L = {w ∈ {a, b, c}∗ | na (w) + nb (w) > 2nc (w) or na (w) + nb (w) < 2nc (w)}.

3.2 Chompsky Normal Form

Definition: A CFG is in Chomsky Normal Form if every rule is of the form

A → BC

A → a
where a is any terminal and A, B, and C are any non-terminal (i.e., variable) except that B and C
may not be the start symbol. In addition, we permit the rule S → , where S is the start symbol.

Theorem 2.9 (pp. 107). Any context-free languages is generated by a context-free grammar in
Chomsky normal form.

16
3.3 CYK Membership Algorithm for Context-Free Grammars

Let G = (V, Σ, R, S) be a CFG in CNF, and consider a string w = a1 a2 · · · an . We define substrings

∗
wij = ai · · · aj and subset Vij = {A ∈ V | A ⇒ wij } of V .

Clearly, w ∈ L(G) if and only if S ∈ V1n . To compute Vij , we observe that A ∈ Vii if and only
if R contains a production A → ai . Therefore, Vii can be computed for all 1 ≤ i ≤ n by inspection
of w and the production rules of G. To continue, notice that for j > i, A derives wij if and only if
∗ ∗
there is a production A → BC with B ⇒ wik and C ⇒ wk+1j for some k with i ≤ k < j. In other
words,
Vij = ∪k∈{i,i+1,··· ,j−1} {A | A → BC, with B ∈ Vik , C ∈ Vk+1j }.
The above equation can be used to compute all the Vij if we proceed in the following sequence:

1. Compute V11 , V22 , · · · , Vnn

2. Compute V12 , V23 , · · · , V(n−1)n

3. Compute V13 , V24 , · · · , V(n−2)n

and so on.

Time Complexity: O(n3 ), where n = |w|.

17
Example: Consider a string w = aabbb and a CFG G with the following production rules:

S → AB
A → BB | a
B → AB | b

j
i 1 2 3 4 5
1 A ∅ S, B A S, B
2 A S, B A B, S
3 B A S, B
4 B A
5 B

Since S ∈ V15 , w ∈ L(G).

18
3.4 Pushdown Automata

A pushdown automaton is a 6-tuples M = (Q, Σ, Γ, δ, q0 , F ) where

1. Q is the finite set of states,

2. Σ is the input alphabet,

3. Γ is the stack alphabet,

4. δ : Q × Σ × Γ → P(Q × Γ ) is the transition function,

5. q0 ∈ Q is the start state, and

6. F ⊆ Q is the set of accept states.

Note: An input is accepted only if (i) input is all read and (ii) the stack is empty.

3.4.1 PDA for CFL

• L = {0n 1n | n ≥ 0}

• L = {ai bj ck | i = j or i = k, where i, j, k ≥ 0}

• L = {w ∈ {a, b}∗ | na (w) = nb (w)}

• L = {wwR | w ∈ {a, b}∗ }

• L = {an b2n | n ≥ 0}

• L = {wcwR | w ∈ {a, b}∗ }

• L = {an bm cn+m | n, m ≥ 0}

• L = {an bm | n ≤ m ≤ 3n}

3.5 Equivalence of PDA and CFG

Theorem 3.1 A language L is a CFL if and only if some PDA recognizes L.

3.6 Pumping Lemma for CFL

Let L be a CFL. Then, there exists a number p, called the pumping length, where for any string
w ∈ L with |w| ≥ p, w may be divided into five substrings w = uvxyz such that

1) |vy| > 0

2) |vxy| ≤ p, and

3) for each i ≥ 0, uv i xy i z ∈ L.

19
3.6.1 Non-Context Free Languages

L = {an bn cn | n ≥ 0}
Let w = ap bp cp and apply the Pumping lemma.

L = {ww | w ∈ 0, 1∗ }
(Try with w = 0p 10p 1. Pumping lemma is not working!)
Let w = 0p 1p 0p 1p and apply Pumping lemma.

L = {ai bj ck | 0 ≤ i ≤ j ≤ k ≤ n}
Let w = ap bp cp and apply Pumping lemma.

L = {an! | n ≥ 0}
(Recall: L is not regular.)
Let w = ap! and apply Pumping lemma.

L = {an bj |n = j 2 }.
2
Let w = ap bp and apply Pumping lemma. We then have w = uvxyz and three cases to
consider.

(i) vy = aα or vy = bβ . Let i = 0 and come up with a contradiction.

(ii) v = aα bβ or y = aα bβ . Let i = 2 and come up with a contradiction.
(iii) v = aα and y = bβ , where α 6= 0 and β 6= 0.
Let’s first consider i = 0. If p2 − α 6= (p − β)2 , then we are done. So assume that
p2 − α = (p − β)2 , i.e., we assume α = 2pβ − β 2 . We then consider i = 2. The number
of a’s in w2 is p2 + α = p2 + 2pβ − β 2 , and the number of b’s in w2 is p + β. Note that
p2 + 2pβ − β 2 6= (p + β)2 since β 6= 0. Therefore, p2 + α 6= (p + β)2 , a contradiction to
the Pumping lemma.

From (i) - (iii), we conclude that L cannot be a CFL.

L = {ar+s | r and s are both prime numbers.}

Let w = a2+p where p is a prime number that is larger than or equal to the pumping length.
Then, by the Pumping lemma, w = uvxyz where v = aα and y = aβ . Consider i = 2p + 1.
Then, |w2p+1 | = 2 + p + 2p(α + β) = 2 + p(1 + 2(α + β)), which is an odd number since
p(1 + 2(α + β) is an odd number (odd * odd). However, p(1 + 2(α + β) is not a prime number;
hence, w2p+1 cannot be in L. Consequently, L cannot be a CFL.

3.7 Closure Properties

• CFL’s are closed under the union operation.

• CFL’s are not closed under the intersection operation.

• CFL’s are not closed under the complementation operation.

• CFL’s are closed under the concatenation operation.

20
• CFL’s are closed under the kleene star operation.

• The intersection of a CFL and a RL is a CFL.

21
3.8 Top-Down Parsing

3.8.1 Transform to Unambiguous Grammar

A grammar is called ambiguous if there is some sentence in its language for which there us more
than one parse tree.

Example: E → E + E | E ∗ E | id;
w = id + id ∗ id.

In general, we may not be able to determine which tree to use. In fact, determining whether a
given arbitrary CFG is ambiguous or not is undecidable.

Solution:

(a) Transform the grammar to an equivalent unambiguous one, or

(b) Use disambiguating rule with the ambiguous grammar to specify, for ambiguous cases, which
parse tree to use.

if then else statement

G1 : stmt → if exp then stmt |

if exp then stmt else stmt

For an input “if E1 then if E2 then S1 else S2 ,” two parse trees can be constructed; hence, G1 is
ambiguous. An unambiguous grammar G2 which is equivalent to G1 can be constructed as follows:

G2 : stmt → matched stmt |

unmatched stmt
matched stmt → if exp then matched stmt else matched stmt |
other stmt
unmatched stmt → if exp then stmt |
if exp then matched stmt else unmatched stmt

22
3.8.2 Left-factoring and Removing left recursions

Consider the following grammar G1 and a token string w = bede.

G1 : S → ee | bAc | bAe
A → d | eA

Since the initial b is in two production rules, S → bAc and S → bAe, the parser cannot make a
correct decision without backtracking. This problem may be solved to redesign the grammar as
shown in G2 .

G2 : S → ee | bAQ
Q→c|e
A → d | eA

In G2 , we have factored out the common prefix bA and used another non-terminal symbol Q to
permit the choice between the final c and a. Such a transformation is called as left factorization or
left factoring.

Now, consider the following grammar G3 and consider a token string w = id + id + id.

G3 : E →E+T | T
T →T ∗F | F
F → id | (E)

A top-down parser for this grammar will start by expanding E with the production E → E + T .
It will then expand E in the same way. In the next step, the parser should expand E by E → T
instead of E → E + T . But there is no way for the parser to know which choice it should make. In
general, there is no solution to this problem as long as the grammar has productions of the form
A → Aα, called left-recursive productions. The solution to this problem is to rewrite the grammar
in such a way to eliminate the left recursions. There are two types of left recursions: immediate
left recursions, where the productions are of the form A → Aα, and non-immediate left recursions,
where the productions are of the form A → Bα; B → Aβ. In the latter case, A will use Bα, and
B will use Aβ, resulting in the same problem as the immediate left recursions have.

We now have the following formal definition: “A grammar is left-recursive if it has a nonterminal
+
A such that there is a derivation A ⇒ Aα for some string α.”

Removing immediate left recursions:

23
Input: A → Aα1 |Aα2 | · · · | Aαm | β1 | β2 | · · · | βn
Output: A → β1 A0 |β2 A0 | · · · | βn A0
A0 → α1 A0 | α2 A0 | · · · | αm A0 | 

Consider the above example G3 in which two productions have left recursions. Applying the
above algorithm to remove immediate left recursions, we have

(i) E → E + T | T
⇒ E → T E0
E 0 → +T E 0 | 

(ii) T → T ∗ F | F
⇒ T → FT0
T 0 → ∗F T 0 | 

Now, we have following grammar G4 which is equivalent to G3 :

G4 : E → T E0
E 0 → +T E 0 | 
T → FT0
T 0 → ∗F T 0 | 
F → (E) | id

The following is an algorithm for eliminating all left recursions including non-immediate left
recursions.

24
Algorithm: Eliminating left recursioin.
Input: Grammar G with no cycles or -productions.
Output: An equivalent grammar with no left recursion.

1. Arrange the nonterminals in some order A1 , A2 , · · · , An .

2. for i = 1 to n begin
for j = 1 to i − 1 do begin
replace each production of the form Ai → Aj γ
by the productions Ai → δ1 γ | δ2 γ | · · · |δk γ.
where Aj → δ1 | δ2 | · · · | δk are all the current Aj -productions;
end
eliminate the immediate left recursion among the Ai -productions
end
end.

Examples

EXAMPLE 1: Consider the following example:

G: S → Ba | b
B → Bc | Sd | e

Let A1 = S and A2 = B. We then have,

G: A1 → A2 a | b
A2 → A2 c | A1 d | e

(i) i=1:
A1 → A2 a | b, OK

(ii) i=2:
A2 → A1 d is replace by A2 → A2 ad | bd

Now, G becomes

G: A1 → A2 a | b
A2 → A2 c | A2 ad | bd | e

25
By eliminating immediate recursions in A2 -productions, we have

(i) A2 → A2 c | bd | e are replaced by

A2 → bdA3
A2 → eA3
A3 → cA3 | 

(ii) A2 → A2 ad | bd | e are replaced by

A2 → bdA4
A2 → eA4
A4 → adA4 | 

(i) and (ii) can be combined as

A2 → bdA3 | eA3
A3 → cA3 | adA3 | 

Therefore, we have

S → Ba | b
B → bdD | eD
D → cD | adD | 

3.8.3 First and Follow Sets

∗
Consider every string derivable from some sentential form α by a leftmost derivation. If α =⇒ β,
where β begins with some terminal, then that terminal is in F IRST (α).

26
Algorithm: Computing F IRST (A).

1. If A is a terminal, F IRST (A) = {A}.

2. If A → , add  to F IRST (A).
3. if A → Y1 Y2 · · · Yk , then
for i = 1 to k − 1 do
∗
if [ ∈ F IRST (Y1 ) ∩ F IRST (Y2 ) ∩ · · · ∩ F IRST (Yi−1 )] (i.e., Y1 Y2 · · · Yi−1 ⇒ ) and
a ∈ F IRST (Yi ), then add a to F IRST (A).
end
if  ∈ F IRST (Y1 ) ∩ · · · ∩ F IRST (Yk ), then add  to F IRST (A).
end.

Now, we define F OLLOW (A) as the set of terminals that cam come right after A in any
sentential form of L(G). If A comes at the end, then F OLLOW (A) includes the end marker $.

Algorithm: Computing F OLLOW (B).

1. $ is in F OLLOW (S).
2. if A → αBβ, then F IRST (β) − {} ⊆ F OLLOW (B).
∗
3. if A → αB or A → αBβ where  ∈ F IRST (β) (i.e., β ⇒ ),
F OLLOW (A) ⊆ F OLLOW (B)
end.

Note: In Step 3, F OLLOW (B) 6⊆ F OLLOW (A). To prove this, consider the following example:
S → Ab | Bc; A → aB; B → c. Clearly, c ∈ F OLLOW (B) but c 6∈ F OLLOW (A).

EXAMPLE:

For the grammar G4 ,

G4 : E → T E0
E 0 → +T E 0 | 
T → FT0
T 0 → ∗F T 0 | 
F → (E) | id

F IRST (E) = F IRST (T ) = F IRST (F ) = {(, id}.

F IRST (E 0 ) = {+, }.

27
F IRST (T 0 ) = {∗, }.
F OLLOW (E) = F OLLOW (E 0 ) = {), $}.
F OLLOW (T ) = F OLLOW (T 0 ) = {+, ), $}.
F OLLOW (F ) = {+, ∗, ), $}.

3.8.4 Constructing a predictive parser

Algorithm: Predictive parser contruction.

Input: Grammar G.
Output: Parsing table M .

1. for each A → α, do Steps 2 & 3.

2. for each terminal a ∈ F IRST (α),
add A → α to M [A, a].
3. 3.1 if  ∈ F IRST (α),
add A → α to M [A, b] for each terminal b ∈ F OLLOW (A).
3.2 if  ∈ F IRST (α) and $ ∈ F OLLOW (A),
add A → α to M [A, $].
end.

EXAMPLE:

G4 : E → T E0
E 0 → +T E 0 | 
T → FT0
T 0 → ∗F T 0 | 
F → (E) | id

Input symbol
id + * ( ) $
E E → T E0 E → T E0
E0 E 0 → +T E 0 E0 →  E0 → 
T T → FT0 T → FT0
T0 T0 → T0 → ∗F T 0 T0 →  T0 → 
F F → id F → (E)

Stack Operation

28
 Stack Input Action
$E id + id ∗ id$ E → T E0
$E 0 T id + id ∗ id$ T → FT0
$E 0 T 0 F id + id ∗ id$ F → id
$E 0 T 0 id id + id ∗ id$ match
$E 0 T 0 +id ∗ id$ T0 → 
$E 0 +id ∗ id$ E 0 → +T E 0
$E 0 T + +id ∗ id$ match
$E 0 T id ∗ id$ T → FT0
$E 0 T 0 F id ∗ id$ F → id
$E 0 T 0 id id ∗ id$ match
$E 0 T 0 ∗id$ T 0 → ∗F T 0
$E 0 T 0 F ∗ ∗id$ match
$E 0 T 0 F id$ F →
$E 0 T 0 id id$ match
$E 0 T 0 $ T0 → 
$E 0 $ E0 → 
$E $ accept

3.8.5 Properties of LL(1) Grammars

A grammar whose parsing table has no multiply-defined entries is said to be LL(1).

Properties:

1. No ambiguous or left-recursive grammar can be LL(1).

2. A grammar G is LL(1) if and only if whenever A → α | β are two distinct productions, the
following conditions hold:
∗ ∗
2.1 For any terminal a, there exist no derivations that α ⇒ aα0 and β ⇒ aβ 0 .

2.2 Either α or β, but not both, can derive .

∗
2.3 If β ⇒ , then α does not derive any string beginning with a terminal in F OLLOW (A).

∗ ∗ ∗
Proof of Condition 2.2: Suppose α ⇒  and β ⇒ . Consider S ⇒ γ1 Aγ2 . Then, two possibilities
∗ ∗ ∗ ∗ ∗ ∗
exist: S ⇒ γ1 Aγ2 ⇒ γ1 αγ2 ⇒ γ1 γ2 and S ⇒ γ1 Aγ2 ⇒ γ1 βγ2 ⇒ γ1 γ2 . G must be then ambiguous.
∗ ∗
Proof of Condition 2.3: Suppose β ⇒  and α ⇒ aα0 , where a ∈ F OLLOW (A). Also, assume
∗ ∗ ∗ ∗
that γ2 ⇒ aγ20 . We then have two possibilities: (i) S ⇒ γ1 Aγ2 ⇒ γ1 αγ2 ⇒ γ1 aα0 γ2 , and (ii)
∗ ∗ ∗ ∗
S ⇒ γ1 Aγ2 ⇒ γ1 βγ2 ⇒ γ1 γ2 ⇒ γ1 aγ20 . Hence, after taking care of the input tokens corresponding
to γ1 , the parser cannot make a clear choice between the two productions A → α and A → β.

29
3.9 Bottom-Up Parsing

3.9.1 SLR Parser

Computation of Closure

If I is a set of items for a grammar G, then closure(I) is the set of items constructed from I
by the two rules.

1. Initially, every item in I is added to closure(I).

2. If A → α · Bβ is in closre(I) and B → γ is a production, then add the item B → ·γ to I, if it

is not already in I. We apply this rule until no more new items can be added to closure(I).

function closure(I):
begin
J = I;
repeat
for each item A → α · Bβ in J and each production
B → γ of G such that B → ·γ is not in J do
add B → ·γ to J
until no more items can be added to J
return
end

We are now ready to give the algorithm to construct C, the canonical collection of stes of LR(0)
items for an augmenting grammar G0 .

procedure items(G0 ):
begin
C = {closure({[S 0 → ·S]})};
repeat
for each set of items I in C and each grammar symbol X
such that goto(I, X) is not empty and not in C do
add goto(I, X) to C
until no more sets of items can be added to C
end

Constructing SLR Parsing Table

30
Algorithm: Constructing an SLR parsing table.
Input: An augmenting grammar G0 .
Output: The SLR parsing table functions action and goto for G0 .

1. Construct C = {I0 , · · · , In }, the collection of sets of LR(0) items for G0 .

2. State i constructed from Ii . The parsing actions for state i are determined as follows:
a) If [A → α · aβ] is in Ii and goto(Ii , a) = Ij , then set action[i, a] to “shift j.”
Here a must be a terminal.
b) If [A → α·] is in Ii , then set action[i, a] to “reduce A → α” for all a in F OLLOW (A);
here A may not be S 0 .
c) If [S 0 → S·] is in Ii , then set action[i, $] to “accept.”
If any conflicting actions are generated by the above rules, we say the grammar is not SLR(0).
The algorithm fails to produce a parser in this case.
3. The goto transitions for state i are constructed for all nonterminals A using the rule:
If goto(Ii , A) = Ij , then goto[i, A] = j.
4. All entries not defined by rules (2) and (3) are made “error.”
5. The initial state of the parser is teh one constructed from the set of items containing [S 0 → ·S].
end.

Example

Consider the following grammar G:

(0) E0 → E
(1) E →E+T
(2) E→T
(3) T →T ∗F
(4) T →F
(5) F → (E)
(6) F → id

31
The canonical LR(0) collection for G is:

I0 : E 0 → ·E I5 : F → id·
E → ·E + T
E → ·T I6 : E → E + ·T
T → ·T ∗ F T → ·T ∗ F
T → ·F T → ·F
F → ·(E) F → ·(E)
F → ·id F → ·id

I1 : E 0 → E· I7 : T → T ∗ ·F
E → E · +T F → ·(E)
F → ·id

I2 : E → T· I8 : F → (E·)
T → T · ∗F E → E · +T

I3 : T → F· I9 : E → E + T·
T → T · ∗F

I4 : F → (·E) I10 T → T ∗ F·
E → ·E + T
E → ·T I11: F → (E)·
T → ·T ∗ F
T → ·F
F → ·(E)
F → ·id

The transition for viable prefixes is:

I0 : goto(I0 , E) = I1 ; goto(I0 , T ) = I2 ; goto(I0 , F ) = I3 ; goto(I0 , () = I4 ; goto(I0 , id) = I5 ;

I1 : goto(I1 , +) = I6 ;

I2 : goto(I2 , ∗) = I7 ;

I4 : goto(I4 , E) = I8 ; goto(I4 , T ) = I2 ; goto(I4 , F ) = I3 ; goto(I4 , () = I4 ;

I6 : goto(I6 , T ) = I9 ; goto(I6 , F ) = I3 ; goto(I6 , () = I4 ; goto(I6 , id) = I5 ;

I7 : goto(I7 , F ) = I10 ; goto(I7 , () = I4 ; goto(I7 , id) = I5 ;

I8 : goto(I8 , ) = I11 ; goto(I8 , +) = I6 ;

I9 : goto(I9 , ∗) = I7 ;

32
The FOLLOW set is: F OLLOW (E 0 ) = {$}; F OLLOW (E) = {+, ), $}; F OLLOW (T ) = F OLLOW (F ) =
{+, ), $, ∗}.

State action goto

id + * ( ) $ E T F
0 s5 s4 1 2 3
1 s6 acc
2 r2 s7 r2 r2
3 r4 r4 r4 r4
4 s5 s4 8 2 3
5 r6 r6 r6 r6
6 s5 s4 9 3
7 s5 s4 10
8 s6 s11
9 r1 s7 r1 r1
10 r3 r3 r3 r3
11 r5 r5 r5 r5

The moves of the SLR parser on input id ∗ id + id is:

Step Stack Input ACtion

(1) 0 id * id + id $ shift
(2) 0id5 *id+id$ reduce by F → id
(3) 0F3 *id+id$ reduce by T →F
(4) 0T2 *id+id$ shift
(5) 0T2*7 id+id$ shift
(6) 0T2*7id5 +id$ reduce by F → id
(7) 0T2*7F10 +id$ reduce by T →T ∗F
(8) 0T2 +id$ reduce by E→T
(9) 0E1 +id$ shift
(10) 0E1+6 id$ shift
(11) 0E1 + 6id5 $ reduce by F → id
(12) 0E1+6F3 $ reduce by T →F
(13) 0E1+6T9 $ reduce by E →E+T
(14) 0E1 $ accept

33
3.9.2 Canonical LR(1) Parser

Consider the following grammar G with productions:

S0 → S
S→L=R
S→R
L → ∗R
L → id
R→L

Let’s construct the canonical sets of LR(0) items for G:

I0 : S 0 → ·S I5 : L → id·
S → ·L = R
S → ·R I6 : S → L = ·R
L→·∗R R → ·L
L → ·id L→·∗R
R → ·L L → ·id

I1 : S 0 → S· I7 : L → ∗R·

I2 : S → L· = R I8 : R → L·
R → L·

I3 : S → R· I9 : S → L = R·

I4 : L → ∗ · R
R → ·L
L→·∗L
L → ·id

Note that =∈ F OLLOW (R) since S ⇒ L = R ⇒ ∗R = R. Consider the state I2 and the input
symbol is “=.” From [R → L·], the parser will reduce by R → L since =∈ F OLLOW (R). But due
to [S → L· = R], it will try to shift the input as well, a conflict. Therefore, this grammar G cannot
be handled by the SLR(0) parser. In fact, G can be parsed using the canonical-LR(1) parser that
will be discussed next.

34
Construction of LR(1) Items

Let G0 be an augmented grammar of G.

function closure(I):
begin
repeat
for each item [A → α · Bβ, a] in I,
each production B → γ in G0 ,
and each terminal b in F IRST (βa)
such that [B → ·γ, b] is not in I do
add [B → ·γ, b] to I;
until no more items can be added to I
return I
end

function goto(I, X):

begin
let J be the set of items [A → αX · β, a] such that
[A → α · Xβ, a] is in I;
return closure(J)
end

procedure items(G0 ):
begin
C = {closure({[S 0 → ·S, $]})};
repeat
for each set of items I in C and each grammar symbol X
such that goto(I, X) is not empty and not in C do
add goto(I, X) to C
until no more sets of items can be added to C
end

35
Construction of canonical-LR(1) parser

Algorithm: Constructing a canonical LR(1) parsing table.

Input: An augmenting grammar G0 .
Output: The canonical LR(1) parsing table functions action and goto for G0 .

1. Construct C = {I0 , · · · , In }, the collection of sets of LR(1) items for G0 .

2. State i constructed from Ii . The parsing actions for state i are determined as follows:
a) If [A → α · aβ, b] is in Ii and goto(Ii , a) = Ij , then set action[i, a] to “shift j.”
Here a must be a terminal.
b) If [A → α·, a] is in Ii , then set action[i, a] to “reduce A → α”;
here A may not be S 0 .
c) If [S 0 → S·, $] is in Ii , then set action[i, $] to “accept.”
If any conflicting actions are generated by the above rules, we say the grammar is not to be LR(1).
The algorithm fails to produce a parser in this case.
3. The goto transitions for state i are constructed for all nonterminals A using the rule:
If goto(Ii , A) = Ij , then goto[i, A] = j.
4. All entries not defined by rules (2) and (3) are made “error.”
5. The initial state of the parser is the one constructed from the set of items containing [S 0 → ·S, $].
end.

Construction of LALR Parsing Table

Algorithm: Constructing an LALR parsing table.

Input: A grammar G.
Output: The LALR parsing table for G.

1. Construct C = {I0 , · · · , In }, the collection of sets of LR(1) items for G.

2. Final all sets having the same core, and replace these sets by their union.
3. Let C 0 = {J1 , J2 , · · · , Jm } be the resulting sets of LR(1) items.
Action table is constructed in the same manner as in Algorithm for Canonical LR(1) parsing table.
4. goto table is constructed as follows.
Note that if Jq = I1 ∪ I2 ∪ · · · ∪ Ik , and for a non-terminal X,
goto(I1 , X) = Jp1 , goto(I2 , X) = Jp2 , · · · , goto(Ik , X) = Jpk ,
then make goto(Jq , X) = s where s = Jp1 ∪ Jp2 ∪ · · · ∪ Jpk .
(Note that Jp1 , · · · , Jpk all have the same core.) end.

36
Example 1: Consider the following grammar G0 .

(0) S0 → S
(1) S→L=R
(2) S→R
(3) L → ∗R
(4) L → id
(5) R→L

The canonical LR(1) collection for G0 is:

I0 : S 0 → ·S, $
S → ·L = R, $
S → ·R, $
L → · ∗ R, =
L → ·id, =
R → ·L, $
L → · ∗ R, $
L → ·id, $

I1 : S 0 → S·, $

I2 : S → L· = R, $
R → L·, $

I3 : S → R·, $

I4 : L → ∗ · R, =
L → ∗ · R, $
R → ·L, = /$
L → · ∗ R, = /$
L → ·id, = /$

I5 : L → id·, = /$

I6 : S → L = ·R, $
R → ·L, $
L → · ∗ R, $
L → ·id, $

I7 : L → ∗R·, = /$

37
I8 : R → L·, = /$

I9 : S → L = R·, $

I10 : R → L·, $

I11 : L → ∗ · R, $
R → ·L, $
L → · ∗ R, $
L → ·id, $

I12 : L → id·, $

I13 : L → ∗R·, $

Example 2:

Consider the following grammar G0 :

(0) S0 → S
(1) S → CC
(2) C → cC
(3) C→d

The canonical LR(1) collection for G0 is:

I0 : S 0 → ·S, $
S → ·CC, $
C → ·cC, c/d
C → ·d, c/d

I1 : S 0 → S·, $

I2 : S → C · C, $
C → ·cC, $
C → ·d, $

I3 : C → c · C, c/d
C → ·cC, c/d
C → ·d, c/d

38
I4 : C → d·, c/d

I5 : S → CC·, $

I6 : C → c · C, $
C → ·cC, $
C → ·d, $

I7 : C → d·, $

I8 : C → cC·, c/d

I9 : C → cC·, $

39
The transition for viable prefixes is:

I0 : goto(I0 , S) = I1 ; goto(I0 , C) = I2 ; goto(I0 , c) = I3 ; goto(I0 , d) = I4 ;

I2 : goto(I2 , C) = I5 ; goto(I2 , c) = I6 ; goto(I2 , d) = I7 ;

I3 : goto(I3 , c) = I3 ; goto(I3 , d) = I4 ; goto(I3 , C) = I8 ;

I6 : goto(I6 , C) = I9 ;

A. Canonical-LR(1) parsing table

State action goto

c d $ S C
0 s3 s4 1 2
1 acc
2 s6 s7 5
3 s3 s4 8
4 r3 r3
5 r1
6 s6 s7 9
7 r3
8 r2 r2
9 r2

B. LALR(1) parsing table

State action goto

c d $ S C
0 s36 s47 1 2
1 acc
2 s36 s47 5
36 s36 s47 89
47 r3 r3 r3
5 r1
89 r2 r2 r2

Note on LALR Parsing Table

Suppose we have an LR(1) grammar, that is, one whose sets of LR(1) items produce no parsing
action conflicts. If we replace all states having the same core with their union, it is possible that
the resulting union wil have a conflict, but it is unlikely for the following reasons.

Suppose in the union there is a conflict on lookahead a because there is an item [B → β · aγ, b]
calling for a reduction by A → α, and there is another item [B → β · aγ, b] calling for a shift. Then,
some set of items from which the union was formed has item [A → α·, a], and since the cores of

40
all these states are the same, it must have an item [B → β · aγ, c] for some c. But then this state
has the same shift/reduce conflict on a, and the grammar was not LR(1) as we assumed. Thus,
the merging of states with common cores can never produce a shift/reduce conflict that was not
present in one of the original states, because shift actions depend only on core, not the lookahead.

It is possible, however, that a merger will produce a reduce/reduce conflict as the following
example shows.

Example:

S0 → S
S → aAd | bBd | aBe | bAe
A→c
B→c

which generates the four strings acd, ace, bcd, bce. This grammar can be checked to be LR(1) by
constructing the sets of items. Upon doing so, we find the set of items {[A → c·, d], [B → c·, e]}
valid for viable prefix ac and {[A → c·, e], [B → c·, d]} valid for bc. Neither of these sets generates
a conflict, and their cores are the same. However, their union, which is
A → c·, d/e
B → c·, d/e
generates a reduce/reduce conflict, since reduction by both A → c and B → c are called for on
input d and e.

41
4 Turing Machine

M = (Q, Σ, Γ, δ, q0 , qaccept , qreject ). where

Σ⊆Γ

δ : Q × Γ → Q × Γ × {L, R}

qaccept 6= qreject

• L is Turing-decidable if some TM decides it (always halts with accept or reject).

• L is Turing-recognizable if some TM recognizes it (accept, reject, or loop).

Examples of Turing-decidable languages:

1. L = {w | |w| is a multiple of three. }

2. L = {an bm | n, m ≥ 1, n 6= m}

3. L = {an bn cn | n ≥ 1}

4. L = {ww | w ∈ {a, b}∗ }

n
5. L = {a2 | n ≥ 1}
2
6. L = {an | n ≥ 1}

7. L = {ai bj ck | i · j = k}

8. L = {an | n is a prime number. }

Hilbert’s 10th problem:

Let D = {P | P is a polynomial with an integral root. } Is D decidable?

• D is not Turing-decidable.

• D is Turing-recognizable.

Church’s Thesis: Turing machine is equivalent in computing power to the digital computers.

4.1 Turing Decidable Languages

1. ADF A = {< M, w > | M is a DFA that accepts w. } (Theorem 4.1, TEXT)

2. AN F A (Theorem 4.2, TEXT)

3. AREX = {< R, w > | R is a regular expression that generates w. } (Theorem 4.3, TEXT)

42
 4. EDF A = {< A > | A is a DFA such that L(A) = ∅. } (Theorem 4.4, TEXT)

5. EQDF A = {< A, B > | A and B are DFAs and L(A) = L(B). } (Theorem 4.5, TEXT)

6. ACF G = {< G, w > | G is a CFG that generates w. } (Theorem 4.7, TEXT)

7. ECF G = {< G > | G is a CFG and L(G) = ∅. } (Theorem 4.8, TEXT)

8. EQCF G = {< G, H > | G and H are CFGs and L(G) = L(H). } (Not decidable)

4.2 Diagonalization Method

Goal: Some languages are not Turing-decidable.

Definition: A set A is countable if and only if either A is finite or A has the same size of N . That
is, there exists a bijection f such that f : N → A.

example: N = {1, 2, 3, · · · , } and E = {2, 4, 6, · · · , }.

1. The set of rational numbers are countable. (Example 4.15, TEXT)

2. The set of real numbers are uncountable. (Theorem 4.17, TEXT)

3. The set of all strings over Σ is countable. (Proof: Corollary 4.18, TEXT)

4. The set of all TMs is countable. (Proof: Corollary 4.18, TEXT)

5. The set of all binary sequences of infinite length is uncountable. (Proof: Corollary 4.18, TEXT)

6. The set of all languages over Σ is uncountable. (Proof: Corollary 4.18, TEXT)

From 4 and 6 above, we have:

Theorem 4.1 There exists a language that is not Turing-recognizable. (Corollary 4.18, TEXT)

5 Turing Undecidable Problems and Reducibility

5.1 AT M

Let AT M = {< M, w > | M is a TM and M accepts w.}

Theorem 5.1 AT M is Turing undecidable.

Proof: Suppose AT M is decidable, and let H be a decider (i.e, H is a TM that decides AT M .) Thus,

accept if M accepts w
H(< M, w >) =
reject if M does not accept w

43
Now, we construct a new TM D with H as a subroutine:

Given a TM M , D take < M > as an input, and (1) run H on input < M, < M >>, (2) output
the opposite of what H outputs, i.e., if H accepts, then “reject” and if H rejects, then “accept.”

In summary, 
accept if M does not accepts < M >
D(< M >) =
reject if M accepts < M >

What happens when we run D with its own description < D > as input? In that case, we get

accept if D does not accepts < D >
D(< D >) =
reject if D accepts < D >

That is, no matter what D does, it is forced to do the opposite, a contradiction. Thus, neither TM
D nor TM H can exist. Therefore, AT M is not Turing-decidable.

However, AT M is Turing-recognizable.

Theorem 5.2 A language is Turing-decidable if and only if it is Turing-recognizable and also co-
Turing-recognizable.

Corollary 5.1 AT M is not Turing-recognizable.

5.2 Halting Problem

Let HALTT M = {< M, w > | M is a TM and M halts on w, }

Theorem 5.3 HALTT M is Turing undecidable.

Proof: Suppose HALTT M is Turing-decidable, and let R be a decider. We then use R as a

subroutine to construct a TM S that decides AT M as follows. S = “On input < M, w >”:

1. Run R on < M, w >

2. If R reject, reject

3. If R accepts, accept, simulate M until it halts.

4. If M has accepted, accept; if M has rejected, reject.

Clearly, if R decides HALTT M , then S decides AT M .Since AT M is undecidable, HALTT M must be

undecidable.

44
Theorem 5.4 (Theorem 5.2, TEXT) ET M is Turing undecidable.

Proof: Suppose ET M is decidable. Let R be a decider. We then construct two TMs M1 and S that
takes < M, w >, an input to AT M and run as follows.

M1 = “On input x”:

1. If x 6= w, reject.

2. If x = w, run M on w and accept if M does.

Note that M1 has w as a part of its description.

S = “On input < M, w >”:

1. Use the description of M and w to construct M1

2. Run R on input < M1 >

3. If R accepts, reject; if R rejects, accept.

Clearly, if ET M is TM decidable, then AT M is also TM decidable. However, we already proved

AT M is not TM decidable. Hence, ET M is TM undecidable.

5.3 More Turing undecidable Problems

• Post Correspondence Problem (PCP)

• Deciding whether an arbitrary CFG G is ambiguous

• Deciding whether L(G1 ) ∩ L(G2 ) = ∅ for arbitrary two CFG G1 and G2 .

6 NP-Completeness

6.1 Problem Transformation (Reduction)

Let A and B be two decision problems. We say problem A is transformed to B using a transfor-
mation algorithm f that takes IA (an arbitrary input to A) and computes f (IA ) (an input to B)
such that problem A with input IA is Y ES if and only if problem B with input f (IA ) is Y ES.

EXAMPLES:

• Hamiltonian Path Problem to Hamiltonian Cycle Problem

• Hamiltonian Cycle Problem to Hamiltonian Path Problem

45
 • 3COLORABILITY to 4COLORABILITY

• SAT to 3SAT

• ···

6.1.1 Upper Bound Analysis

Suppose A is a new problem for which we are interested in computing an upper bound, i.e., finding
an algorithm to solve A. Assume we have an algorithm ALGOB to solve B in O(nB ) time where
nB is the size of an input to B. We can then solve A using the following steps: (i) for an arbitrary
instance IA to A, transform IA to f (IA ) where f (IA ) is an instance to B; (ii) solve f (IA ) to B using
ALGOB ; (iii) if ALGOB taking f (IA ) as an input reports YES, we report IA is YES; otherwise,
NO.

6.1.2 Lower Bound Analysis

6.2 Satisfiability Problem

Let U = {u1 , u2 , · · · , un } be a set of boolean variables. A truth assignment for U is a function

f : U → {T, F }. If f (ui ) = T , we say ui is true under f ; and if f (ui ) = F , we say ui is false under
f . For each ui ∈ U , ui and ui are literals over U . The literal ui is true under f if and only if
the variable ui is false under f . A clause over U is a set of literals over U such as {u1 , u3 , u8 , u9 }.
Each clause represents the disjunction of its literals, and we say it is satisfied by a truth assignment
function if and only if at least one of its members is true under that assignment. A collection C
over U is satisfiable if and only if there exists a truth assignment for U that simultaneously satisfies
all the clauses in C.

Satisfiability (SAT) Problem

Given: a set U of variable and a collection C of clauses over U

Question: is there a satisfying truth assignment for C?

Example:
U = {x1 , x2 , x3 , x4 }
C = {{x1 , x2 , x3 }, {x1 , x3 , x4 }, {x2 , x3 , x4 }, {x1 , x2 , x4 }}.

The input to SAT is also given as a well-formed formula in conjunctive normal form (i.e.,
sum-of-product form:

w = (x1 + x2 + x3 )(x1 + x3 + x4 )(x2 + x3 + x4 )(x1 + x2 + x4 )

46
 Let x1 = T , x2 = F , x3 = F , x4 = T . Then, w = T .

Ans: yes

Reduction from SAT to 3SAT:

(1) (x1 ) → (x1 + a + b)(x1 + a + b)(x1 + a + b)(x1 + a + b)

(2) (x1 + x2 ) → (x1 + x2 + a)(x1 + x2 + a)

(3) (x1 + x2 + x3 + x4 + x5 ) → (x1 + x2 + a1 )(a1 + x3 + a2 )(a2 + x4 + x5 )

• 3SAT

• Not-All-Equal 3SAT: Each clause has at least one true literal and one false literal, i,e, not
all three literals can be true.

• One-In-Three 3SAT: Each clause has exactly one true literal and two false literals.

47
Definition:

P: a set of problems that can be solved deterministically in polynomial time.

NP: a set of problems that can be solved nondeterministically in polynomial time.

NPC: a problem B is called NP-complete or a NP-complete problem if (i) B ∈ N P , i.e., B can

be solved nondeterministically in polynomial time, and (ii) for all B 0 ∈ N P , B 0 ≤P B, i.e.,
any problem in NP can be transformed to B deterministically in polynomial time.

Cook’s Theorem: Every problem in NP can be transformed to the Satisfiability problem deter-
ministicall in polynomial time.

Note:

(i) The SAT is the first problem belonging to NPC.

(ii) To prove a new problem, say B, being NPC, we need to show (1) B is in NP and (2) any
known NPC problem, say B 0 , can be transformed to B deterministically in polynomial time.
(By definition of B 0 ∈ N P C, every problem in NP can be transformed to B in polynomial
time. As polynomial time transformation is transitive, it implies that every problem in NP
can be transformed to B in polynomial time.)

Theorem: P = N P if and only if there exists a problem B ∈ N P C ∩ P .

Proof: If P = N P , it is clear that every problem in N P C belongs to P . Now assume that there is
a problem B ∈ N P C that can be solved in polynomial time deterministically. Then by definition
of B ∈ N P C, any problem in N P can be transformed to B in polynomial time derterministically,
which can then be solved in polynomial time deterministically using the algorithm for B. Hence,
N P ⊆ P . Since P ⊆ N P , we conclude that P = N P , which completes the proof of the theorem.

48
 Problem Transformations:

Node Cover Problem:

Given: a graph G and an integer k,

Objective: to find a subset S ⊆ V such that (i) for each (u, v) ∈ E, either u or v (or both) is in S,
and (ii) |S| ≤ k.

Hamiltonian Cycle Problem:

Given: a graph G

Objective: to find a simple cycle of G that goes through every vertex exactly once.

Hamiltonian Path Problem:

Given: a graph G

Objective: to find a simple path of G that goes through every vertex exactly once.

Vertex Coloring Problem:

Given: a graph G and an integer k

Objective: to decide if there exists a proper coloring of V (i.e., a coloring of vertices in V such that
no two adjacent vertices receive the same color) using k colors.

• 3SAT ≤P N ode − Cover

Let W be an arbitrary well-formed formula in conjunctive normal form, i.e., in sum-of-product

form, where W has n variables and m clauses. We then construct a graph G from W as follows.

The vertex set V (G) is defined as V (G) = X ∪ Y , where X = {xi , xi | 1 ≤ i ≤ n} and

Y = {pj , qj , rj | 1 ≤ j ≤ m}. The edge set of G is defined to be E(G) = E1 ∪ E2 ∪, E3 , where
E1 = {(xi , xi ) | 1 ≤ i ≤ n}, E2 = {(pj , qj ), (qj , rj ), (rj , pj ) | 1 ≤ j ≤ m}, and E3 is defined to be a
set of edges such that pj , qj , and rj are respectively connected to c1j , c2j , and c3j , where c1j , c2j , and
c3j denote the first, second and the third literals in clause Cj .

For example, let W = (x1 + x2 + x3 )(x1 + x2 + x3 )(x1 + x2 + x3 ). Then G is defined such that
V (G) = {x1 , x1 , x2 , x2 , x3 , x3 , p1 , q1 , r1 , p2 , q2 , r2 , p3 , q3 , r3 } and E(G) = {(x1 , x1 ), (x2 , x2 ), (x3 , x3 ),
(p1 , q1 ), (q1 , r1 ), (r1 , q1 ), (p2 , q2 ), (q2 , r2 ), (r2 , p2 ), (p3 , q3 ), (q3 , r3 ), (r3 , p3 ), (p1 , x1 ), (q1 , x2 ), (r1 , x3 ),
(p2 , x1 ), (q2 , x2 ), (r3 , x3 ), (p3 , x1 ), (q3 , x2 ), (r3 , x3 )}.

49
 We now claim that there exists a truth assignment to make W = T if and only if G has a node
cover of size k = n + 2m.

To prove this claim, suppose there exists a truth assignment. We then construct a node cover S
such that xi ∈ S if xi = T and xi ∈ S if xi = F . Since at least one literal in each clause Cj must be
true, we include the other two nodes in each triangle (i.e., pj , qj , rj ) in S. Conversely, assume that
there exists a node cover of size n + 2m. We then note that exactly one of xi , xi for each 1 ≤ i ≤ n
must be in S, and exactly two nodes in pj , qj , rj for each 1 ≤ j ≤ m must be in S. It is then easy
to see the S must be such that at least one node in each pj , qj , rj for 1 ≤ j ≤ m must be connected
to a node xi or xi for 1 ≤ i ≤ n. Hence we can find a truth assignment to W by assigning xi true
if xi ∈ S and false xi ∈ S.

50