Introduction

Theory of Computation (CSE 315)

Papia Akter
Lecturer, Dept. of CSE
Prime University
 Purpose and motivation
This course is on the Theory of Computation, which tries to answer the following
questions:
• What are the mathematical properties of computer hardware and software?
• What is a computation and what is an algorithm? Can we give rigorous
mathematical definitions of these notions?
• What are the limitations of computers? Can “everything” be computed? (As we
will see, the answer to this question is “no”.)
So, the main purpose of the Theory of Computation is to Develop formal
mathematical models of computation that reflect real-world computers.
Nowadays, the Theory of Computation can be divided into the following three areas:
Complexity Theory, Computability Theory, and Automata Theory.

Complexity theory
The main question asked in this area is “What makes some problems computationally
hard and other problems easy?”
Informally, a problem is called “easy”, if it is efficiently solvable. Examples of “easy”
problems are (i) sorting a sequence of, say, 1,000,000 numbers, (ii) searching for a name in
a telephone directory, and (iii) computing the fastest way to drive from Ottawa to Miami.
On the other hand, a problem is called “hard”, if it cannot be solved efficiently, or if we
don’t know whether it can be solved efficiently. Examples of “hard” problems are (i) time
table scheduling for all courses at Carleton, (ii) factoring a 300-digit integer into its prime
factors, and (iii) computing a layout for chips in VLSI.
Central Question in Complexity Theory: Classify problems according to their degree of
“difficulty”. Give a rigorous proof that problems that seem to be “hard” are really “hard”.
 Computability Theory
In the 1930’s, G¨odel, Turing, and Church discovered that some of the fundamental
mathematical problems cannot be solved by a “computer”. (This may sound strange,
because computers were invented only in the 1940’s). An example of such a problem is
“Is an arbitrary mathematical statement true or false?” To attack such a problem, we
need formal definitions of the notions of
• computer,
• algorithm, and
• computation.
The theoretical models that were proposed in order to understand solvable and
unsolvable problems led to the development of real computers.

Central Question in Computability Theory: Classify problems as being solvable or

unsolvable.
 Automata Theory
Automata Theory deals with definitions and properties of different types of
“computation models”. Examples of such models are:
• Finite Automata: These are used in text processing, compilers, and hardware design.
• Context-Free Grammars: These are used to define programming languages and in
Artificial Intelligence.
• Turing Machines: These form a simple abstract model of a “real” computer, such as
your PC at home.

Central Question in Automata Theory: Do these models have the same power, or can
one model solve more problems than the other?
 Theory of Automata
Theory of automata is a theoretical branch of computer science and mathematical. It is
the study of abstract machines and the computation problems that can be solved using
these machines. The abstract machine is called the automata. The main motivation
behind developing the automata theory was to develop methods to describe and
analyse the dynamic behavior of discrete systems.
This automaton consists of states and transitions. The State is represented by circles,
and the Transitions is represented by arrows.
Automata is the kind of machine which takes some string as input and this input goes
through a finite number of states and may enter in the final state.
 Basic Terminologies
There are the basic terminologies that are important and frequently used in automata:
Symbols:
Symbols are an entity or individual objects, which can be any letter, alphabet or any
picture.
Example:
1, a, b, #
Alphabets:
Alphabets are a finite set of symbols. It is denoted by ∑.
Examples:
1.∑ = {a, b}
2.∑ = {A, B, C, D}
3.∑ = {0, 1, 2}
4.∑ = {0, 1, ....., 5]
5.∑ = {#, β, Δ}
String:
It is a finite collection of symbols from the alphabet. The string is denoted by w.
Example 1:
If ∑ = {a, b}, various string that can be generated from ∑ are {ab, aa, aaa, bb, bbb, ba,
aba.....}.
 A string with zero occurrences of symbols is known as an empty string. It is represented
by ε.

Length of string
The number of symbols present in the string is known as the Length of the string. The
length of the string is denoted by |w|.
For the string S = 'adadda', the length of the string is |w| = 6.
For the string S = '010', the length of string is |w| = 3
Language:
A language is a collection of appropriate string. A language which is formed over Σ can
be Finite or Infinite.
Example: 1
L1 = {Set of string of length 2} = {aa, bb, ba, bb} Finite Language
Example: 2
L2 = {Set of all strings starts with 'a'} = {a, aa, aaa, abb, abbb, ababb} Infinite Language

Kleene Closure/Star
•Definition − The Kleene star, ∑*, is a unary operator on a set of symbols or strings, ∑, that
gives the infinite set of all possible strings of all possible lengths over ∑ including λ.
•Representation − ∑* = ∑0 ∑1 ∑2 ……. where ∑p is the set of all possible strings of
length p.
•Example − If ∑ = {a, b}, ∑* = {λ, a, b, aa, ab, ba, bb,………..}
 Kleene Closure / Plus
•Definition − The set ∑+ is the infinite set of all possible strings of all possible lengths
over ∑ excluding λ.
•Representation − ∑+ = ∑1 ∑2 ∑3 …….
•∑+ = ∑* − { λ }
•Example − If ∑ = { a, b } , ∑+ = { a, b, aa, ab, ba, bb,………..}

Finite Automata
•Finite automata are used to recognize patterns.
•It takes the string of symbol as input and changes its state accordingly. When the desired
symbol is found, then the transition occurs.
•At the time of transition, the automata can either move to the next state or stay in the
same state.
•Finite automata have two states, Accept state or Reject state. When the input string is
processed successfully, and the automata reached its final state, then it will accept.
 Formal Definition of FA
A finite automaton is a collection of 5-tuple (Q, ∑, δ, q0, F), where:
• Q: finite set of states
• ∑: finite set of the input symbol
• q0: initial state
• F: final state
• δ: Transition function

Finite Automata Model:

Finite automata can be represented by input tape and finite control.
Input tape: It is a linear tape having some number of cells. Each input symbol is placed
in each cell.
Finite control: The finite control decides the next state on receiving particular input from
input tape. The tape reader reads the cells one by one from left to right, and at a time only
one input symbol is read.
 Types of Finite Automata:
There are two types of finite automata:
1.DFA(deterministic finite automata)
2.NFA(non-deterministic finite automata)
1. DFA
DFA refers to deterministic finite automata. Deterministic refers to the uniqueness of the
computation. In the DFA, the machine goes to one state only for a particular input character.
DFA does not accept the null move.
2. NFA
NFA stands for non-deterministic finite automata. It is used to transmit any number of states
for a particular input. It can accept the null move.
Some important points about DFA and NFA:
1.Every DFA is NFA, but NFA is not DFA.
2.There can be multiple final states in both NFA and DFA.
3.DFA is used in Lexical Analysis in Compiler.
4.NFA is more of a theoretical concept.
 Transition Diagram
A transition diagram or state transition diagram is a
directed graph which can be constructed as follows:
 There is a node for each state in Q, which is
represented by the circle.
 There is a directed edge from node q to node p
labeled a if δ(q, a) = p.
 In the start state, there is an arrow with no source.
 Accepting states or final states are indicating by a
double circle.
Some Notations that are used in the transition diagram:
Example of Transition Diagram :
DFA with ∑ = {0, 1} accepts all strings starting with 1.
Solution:
∑ = {0, 1}
L = { 1, 10, 11, 100, 101, 110, 111, ……}

Fig: Transition Diagram

The finite automata can be represented using a transition graph. In the above diagram, the
machine initially is in start state q0 then on receiving input 1 the machine changes its state to
q1. From q0 on receiving 0, the machine changes its state to q2, which is the dead state. From
q1 on receiving input 0, 1 the machine changes its state to q1, which is the final state. The
possible input strings that can be generated are 10, 11, 110, 101, 111......., that means all string
starts with 1.
 Transition Table
The transition table is basically a tabular representation of the transition function. It
takes two arguments (a state and a symbol) and returns a state (the "next state").
A transition table is represented by the following things:
• Columns correspond to input symbols.
• Rows correspond to states.
• Entries correspond to the next state.
• The start state is denoted by an arrow with no source.
• The accept state is denoted by a star.
Example 1:

Solution:
Transition table of given DFA is as follows:

Present State Next state for Input 0 Next State of Input 1

→q0 q1 q2
q1 q0 q2
*q2 q2 q2
Example 2:

Solution:
Transition table of given NFA is as follows:

Present State Next state for Input 0 Next State of Input 1

→q0 q0 q1
q1 q1, q2 q2
q2 q1 q3
*q3 q2 q2
 DFA (Deterministic finite automata)
•DFA refers to deterministic finite automata. Deterministic refers to the uniqueness of the
computation. The finite automata are called deterministic finite automata if the machine is
read an input string one symbol at a time.
•In DFA, there is only one path for specific input from the current state to the next state.
•DFA does not accept the null move, i.e., the DFA cannot change state without any input
character.
•DFA can contain multiple final states. It is used in Lexical Analysis in Compiler.

In the following diagram, we can see that from state q0

for input a, there is only one path which is going to q1.
Similarly, from q0, there is only one path for input b
going to q2.
 Formal Definition of DFA
A DFA is a collection of 5-tuples same as we described in the definition of FA.
1. Q: finite set of states
2. ∑: finite set of the input symbol
3. q0: initial state
4. F: final state
5. δ: Transition function

Transition function can be defined as:

δ: Q x ∑→Q
 Graphical Representation of DFA
A DFA can be represented by digraphs called state diagram. In which:
1. The state is represented by vertices.
2. The arc labeled with an input character show the transitions.
3. The initial state is marked with an arrow.
4. The final state is denoted by a double circle.

Example 1:
1. Q = {q0, q1, q2}
2. ∑ = {0, 1}
3. q0 = {q0}
4. F = {q2}
5. Transition Table:

Present State Next state for Input 0 Next State of Input 1

→q0 q0 q1
q1 q2 q1
*q2 q2 q2

Solution:
Transition Diagram:
Give a Formal Definition of DFA from the following language.

Example1: DFA with ∑ = {0, 1} accepts all starting with 0.

Solution:
∑ = {0, 1}
L= {0, 00, 01, 000, 001, 010, 011.

0
A B 0,1

C 0,1
Formal Definition of DFA:
A DFA is a collection of 5-tuples same as we described in the definition of FA.
Q = finite set of states
= {q0, q1, q2}
∑ = finite set of the input symbol
= {0, 1}
q0 = initial state
= {q0}
F = final state
= {q1}
δ = Transition function
Transition function can be defined as:
δ: Q x ∑→Q
Transition Table:

Present State Next state for Input 0 Next State of Input 1

→q0 q0 q2
*q1 q1 q1
q2 q2 q2
Example 2: DFA with ∑ = {0, 1} accepts all ending with 0.
Solution:
∑ = {0, 1}
L={0,00,10,100,110,….}

Formal Definition of DFA:

A DFA is a collection of 5-tuples same as we described in the
definition of FA.
Q = finite set of states
= {q0, q1}
∑ = finite set of the input symbol
= {0, 1}
q0 = initial state
= {q0}
F = final state
= {q1}
δ = Transition function
Transition function can be defined as:
δ: Q x ∑→Q

Transition Table:

Present State Next state for Input 0 Next State of Input 1

→q0 q1 q0
*q1 q1 q0
 Some Examples of DFA
Example 1:
Design a DFA with ∑ = {0, 1}, accepts those string which starts with 1.
Solution:
∑ = {0, 1}, 1
L={1,10,11,111,….}
A B 0,1

C 0,1
Example 2:
Design a DFA, which accepts set of all strings over ∑ = {0, 1}, where each string starts with
‘00’
0 0
Solution: A B C 0,1
∑ = {0, 1},
1 1
L= {00,000,001,0000,0001,…}
D 0,1
Example 3:
Design a DFA with ∑ = {0, 1} accepts those set of strings which starts with ‘001’
Solution:
0 0 1 0,1
∑ = {0, 1} A B C D
L = {001,0010,0011,00100, 00101,…} 1
1
0
E 0,1
Example 4:
Design a DFA with ∑ = {0, 1} accepts those set of strings which starts with ‘110’
Solution:
∑ = {0, 1} 1 1 0 0,1
L = {001,0010,0011,00100, 00101,…}
A B C D
0
0
1
E 0,1