UNIT 1 : INTRODUCTION TO ALGORITHM

UNIT 1 : INTRODUCTION TO ALGORITHM

DEFINITION OF ALGORITHM
An algorithm is a well-organized, pre-arranged and well-defined step by step
procedure solve a given problem by receiving input and providing output.
Algorithms are used to solve complex problems. An algorithm is said to be
accurate only when it provides the exact output. Every problem can be solved using
various different methods.

The best method or the best algorithm depends on the following factors.
1. An algorithm should be easy to understand, code and debug (design of algorithms).
2. An algorithm should make use of the computer resources efficiently (analysis of
Algorithms).

CHARACTERISTICS OF AN ALGORITHM

1. Input: Zero or more input values are externally supplied initially before the
algorithm begins or dynamically as algorithm is executed.
2. Output: At least one output value is produced. One or more values are produced.
3. Definiteness: Each instruction should be clear and unambiguous.
4. Finiteness: For all cases, the algorithm terminates after a finite number of steps.
5. Effectiveness: Every instruction must be very basic so that it can be carried out
easily in a finite amount of time.

NOTION OF ALGORITHM

1. Algorithm should not be ambiguous.

2. The range of inputs for which an algorithm works has to be specified carefully.
3. The same algorithm can be represented in several different ways.
4. Several algorithms for solving the same problem may exist.
5. Algorithms for the same problem can be based on different ideas and with
different speeds.

FUNDAMENTALS OF ALGORITHMIC PROBLEM SOLVING

The sequence of steps that are followed in the design and analysis of algorithms are
listed below.

i. Understanding the problem (Statement of the problem)

ii. Ascertaining the capabilities of a computational device
iii. Choosing between Exact and Approximate Algorithms
iv. Deciding on appropriate Data Structures
v. Designing an algorithm
vi. Method of specifying an algorithm

S LAVANYA , DEPARTMENT OF COMPUTER APPLICATION, CHICKBALLAPUR 1

 UNIT 1 : INTRODUCTION TO ALGORITHM

vii. Proving the correctness of an algorithm (Algorithm Validation)

viii. Analyzing an algorithm (Performance Analysis)
ix. Coding an algorithm (Writing a computer program)
x. Testing the computer program (Program Testing)

i. Understanding the problem (Statement of the problem)

The problem description should be read carefully ,understood completely and

clearly.
The problem should be clarified. Missing information should be identified. The
inputs to be given should be identified. The output that is to be obtained is to be
determined. The exact range of inputs should be specified because the algorithm
may work correctly.

ii. Ascertaining the capabilities of a computational device

It is important to ascertain the capabilities of the computational device for which the
algorithm is intended. This is based on the following factors.

• Architecture of the device

• Speed of the device
• Memory space

1.Architecture of the device

Algorithms may be written for devices based on Von Neumannn architecture where
instructions are executed sequentially, one operation at a time. These are sequential
algorithms.
In newer computers, operations take place concurrently, i.e. in parallel. Such
algorithms are called parallel algorithms.

2.Speed of the device

In military applications, communication applications and real time applications, where
time is a critical factor, the speed of the device is important and is to be considered.

3. Memory space
In applications that involve large volumes of data and complex problems, it is
important to choose a computer that has large memory space.

iii. Choosing between Exact and Approximate Algorithms

There are two types of algorithms based on whether we want the output to be exact
solution or an approximate solution: 1. EXACT 2.APPROXIMATE

S LAVANYA , DEPARTMENT OF COMPUTER APPLICATION, CHICKBALLAPUR 2

 UNIT 1 : INTRODUCTION TO ALGORITHM

1.Exact algorithm: Exact algorithms are the algorithms that solve the problem
exactly.
Examples : algorithms for sorting, searching, pattern matching, traveling salesman
problem, etc, that give the exact result.

2.Approximate algorithm : Approximate algorithms solve the problem

approximately.
Examples are the algorithms for finding square root, solving non-linear equations,
evaluating definite integrals, etc.

iv. Deciding on appropriate Data Structures

Proper data structure is to be chosen so that with well defined efficient algorithm and
suitable data structure, efficient computer programs can be developed.

Algorithms + Data Structures = Programs

V. Designing an algorithm

An algorithm design technique (strategy or paradigm) is a general approach to

solving problems algorithmically that is applicable to a variety of problems from
different areas of computing.

The various algorithms design techniques are:

a. Brute force
b. Divide and conquer
c. Decrease and conquer
d. Transform and conquer
e. Greedy method
f. Dynamic programming
g. Iterative improvement

vi. Method of specifying an algorithm

An algorithm can be specified in the following ways.

• Natural language
• Flowchart
• Pseudocode

Natural Language
Algorithms that are written in English like statements with few mathematical
expressions are not preferred because of the inherent ambiguity in natural language
representation. So it is rarely used.

S LAVANYA , DEPARTMENT OF COMPUTER APPLICATION, CHICKBALLAPUR 3

 UNIT 1 : INTRODUCTION TO ALGORITHM

Flowchart
It is a pictorial representation of an algorithm. All the steps of an algorithm are drawn
using geometric shapes such as boxes, circles, connecting arrows etc. that describe
the steps of algorithm. Flowcharts are used to help the programmer to understand
the logic of the program.

Pseudocode
It is a method of representing an algorithm using natural language and
programming language constructs. It is more precise than a natural language.
There is no single form of pseudocode.

STEP 1: Start
STEP 2: Read n
STEP 3: Initialize sum=0
STEP 4: for i  1 to n do
STEP 5: sum  sum + i
STEP 6: Print ” sum of ”, n, ”natural numbers is”, sum
STEP 7: Stop

vii. Proving the correctness of an algorithm (Algorithm Validation)

The correctness of the algorithm can be proved by finding whether the algorithm
gives a required result for every valid input in a finite amount of time. This process
is called Algorithm Validation.
The purpose of validation is to assure that the algorithm will work correctly
independent of the programming language that is to be used to implement it.

viii. Analyzing an algorithm (Performance Analysis)

Analyzing an algorithm refers to calculating or guessing resources (computer

memory, processing time) needful for the algorithm.
Performance analysis refers to the tasks of determining how much
computing time and storage an algorithm requires.

The time and space the algorithm uses are two major measures of the efficiency of
an algorithm. They are
a. Time efficiency
b. Space efficiency

Time efficiency indicates how fast the algorithm runs.

Space efficiency indicates how much extra memory the algorithm needs.

Simplicity
Simpler algorithms are easier to understand and easier to program.

S LAVANYA , DEPARTMENT OF COMPUTER APPLICATION, CHICKBALLAPUR 4

 UNIT 1 : INTRODUCTION TO ALGORITHM

Simpler algorithms are sometimes more efficient than more complicated algorithms.

Generality
The algorithm developed for the problem that is general and the range of inputs it
accepts are to be considered.

Analysis of algorithm can be of two phases

1) Priori analysis
2) Posteriori analysis

ix. Coding an algorithm (Writing a computer program)

The algorithm should be converted into a computer program. Usage of different

languages such as C or C++ or C# for implementing the algorithm results in different
memory requirements and affects the speed of the program. The syntax varies from
one language to another.

X. Testing the computer program

Testing is the process of identifying errors in a program (debugging) and finding how
well the program works (performance measurement).

Profiling or performance measurement is the process of executing a correct

program on data sets and measuring the time and space it takes to compute the
results.

Comparison between Analysis and Profiling :

Analysis Profiling
Analysis is the process of determining Profiling is the process of executing the
how much computing time and storage correct program on data sets and
an algorithm will require. measuring the time and space it takes to
compute the results.
This is independent of machine, This is dependent on machine,
programming language and will not programming language and the compiler
involve the execution of program. used.
This allows making quantitative It helps in finding the logical places to
judgements about the value of one perform optimization and also confirms
algorithm over the other. the previously done analysis.

FUNDAMENTALS OF ANALYSIS OF ALGORITHMS

'Analysis' defines the separation of an intellectual or substantial whole into its

constituent parts for individual study.
Once an algorithm is developed, we would like to judge certain criteria.

S LAVANYA , DEPARTMENT OF COMPUTER APPLICATION, CHICKBALLAPUR 5

 UNIT 1 : INTRODUCTION TO ALGORITHM

The questions that arise are:

i. Does the algorithm do what it is supposed to do?

ii. Does it work according to the original specifications of the task?
iii. Is there any documentation mentioning about how to use it and how it works?
iv. Do the procedures developed perform logical sub-functions?
V. Is the code readable?

The other criteria for judging algorithms that have a more direct relationship to
performance are
• Running time (computing time)
• Memory space (storage requirements)

Performance evaluation can be divided into two major phases

• A priori estimates (performance analysis)
• A posteriori testing (performance testing)

Efficiency can be studied in precise quantitative terms. There are two kinds of
efficiency.
1. Time efficiency (Time complexity)
2. Space efficiency (Space complexity)

There are three different notations

1. Big Oh O
2. Big Omega Ω
3. Big Theta

ANALYSIS FRAMEWORK

Efficiency :The main purpose of algorithm analysis is to design most efficient

algorithms. Efficiency can be studied in precise quantitative terms. There are two
kinds of efficiency

1. Space efficiency (Space complexity)

2. Time efficiency (Time complexity)

1. Space efficiency
The space efficiency of an algorithm is the amount of memory required to run the
program completely and efficiently.

2. Time efficiency
The time efficiency of an algorithm is the amount of time taken to execute an
algorithm.

S LAVANYA , DEPARTMENT OF COMPUTER APPLICATION, CHICKBALLAPUR 6

 UNIT 1 : INTRODUCTION TO ALGORITHM

Measuring Input size

All algorithms run longer on larger inputs. It takes longer to sort larger arrays,
multiply larger matrices, and so on. So it is important to investigate an algorithm's
efficiency as a function of some parameter 'n' indicating the algorithm's input size.

Units for measuring running times

For the analysis of time efficiency of algorithm, it is measured by counting the number
of times the algorithm's basic operation is executed on inputs of size 'n'.

• Dependence on the speed of the computer

• Dependence on the quality of program implementing the algorithm
• Dependence on the compiler used in generating machine code

Order of Growth

1) 1 : Running time of a program is constant.

2) log n : Running time of a program is logarithmic. This running time occurs in

programs that solve larger problems by reducing the problem size by a constant
factor at each iteration of the loop (ex. Binary search)

3) n: Running time of a algorithm is linear. (Linear search)

4) n log n : Running time of a algorithm is n log n. (divide and conquer algorithms

like quick sort, merge sort)

5) n² : Running time of a algorithm is quadratic. The algorithms will have 2 loops like
bubble sort, selection sort, addition and subtraction of two matrices etc.

6) n³ : Running time of algorithm is cubic. The algorithms will have 3 loops, like
matrix multiplication, Gauss Elimination method of solving simultaneous equations

7) 2n : Running time of an algorithm is exponential. Algorithms that generate subsets

of a given set fall under this category.

8) n! : Running time of an algorithm is factorial. Algorithms that generate all

permutations of set using brute force technique fall under this category.

Note: All these functions can be ordered according to their order of Growth (from
lowest to highest) 1 < log n < n < n log n < n² < n³ < 2n < n!

S LAVANYA , DEPARTMENT OF COMPUTER APPLICATION, CHICKBALLAPUR 7

 UNIT 1 : INTRODUCTION TO ALGORITHM

WORST-CASE, BEST-CASE AND AVERAGE-CASE EFFICIENCIES

1.Worst-Case Efficiency:
The efficiency of an algorithm for the input of size n for which the algorithm takes
longest time to execute among all possible inputs is called Worst Case efficiency.

2.Best-Case Efficiency:
The efficiency of an algorithm for the input of size n for which the algorithm takes
least time to execute among all possible inputs of that size is called Best Case
Efficiency.

3.Average-Case Efficiency:
The efficiency of an algorithm for the input of size n for which the algorithm takes
average time to execute among all possible inputs of that size is called Average
Case Efficiency.

S LAVANYA , DEPARTMENT OF COMPUTER APPLICATION, CHICKBALLAPUR 8