Introduction to Algorithms

The Role of Algorithms in Computing

Computational problems
 A computational problem specifies an
input-output relationship
 What does the input look like?
 What should the output be for each input?
 Example1:
 Input: an integer number n
 Output: Is the number prime?
 Example2:
 Input: A list of names of people
 Output: The same list sorted alphabetically
2
 Algorithm
 The word Algorithm means “a process or set
of rules to be followed in calculations or other
problem-solving operations”.
 Therefore Algorithm refers to a set of
rules/instructions that step-by-step define
how a work is to be executed upon inorder to
get the expected results.

3
Algorithms
 A tool for solving a well-specified
computational problem

Input Algorithm Output

 Algorithms must be:

 Correct: For each input produce an appropriate output
 Efficient: run as quickly as possible, and use as little
memory as possible – more about this later

4
.Algorithms Cont
 A well-defined computational procedure that
takes some value, or set of values, as input and
produces some value, or set of values, as output.

 Written in a pseudo code which can be

implemented in the language of programmer’s
choice.

5
Correct and incorrect algorithms
 Algorithm is correct if, for every input instance, it ends
with the correct output. We say that a correct algorithm
solves the given computational problem.

 An incorrect algorithm might not end at all on some input

instances, or it might end with an answer other than the
desired one.

 We shall be concerned only with correct algorithms.

6
Problems and Algorithms
 We need to solve a computational problem
 “Convert a weight in pounds to Kg”

 An algorithm specifies how to solve it, e.g.:

 1. Read weight-in-pounds
 2. Calculate weight-in-Kg = weight-in-pounds *
0.455
 3. Print weight-in-Kg

 A computer program is a computer-

executable description of an algorithm

7
The Problem-solving Process

Analysis
Problem
specification
Design

Algorithm
Implementation
Program

Compilation
Executable
(solution)

8
From Algorithms to Programs
Problem
Algorithm:
Algorithm A sequence
of instructions describing
how to do a task (or
process)

C++/C/Java Program
9
Practical Examples(Applications)
 Internet and Networks
􀂄 The need to access large amount of information with the shortest
time.
􀂄 Problems of finding the best routs for the data to travel.
􀂄 Algorithms for searching this large amount of data to quickly find
the pages on which particular information resides.
 DNA Analysis in Medical Field
 Electronic Commerce
􀂄 The ability of keeping the information (credit card numbers,
passwords, bank statements) private, safe, and secure.
􀂄 Algorithms involves encryption/decryption techniques.

10
Hard problems
 We can identify the Efficiency of an algorithm
from its speed (how long does the algorithm take
to produce the result).
 Some problems have unknown efficient solution.
 These problems are called NP-complete problems.
 If we can show that the problem is NP-complete,
we can spend our time developing an efficient
algorithm that gives a good, but not the best
possible solution.

11
Components of an Algorithm
 Variables and values
 Instructions
 Sequences
 A series of instructions
 Procedures
 A named sequence of instructions
 we also use the following words to refer to a
“Procedure” :
 Sub-routine
 Module
 Function
12
Components of an Algorithm Cont.
 Selections
 An instruction that decides which of two possible
sequences is executed
 The decision is based on true/false condition
 Repetitions
 Also known as iteration or loop
 Documentation
 Records what the algorithm does

13
A Simple Algorithm
 INPUT: a sequence of n numbers
 T is an array of n elements
 T[1], T[2], …, T[n]
 OUTPUT: the smallest number among them

min = T[1]
for i = 2 to n do
{
if T[i] < min
min = T[i]
}
Output min

 Performance of this algorithm is a function of n

14
Analysis of algorithms
 The theoretical study of computer-program
performance and resource usage.
 Algorithm features
 modularity
 correctness
 maintainability
 user-friendliness
 programmer time
 extensibility

15
Analysis of algorithms
 The term "analysis of algorithms" was coined
by Donald Knuth.
 Algorithm analysis is an important part of
computational complexity theory, which
provides theoretical estimation for the required
resources of an algorithm to solve a specific
computational problem.
 Analysis of algorithms is the determination of
the amount of time and space resources
required to execute it.
16
Time Complexity
 Time complexity of an algorithm quantifies
the amount of time taken by an algorithm to
run as a function of the length of the input. 

17
Space Complexity
 Space complexity of an algorithm quantifies
the amount of space or memory taken by an
algorithm to run as a function of the length of
the input.
 Time and space complexity depends on lots
of things like hardware, operating system,
processors, etc.
 But we will only consider the execution time
of an algorithm.

18
 Lets start with a simple example.
 Suppose you are given an array A and an
integer x and you have to find if x exists in
array A.
 Simple solution to this problem is traverse the
whole array A and check if the any element is
equal to x
 for i : 1 to length of A
if A[i] is equal to x
return TRUE
return FALSE

19
Types of Analysis
 Analysis of algorithm is the process of
analyzing the problem-solving capability of
the algorithm in terms of the time and size
required
 Worst-case − The maximum number of
steps taken on any instance of size a.
 Best-case − The minimum number of steps
taken on any instance of size a.
 Average case − An average number of steps
taken on any instance of size a.
20
-Time Complexity Analysis
Linear Search time complexity analysis is done
below-
  Best case- 
 In the best possible case,
 The element being searched may be found at
the first position.
 In this case, the search terminates in success
with just one comparison.
 Thus in best case, linear search algorithm takes
O(1) operations.
21
Worst Case-Linear Search
  In the worst possible case,
 The element being searched may be present at the last position or
not present in the array at all.
 In the former case, the search terminates in success with n
comparisons.
 In the later case, the search terminates in failure with n
comparisons.
 Thus in worst case, linear search algorithm takes O(n) operations.
  
 Thus, we have-
  
 Time Complexity of Linear Search Algorithm is O(n).
 Here, n is the number of elements in the linear array.

22
 To solve a problem, we need to consider time
as well as space complexity as the program
may run on a system where memory is
limited but adequate space is available or
may be vice-versa.

23
 In this context, if we compare bubble
sort and merge sort.
 Bubble sort does not require additional
memory, but merge sort requires additional
space.
 Though time complexity of bubble sort is
higher compared to merge sort, we may need
to apply bubble sort if the program needs to
run in an environment, where memory is very
limited.

24
Algorithm Design Techniques
 1. Divide and Conquer Approach: It is a top-
down approach. The algorithms which follow the
divide & conquer techniques involve three steps:
 Divide the original problem into a set of
subproblems.
 Solve every subproblem individually, recursively.
 Combine the solution of the subproblems (top
level) into a solution of the whole original
problem.

25
Greedy Technique .2
 Greedy method is used to solve the
optimization problem.
 An optimization problem is one in which we
are given a set of input values, which are
required either to be maximized or minimized
(known as objective), i.e. some constraints or
conditions.

26
 :Dynamic Programming .3
 Dynamic Programming is a bottom-up
approach we solve all possible small
problems and then combine them to obtain
solutions for bigger problems.
 Eg: Matrix multiplication

27
:Branch and Bound .4
  In Branch & Bound algorithm a given
subproblem, which cannot be bounded, has
to be divided into at least two new restricted
subproblems.
 Used for solving combinatorial optimization
problems

28
Randomized Algorithm.5
  A randomized algorithm uses a random
number at least once during the computation
make a decision.

29
:Backtracking Algorithm .6
 Backtracking Algorithm tries each possibility
until they find the right one.
 It is a depth-first search of the set of possible
solution.
 During the search, if an alternative doesn't
work, then backtrack to the choice point, the
place which presented different alternatives,
and tries the next alternative.

30