Chapter 1

Algorithms Basic

An algorithm is an unambiguous specification of how to solve a class of problems. The

performance of an algorithm is measured on the basis of following properties :
Time Complexity - The total amount of time required by an algorithm to complete its
execution.
Space Complexity - Space complexity is an amount of memory used by the algorithm, to
execute it completely.

Asymptotic Notation is used to describe the running time of an algorithm - how much time an
algorithm takes with a given input, n.
There are three different notations:
big O - for the worst case running time
big Theta (Θ) - the running time is the same for all cases (average case).
big Omega (Ω) - for the best case running time

Following is a list of some common asymptotic notations −

constant Ο(1)

logarithmic Ο(log n)

linear Ο(n)

quadratic Ο(n2)

cubic Ο(n3)

polynomial nΟ(1)
exponential 2O(n)

How to Write an Algorithm?

There are no well-defined standards for writing algorithms. Rather, it is problem and resource
dependent. Algorithms are never written to support a particular programming code.
As we know that all programming languages share basic code constructs like loops (do, for,
while), flow-control (if-else), etc. These common constructs can be used to write an algorithm.

Problem − Design an algorithm to add two numbers and display the result. Writing step
numbers, is optional.
 Algorithm Analysis
Efficiency of an algorithm can be analyzed at two different stages, before implementation
and after implementation. They are the following –

A Priori Analysis − This is a theoretical analysis of an algorithm. Efficiency of an algorithm

is measured by assuming that all other factors, for example, processor speed, are constant and
have no effect on the implementation.
A Posterior Analysis − This is an empirical analysis of an algorithm. The selected algorithm is
implemented using programming language. This is then executed on target computer machine. In
this analysis, actual statistics like running time and space required, are collected.

Chapter 2

Sorting Algorithms

A sorting algorithm is used to arrange elements of an array/list in a specific order. There are many
types of sorting algorithms.
 Bubble Sort
 Selection Sort
 Insertion Sort
 Merge Sort
 Quicksort
 Counting Sort
 Radix Sort

Quick Sort
Quick sort is based on the divide-and-conquer approach based on the idea of choosing one element
as a pivot element and partitioning the array around it such that: Left side of pivot contains all the
elements that are less than the pivot element Right side contains all elements greater than the pivot.
Example. Sort {1, 12, 5, 26, 7, 14, 3, 7, 2} using quicksort.
Ans:
Notice, that we show here only the first recursion step, in order not to make example too long. But,
in fact, {1, 2, 5, 7, 3} and {14, 7, 26, 12} are sorted then recursively.
 Practice Exercise
a. Sort {24,9,29,14,19,27} using quicksort where pivot=24.
b. Sort {4,2,6,5,3,9} using quicksort where pivot=5.

Counting Sort
Counting sort is a sorting algorithm that sorts the elements of an array by counting the number of
occurrences of each unique element in the array. The count is stored in an auxiliary array and the
sorting is done by mapping the count as an index of the auxiliary array.
Example- Sort {4,2,2,8,3,3,1} using counting sort.
Ans:
Step1: Find max element from the given array.
4 2 2 8 3 3 1
Here, max=8
Step2: Initialize count array of length max+1 with all elements 0.
0 0 0 0 0 0 0 0 0
0 1 2 3 4 5 6 7 8
Step 3: Store the count of each element at their respective index in count array.
0 1 2 2 1 0 0 0 1
0 1 2 3 4 5 6 7 8
Step4: Store cumulative sum of the elements of the count array.
0 1 3 5 6 6 6 6 7
0 1 2 3 4 5 6 7 8
Step5: Find the index of each element of the original array in the count array. This gives the
cumulative count. After placing each element at its correct position, decrease its count by one.

Time Complexity:
Worst Case Complexity: O(n+k)
Best Case Complexity: O(n+k)
Average Case Complexity: O(n+k)
where n is the number of elements in the array and k is the range of the elements.
Practice Exercise
a. Sort {2,5,0,3,2,3,0,3} using countingsort.
 Chapter 3

Divide and Conquer

In divide and conquer approach, the problem in hand, is divided into smaller sub-problems and then
each problem is solved independently. When we keep on dividing the sub-problems into even smaller
sub-problems, we may eventually reach a stage where no more division is possible. Those "atomic"
smallest possible sub-problem (fractions) are solved. The solution of all sub- problems is finally
merged in order to obtain the solution of an originalproblem.

Examples of Divide and Conquer Algorithms

Merge Sort
Quick Sort
Binary Search
Strassen's Matrix Multiplication
Closest pair (points)

Strassen's Matrix Multiplication

 Chapter 4

Greedy Algorithms

Greedy algorithms try to find a localized optimum solution, which may eventually lead to globally
optimized solutions. However, generally greedy algorithms do not provide globally optimized
solutions.
To solve a problem based on the greedy approach, there are two stages
Feasible solution
 Optimization

Examples of Greedy Algorithms

Kruskal’s Minimum Spanning Tree (MST)
Prim’s Minimum Spanning Tree
Shortest Path Algorithms (Dijkstra, Bellman Ford, Floyd Warshall )
Huffman Coding
Travelling Salesman Problem
Job Scheduling Problem

Shortest Path Problem

 Shortest path problem is a problem of finding the shortest path(s) betweenvertices of a
given graph.
 Shortest path between two vertices is a path that has the least cost ascompared to all other
existing paths.
Shortest path algorithms have a wide range of applications such as in-
Google Maps
Road Networks
Logistics Research
Single-Pair Shortest Path Problem

It is a shortest path problem where the shortest path between a given pairof vertices is
computed.
A* Search Algorithm is a famous algorithm used for solving single-pairshortest
path problem.

Single-Source Shortest Path Problem

It is a shortest path problem where the shortest path from a given sourcevertex to all
other remaining vertices is computed.
Dijkstra’s Algorithm and Bellman Ford Algorithm are the famous algorithmsused for
solving single-source shortest path problem.

Single-Destination Shortest Path Problem

It is a shortest path problem where the shortest path from all the vertices toa single
destination vertex is computed.
By reversing the direction of each edge in the graph, this problem reducesto single-
source shortest path problem.
Dijkstra’s Algorithm is a famous algorithm adapted for solving single-destination
shortest path problem.

All Pairs Shortest Path Problem

It is a shortest path problem where the shortest path between every pair ofvertices is
computed.
Floyd-Warshall Algorithm and Johnson’s Algorithm are the famous
algorithms used for solving All pairs shortest path problem.

Dijkstra Algorithm
 Dijkstra Algorithm is a very famous greedy algorithm.
 It is used for solving the single source shortest path problem.
 It computes the shortest path from one particular source node to all otherremaining
nodes of the graph.
Conditions

It is important to note the following points regarding Dijkstra Algorithm-

Dijkstra algorithm works only for connected graphs.
Dijkstra algorithm works only for those graphs that do not contain any negative
weight edge.
It only provides the value or cost of the shortest paths.
Dijkstra algorithm works for directed as well as undirected graphs.
Limitations
The graph should be weighted.
The weights should be non-negative.
Time Complexity
Time complexity O(ElogV)
This time complexity can be reduced to O(E+VlogV) using Fibonacci heap.
O(V + E*log(V)), when priority queue is used
Problem: Using Dijkstra’s Algorithm, find the shortest distance from source vertex ‘S’ to remaining
vertices in the following graph-

s A B c d e
0 infinite infinite infinite infinite infinite
S 0 1 5 infinite infinite infinite
A 0 1 3 3 2 infinite
D 0 1 3 3 2 4
B 0 1 3 3 2 4
C 0 1 3 3 2 4
E 0 4

The shortest path is S->A->D->E

 Why does Dijkstra’s Algorithm fail on negative weights?
In a Dijkstra algorithm, once a node is marked as visited it cannot be reconsidered even if there is
another path with less cost or distance. This issue arises only if there exists a negative weight or edge
in the graph. For this reason, this algorithm fails to find the minimum distance in case of negative
weights.

In dijkstra algorithm, the shortest distance from A –> B is 5 but if traveled the distance via node C that is
the path A –> C –> B the distance will be as 3. As 3 is less than 5, but Dijkstra’s algorithm gives the
incorrect answer as 5, which is not the shortest distance. Therefore Dijkstra’s Algorithm fails for
negative cases.
  Does Dijkstra's algorithm work with negative weights?
May or may not , Dijkstra also works for some of the graphs with negative weighted cycle too as long
as the element that is already considered shortest is not relaxedanymore.

Actually , Dijkstra's algorithm fails to work for most of the negative weight edged graphs , but
sometimes it works with some of the graphs with negative weighted edges too provided the graph
doesn't have negative weight cycles.

Bellman Ford's Algorithm

Bellman Ford algorithm helps us find the shortest path from a vertex to all other vertices of a
weighted graph. It is similar to Dijkstra's algorithm but it can work with graphs in which edges can
have negative weights.
 What is a Negative cycle?
A negative cycle in a weighted graph is a cycle whose total weight is negative.
Let’s see two examples:

Consider the following graph: Weight of the graph is equal to the weight of its edges. So, weight = 1
+ 2 + 3= 6, Positive value, so we don’t have a negative cycle.

Let us consider another graph: Weight of the graph is equal to the weight of its edges.So, weight
= 3 + 2 + (-6)= -1, Negative value, so we have a negative cycle.
Point to note

If there are N vertices then we will iterate N - 1 times to get the shortest distance.
And we do the Nth iteration to check if there is any negative cycle.
If there is no change in the value of N-1 iteration then the graph has no negative
 cycle which means we can find the shortest path. Otherwise the graph has negative
cycle and we cannot find the shortest path
Complexity

Bellman Ford algorithm has a time complexity of O(VE).

The space complexity is O(V).
Applications

For calculating shortest paths in routing algorithms

For finding the shortest path
Limitation
It can only work for directed graphs
If there are negative cycles (reachable from the source), Bellman-Ford can be
considered to fail.
 What are the differences between Bellman Ford’s and Dijkstra’s algorithms?

Bellman Ford’s Dijkstra’s

Algorithm Algorithm
Bellman Ford’s Algorithm Dijkstra’s Algorithm doesn’t
works whenthere is negative work whenthere is negative
weight edge. weight edge.
It is more time It is less time
consuming than consuming.
Dijkstra’s algorithm.
Its time The time
complexity is complexity
O(VE). is O(E
logV).
Bellman Ford’s Algorithm use Dijkstra
DynamicProgramming Algorithm
use Greedy
approach
It can be It can not be implemented
easily easily in adistributed way.
implemented
in adistributed
way.

Till Mid Exam

 Dynamic Programming

Dynamic programming approach is similar to divide and conquer in breaking down the problem
into smaller and yet smaller possible sub-problems. But unlike, divide and conquer, these sub-
problems are not solved independently. Rather, results of these smaller sub-problems are
remembered and used for similar or overlapping sub-problems.
Dynamic programming is used where we have problems, which can be divided into similar sub-
problems, so that their results can be re-used. Mostly, these algorithms are used for optimization.
Before solving the in-hand sub-problem,dynamic algorithm will try to examine the results of the
previously solved sub-problems. The solutions of sub-problems are combined in order to achieve
thebest solution.

So we can say that −

The problem should be able to be divided into smaller overlapping sub-problem.
An optimum solution can be achieved by using an optimum solution of smaller sub-
problems.
Dynamic algorithms use Memoization.

Examples of Dynamic Algorithms

Fibonacci number series
0/1 Knapsack problem
Tower of Hanoi
All pair shortest path by Floyd-Warshall

Floyd Warshall Algorithm

 Floyd Warshall Algorithm is a famous algorithm.
 It is used to solve All Pairs Shortest Path Problem.
 It computes the shortest path between every pair of vertices of the given graph.
 Floyd Warshall Algorithm is an example of dynamic programming approach.
Time Complexity
Floyd Warshall Algorithm has a time complexity of O(n3).
When Floyd Warshall Algorithm is used?

Floyd Warshall Algorithm is best suited for dense graphs.

This is because its complexity depends only on the number of vertices in thegiven
graph.
For sparse graphs, Johnson’s Algorithm is more suitable.

Problem:- Consider the following directed weighted graph-

 Do(2,3) < Do(2,1)+Do(1,3)
so, place Do(2,3)

Using Floyd Warshall Algorithm, find the shortest path distance between every pairof vertices.
Solution: Initial distance matrix for the given graph is-

Using Floyd Warshall Algorithm, write the following 4 matrices-

The last matrix D4 represents the shortest path distance between every pair ofvertices.