What is Algorithm ?

Algorithm is a set of steps to complete tasks.

For example
Tasks:- Make a cap of tea.
Maintain engine of the cars.
Maintain/Troubleshoot field computers.
Find the sum of two numbers.

So What is computer algorithm?

1
Computer Algorithm
Computer Algorithm is a set of steps to accomplish a tasks that is
described precisely enough that a computer can run it.
 Which is clearly specified set of simple instructions to be
followed to solve a problem or tasks.
 Computer Algorithm related with two main concepts.
 Data Structure: a method used to organize/manage a data
 Program = Algorithm + Data Structure.

2
Computer Algorithm
Computer Algorithm may be specified as.
 pseudo code
 an English words
 flow chart
 a computer program

3
Computer Algorithm to Add Two integer numbers
Pseudo code
 BEGIN
NUMBER S1, S2, SUM
OUPUT(“Insert first number ”)
INPUT S1
OUPUT(“Insert second number ”)
INPUT S2
SUM = S1 + S2
OUTPUT SUM

END

4
Computer Algorithm to Add Two integer numbers
English Words

1. Start

2. Declare 3 variables of type integer namely x, y, z

3. Input values for x, y from keyboard

4. Do the operation x + y and assign the sum to variable z

5. Display value of variable z

6. Stop

5
Computer Algorithm to Add Two integer numbers
Flow Chart Start/
BEGIN
Declare S1, S2,
Sum

Input S1, S2

Sum = S1 + S2

Display Sum

Stop/ END

6
Computer Algorithm
Characteristics of an Algorithm
 Must take an input.

 Must give some output(Yes/NO).

 Definiteness: each instruction must be clear & unambiguous.

 Finiteness: algorithm must be terminated after a finite no of steps.

 Effectiveness: every instruction must be basic/simple to understand.

7
Computer Algorithm
Expectation from an Algorithm
 Correctness.
 Correct: Algorithm must produce correct result.
 Approximation algorithm: Exact solution is not found but near optimal solution can be found
out.
 Less Resource Usage
Algorithm should be use less resource (Time and Space)
Running Time of algorithm depend on the following variety of computer features .
Speed of computer
language in which the algorithm is implemented
Compiler/ interpreter and
Skill of the programmers.

8
Computer Algorithm
Why we need algorithm analysis
 Writing a working program is not good enough.

 The program my be inefficient.

 If the program is run on a large dataset, then the running time become an issue.

9
Order Notation/Asymptotic Notation
Introduction
 Big-oh(O): Used to represent the upper-bound of the algorithm.
 Big-Omega(Ω): Used to represent the lower-bound of the algorithm.
 Big-theta(ϴ): Used to represent the average-bound of the algorithm.
 The property of time complicity of an algorithm is as follow
 1< logn < < n < nlogn < n2 < n3 < ….. < 2n < 3n …….< nn

10
Order Notation/Asymptotic Notation
Big-oh(O)
 The function f(n)=O(g(n)) if only if there exist positive constant c and
n0, such that f(n) <= c*g(n) for all n>n0
 e.g. f(n) = 3n+1 assume g(n) any value which is greater than f(n).
 3n +1 <= 5n

f(n) c g(n
)
 let substitute n by value grater than or equal to 1.
 f(n) = 3+1 = 4
 g(n) = 5*1 = 5
 There for 4 <= 5 so f(n) = O(n)
11
Order Notation/Asymptotic Notation
Big-Omega(Ω)
 The function f(n)= Ω(g(n)) if only if there exist positive constant c and
n0, such that f(n) >= c*g(n) for all n>n0
 e.g. f(n) = 3n+1 assume g(n) any value which is less than f(n).
 3n +1 >= 2n

f(n) c g(n
 let substitute n by value) grater than or equal to 1.
 f(n) = 3+1 = 4
 g(n) = 2*1 = 2
 There for 4 >= 2 so f(n) = Ω(n)
12
Order Notation/Asymptotic Notation
Big-theta(ϴ)

 The function f(n)= ϴ(g(n)) if only if there exist positive constant c1, c2 and n0, such that
c1*g(n) <= f(n) <= c2*g(n) for all n>n0
 e.g. f(n) = 3n+1 assume c1*g(n) any value which is less than f(n) and c2*g(n) greater than
f(n).
 2n <= 3n +1 <= 5n

c1 f(n) c2 g(n)
 let substitute n by value grater than or equal to 1.
 c1 * g(n) = 2*1 = 2
 c2 * g(n) = 5*1 = 5
 f(n) = 3*1 +1 = 4 There for 2 <= 4 <= 5 so f(n) = ϴ(n) (when the upper and lower bound is equal)

13
Review of Elementary Data Structure
Introduction
 Data structures for dynamic sets
 Set in mathematics
 {1,2, 5, 4, 3, 6}

 Set in algorithms
 Allow repetition in set elements: {1,2, 5, 4, 3, 6, 4}
 Dynamic: can grow, shrink, or change over time
 Set operations: insert, delete, test membership

14
Review of Elementary Data Structure
Elements of A Dynamic Set
Each element is represented by an object (or record)
An object may consists of many fields
Need a pointer to an object to examine and manipulate its fields
Key field for identifying objects and for the set manipulation
 Keys are usually drawn from a totally ordered set
Satellite fields: all the fields relevant for the set

15
Review of Elementary Data Structure
Record(Object) and Pointer

16
 Review of Elementary Data Structure
Operations on Dynamic Sets
 Query operations: return information about a set
 SEARCH(S, k): given a set S and key value k, returns a pointer x to an element in S such
that key[x] = k, or NIL if no such element belongs to S
MINIMUM(S): returns a pointer to the element of S with the smallest key
MAXIMUM(S): returns a pointer to the element of S with the largest key
SUCCESSOR(S, x): returns a pointer to the next larger element in S, or NIL if x is the
maximum element
PREDECESSOR(S, x): returns a pointer to the next smaller element in S, or NIL if x is
the minimum element

17
Review of Elementary Data Structure
Operations on Dynamic Sets
 Modifying operations: change a set
INSERT(S, x): augments the set S with the element pointed to by x. We usually assume
that any fields in element x needed by the set implementation have already initialized.
DELETE(S, x): given a pointer x to an element in the set S, removes x from S.

18
 Review of Elementary Data Structure
Stack
 Stack
 The element deleted from the set is the one most recently inserted
 Last-in, First-out (LIFO)

Stack operations
 PUSH: Insert
 DELETE: Delete
 TOP: return the key value of the most recently inserted element
 POP: removes and returns the last inserted element
 STACK-EMPTY: check if the stack is empty
 STACK-FULL: check if the stack is full

19
Review of Elementary Data Structure
Applications of Stacks
Direct applications
 Page-visited history in a Web browser
 Undo sequence in a text editor
 Chain of method calls in the Java Virtual
 Machine or C++ runtime environment

Indirect applications
 Auxiliary data structure for algorithms
 Component of other data structures

20
 Review of Elementary Data Structure
Queue
 Queue
 The element deleted is always the one that has been in the set for the longest time
 First-in, First-out (FIFO)

 Queue operations
 ENQUEUE: Insert
 DEQUEUE: Delete
 HEAD/front: return the key value of the element that has been in the set for the longest time
 TAIL/rear: return the key value of the element that has been in the set for the shortest time
 QUEUE-EMPTY: check if the queue is empty
 QUEUE-FULL: check if the queue is full

21
Review of Elementary Data Structure
Applications of Queue
Direct applications
 Waiting lines
 Access to shared resources (e.g., printer)
 Multiprogramming

Indirect applications
 Auxiliary data structure for algorithms
 Component of other data structures

22
Heap Sort
 A heap is a data structure that stores a collection of objects (with keys),
and has the following properties:
 Complete Binary tree
 Heap Order

 It is implemented as an array where each node in the tree

corresponds to an element of the array.

23
Heap Sort
 Array representation using binary tree
 How to identify the r/ship between each nodes in binary tree
 if a node is at index –> i
 its left child is at –> 2*i 11

 its right child is at –> 2*i+1

 its parent is at –> i/2 12 13

 e.g. array 1 12 13 14 15 16 17 17
1 14 15 16
index
1 2 3 4 5 6 7
Binary Tree

24
Heap Sort
 Full Binary Tree : a binary tree with maximum number of nodes.
 Complete Binary Tree : a binary tree until height minus one and on the last level
elements are filled from left to right.
 e.g.

Complete binary tree Complete binary tree Not complete binary tree

25
Heap order property
 For every node v, other than the root, the key stored in v is greater or equal (smaller or
equal for max heap) than the key stored in the parent of v.
 In this case the maximum value is stored in the root
Max Heap
Store data in ascending order
Has property of
A[Parent(i)] ≥ A[i]
Min Heap
Store data in descending order
Has property of
A[Parent(i)] ≤ A[i]

26
Heap Sort
 Max Heap  Min Heap
19 1

12 16 4 16

7 12 19
1 4 7

1 12 16 1 4 7
9 1 4 16 7 1 1
Array A 2 9
Array A

27
Heap Insertion
Algorithm
1. Add the new element to the next available position at the lowest level
2. Restore the max-heap property if violated
 General strategy is bubble up : if the parent of the element is smaller than the
element, then interchange the parent and child.
OR
3. Restore the min-heap property if violated
 General strategy is bubble up : if the parent of the element is larger than the element,
then interchange the parent and child.

28
Heap Insertion
19 19

12 16 12 16

1 4 7 1 4 7 1
7
19

12 17
swap

1 4 7 1
6
Percolate up to maintain the
heap property
29
Heap Deletion/Take Out
 In heap sort the element must be deleted is the root element.
 Swap the last element at the index of root element.
 The heap sort algorithm consists of two phases:
build a heap from an arbitrary array
use the heap to sort the data

To sort the elements in the decreasing order, use a min heap

To sort the elements in the increasing order, use a max heap

30
Example of Heap Sort
Take out biggest
19

12 16
Move the last element
to the root

1 4 7

Sorted:
Array A

12 1 1 4 7 19
6

31
Example of Heap Sort
7
swap
HEAPIFY()
12 16

1 4

Sorted:
Array A
7 1 16 1 4 19
2

32
Example of Heap Sort
16

12 7

1 4

Sorted:
Array A
1 12 7 1 4 19
6

33
Example of Heap Sort
Take out biggest
16
Move the last element
to the root
12 7

1 4

Sorted:
Array A
12 7 1 4 16 1
9

34
Example of Heap Sort

12 7

Sorted:
Array A
4 12 7 1 16 19

35
Example of Heap Sort
swap 4

HEAPIFY()
12 7

Sorted:
Array A

4 12 7 1 1 19
6
36
Example of Heap Sort
12

4 7

Sorted:
Array A
12 4 7 1 1 19
6

37
Example of Heap Sort
Take out biggest
12
Move the last
element to the
root 4 7

Sorted:
Array A
4 7 1 12 1 19
6
38
Example of Heap Sort
1 swap

4 7

Sorted:
Array A
1 4 7 12 1 19
6

39
Example of Heap Sort
7

4 1

Sorted:
Array A

7 4 1 12 1 19
6

40
Example of Heap Sort
Take out biggest
7
Move the last
element to the
4 1 root

Sorted:
Array A
1 4 7 12 1 19
6

41
Example of Heap Sort
swap 1

HEAPIFY()
4

Sorted:
Array A

4 1 7 12 1 19
6
42
Example of Heap Sort
Take out biggest
Move the last 4
element to the
root
1

Sorted:
Array A

1 4 7 12 1 19
6

43
Example of Heap Sort
Take out biggest
1
Sorted:

1 4 7 12 16 19

Sorted:
Array A
1 4 7 12 1 19
6

44
Hashing
 Useful for the searching purpose.
 It is a techniques to store value on index equal to a value to be stored.
 Hashing techniques used to reduce the time complexity of search.
 a lot of memory wasted.
 Example:
 values to be stored = 2, 3, 4, 5, 6, 8, 0, 7, 10 …….
index 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 . . .
value 0 2 3 4 5 6 7 8 10

45
Hashing in function representation
Hash Table
0
Key Space 1
h(x) = x
2
2 2
3
 What if we want to store 60?
5 4 4

8 5 5 The function must be modified to

10 6 h(x) = x % n
7
4 7
Where % is mod and
8 8
15
7
9 n is length of hash table.
13
10 10
11
 To proper space/ memory utilization.
12

13 13
14

15 15
16

46
Hashing in function representation
Key Space Hash Table
h(x) = x % 10 10 0 x=2
2 1 h(2) = 2 % 10 = 2 then
5 2 2 x=5
8
3 h(5) = 5 % 10 = 5 then
10
15 4 . Collision between 5 and 15
7 515 5 .
12 6 x = 15
7 h(15) = 15 % 10 = 5.
8
 So we have to use conflict/collision resolution
8
9
methods.

47
Collision Resolution Methods in Hashing
 Open Hashing : Store the collision value outside the hash table
 Chaining : use linked list as a chain on the index.

 Closed Hashing : Store the collision value in side the hash table.
 Open Addressing
 Linear Probing : use next free space in the hash table.
 Mathematical representation
 h(x) = x % size and h’(x) = [h(x) + f(i)] % size where i = 0, 1, 2, 3, …….
 Quadratic Probing : use quadratic free space in the hash table
 Mathematical representation
h(x) = x % size and h’(x) = [h(x) + f(i)] % size where f(i) = and i = 0, 1, 2, 3, …….

48
Example for chaining collision resolution
Key Space Hash Table
h(x) = x % 10 10 0 x=2
2 1 h(2) = 2 % 10 = 2 then
5 2 2 x = 5 12 /
8
3 h(5) = 5 % 10 = 5 then
10
15 4 .
7 5 5 . 15 /
12 6 x = 15
7 7 h(15) = 15 % 10 = 5.
8 8 Finally x = 12
9
h(12) = 12%10 = 2

49
Example for linear probing collision resolution
Key Space Hash Table
h(x) = x % 10 10 0 x = 15 and i = 0
2 1 h’(15) = [h(15) + f(0)] % 10 = 5 still collision
5 2 2 x = 15 and i = 1
8
12 3 h’(15) = [h(15) + f(1)] % 10 = 6 then store 15 at index
10
15 4 6 if it is free else calculate until you find free space.
7 5 5 Finally x = 12 and i = 0
12 15 6 h’(12) = [h(12) + f(0)] % 10 = 2 still collision
7 7 x = 12 and i = 1
8 8 h’(12) = [h(12) + f(1)] % 10 = 3 then store 12 at index
9
3 if it is free else calculate until you find free space.

50
Example for Quadratic probing collision resolution

Key Space Hash Table

h(x) = x % 10 10 0 x = 15 and i = 0
2 1 h’(15) = [h(15) + f(0)] % 10 = 5 still collision
5
2 2 x = 15 and i = 1
8
10
h’(15) = [h(15) + f(1)] % 10 = 6 then store 15 at index 6 if it
12 3
15 is free else calculate until you find free space.
4
25 x = 25 and i = 0
5 5 h’(15) = [h(15) + f(0)] % 10 = 6 still collision
7
12 15 6 x = 25 and i = 1
7 7 h’(15) = [h(15) + f(1)] % 10 = 6 still collision
8 8 x = 25 and i = 2
9 h’(15) = [h(15) + f(2)] % 10
h’(15) = [5+ 4] % 10 = 9 store the value at index 9
51
Set Representation
 Let have 3 sets s1= {1, 7, 8, 9} s2 = {2, 5, 10} s3 = {3, 4, 6}
1. Graphical/Tree representation of set.
1 5 3

7 8 2 10 4 6
9

2. Array Representation of a set

i 1 2 3 4 5 6 7 8 9
p -1 5 -1 3 -1 3 1 1 1
Where i = the element in the set and p = parent of the element in the set

52
Disjoint Set-Union
 If and are two disjoint sets, then their union U equal to all elements x
such that x is in or .
 Example:
S1 = {1, 7, 8, 9} S2 = {2, 5, 10}
S1 U S2 = {1, 2, 5, 7, 8, 9}

53
Disjoint Set-Union
Simple Find Operation
 Find(i) : given the element i, find the set containing i, thus 4 is in set and 9 is in set
 example if i = 9 then it return 1
 Algorithm
 Algorithm SimpleFind(i){
while (p[i] >= 0){
i = p[i];
}
return i;
}

54
Disjoint Set-Union
 Simple Union Operation
 Used to find value from different sets.
 Example : Find(9,3), then you need to find Union of set s1 and s3.
which is s1 = {1, 7, 8, 9}, s3 = {3, 4, 6}
s1 U s3 = {1, 3, 4, 6, 7, 8, 9}
 Algorithm
 Algorithm SimpleUnion(i, j){
p[i] = j;
}
Where i is parent of first set and j is parent of second set

55
The execution time of algorithm
 Each operation in an algorithm has a cost(time).
 Assignments have cost of 1
 sum = 0; …………….. 1
 Comparisons have a cost of 1
 i = 0; …………….. 1
 I >= 10 …………….. 1 total cost 2
 Input and Output have a cost of 1
 cin >> i; …………….. 1
 cout << i; …………….. 1 total cost 2
 Execution cost have a cost of 1
 sum = sum + next; …………….. 1

56
General Rules for Time Estimation
 Loops : at most the running time of statement inside of the loop times the no of iterations
 Example:
 for (int i =0; i<=0;i++){

cout << i;
}

 Nested Loops: the inner most loop running time multiplied by the product of the sized of all loops.
 Example
 for (int i =0; i<=0;i++){
for(int j=0<=n;i++){
cout << i;
}
}

57
General Rules for Time Estimation
 Consecutive Statements : just add the running times of those consecutive statements.
 Example:
int a; // Declaration
a = 5;. // Assignment
a= a++; // Increment
cout<<a; // Display

 If/Else : Running time of the comparison plus the larger of running time of s1 and s2.
 Example
 if(a < 0){
print(“Negative”);
}
else{
print(“Positive”);
}

58
Algorithm Growth Rate
 An algorithms proportional time requirement is known as growth rate.
 Example: the growth rate of function:
4n+3 is = n
4+3 is =
5 + 1000 =

59
Properties of Growth Rate
 Ignore low order terms
If an algorithm with (O( + 2n + 3) execution time, the time complexity of algorithm is O( )

 Ignore a constant term with higher-order term

If a higher term is O(3 only take O(

 Summation of Different order O(f(n))+ O(g(n)) = O(f(n) + g(n))

If O(f(n)) = O(2) and O(g(n)) = O(2), by combining both O(2 + 2) = O()

 Product of different order O(f(n))+ O(g(n)) = O(f(n) * g(n))

If O(f(n)) = O(2) and O(g(n)) = O(2), by combining both O(2 * 2) = O()

60
Analyze the following Algorithm
1. for (int j = 0; j < n; j++) 3. for (int j = 0; j < n; j++)
for (int k = 0; k < n; k++) for (int k = 0; k < n; k++)
sum += k + l; for (int l = 0; l < n; l++)
for (int l = 0; l < n; l++) sum += j * k * l;
sum += l;

4. for (int j = 0; j < n; j++)

2. for (int j = 0; j < n; j++)
sum++;
sum += l;
for (int j = 0; j < n; j++)
for (int k = 0; k < n; k++)
sum--;
sum += l;
for (int l = 0; l < n; l++)
sum += l;
61
 Analyze the following Algorithm
1. for (int j = n; j > 0; j--) 5. for (int j = 0; j < n; j++) 9. for (int j = 0; j*j < n; j++) 1. O()
for (int k = 0; k < n; k++) cout<<k; 2. O(n)
for (int k = 0; k < j; k++) 3. O(n)
sum += k + l; 10. for (int j = 0; j < n; j++) 4. O(n)
cout<<k; for (int k = 0; k < n; k=k*2) 5. O(n)
2. for (int j = n; j >= 0; j--) cout<<k; 6. O( )
6. m = 0; 7. O(logn)
sum += j;
for (int j = 1; m <= n; j++) 11. for (int j = 0; j < n; j=j/2) 8. O(logn)
m = m+j 9. O( )
3. for (int k = n; k >= 0; k-2) for (int k = 0; k < n; k=k*2)
sum += k; cout<<k; 10. O(nlogn)
7. for (int i = 0; i < n; i = i*2) 11. O(loglogn)
4. for (int l = 0; l < n; l+10) cout<<i;
sum += l;
8. for (int j = 0; j < n; j = j/2) 62
Analyze the following Algorithm
1. while(m!=n) { 3. i = 1; 1. O(n)
if(m>n) k = 0; 2. O(logn)
m=m-n while(k < n){ 3. O( )
else stmt;
n = n-m k=k + i;
} i++;
2. a = 1; }
while(a<b){
stmt;
a = a*2;
}

63