Data Structures And

Algorithms

Autor: Dr. N.Mahamatov

 Table of Contents

1. Introduction
1.1 Variables
1.2 Data Types
1.3 Data Structures
1.4 Abstract Data Types (ADTs)
1.5 What is an Algorithm?
1.6 Why the Analysis of Algorithms?
1.7 Goal of the Analysis of Algorithms
1.8 What is Running Time Analysis?
1.9 How to Compare Algorithms
1.10 What is Rate of Growth?
1.11 Commonly Used Rates of Growth
1.12 Types of Analysis
1.13 Asymptotic Notation
1.14 Big-O Notation

2. Recursion and Backtracking

2.1 Introduction
2.2 What is Recursion?
2.3 Why Recursion?
2.4 Format of a Recursive Function
2.5 Recursion and Memory (Visualization)
2.6 Recursion versus Iteration
2.7 Notes on Recursion
2.8 Example Algorithms of Recursion
2.9 Recursion: Problems & Solutions
2.10 What is Backtracking?
2.11 Example Algorithms of Backtracking
3. Linked Lists
3.1 What is a Linked List?
3.2 Linked Lists ADT
3.3 Why Linked Lists?
3.4 Arrays Overview
3.5 Comparison of Linked Lists with Arrays & Dynamic Arrays
3.6 Singly Linked Lists
3.7 Doubly Linked Lists
3.8 Circular Linked Lists
3.9 A Memory-efficient Doubly Linked List
3.10 Unrolled Linked Lists

4. Stacks
4.1 What is a Stack?
4.2 How Stacks are used
4.3 Stack ADT
4.4 Applications
4.5 Implementation
4.6 Comparison of Implementations
4.7 Stacks: Problems & Solutions

5. Queues
5.1 What is a Queue?
5.2 How are Queues Used?
5.3 Queue ADT
5.4 Exceptions
5.5 Applications
5.6 Implementation
5.7 Queues: Problems & Solutions
6. Trees
6.1 What is a Tree?
6.2 Glossary
6.3 Binary Trees
6.4 Types of Binary Trees
6.5 Properties of Binary Trees

7. Sorting
7.1 What is Sorting?
7.2 Why is Sorting Necessary?
7.3 Classification of Sorting Algorithms
7.4 Other Classifications
7.5 Bubble Sort
7.6 Selection Sort
7.7 Insertion Sort
7.8 Shell Sort
7.9 Merge Sort
7.10 Heap Sort
7.11 Quick Sort
7.12 Tree Sort
7.13 Comparison of Sorting Algorithms

8. Searching
8.1 What is Searching?
8.2 Why do we need Searching?
8.3 Types of Searching
8.4 Unordered Linear Search
8.5 Sorted/Ordered Linear Search
8.6 Binary Search
8.7 Interpolation Search
8.8 Comparing Basic Searching Algorithms
8.9 Symbol Tables and Hashing
The objective of this chapter is to explain the importance of the analysis of
algorithms, their notations, relationships and solving as many problems as
possible. Let us first focus on understanding the basic elements of algorithms,
the importance of algorithm analysis, and then slowly move toward the other
topics as mentioned above. After completing this chapter, you should be able to
find the complexity of any given algorithm (especially recursive functions).

1.1 Variables
Before going to the definition of variables, let us relate them to old mathematical
equations. All of us have solved many mathematical equations since childhood.
As an example, consider the below equation:

We don’t have to worry about the use of this equation. The important thing that
we need to understand is that the equation has names (x and y), which hold values
(data). That means the names (x and y) are placeholders for representing data.
Similarly, in computer science programming we need something for holding
data, and variables is the way to do that.

1.2 Data Types

In the above-mentioned equation, the variables x and y can take any values such
as integral numbers (10, 20), real numbers (0.23, 5.5), or just 0 and 1. To solve
the equation, we need to relate them to the kind of values they can take, and data
type is the name used in computer science programming for this purpose. A data
type in a programming language is a set of data with predefined values.
Examples of data types are: integer, floating point, unit number, character,
string, etc.

Computer memory is all filled with zeros and ones. If we have a problem and we
want to code it, it’s very difficult to provide the solution in terms of zeros and
ones. To help users, programming languages and compilers provide us with data
types. For example, integer takes 2 bytes (actual value depends on compiler),
float takes 4 bytes, etc. This says that in memory we are combining 2 bytes (16
bits) and calling it an integer. Similarly, combining 4 bytes (32 bits) and calling
it a float. A data type reduces the coding effort. At the top level, there are two
types of data types:

• System-defined data types (also called Primitive data types)

• User-defined data types

System-defined data types (Primitive data types)

Data types that are defined by system are called primitive data types. The
primitive data types provided by many programming languages are: int, float,
char, double, bool, etc. The number of bits allocated for each primitive data type
depends on the programming languages, the compiler and the operating system.
For the same primitive data type, different languages may use different sizes.
Depending on the size of the data types, the total available values (domain) will
also change.

For example, “int” may take 2 bytes or 4 bytes. If it takes 2 bytes (16 bits), then
the total possible values are minus 32,768 to plus 32,767 (-215 to 215-1). If it takes
4 bytes (32 bits), then the possible values are between -2,147,483,648 and
+2,147,483,647 (-231 to 231-1). The same is the case with other data types.

User defined data types

If the system-defined data types are not enough, then most programming
languages allow the users to define their own data types, called user – defined
data types. Good examples of user defined data types are: structures in C/C + +
and classes in Java. For example, in the snippet below, we are combining many
system-defined data types and calling the user defined data type by the name
“newType”. This gives more flexibility and comfort in dealing with computer
memory.

1.3 Data Structures

Based on the discussion above, once we have data in variables, we need some
mechanism for manipulating that data to solve problems. Data structure is a
particular way of storing and organizing data in a computer so that it can be used
efficiently. A data structure is a special format for organizing and storing data.
General data structure types include arrays, files, linked lists, stacks, queues,
trees, graphs and so on.
Depending on the organization of the elements, data structures are classified into
two types:

1) Linear data structures: Elements are accessed in a sequential order but

it is not compulsory to store all elements sequentially. Examples:
Linked Lists, Stacks and Queues.
2) Non – linear data structures: Elements of this data structure are
stored/accessed in a non-linear order. Examples: Trees and graphs.

1.4 Abstract Data Types (ADTs)

Before defining abstract data types, let us consider the different view of system-
defined data types. We all know that, by default, all primitive data types (int,
float, etc.) support basic operations such as addition and subtraction. The system
provides the implementations for the primitive data types. For user-defined data
types we also need to define operations. The implementation for these operations
can be done when we want to actually use them. That means, in general, user
defined data types are defined along with their operations.

To simplify the process of solving problems, we combine the data structures

with their operations and we call this Abstract Data Types (ADTs). An ADT
consists of two parts:

1. Declaration of data
2. Declaration of operations

Commonly used ADTs include: Linked Lists, Stacks, Queues, Priority Queues,
Binary Trees, Dictionaries, Disjoint Sets (Union and Find), Hash Tables,
Graphs, and many others. For example, stack uses LIFO (Last-In-First-Out)
mechanism while storing the data in data structures. The last element inserted
into the stack is the first element that gets deleted. Common operations of it are:
creating the stack, pushing an element onto the stack, popping an element from
stack, finding the current top of the stack, finding number of elements in the
stack, etc.

While defining the ADTs do not worry about the implementation details. They
come into the picture only when we want to use them. Different kinds of ADTs
are suited to different kinds of applications, and some are highly specialized to
specific tasks. By the end of this book, we will go through many of them and
you will be in a position to relate the data structures to the kind of problems they
solve.

1.5 What is an Algorithm?

Let us consider the problem of preparing an omelette. To prepare an omelette,
we follow the steps given below:

1) Get the frying pan.

2) Get the oil.
a. Do we have oil?
i. If yes, put it in the pan.
ii. If no, do we want to buy oil?
1. If yes, then go out and buy.
2. If no, we can terminate.
3) Turn on the stove, etc...

What we are doing is, for a given problem (preparing an omelette), we are
providing a step-by-step procedure for solving it. The formal definition of an
algorithm can be stated as:
An algorithm is the step-by-step unambiguous instructions to solve a given
problem.

In the traditional study of algorithms, there are two main criteria for judging the
merits of algorithms: correctness (does the algorithm give solution to theproblem
in a finite number of steps?) and efficiency (how much resources (in terms of
memory and time) does it take to execute the).

Note: We do not have to prove each step of the algorithm.

1.6 Why the Analysis of Algorithms?

To go from city “A” to city “B”, there can be many ways of accomplishing this:
by flight, by bus, by train and also by bicycle. Depending on the availability and
convenience, we choose the one that suits us. Similarly, in computer science,
multiple algorithms are available for solving the same problem (for example, a
sorting problem has many algorithms, like insertion sort, selection sort, quick
sort and many more). Algorithm analysis helps us to determine which algorithm
is most efficient in terms of time and space consumed.

1.7 Goal of the Analysis of Algorithms

The goal of the analysis of algorithms is to compare algorithms (or solutions)
mainly in terms of running time but also in terms of other factors (e.g., memory,
developer effort, etc.)

1.8 What is Running Time Analysis?

It is the process of determining how processing time increases as the size of the
problem (input size) increases. Input size is the number of elements in the input,
and depending on the problem type, the input may be of different types. The
following are the common types of inputs.

• Size of an array
• Polynomial degree
• Number of elements in a matrix
• Number of bits in the binary representation of the input
• Vertices and edges in a graph.

1.9 How to Compare Algorithms

To compare algorithms, let us define a few objective measures:

Execution times? Not a good measure as execution times are specific to a

particular computer.

Number of statements executed? Not a good measure, since the number of

statements varies with the programming language as well as the style of the
individual programmer.

Ideal solution? Let us assume that we express the running time of a given
algorithm as a function of the input size n (i.e., f(n)) and compare these different
functions corresponding to running times. This kind of comparison is
independent of machine time, programming style, etc.

1.10 What is Rate of Growth?

The rate at which the running time increases as a function of input is called rate
of growth. Let us assume that you go to a shop to buy a car and a bicycle. If your
friend sees you there and asks what you are buying, then in general you say
buying a car. This is because the cost of the car is high compared to the cost of
the bicycle (approximating the cost of the bicycle to the cost of the car).

For the above-mentioned example, we can represent the cost of the car and the
cost of the bicycle in terms of function, and for a given function ignore the low
order terms that are relatively insignificant (for large value of input size, n). As
an example, in the case below, n4, 2n2, 100n and 500 are the individual costs of
some function and approximate to n4 since n4 is the highest rate of growth.

1.11 Commonly Used Rates of Growth

The diagram below shows the relationship between different rates of growth.
Below is the list of growth rates you will come across in the following chapters.
1.12 Types of Analysis
To analyze the given algorithm, we need to know with which inputs the
algorithm takes less time (performing wel1) and with which inputs the algorithm
takes a long time. We have already seen that an algorithm can be represented in
the form of an expression. That means we represent the algorithm with multiple
expressions: one for the case where it takes less time and another for the case
where it takes more time.

In general, the first case is called the best case and the second case is called the
worst case for the algorithm. To analyze an algorithm we need some kind of
syntax, and that forms the base for asymptotic analysis/notation. There are three
types of analysis:

• Worst case
○ Defines the input for which the algorithm takes a long time
(slowest time to complete).
○ Input is the one for which the algorithm runs the slowest.
• Best case
○ Defines the input for which the algorithm takes the least time
(fastest time to complete).
○ Input is the one for which the algorithm runs the fastest.
• Average case
○ Provides a prediction about the running time of the algorithm.
 ○ Run the algorithm many times, using many different inputs
that come from some distribution that generates these
inputs, compute the total running time (by adding the
individual times), and divide by the number of trials.
○ Assumes that the input is random.

Lower Bound <= Average Time <= Upper Bound

For a given algorithm, we can represent the best, worst and average cases in the
form of expressions. As an example, let f(n) be the function which represents the
given algorithm.

Similarly for the average case. The expression defines the inputs with which the
algorithm takes the average running time (or memory).

1.13 Asymptotic Notation

Having the expressions for the best, average and worst cases, for all three cases
we need to identify the upper and lower bounds. To represent these upper and
lower bounds, we need some kind of syntax, and that is the subject of the
following discussion. Let us assume that the given algorithm is represented in
the form of function f(n).

1.14 Big-O Notation [Upper Bounding Function]

This notation gives the tight upper bound of the given function. Generally, it is
represented as f(n) = O(g(n)). That means, at larger values of n, the upper bound
of f(n) is g(n). For example, if f(n) = n4 + 100n2 + 10n + 50 is the given algorithm,
then n4 is g(n). That means g(n) gives the maximum rate of growth for f(n) at
larger values of n.
Let us see the O–notation with a little more detail. O–notation defined as O(g(n))
= {f(n): there exist positive constants c and n0 such that 0 ≤ f(n) ≤ cg(n) for all n
> n0}. g(n) is an asymptotic tight upper bound for f(n). Our objective is to give
the smallest rate of growth g(n) which is greater than or equal to the given
algorithms’ rate of growth /(n).

Generally we discard lower values of n. That means the rate of growth at lower
values of n is not important. In the figure, n0 is the point from which we need to
consider the rate of growth for a given algorithm. Below n0, the rate of growth
could be different. n0 is called threshold for the given function.

Big-O Visualization
O(g(n)) is the set of functions with smaller or the same order of growth as g(n).
For example; O(n2) includes O(1), O(n), O(nlogn), etc.

Note: Analyze the algorithms at larger values of n only. What this means is,
below n0 we do not care about the rate of growth.
Big-O Examples
Example-1 Find upper bound for f(n) = 3n + 8
Solution: 3n + 8 ≤ 4n, for all n ≥ 8
∴ 3n + 8 = O(n) with c = 4 and n0 = 8
Example-2 Find upper bound for f(n) = n2 + 1
Solution: n2 + 1 ≤ 2n2, for all n ≥ 1
∴ n2 + 1 = O(n2) with c = 2 and n0 = 1

Example-3 Find upper bound for f(n) = n4 + 100n2 + 50

Solution: n4 + 100n2 + 50 ≤ 2n4, for all n ≥ 11
∴ n4 + 100n2 + 50 = O(n4 ) with c = 2 and n0 = 11

Example-4 Find upper bound for f(n) = 2n3 – 2n2

Solution: 2n3 – 2n2 ≤ 2n3, for all n > 1
∴ 2n3 – 2n2 = O(n3 ) with c = 2 and n0 = 1
Example-5 Find upper bound for f(n) = n
Solution: n ≤ n, for all n ≥ 1
∴ n = O(n) with c = 1 and n0 = 1
Example-6 Find upper bound for f(n) = 410
Solution: 410 ≤ 410, for all n > 1
∴ 410 = O(1) with c = 1 and n0 = 1

No Uniqueness?
There is no unique set of values for n 0 and c in proving the asymptotic bounds.
Let us consider, 100n + 5 = O(n). For this function there are multiple n0 and c
values possible.
Solution1: 100n + 5 ≤ 100n + n = 101n ≤ 101n, for all n ≥ 5, n0 = 5 and c = 101
is a solution.
Solution2: 100n + 5 ≤ 100n + 5n = 105n ≤ 105n, for all n > 1, n0 = 1 and c =
105 is also a solution.
2.1 Introduction
In this chapter, we will look at one of the important topics, “recursion”, which
will be used in almost every chapter, and also its relative “backtracking”.

2.2 What is Recursion?

Any function which calls itself is called recursive. A recursive method solves a
problem by calling a copy of itself to work on a smaller problem. This is called
the recursion step. The recursion step can result in many more such recursive
calls.

It is important to ensure that the recursion terminates. Each time the function
calls itself with a slightly simpler version of the original problem. The sequence
of smaller problems must eventually converge on the base case.

2.3 Why Recursion?

Recursion is a useful technique borrowed from mathematics. Recursive code is
generally shorter and easier to write than iterative code. Generally, loops are
turned into recursive functions when they are compiled or interpreted.

Recursion is most useful for tasks that can be defined in terms of similar
subtasks. For example, sort, search, and traversal problems often have simple
recursive solutions.

2.4 Format of a Recursive Function

A recursive function performs a task in part by calling itself to perform the
subtasks. At some point, the function encounters a subtask that it can perform
without calling itself. This case, where the function does not recur, is called the
base case. The former, where the function calls itself to perform a subtask, is
referred to as the ecursive case. We can write all recursive functions using the
format:

As an example consider the factorial function: n! is the product of all integers

between n and 1. The definition of recursive factorial looks like:
This definition can easily be converted to recursive implementation. Here the
problem is determining the value of n!, and the subproblem is determining the
value of (n – l)!. In the recursive case, when n is greater than 1, the function calls
itself to determine the value of (n – l)! and multiplies that with n.

In the base case, when n is 0 or 1, the function simply returns 1. This looks like
the following:

2.5 Recursion and Memory (Visualization)

Each recursive call makes a new copy of that method (actually only the
variables) in memory. Once a method ends (that is, returns some data), the copy
of that returning method is removed from memory. The recursive solutions look
simple but visualization and tracing takes time. For better understanding, let us
consider the following example.
For this example, if we call the print function with n=4, visually our memory
assignments may look like:

Now, let us consider our factorial function. The visualization of factorial

function with n=4 will look like:
2.6 Recursion versus Iteration
While discussing recursion, the basic question that comes to mind is: which way
is better? – iteration or recursion? The answer to this question depends on what
we are trying to do. A recursive approach mirrors the problem that we are trying
to solve. A recursive approach makes it simpler to solve a problem that may not
have the most obvious of answers. But, recursion adds overhead for each
recursive call (needs space on the stack frame).

Recursion
• Terminates when a base case is reached.
• Each recursive call requires extra space on the stack frame (memory).
• If we get infinite recursion, the program may run out of memory and
result in stack overflow.
• Solutions to some problems are easier to formulate recursively.

Iteration
• Terminates when a condition is proven to be false.
 • Each iteration does not require extra space.
• An infinite loop could loop forever since there is no extra memory being
created.
• Iterative solutions to a problem may not always be as obvious as a
recursive solution.

2.7 Notes on Recursion

• Recursive algorithms have two types of cases, recursive cases and base
cases.
• Every recursive function case must terminate at a base case.
• Generally, iterative solutions are more efficient than recursive solutions
[due to the overhead of function calls].
• A recursive algorithm can be implemented without recursive function
calls using a stack, but it’s usually more trouble than its worth. That
means any problem that can be solved recursively can also be solved
iteratively.
• For some problems, there are no obvious iterative algorithms.
• Some problems are best suited for recursive solutions while others are
not.

2.8 Example Algorithms of Recursion

• Fibonacci Series, Factorial Finding
• Merge Sort, Quick Sort
• Binary Search
• Tree Traversals and many Tree Problems: InOrder, PreOrder PostOrder
• Graph Traversals: DFS [Depth First Search] and BFS [Breadth First
Search]
• Dynamic Programming Examples
• Divide and Conquer Algorithms
• Towers of Hanoi
• Backtracking Algorithms [we will discuss in next section]

2.9 Recursion: Problems & Solutions

In this chapter we cover a few problems with recursion and we will discuss the
rest in other chapters. By the time you complete reading the entire book, you will
encounter many recursion problems.
Problem-1 Discuss Towers of Hanoi puzzle.
Solution: The Towers of Hanoi is a mathematical puzzle. It consists of three
rods (or pegs or towers), and a number of disks of different sizes which can slide
onto any rod. The puzzle starts with the disks on one rod in ascending order of
size, the smallest at the top, thus making a conical shape. The objective of the
puzzle is to move the entire stack to another rod, satisfying the following rules:
• Only one disk may be moved at a time.
• Each move consists of taking the upper disk from one of the rods and
sliding it onto another rod, on top of the other disks that may already
be present on that rod.
• No disk may be placed on top of a smaller disk.

Algorithm:

• Move the top n – 1 disks from Source to Auxiliary tower, • Move the nth
disk from Source to Destination tower, • Move the n – 1 disks from
Auxiliary tower to Destination tower.
• Transferring the top n – 1 disks from Source to Auxiliary tower can again
be thought of as a fresh problem and can be solved in the same
manner. Once we solve Towers of Hanoi with three disks, we can
solve it with any number of disks with the above algorithm.
Problem-2 Given an array, check whether the array is in sorted order with
recursion.
Solution:

Time Complexity: O(n). Space Complexity: O(n) for recursive stack space.

2.10 What is Backtracking?

Backtracking is an improvement of the brute force approach. It systematically
searches for a solution to a problem among all available options. In backtracking,
we start with one possible option out of many available options and try to solve
the problem if we are able to solve the problem with the selected
move then we will print the solution else we will backtrack and select some other
option and try to solve it. If none if the options work out we will claim thatthere
is no solution for the problem.

Backtracking is a form of recursion. The usual scenario is that you are faced with
a number of options, and you must choose one of these. After you make your
choice you will get a new set of options; just what set of options you get depends
on what choice you made. This procedure is repeated over and over until you
reach a final state. If you made a good sequence of choices, your final state is a
goal state; if you didn’t, it isn’t.

Backtracking can be thought of as a selective tree/graph traversal method. The

tree is a way of representing some initial starting position (the root node) and a
final goal state (one of the leaves). Backtracking allows us to deal with situations
in which a raw brute-force approach would explode into an impossible number
of options to consider. Backtracking is a sort of refined brute force. At each node,
we eliminate choices that are obviously not possible and proceed to recursively
check only those that have potential.

What’s interesting about backtracking is that we back up only as far as needed to

reach a previous decision point with an as-yet-unexplored alternative. In general,
that will be at the most recent decision point. Eventually, more and more of these
decision points will have been fully explored, and we will have to backtrack
further and further. If we backtrack all the way to our initial state and have
explored all alternatives from there, we can conclude the particular problem is
unsolvable. In such a case, we will have done all the work of the exhaustive
recursion and known that there is no viable solution possible.

• Sometimes the best algorithm for a problem is to try all possibilities.

• This is always slow, but there are standard tools that can be used to help.
• Tools: algorithms for generating basic objects, such as binary strings [2 n
possibilities for n-bit string], permutations [n!], combinations [n!/r!(n
– r)!], general strings [k –ary strings of length n has kn possibilities],
etc...
• Backtracking speeds the exhaustive search by pruning.

2.11 Example Algorithms of Backtracking

 • Binary Strings: generating all binary strings
• Generating k – ary Strings
• N-Queens Problem
• The Knapsack Problem
• Generalized Strings
• Hamiltonian Cycles [refer to Graphs chapter]
• Graph Coloring Problem

2.12 Backtracking: Problems & Solutions

Problem-3 Generate all the strings of n bits. Assume A[0..n – 1] is an array
of size n.
Solution:

Let T(n) be the running time of binary(n). Assume function printf takes time
O(1).

Using Subtraction and Conquer Master theorem we get: T(n) = O(2n). This
means the algorithm for generating bit-strings is optimal.
Problem-4 Generate all the strings of length n drawn from 0... k – 1.
Solution: Let us assume we keep current k-ary string in an array A[0.. n – 1].
Call function k-string(n, k):

Let T(n) be the running time of k – string(n). Then,

Using Subtraction and Conquer Master theorem we get: T(n) = O(kn).

Note: For more problems, refer to String Algorithms chapter.

Problem-5 Finding the length of connected cells of 1s (regions) in an matrix
of Os and 1s: Given a matrix, each of which may be 1 or 0. The filled cells
that are connected form a region. Two cells are said to be connected if they
are adjacent to each other horizontally, vertically or diagonally. There may
be several regions in the matrix. How do you find the largest region (in
terms of number of cells) in the matrix?

Solution: The simplest idea is: for each location traverse in all 8 directions and
in each of those directions keep track of maximum region found.
3.1 What is a Linked List?
A linked list is a data structure used for storing collections of data. A linked list
has the following properties.

• Successive elements are connected by pointers

• The last element points to NULL
• Can grow or shrink in size during execution of a program
• Can be made just as long as required (until systems memory exhausts)
• Does not waste memory space (but takes some extra memory for
pointers). It allocates memory as list grows.
3.2 Linked Lists ADT
The following operations make linked lists an ADT:

Main Linked Lists Operations

• Insert: inserts an element into the list
• Delete: removes and returns the specified position element from the list

Auxiliary Linked Lists Operations

• Delete List: removes all elements of the list (disposes the list)
• Count: returns the number of elements in the list
• Find nth node from the end of the list

3.3 Why Linked Lists?

There are many other data structures that do the same thing as linked lists. Before
discussing linked lists it is important to understand the difference between linked
lists and arrays. Both linked lists and arrays are used to store collections of data,
and since both are used for the same purpose, we need to differentiate their
usage. That means in which cases arrays are suitable and in which cases linked
lists are suitable.

3.4 Arrays Overview

One memory block is allocated for the entire array to hold the elements of the
array. The array elements can be accessed in constant time by using the index of
the particular element as the subscript.

Why Constant Time for Accessing Array Elements?

To access an array element, the address of an element is computed as an offset
from the base address of the array and one multiplication is needed to compute
what is supposed to be added to the base address to get the memory address of
the element. First the size of an element of that data type is calculated and then it
is multiplied with the index of the element to get the value to be added to the
base address.

This process takes one multiplication and one addition. Since these two
operations take constant time, we can say the array access can be performed in
constant time.

Advantages of Arrays
• Simple and easy to use
• Faster access to the elements (constant access)

Disadvantages of Arrays
• Preallocates all needed memory up front and wastes memory space for
indices in the array that are empty.
• Fixed size: The size of the array is static (specify the array size before
using it).
• One block allocation: To allocate the array itself at the beginning,
sometimes it may not be possible to get the memory for the complete
array (if the array size is big).
• Complex position-based insertion: To insert an element at a given
position, we may need to shift the existing elements. This will create
a position for us to insert the new element at the desired position. If
 the position at which we want to add an element is at the beginning,
then the shifting operation is more expensive.

Dynamic Arrays
Dynamic array (also called as growable array, resizable array, dynamic table, or
array list) is a random access, variable-size list data structure that allows
elements to be added or removed.

One simple way of implementing dynamic arrays is to initially start with some
fixed size array. As soon as that array becomes full, create the new array double
the size of the original array. Similarly, reduce the array size to half if the
elements in the array are less than half.

Note: We will see the implementation for dynamic arrays in the Stacks, Queues
and Hashing chapters.

Advantages of Linked Lists

Linked lists have both advantages and disadvantages. The advantage of linked
lists is that they can be expanded in constant time. To create an array, we must
allocate memory for a certain number of elements. To add more elements to the
array when full, we must create a new array and copy the old array into the new
array. This can take a lot of time.

We can prevent this by allocating lots of space initially but then we might
allocate more than we need and waste memory. With a linked list, we can start
with space for just one allocated element and add on new elements easily without
the need to do any copying and reallocating.

Issues with Linked Lists (Disadvantages)

There are a number of issues with linked lists. The main disadvantage of linked
lists is access time to individual elements. Array is random-access, which means
it takes O(1) to access any element in the array. Linked lists take O(n) for access
to an element in the list in the worst case. Another advantage of arrays in access
time is spacial locality in memory. Arrays are defined as contiguous blocks of
memory, and so any array element will be physically near its neighbors. This
greatly benefits from modern CPU caching methods.

Although the dynamic allocation of storage is a great advantage, the overhead

with storing and retrieving data can make a big difference. Sometimes linked
lists are hard to manipulate. If the last item is deleted, the last but one must then
have its pointer changed to hold a NULL reference. This requires that the list is
traversed to find the last but one link, and its pointer set to a NULL reference.

Finally, linked lists waste memory in terms of extra reference points.

3.5 Comparison of Linked Lists with Arrays & Dynamic Arrays

3.6 Singly Linked Lists

Generally “linked list” means a singly linked list. This list consists of a number
of nodes in which each node has a next pointer to the following element. The
link of the last node in the list is NULL, which indicates the end of the list.
Following is a type declaration for a linked list of integers:

Basic Operations on a List

• Traversing the list
• Inserting an item in the list
• Deleting an item from the list

Traversing the Linked List

Let us assume that the head points to the first node of the list. To traverse the list
we do the following
• Follow the pointers.
• Display the contents of the nodes (or count) as they are traversed.
• Stop when the next pointer points to NULL.

The ListLength() function takes a linked list as input and counts the number of
nodes in the list. The function given below can be used for printing the list data
with extra print function.

Time Complexity: O(n), for scanning the list of size n.

Space Complexity: O(1), for creating a temporary variable.

Singly Linked List Insertion

Insertion into a singly-linked list has three cases:

• Inserting a new node before the head (at the beginning)

• Inserting a new node after the tail (at the end of the list)
• Inserting a new node at the middle of the list (random location)

Note: To insert an element in the linked list at some position p, assume that after
inserting the element the position of this new node is p.

Inserting a Node in Singly Linked List at the Beginning

In this case, a new node is inserted before the current head node. Only one next
pointer needs to be modified (new node’s next pointer) and it can be done in two
steps:
• Update the next pointer of new node, to point to the current head.
 • Update head pointer to point to the new node.

Inserting a Node in Singly Linked List at the Ending

In this case, we need to modify two next pointers (last nodes next pointer and
new nodes next pointer).

• New nodes next pointer points to NULL.

 • Last nodes next pointer points to the new node.

Inserting a Node in Singly Linked List at the Middle

Let us assume that we are given a position where we want to insert the new
node. In this case also, we need to modify two next pointers.

• If we want to add an element at position 3 then we stop at position 2. That

means we traverse 2 nodes and insert the new node. For simplicity let
us assume that the second node is called position node. The new node
points to the next node of the position where we want to add this
node.
 • Position node’s next pointer now points to the new node.

Let us write the code for all three cases. We must update the first element pointer
in the calling function, not just in the called function. For this reason we need to
send a double pointer. The following code inserts a node in the singly linked list.
Note: We can implement the three variations of the insert operation separately.

Time Complexity: O(n), since, in the worst case, we may need to insert the node
at the end of the list.
Space Complexity: O(1), for creating one temporary variable.

Singly Linked List Deletion

Similar to insertion, here we also have three cases.

• Deleting the first node

• Deleting the last node
• Deleting an intermediate node.

Deleting the First Node in Singly Linked List

First node (current head node) is removed from the list. It can be done in two
steps:

• Create a temporary node which will point to the same node as that of
head.

• Now, move the head nodes pointer to the next node and dispose of the
temporary node.
Deleting the Last Node in Singly Linked List
In this case, the last node is removed from the list. This operation is a bit trickier
than removing the first node, because the algorithm should find a node, which is
previous to the tail. It can be done in three steps:
• Traverse the list and while traversing maintain the previous node address
also. By the time we reach the end of the list, we will have two
pointers, one pointing to the tail node and the other pointing to the
node before the tail node.

• Update previous node’s next pointer with NULL.

 • Dispose of the tail node.

Deleting an Intermediate Node in Singly Linked List

In this case, the node to be removed is always located between two nodes. Head
and tail links are not updated in this case. Such a removal can be done in two
steps:
• Similar to the previous case, maintain the previous node while traversing
the list. Once we find the node to be deleted, change the previous
node’s next pointer to the next pointer of the node to be deleted.
• Dispose of the current node to be deleted.
Time Complexity: O(n). In the worst case, we may need to delete the node at the
end of the list.
Space Complexity: O(1), for one temporary variable.

Deleting Singly Linked List

This works by storing the current node in some temporary variable and freeing
the current node. After freeing the current node, go to the next node with a
temporary variable and repeat this process for all nodes.

Time Complexity: O(n), for scanning the complete list of size n.

Space Complexity: O(1), for creating one temporary variable.

3.7 Doubly Linked Lists

The advantage of a doubly linked list (also called two – way linked list) is that
given a node in the list, we can navigate in both directions. A node in a singly
linked list cannot be removed unless we have the pointer to its predecessor. But
in a doubly linked list, we can delete a node even if we don’t have the previous
node’s address (since each node has a left pointer pointing to the previous node
and can move backward).

The primary disadvantages of doubly linked lists are:

 • Each node requires an extra pointer, requiring more space.
• The insertion or deletion of a node takes a bit longer (more pointer
operations).

Similar to a singly linked list, let us implement the operations of a doubly linked
list. If you understand the singly linked list operations, then doubly linked list
operations are obvious. Following is a type declaration for a doubly linked list of
integers:

Doubly Linked List Insertion

Insertion into a doubly-linked list has three cases (same as singly linked list):

• Inserting a new node before the head.

• Inserting a new node after the tail (at the end of the list).
• Inserting a new node at the middle of the list.

Inserting a Node in Doubly Linked List at the Beginning

In this case, new node is inserted before the head node. Previous and next
pointers need to be modified and it can be done in two steps:
• Update the right pointer of the new node to point to the current head node
(dotted link in below figure) and also make left pointer of new node
as NULL.
 • Update head node’s left pointer to point to the new node and make new
node as head. Head

Inserting a Node in Doubly Linked List at the Ending

In this case, traverse the list till the end and insert the new node.

• New node right pointer points to NULL and left pointer points to the end
of the list.
 • Update right pointer of last node to point to new node.

Inserting a Node in Doubly Linked List at the Middle

As discussed in singly linked lists, traverse the list to the position node and insert
the new node.

• New node right pointer points to the next node of the position node where
we want to insert the new node. Also, new node left pointer points to
the position node.
 • Position node right pointer points to the new node and the next node of
position node left pointer points to new node.

Now, let us write the code for all of these three cases. We must update the first
element pointer in the calling function, not just in the called function. For this
reason we need to send a double pointer.
Time Complexity: O(n). In the worst case, we may need to insert the node at the
end of the list.
Space Complexity: O(1), for creating one temporary variable.

Doubly Linked List Deletion

Similar to singly linked list deletion, here we have three cases:

• Deleting the first node

• Deleting the last node
• Deleting an intermediate node

Deleting the First Node in Doubly Linked List

In this case, the first node (current head node) is removed from the list. It can be
done in two steps:
• Create a temporary node which will point to the same node as that of
head.
 • Now, move the head nodes pointer to the next node and change the heads
left pointer to NULL. Then, dispose of the temporary node.

Deleting the Last Node in Doubly Linked List

This operation is a bit trickier than removing the first node, because the
algorithm should find a node, which is previous to the tail first. This can be done
in three steps:
• Traverse the list and while traversing maintain the previous node address
also. By the time we reach the end of the list, we will have two
pointers, one pointing to the tail and the other pointing to the node
before the tail.
• Update the next pointer of previous node to the tail node with NULL.

• Dispose the tail node.

Deleting an Intermediate Node in Doubly Linked List
In this case, the node to be removed is always located between two nodes, and
the head and tail links are not updated. The removal can be done in two steps:
• Similar to the previous case, maintain the previous node while also
traversing the list. Upon locating the node to be deleted, change the
previous node’s next pointer to the next node of the node to be
deleted.

• Dispose of the current node to be deleted.

Time Complexity: O(n), for scanning the complete list of size n.

Space Complexity: O(1), for creating one temporary variable.
3.8 Circular Linked Lists
In singly linked lists and doubly linked lists, the end of lists are indicated with
NULL value. But circular linked lists do not have ends. While traversing the
circular linked lists we should be careful; otherwise we will be traversing the list
infinitely. In circular linked lists, each node has a successor. Note that unlike
singly linked lists, there is no node with NULL pointer in a circularly linked list.
In some situations, circular linked lists are useful.

For example, when several processes are using the same computer resource
(CPU) for the same amount of time, we have to assure that no process accesses
the resource before all other processes do (round robin algorithm). The following
is a type declaration for a circular linked list of integers:

In a circular linked list, we access the elements using the head node (similar to
head node in singly linked list and doubly linked lists).

Counting Nodes in a Circular Linked List

The circular list is accessible through the node marked head. To count the nodes,
the list has to be traversed from the node marked head, with the help of a
dummy node current, and stop the counting when current reaches the starting
node head.

If the list is empty, head will be NULL, and in that case set count = 0.
Otherwise, set the current pointer to the first node, and keep on counting till the
current pointer reaches the starting node.

Time Complexity: O(n), for scanning the complete list of size n.

Space Complexity: O(1), for creating one temporary variable.

Printing the Contents of a Circular Linked List

We assume here that the list is being accessed by its head node. Since all the
nodes are arranged in a circular fashion, the tail node of the list will be the node
previous to the head node. Let us assume we want to print the contents of the
nodes starting with the head node. Print its contents, move to the next node and
continue printing till we reach the head node again.
Time Complexity: O(n), for scanning the complete list of size n.
Space Complexity: O(1), for temporary variable.

Inserting a Node at the End of a Circular Linked List

Let us add a node containing data, at the end of a list (circular list) headed by
head. The new node will be placed just after the tail node (which is the last node
of the list), which means it will have to be inserted in between the tail node and
the first node.

• Create a new node and initially keep its next pointer pointing to itself.
• Update the next pointer of the new node with the head node and also
traverse the list to the tail. That means in a circular list we should stop
at the node whose next node is head.

• Update the next pointer of the previous node to point to the new node and
we get the list as shown below.
Time Complexity: O(n), for scanning the complete list of size n.
Space Complexity: O(1), for temporary variable.

Inserting a Node at the Front of a Circular Linked List

The only difference between inserting a node at the beginning and at the end is
that, after inserting the new node, we just need to update the pointer. The steps
for doing this are given below:
• Create a new node and initially keep its next pointer pointing to itself.

• Update the next pointer of the new node with the head node and also
traverse the list until the tail. That means in a circular list we should
stop at the node which is its previous node in the list.
• Update the previous head node in the list to point to the new node.

• Make the new node as the head.

Time Complexity: O(n), for scanning the complete list of size n.
Space Complexity: O(1), for temporary variable.

Deleting the Last Node in a Circular Linked List

The list has to be traversed to reach the last but one node. This has to be named
as the tail node, and its next field has to point to the first node. Consider the
following list.

To delete the last node 40, the list has to be traversed till you reach 7. The next
field of 7 has to be changed to point to 60, and this node must be renamed pTail.
• Traverse the list and find the tail node and its previous node.

• Update the next pointer of tail node’s previous node to point to head.

• Dispose of the tail node.

Time Complexity: O(n), for scanning the complete list of size n. Space
Complexity: O(1), for a temporary variable.

Deleting the First Node in a Circular List

The first node can be deleted by simply replacing the next field of the tail node
with the next field of the first node.
• Find the tail node of the linked list by traversing the list. Tail node is the
previous node to the head node which we want to delete.

• Create a temporary node which will point to the head. Also, update the
tail nodes next pointer to point to next node of head (as shown
below).

• Now, move the head pointer to next node. Create a temporary node which
will point to head. Also, update the tail nodes next pointer to point to
next node of head (as shown below).
Time Complexity: O(n), for scanning the complete list of size n.
Space Complexity: O(1), for a temporary variable.

Applications of Circular List

Circular linked lists are used in managing the computing resources of a
computer. We can use circular lists for implementing stacks and queues.

3.9 A Memory-efficient Doubly Linked List

In conventional implementation, we need to keep a forward pointer to the next
item on the list and a backward pointer to the previous item. That means
elements in doubly linked list implementations consist of data, a pointer to the
next node and a pointer to the previous node in the list as shown below.

Conventional Node Definition

Recently a journal (Sinha) presented an alternative implementation of the doubly

linked list ADT, with insertion, traversal and deletion operations. This
implementation is based on pointer difference. Each node uses only one pointer
field to traverse the list back and forth.

New Node Definition

The ptrdiff pointer field contains the difference between the pointer to the next
node and the pointer to the previous node. The pointer difference is calculated by
using exclusive-or (⊕) operation.
ptrdiff = pointer to previous node ⊕ pointer to next node.

The ptrdiff of the start node (head node) is the ⊕ of NULL and next node (next
node to head). Similarly, the ptrdiff of end node is the ⊕ of previous node
(previous to end node) and NULL. As an example, consider the following linked
list.

In the example above,

• The next pointer of A is: NULL ⊕ B

• The next pointer of B is: A ⊕ C
• The next pointer of C is: B ⊕ D
• The next pointer of D is: C ⊕ NULL

Why does it work?

To find the answer to this question let us consider the properties of ⊕:

X ⊕ X=0
X ⊕0 = X
X ⊕ Y = Y ⊕ X (symmetric)
(X ⊕ Y) ⊕ Z = X ⊕ (Y ⊕ Z) (transitive)
For the example above, let us assume that we are at C node and want to move to
B. We know that C’s ptrdiff is defined as B ⊕ D. If we want to move to B,
performing ⊕ on C’s ptrdiff with D would give B. This is due to the fact that (B
⊕ D) ⊕ D = B(since, D ⊕ D= 0)
Similarly, if we want to move to D, then we have to apply ⊕ to C’s ptrdiff with
B to give D.
(B ⊕ D) ⊕ B = D (since, B © B=0)

From the above discussion we can see that just by using a single pointer, we can
move back and forth. A memory-efficient implementation of a doubly linked list
is possible with minimal compromising of timing efficiency.
3.10 Unrolled Linked Lists
One of the biggest advantages of linked lists over arrays is that inserting an
element at any location takes only O(1) time. However, it takes O(n) to search
for an element in a linked list. There is a simple variation of the singly linked list
called unrolled linked lists.

An unrolled linked list stores multiple elements in each node (let us call it a block
for our convenience). In each block, a circular linked list is used to connect all
nodes.

Assume that there will be no more than n elements in the unrolled linked list at
any time. To simplify this problem, all blocks, except the last one, should contain
exactly elements. Thus, there will be no more than blocks at any time.

Searching for an element in Unrolled Linked Lists

In unrolled linked lists, we can find the kth element in O( ): 1. Traverse the
list of blocks to the one that contains the kth node, i.e., the block. It takes
O( ) since we may find it by going through no more than blocks.
2. Find the (k mod )th node in the circular linked list of this block. It
also takes O( ) since there are no more than nodes in a single
block.
Inserting an element in Unrolled Linked Lists

When inserting a node, we have to re-arrange the nodes in the unrolled linked
list to maintain the properties previously mentioned, that each block contains
nodes. Suppose that we insert a node x after the ith node, and x should be
placed in the jth block. Nodes in the jth block and in the blocks after the jth block
have to be shifted toward the tail of the list so that each of them still have
nodes. In addition, a new block needs to be added to the tail if the last block of
the list is out of space, i.e., it has more than nodes.
Performing Shift Operation
Note that each shift operation, which includes removing a node from the tail of
the circular linked list in a block and inserting a node to the head of the circular
linked list in the block after, takes only O(1). The total time complexity of an
insertion operation for unrolled linked lists is therefore O( ); there are at most
O( ) blocks and therefore at most O( ) shift operations.
1. A temporary pointer is needed to store the tail of A.

2. In block A, move the next pointer of the head node to point to the
second-to-last node, so that the tail node of A can be removed.

3. Let the next pointer of the node, which will be shifted (the tail
node of A), point to the tail node of B.
 4. Let the next pointer of the head node of B point to the node temp
points to.

5. Finally, set the head pointer of B to point to the node temp points
to. Now the node temp points to becomes the new head node of
B.

6. temp pointer can be thrown away. We have completed the shift

operation to move the original tail node of A to become the new
head node of B.

Performance
With unrolled linked lists, there are a couple of advantages, one in speed and one
in space. First, if the number of elements in each block is appropriately sized
(e.g., at most the size of one cache line), we get noticeably better cache
performance from the improved memory locality. Second, since we have O(n/m)
links, where n is the number of elements in the unrolled linked list and m is the
number of elements we can store in any block, we can also save an appreciable
amount of space, which is particularly noticeable if each element is small.

Comparing Linked Lists and Unrolled Linked Lists

To compare the overhead for an unrolled list, elements in doubly linked list
implementations consist of data, a pointer to the next node, and a pointer to the
previous node in the list, as shown below.

Assuming we have 4 byte pointers, each node is going to take 8 bytes. But the
allocation overhead for the node could be anywhere between 8 and 16 bytes.
Let’s go with the best case and assume it will be 8 bytes. So, if we want to store
IK items in this list, we are going to have 16KB of overhead.

Now, let’s think about an unrolled linked list node (let us call it LinkedBlock). It
will look something like this:

Therefore, allocating a single node (12 bytes + 8 bytes of overhead) with an array
of 100 elements (400 bytes + 8 bytes of overhead) will now cost 428 bytes,or 4.28
bytes per element. Thinking about our IK items from above, it would take about
4.2KB of overhead, which is close to 4x better than our original list. Even if the
list becomes severely fragmented and the item arrays are only 1/2
full on average, this is still an improvement. Also, note that we can tune the
array size to whatever gets us the best overhead for our application.

Implementation

3.11 Linked Lists: Problems & Solutions

Problem-1 Implement Stack using Linked List.
Solution: Refer to Stacks chapter.

Problem-2 Find nth node from the end of a Linked List.

Solution: Brute-Force Method: Start with the first node and count the number
of nodes present after that node. If the number of nodes is < n – 1 then return
saying “fewer number of nodes in the list”. If the number of nodes is > n – 1 then
go to next node. Continue this until the numbers of nodes after current nodeare n
– 1.

Time Complexity: O(n2), for scanning the remaining list (from current node) for
each node.
Space Complexity: O(1).
Problem-3 Can we improve the complexity of Problem-2?

Solution: Yes, using hash table. As an example consider the following list.
In this approach, create a hash table whose entries are < position of node, node
address >. That means, key is the position of the node in the list and value is the
address of that node.

Position in List Address of Node

1 Address of 5 node
2 Address of 1 node
3 Address of 17 node
 4 Address of 4 node

By the time we traverse the complete list (for creating the hash table), we can
find the list length. Let us say the list length is M. To find nth from the end of
linked list, we can convert this to M-n + 1th from the beginning. Since we
already know the length of the list, it is just a matter of returning M-n + 1th key
value from the hash table.

Time Complexity: Time for creating the hash table, T(m) = O(m).
Space Complexity: Since we need to create a hash table of size m, O(m).
Problem-4 Can we use the Problem-3 approach for solving Problem-2
without creating the hash table?

Solution: Yes. If we observe the Problem-3 solution, what we are actually doing
is finding the size of the linked list. That means we are using the hash table to
find the size of the linked list. We can find the length of the linked list just by
starting at the head node and traversing the list.

So, we can find the length of the list without creating the hash table. After finding
the length, compute M – n + 1 and with one more scan we can get the M
– n+ 1th node from the beginning. This solution needs two scans: one for finding
the length of the list and the other for finding M – n+ 1th node from the beginning.

Time Complexity: Time for finding the length + Time for finding the M – n +
1th node from the beginning. Therefore, T(n) = O(n) + O(n) ≈ O(n). Space
Complexity: O(1). Hence, no need to create the hash table.
Problem-5 Can we solve Problem-2 in one scan?

Solution: Yes. Efficient Approach: Use two pointers pNthNode and pTemp.
Initially, both point to head node of the list. pNthNode starts moving only after
pTemp has made n moves.

From there both move forward until pTemp reaches the end of the list. As a result
pNthNode points to nth node from the end of the linked list.

Note: At any point of time both move one node at a time.

Time Complexity: O(n). Space Complexity: O(1).
Problem-6 Check whether the given linked list is either NULL-terminated
or ends in a cycle (cyclic).

Solution: Brute-Force Approach. As an example, consider the followinglinked

list which has a loop in it. The difference between this list and the regularlist is
that, in this list, there are two nodes whose next pointers are the same. In regular
singly linked lists (without a loop) each node’s next pointer is unique.

That means the repetition of next pointers indicates the existence of a loop.
One simple and brute force way of solving this is, start with the first node and
see whether there is any node whose next pointer is the current node’s address. If
there is a node with the same address then that indicates that some other node is
pointing to the current node and we can say a loop exists. Continue this process
for all the nodes of the linked list.

Does this method work? As per the algorithm, we are checking for the next
pointer addresses, but how do we find the end of the linked list (otherwise we
will end up in an infinite loop)?

Note: If we start with a node in a loop, this method may work depending on the
size of the loop.
Problem-7 Can we use the hashing technique for solving Problem-6?

Solution: Yes. Using Hash Tables we can solve this problem.

Algorithm:

• Traverse the linked list nodes one by one.

• Check if the address of the node is available in the hash table or not.
• If it is already available in the hash table, that indicates that we are
visiting the node that was already visited. This is possible only if the
given linked list has a loop in it.
• If the address of the node is not available in the hash table, insert that
 node’s address into the hash table.
• Continue this process until we reach the end of the linked list or we find
the loop.

Time Complexity; O(n) for scanning the linked list. Note that we are doing a
scan of only the input.
Space Complexity; O(n) for hash table.
Problem-8 Can we solve Problem-6 using the sorting technique?

Solution: No. Consider the following algorithm which is based on sorting. Then
we see why this algorithm fails.

Algorithm:

• Traverse the linked list nodes one by one and take all the next pointer
values into an array.
• Sort the array that has the next node pointers.
• If there is a loop in the linked list, definitely two next node pointers will
be pointing to the same node.
• After sorting if there is a loop in the list, the nodes whose next pointers
are the same will end up adjacent in the sorted list.
• If any such pair exists in the sorted list then we say the linked list has a
loop in it.

Time Complexity; O(nlogn) for sorting the next pointers array.

Space Complexity; O(n) for the next pointers array.

Problem with the above algorithm: The above algorithm works only if we can
find the length of the list. But if the list has a loop then we may end up in an
infinite loop. Due to this reason the algorithm fails.
Problem-9 Can we solve the Problem-6 in O(n)?

Solution: Yes. Efficient Approach (Memoryless Approach): This problem

was solved by Floyd. The solution is named the Floyd cycle finding algorithm. It
uses two pointers moving at different speeds to walk the linked list. Once they
enter the loop they are expected to meet, which denotes that there is a loop.
This works because the only way a faster moving pointer would point to the
same location as a slower moving pointer is if somehow the entire list or a part
of it is circular. Think of a tortoise and a hare running on a track. The faster
running hare will catch up with the tortoise if they are running in a loop. As an
example, consider the following example and trace out the Floyd algorithm.
From the diagrams below we can see that after the final step they are meeting at
some point in the loop which may not be the starting point of the loop.

Note: slowPtr (tortoise) moves one pointer at a time and fastPtr (hare) moves
two pointers at a time.
slowPtr

fastPtr

fastPtr

slowPtr

fastPt

fastPtr slowPt
slowPtrfastPtr
Time Complexity: O(n). Space Complexity: O(1).
Problem-10 are given a pointer to the first element of a linked list L. Thereare
two possibilities for L: it either ends (snake) or its last element points back
to one of the earlier elements in the list (snail). Give an algorithm thattests
whether a given list L is a snake or a snail.
Solution: It is the same as Problem-6.
Problem-11 Check whether the given linked list is NULL-terminated or not.
If there is a cycle find the start node of the loop.
Solution: The solution is an extension to the solution in Problem-9. After finding
the loop in the linked list, we initialize the slowPtr to the head of the linked list.
From that point onwards both slowPtr and fastPtr move only one node at a time.
The point at which they meet is the start of the loop. Generally we use this
method for removing the loops.
Time Complexity: O(n). Space Complexity: O(1).
Problem-12 From the previous discussion and problems we understand thatthe
meeting of tortoise and hare concludes the existence of the loop, but how
does moving the tortoise to the beginning of the linked list while keeping
the hare at the meeting place, followed by moving both one step ata time,
make them meet at the starting point of the cycle?
Solution: This problem is at the heart of number theory. In the Floyd cycle
finding algorithm, notice that the tortoise and the hare will meet when they are n
× L, where L is the loop length. Furthermore, the tortoise is at the midpoint
between the hare and the beginning of the sequence because of the way they
move. Therefore the tortoise is n × L away from the beginning of the sequence as
well. If we move both one step at a time, from the position of the tortoise and
from the start of the sequence, we know that they will meet as soon as both are
in the loop, since they are n × L, a multiple of the loop length, apart. One of them
is already in the loop, so we just move the other one in single step until it enters
the loop, keeping the other n × L away from it at all times.
Problem-13 In the Floyd cycle finding algorithm, does it work if we use steps
2 and 3 instead of 1 and 2?

Solution: Yes, but the complexity might be high. Trace out an example.
Problem-14 Check whether the given linked list is NULL-terminated. If there
is a cycle, find the length of the loop.
Solution: This solution is also an extension of the basic cycle detection problem.
After finding the loop in the linked list, keep the slowPtr as it is. The fastPtr
keeps on moving until it again comes back to slowPtr. While moving fastPtr,
use a counter variable which increments at the rate of 1.
4.1 What is a Stack?
A stack is a simple data structure used for storing data (similar to Linked Lists).
In a stack, the order in which the data arrives is important. A pile of plates in a
cafeteria is a good example of a stack. The plates are added to the stack as they
are cleaned and they are placed on the top. When a plate, is required it is taken
from the top of the stack. The first plate placed on the stack is the last one to be
used.

Definition: A stack is an ordered list in which insertion and deletion are done at
one end, called top. The last element inserted is the first one to be deleted. Hence,
it is called the Last in First out (LIFO) or First in Last out (FILO) list.
Special names are given to the two changes that can be made to a stack. When
an element is inserted in a stack, the concept is called push, and when an element
is removed from the stack, the concept is called pop. Trying to pop out an empty
stack is called underflow and trying to push an element in a full stack is called
overflow. Generally, we treat them as exceptions. As an example, consider the
snapshots of the stack.

4.2 How Stacks are used

Consider a working day in the office. Let us assume a developer is working on a
long-term project. The manager then gives the developer a new task which is
more important. The developer puts the long-term project aside and begins work
on the new task. The phone rings, and this is the highest priority as it must be
answered immediately. The developer pushes the present task into the pending
tray and answers the phone.

When the call is complete the task that was abandoned to answer the phone is
retrieved from the pending tray and work progresses. To take another call, it may
have to be handled in the same manner, but eventually the new task will be
finished, and the developer can draw the long-term project from the pending tray
and continue with that.
4.3 Stack ADT
The following operations make a stack an ADT. For simplicity, assume the data
is an integer type.

Main stack operations

• Push (int data): Inserts data onto stack.
• int Pop(): Removes and returns the last inserted element from the stack.

Auxiliary stack operations

• int Top(): Returns the last inserted element without removing it.
• int Size(): Returns the number of elements stored in the stack.
• int IsEmptyStack(): Indicates whether any elements are stored in the
stack or not.
• int IsFullStack(): Indicates whether the stack is full or not.

Exceptions
Attempting the execution of an operation may sometimes cause an error
condition, called an exception. Exceptions are said to be “thrown” by an
operation that cannot be executed. In the Stack ADT, operations pop and top
cannot be performed if the stack is empty. Attempting the execution of pop (top)
on an empty stack throws an exception. Trying to push an element in a full stack
throws an exception.

4.4 Applications
Following are some of the applications in which stacks play an important role.

Direct applications
• Balancing of symbols
• Infix-to-postfix conversion
• Evaluation of postfix expression
 • Implementing function calls (including recursion)
• Finding of spans (finding spans in stock markets, refer to Problems
section)
• Page-visited history in a Web browser [Back Buttons]
• Undo sequence in a text editor
• Matching Tags in HTML and XML

Indirect applications
• Auxiliary data structure for other algorithms (Example: Tree traversal
algorithms)
• Component of other data structures (Example: Simulating queues, refer
Queues chapter)

4.5 Implementation
There are many ways of implementing stack ADT; below are the commonly
used methods.

• Simple array based implementation

• Dynamic array based implementation
• Linked lists implementation

Simple Array Implementation

This implementation of stack ADT uses an array. In the array, we add elements
from left to right and use a variable to keep track of the index of the top element.

The array storing the stack elements may become full. A push operation will
then throw a full stack exception. Similarly, if we try deleting an element from
an empty stack it will throw stack empty exception.
Performance & Limitations
Performance

Let n be the number of elements in the stack. The complexities of stack

operations with this representation can be given as:

Space Complexity (for n push operations) O(n)

Time Complexity of Push() O(1)
Time Complexity of Pop() O(1)
Time Complexity of Size() O(1)
Time Complexity of IsEmptyStack() O(1)
Time Complexity of IsFullStackf) O(1)
Time Complexity of DeleteStackQ O(1)

Limitations
The maximum size of the stack must first be defined and it cannot be changed.
Trying to push a new element into a full stack causes an implementation-specific
exception.

Dynamic Array Implementation

First, let’s consider how we implemented a simple array based stack. We took
one index variable top which points to the index of the most recently inserted
element in the stack. To insert (or push) an element, we increment top index and
then place the new element at that index.

Similarly, to delete (or pop) an element we take the element at top index and
then decrement the top index. We represent an empty queue with top value equal
to –1. The issue that still needs to be resolved is what we do when all the slots in
the fixed size array stack are occupied?
First try: What if we increment the size of the array by 1 every time the stack is
full?
• Push(); increase size of S[] by 1
• Pop(): decrease size of S[] by 1

Problems with this approach?

This way of incrementing the array size is too expensive. Let us see the reason
for this. For example, at n = 1, to push an element create a new array of size 2
and copy all the old array elements to the new array, and at the end add the new
element. At n = 2, to push an element create a new array of size 3 and copy all
the old array elements to the new array, and at the end add the new element.

Similarly, at n = n – 1, if we want to push an element create a new array of size n

and copy all the old array elements to the new array and at the end add the new
element. After n push operations the total time T(n) (number of copy operations)
is proportional to 1 + 2 + ... + n ≈ O(n2).

Alternative Approach: Repeated Doubling

Let us improve the complexity by using the array doubling technique. If the array
is full, create a new array of twice the size, and copy the items. With this
approach, pushing n items takes time proportional to n (not n2).

For simplicity, let us assume that initially we started with n = 1 and moved up to
n = 32. That means, we do the doubling at 1,2,4,8,16. The other way of
analyzing the same approach is: at n = 1, if we want to add (push) an element,
double the current size of the array and copy all the elements of the old array to
the new array.

At n = 1, we do 1 copy operation, at n = 2, we do 2 copy operations, and at n =

4, we do 4 copy operations and so on. By the time we reach n = 32, the total
number of copy operations is 1+2 + 4 + 8+16 = 31 which is approximately equal
to 2n value (32). If we observe carefully, we are doing the doubling operation
logn times. Now, let us generalize the discussion. For n push operations we
double the array size logn times. That means, we will have logn terms in the
expression below. The total time T(n) of a series of n push operations is
proportional to
 T(n) is O(n) and the amortized time of a push operation is O(1) .

Performance
Let n be the number of elements in the stack. The complexities for operations
with this representation can be given as:

Space Complexity (for n push operations) O(n)

Time Complexity of CreateStack() O(1)
Time Complexity of PushQ O(1) (Average)
Time Complexity of PopQ O(1)
Time Complexity of Top() O(1)
Time Complexity of IsEmpryStackf) O(1))
Time Complexity of IsFullStackf) O(1)
Time Complexity of DeleteStackQ O(1)

Note: Too many doublings may cause memory overflow exception.

Linked List Implementation

The other way of implementing stacks is by using Linked lists. Push operation is
implemented by inserting element at the beginning of the list. Pop operation is
implemented by deleting the node from the beginning (the header/top node).
4.6 Stacks: Problems & Solutions
Problem-1 Discuss how stacks can be used for checking balancing ofsymbols.
Solution: Stacks can be used to check whether the given expression has
balanced symbols. This algorithm is very useful in compilers. Each time the
parser reads one character at a time. If the character is an opening delimiter such
as (, {, or [- then it is written to the stack. When a closing delimiter isencountered
like ), }, or ]-the stack is popped.

The opening and closing delimiters are then compared. If they match, the parsing
of the string continues. If they do not match, the parser indicates that there is an
error on the line. A linear-time O(n) algorithm based on stack can be given as:

Algorithm:
a) Create a stack.
b) while (end of input is not reached) {
1) If the character read is not a symbol to be balanced, ignore it.
2) If the character is an opening symbol like (, [, {, push it onto
 the stack
3) If it is a closing symbol like ),],}, then if the stack is empty
report an error. Otherwise pop the stack.
4) If the symbol popped is not the corresponding opening
symbol, report an error.

c) At end of input, if the stack is not empty report an error

Examples:

For tracing the algorithm let us assume that the input is: () (() [()])
Time Complexity: O(n). Since we are scanning the input only once. Space
Complexity: O(n) [for stack].
Problem-2 Discuss infix to postfix conversion algorithm using stack.
Solution: Before discussing the algorithm, first let us see the definitions of infix,
prefix and postfix expressions.

Infix: An infix expression is a single letter, or an operator, proceeded by one

infix string and followed by another Infix string.

Prefix: A prefix expression is a single letter, or an operator, followed by two

prefix strings. Every prefix string longer than a single variable contains an
operator, first operand and second operand.

Postfix: A postfix expression (also called Reverse Polish Notation) is a single

letter or an operator, preceded by two postfix strings. Every postfix string longer
than a single variable contains first and second operands followed by an
operator.

Prefix and postfix notions are methods of writing mathematical expressions

without parenthesis. Time to evaluate a postfix and prefix expression is O(n),
where n is the number of elements in the array.

Now, let us focus on the algorithm. In infix expressions, the operator precedence
is implicit unless we use parentheses. Therefore, for the infix to postfix
conversion algorithm we have to define the operator precedence (or priority)
inside the algorithm.

The table shows the precedence and their associativity (order of evaluation)
among operators.
Token Operator Precedence Associativity
() fu n ction call 17 left-to-r igh t
array element
-
[I
.
-- ++
-- ++
struct or union member
increment, decrement
dec re me n t, inc rem ent
16
15
left-to-r igh t
right-to-left
! logical not
- one's complement
-+ unary minus or plus
&* address or indirection
sizeof size (in bytes)
(type) type cast 14 right-to-left
k I% multiplicative 13 Left-t o-righ t
+ - bina ry add or subtract 12 left-to -righ t
<< >> shift 11 left-to-righ t
> >= relational 10 left-to-righ t
< <=
== 1= equalitv 9 left-to-r igh t
& bitwise and 8 left-to-righ t
I\ bitwise exclusive or 7 left-to-righ t
I bitv.rise or 6 left -to-righ t
&& logical and 5 left-to -righ t
logical or 4 left-to-righ t
11
?: conditional 3 right-to-left
= += -= / = *= %= assignment 2 right-to-left
<<= >>=
& = /:\
comma 1 left-to-righ t
'
Important Properties

• Let us consider the infix expression 2 + 3*4 and its postfix equivalent
234*+. Notice that between infix and postfix the order of the numbers
(or operands) is unchanged. It is 2 3 4 in both cases. But theorder of
the operators * and + is affected in the two expressions.
• Only one stack is enough to convert an infix expression to postfix
expression. The stack that we use in the algorithm will be used to
change the order of operators from infix to postfix. The stack we use
will only contain operators and the open parentheses symbol ‘(‘.

Postfix expressions do not contain parentheses. We shall not output the

parentheses in the postfix output.

Algorithm:
a) Create a stack
b) for each character t in the input stream}

c) pop and output tokens until the stack is empty

For better understanding let us trace out an example: A * B- (C + D) + E

Problem-3 Discuss postfix evaluation using stacks?
Solution:

Algorithm:
1 Scan the Postfix string from left to right.
2 Initialize an empty stack.
3 Repeat steps 4 and 5 till all the characters are scanned.
4 If the scanned character is an operand, push it onto the stack.
5 If the scanned character is an operator, and if the operator is a unary
operator, then pop an element from the stack. If the operator is a
binary operator, then pop two elements from the stack. After popping
the elements, apply the operator to those popped elements. Let the
result of this operation be retVal onto the stack.
6 After all characters are scanned, we will have only one element in the
stack.
7 Return top of the stack as result.

Example: Let us see how the above-mentioned algorithm works using an

example. Assume that the postfix string is 123*+5-.

Initially the stack is empty. Now, the first three characters scanned are 1, 2 and
3, which are operands. They will be pushed into the stack in that order.

The next character scanned is “”, which is an operator. Thus, we pop the top two
elements from the stack and perform the “” operation with the two operands.
The second operand will be the first element that is popped.

The value of the expression (2*3) that has been evaluated (6) is pushed into the
stack.
The next character scanned is “+”, which is an operator. Thus, we pop the top
two elements from the stack and perform the “+” operation with the two
operands. The second operand will be the first element that is popped.

The value of the expression (1+6) that has been evaluated (7) is pushed into the
stack.
The next character scanned is “5”, which is added to the stack.

The next character scanned is “-”, which is an operator. Thus, we pop the top
two elements from the stack and perform the “-” operation with the twooperands.
The second operand will be the first element that is popped.
The value of the expression(7-5) that has been evaluated(23) is pushed into the
stack.

Now, since all the characters are scanned, the remaining element in the stack
(there will be only one element in the stack) will be returned. End result:

• Postfix String : 123*+5-

• Result : 2
Problem-4 Can we evaluate the infix expression with stacks in one pass?
Solution: Using 2 stacks we can evaluate an infix expression in 1 pass without
converting to postfix.

Algorithm:
1) Create an empty operator stack
2) Create an empty operand stack
3) For each token in the input string
a. Get the next token in the infix string
b. If next token is an operand, place it on the operand stack
c. If next token is an operator
i. Evaluate the operator (next op)
4) While operator stack is not empty, pop operator and operands (left and
right), evaluate left operator right and push result onto operand stack
 5) Pop result from operator stack
Problem-5 How to design a stack such that GetMinimum( ) should be O(1)?
Solution: Take an auxiliary stack that maintains the minimum of all values in
the stack. Also, assume that each element of the stack is less than its below
elements. For simplicity let us call the auxiliary stack min stack.

When we pop the main stack, pop the min stack too. When we push the main
stack, push either the new element or the current minimum, whichever is lower.
At any point, if we want to get the minimum, then we just need to return the top
element from the min stack. Let us take an example and trace it out. Initially let
us assume that we have pushed 2, 6, 4, 1 and 5. Based on the above-mentioned
algorithm the min stack will look like:

Main stack Min stack

5 → top 1 → top
1 1
4 2
6 2
2 2

After popping twice we get:

Main stack Min stack

4 -→ top 2 → top
6 2
2 2

Based on the discussion above, now let us code the push, pop and
GetMinimum() operations.

Time complexity: O(1). Space complexity: O(n) [for Min stack]. This algorithm
has much better space usage if we rarely get a “new minimum or equal”.
Problem-6 For Problem-5 is it possible to improve the space complexity?
Solution: Yes. The main problem of the previous approach is, for each push
operation we are pushing the element on to min stack also (either the new
element or existing minimum element). That means, we are pushing the
duplicate minimum elements on to the stack.

Now, let us change the algorithm to improve the space complexity. We still have
the min stack, but we only pop from it when the value we pop from the main
stack is equal to the one on the min stack. We only push to the min stack when
the value being pushed onto the main stack is less than or equal to the current
min value. In this modified algorithm also, if we want to get the minimum then
we just need to return the top element from the min stack. For example, taking
the original version and pushing 1 again, we’d get:

Main stack Min stack

1 → top
5
1
4 1 → top
6 1
2 2

Popping from the above pops from both stacks because 1 == 1, leaving:

Main stack Min stack

5 → top
1
4
6 1 → top

2 2

Popping again only pops from the main stack, because 5 > 1:

Main stack Min stack

1 → top
4
 6 1 → top
2 2

Popping again pops both stacks because 1 == 1:

Main stack Min stack

4 → top
6
2 2 → top
Note: The difference is only in push & p
5.1 What is a Queue?
A queue is a data structure used for storing data (similar to Linked Lists and
Stacks). In queue, the order in which data arrives is important. In general, a
queue is a line of people or things waiting to be served in sequential order
starting at the beginning of the line or sequence.

Definition: A queue is an ordered list in which insertions are done at one end
(rear) and deletions are done at other end (front). The first element to be inserted
is the first one to be deleted. Hence, it is called First in First out (FIFO) or Last
in Last out (LILO) list.
Similar to Stacks, special names are given to the two changes that can be made
to a queue. When an element is inserted in a queue, the concept is called
EnQueue, and when an element is removed from the queue, the concept is called
DeQueue.

DeQueueing an empty queue is called underflow and EnQueuing an element in a

full queue is called overflow. Generally, we treat them as exceptions. As an
example, consider the snapshot of the queue.

5.2 How are Queues Used?

The concept of a queue can be explained by observing a line at a reservation
counter. When we enter the line we stand at the end of the line and the person
who is at the front of the line is the one who will be served next. He will exit the
queue and be served.

As this happens, the next person will come at the head of the line, will exit the
queue and will be served. As each person at the head of the line keeps exiting the
queue, we move towards the head of the line. Finally we will reach the head of
the line and we will exit the queue and be served. This behavior is very useful in
cases where there is a need to maintain the order of arrival.

5.3 Queue ADT

The following operations make a queue an ADT. Insertions and deletions in the
queue must follow the FIFO scheme. For simplicity we assume the elements are
integers.

Main Queue Operations

• EnQueue(int data): Inserts an element at the end of the queue

• int DeQueue(): Removes and returns the element at the front of the queue
Auxiliary Queue Operations

• int Front(): Returns the element at the front without removing it

• int QueueSize(): Returns the number of elements stored in the queue
• int IsEmptyQueueQ: Indicates whether no elements are stored in the
queue or not

5.4 Exceptions
Similar to other ADTs, executing DeQueue on an empty queue throws an
“Empty Queue Exception” and executing EnQueue on a full queue throws “Full
Queue Exception”.

5.5 Applications
Following are some of the applications that use queues.

Direct Applications

• Operating systems schedule jobs (with equal priority) in the order of

arrival (e.g., a print queue).
• Simulation of real-world queues such as lines at a ticket counter or any
other first-come first-served scenario requires a queue.
• Multiprogramming.
• Asynchronous data transfer (file IO, pipes, sockets).
• Waiting times of customers at call center.
• Determining number of cashiers to have at a supermarket.

Indirect Applications

• Auxiliary data structure for algorithms

• Component of other data structures
5.6 Implementation
There are many ways (similar to Stacks) of implementing queue operations and
some of the commonly used methods are listed below.

• Simple circular array based implementation

• Dynamic circular array based implementation
• Linked list implementation

Why Circular Arrays?

First, let us see whether we can use simple arrays for implementing queues as we
have done for stacks. We know that, in queues, the insertions are performed at
one end and deletions are performed at the other end. After performing some
insertions and deletions the process becomes easy to understand.

In the example shown below, it can be seen clearly that the initial slots of the
array are getting wasted. So, simple array implementation for queue is not
efficient. To solve this problem we assume the arrays as circular arrays. That
means, we treat the last element and the first array elements as contiguous. With
this representation, if there are any free slots at the beginning, the rear pointer
can easily go to its next free slot.

Note: The simple circular array and dynamic circular array implementations are
very similar to stack array implementations. Refer to Stacks chapter for analysis
of these implementations.
Simple Circular Array Implementation

This simple implementation of Queue ADT uses an array. In the array, we add
elements circularly and use two variables to keep track of the start element and
end element. Generally, front is used to indicate the start element and rear is
used to indicate the end element in the queue. The array storing the queue
elements may become full. An EnQueue operation will then throw a full queue
exception. Similarly, if we try deleting an element from an empty queue it will
throw empty queue exception.

Note: Initially, both front and rear points to -1 which indicates that the queue is
empty.
Performance and Limitations
Performance: Let n be the number of elements in the queue:
Space Complexity (for n EnQueue operations) O(n)
Time Complexity of EnQueue() O(1)
Time Complexity of DeQueue() O(1)
Time Complexity of IsEmptyQueue() O(1)
Time Complexity of IsFullQueue() O(1)
Time Complexity of QueueSize() O(1)
Time Complexity of DeleteQueue() O(1)
Limitations: The maximum size of the queue must be defined as prior and
cannot be changed. Trying to EnQueue a new element into a full queue causes
an implementation-specific exception.
Linked List Implementation
Another way of implementing queues is by using Linked lists. EnQueue
operation is implemented by inserting an element at the end of the list. DeQueue
operation is implemented by deleting an element from the beginning of the list.

Performance
Let n be the number of elements in the queue, then

Space Complexity (for n EnQueue operations) O(n)

Time Complexity of EnQueue() O(1) (Average)
Time Complexity of DeQueue() O(1)
Time Complexity of IsEmptyQueue() O(1)
Time Complexity of DeleteQueue() O(1)

Comparison of Implementations
Note: Comparison is very similar to stack implementations and Stacks chapter.

5.7 Queues: Problems & Solutions

Problem-1 Give an algorithm for reversing a queue Q. To access the queue,
 we are only allowed to use the methods of queue ADT.
Solution:

Time Complexity: O(n).

Problem-2 How can you implement a queue using two stacks?
Solution: Let SI and S2 be the two stacks to be used in the implementation of
queue. All we have to do is to define the EnQueue and DeQueue operations for
the queue.

EnQueue Algorithm

• Just push on to stack S1

Time Complexity: O(1).

DeQueue Algorithm

• If stack S2 is not empty then pop from S2 and return that element.
• If stack is empty, then transfer all elements from SI to S2 and pop the top
element from S2 and return that popped element [we can optimize
the code a little by transferring only n – 1 elements from SI to S2 and
pop the nth element from SI and return that popped element].
• If stack S1 is also empty then throw error.
Time Complexity: From the algorithm, if the stack S2 is not empty then the
complexity is O(1). If the stack S2 is empty, then we need to transfer the
elements from SI to S2. But if we carefully observe, the number of transferred
elements and the number of popped elements from S2 are equal. Due to this the
average complexity of pop operation in this case is O(1).The amortized
complexity of pop operation is O(1).
Problem-3 Show how you can efficiently implement one stack using two
queues. Analyze the running time of the stack operations.
Solution: Yes, it is possible to implement the Stack ADT using 2
implementations of the Queue ADT. One of the queues will be used to store the
elements and the other to hold them temporarily during the pop and top methods.
The push method would enqueue the given element onto the storage queue. The
top method would transfer all but the last element from the storage queue onto
the temporary queue, save the front element of the storage queue to be returned,
transfer the last element to the temporary queue, then transfer all elements back
to the storage queue. The pop method would do the same as top, except instead
of transferring the last element onto the temporary queue after saving it for
return, that last element would be discarded. Let Q1 and Q2 be the two queues to
be used in the implementation of stack. All we have to do is to define the push
and pop operations for the stack.
In the algorithms below, we make sure that one queue is always empty.

Push Operation Algorithm: Insert the element in whichever queue is not

empty.

• Check whether queue Q1 is empty or not. If Q1 is empty then Enqueue

the element into Q2.
• Otherwise EnQueue the element into Q1.

Time Complexity: O(1).

Pop Operation Algorithm: Transfer n – 1 elements to the other queue and

delete last from queue for performing pop operation.

• If queue Q1 is not empty then transfer n – 1 elements from Q1 to Q2 and

then, DeQueue the last element of Q1 and return it.
• If queue Q2 is not empty then transfer n – 1 elements from Q2 to Q1 and
then, DeQueue the last element of Q2 and return it.
Time Complexity: Running time of pop operation is O(n) as each time pop is
called, we are transferring all the elements from one queue to the other.
Problem-4 Maximum sum in sliding window: Given array A[] with sliding
window of size w which is moving from the very left of the array tothe very
right. Assume that we can only see the w numbers in the window.Each time
the sliding window moves rightwards by one position. For example: The
array is [1 3 -1 -3 5 3 6 7], and w is 3.

Window position Max

[1 3 -1] -3 5 3 6 7 3
1 [3 -1 -3] 5 3 6 7 3
1 3 [-1 -3 5] 3 6 7 5
 1 3 -1 [-3 5 3] 6 7 5
1 3 -1 -3 [5 3 6] 7 6
1 3 -1 -3 5 [3 6 7] 7

Input: A long array A[], and a window width w. Output: An array B[],
B[i] is the maximum value from A[i] to A[i+w-1]. Requirement: Find a
good optimal way to get B[i]
Solution: This problem can be solved with doubly ended queue (which supports
insertion and deletion at both ends). Refer Priority Queues chapter for
algorithms.
Problem-5 Given a queue Q containing n elements, transfer these items on
to a stack S (initially empty) so that front element of Q appears at the top
of the stack and the order of all other items is preserved. Using enqueue
and dequeue operations for the queue, and push and pop operations for the
stack, outline an efficient O(n) algorithm to accomplish the above task,
using only a constant amount of additional storage.
Solution: Assume the elements of queue Q are a1:a2 ...an. Dequeuing all
elements and pushing them onto the stack will result in a stack with an at the top
and a1 at the bottom. This is done in O(n) time as dequeue and each push require
constant time per operation. The queue is now empty. By popping all elements
and pushing them on the queue we will get a1 at the top of the stack. This is done
again in O(n) time.

As in big-oh arithmetic we can ignore constant factors. The process is carried out
in O(n) time. The amount of additional storage needed here has to be big enough
to temporarily hold one item.
Problem-6 A queue is set up in a circular array A[O..n - 1] with front and rear
defined as usual. Assume that n – 1 locations in the array are availablefor
storing the elements (with the other element being used to detect full/empty
condition). Give a formula for the number of elements in the queue in terms
of rear, front, and n.
Solution: Consider the following figure to get a clear idea of the queue.
 • Rear of the queue is somewhere clockwise from the front.
• To enqueue an element, we move rear one position clockwise and write
the element in that position.
• To dequeue, we simply move front one position clockwise.
• Queue migrates in a clockwise direction as we enqueue and dequeue.
• Emptiness and fullness to be checked carefully.
• Analyze the possible situations (make some drawings to see where front
and rear are when the queue is empty, and partially and totally filled).
We will get this:

Problem-7 What is the most appropriate data structure to print elements of

queue in reverse order?
Solution: Stack.
6.1 What is a Tree?
A tree is a data structure similar to a linked list but instead of each node pointing
simply to the next node in a linear fashion, each node points to a number of
nodes. Tree is an example of a non-linear data structure. A tree structure is a way
of representing the hierarchical nature of a structure in a graphical form.

In trees ADT (Abstract Data Type), the order of the elements is not important. If
we need ordering information, linear data structures like linked lists, stacks,
queues, etc. can be used.
6.2 Glossary

• The root of a tree is the node with no parents. There can be at most one
root node in a tree (node A in the above example).
• An edge refers to the link from parent to child (all links in the figure).
• A node with no children is called leaf node (E,J,K,H and I).
• Children of same parent are called siblings (B,C,D are siblings of A, and
E,F are the siblings of B).
• A node p is an ancestor of node q if there exists a path from root to q and
p appears on the path. The node q is called a descendant of p. For
example, A,C and G are the ancestors of if.
• The set of all nodes at a given depth is called the level of the tree (B, C
and D are the same level). The root node is at level zero.
• The depth of a node is the length of the path from the root to the node
(depth of G is 2, A – C – G).
• The height of a node is the length of the path from that node to the deepest
node. The height of a tree is the length of the path from the root to
the deepest node in the tree. A (rooted) tree with only one node (the
root) has a height of zero. In the previous example, the height of B is
2 (B – F – J).
• Height of the tree is the maximum height among all the nodes in the tree
and depth of the tree is the maximum depth among all the nodes in
the tree. For a given tree, depth and height returns the same value.
But for individual nodes we may get different results.
• The size of a node is the number of descendants it has including itself
(the size of the subtree C is 3).
• If every node in a tree has only one child (except leaf nodes) then we call
such trees skew trees. If every node has only left child then we call
them left skew trees. Similarly, if every node has only right child then
we call them right skew trees.
6.3 Binary Trees
A tree is called binary tree if each node has zero child, one child or two children.
Empty tree is also a valid binary tree. We can visualize a binary tree as
consisting of a root and two disjoint binary trees, called the left and right subtrees
of the root.

Generic Binary Tree

6.4 Types of Binary Trees
Strict Binary Tree: A binary tree is called strict binary tree if each node has
exactly two children or no children.

Full Binary Tree: A binary tree is called full binary tree if each node has
exactly two children and all leaf nodes are at the same level.
Complete Binary Tree: Before defining the complete binary tree, let us assume
that the height of the binary tree is h. In complete binary trees, if we give
numbering for the nodes by starting at the root (let us say the root node has 1)
then we get a complete sequence from 1 to the number of nodes in the tree. While
traversing we should give numbering for NULL pointers also. A binary tree is
called complete binary tree if all leaf nodes are at height h or h – 1 and also
without any missing number in the sequence.

6.5 Properties of Binary Trees

For the following properties, let us assume that the height of the tree is h. Also,
assume that root node is at height zero.
From the diagram we can infer the following properties:

• The number of nodes n in a full binary tree is 2h+1 – 1. Since, there are h
levels we need to add all nodes at each level [20 + 21+ 22 + ··· + 2h =
2h+1 – 1].
• The number of nodes n in a complete binary tree is between 2h
(minimum) and 2h+1 – 1 (maximum). For more information on this,
 refer to Priority Queues chapter.
• The number of leaf nodes in a full binary tree is 2h.
• The number of NULL links (wasted pointers) in a complete binary tree of
n nodes is n + 1.

Structure of Binary Trees

Now let us define structure of the binary tree. For simplicity, assume that the
data of the nodes are integers. One way to represent a node (which contains data)
is to have two links which point to left and right children along with data fields
as shown below:

Note: In trees, the default flow is from parent to children and it is not mandatory
to show directed branches. For our discussion, we assume both the
representations shown below are the same.

Operations on Binary Trees

Basic Operations

• Inserting an element into a tree

• Deleting an element from a tree
• Searching for an element
• Traversing the tree

Auxiliary Operations

• Finding the size of the tree

• Finding the height of the tree
• Finding the level which has maximum sum
• Finding the least common ancestor (LCA) for a given pair of nodes, and
many more.

Applications of Binary Trees

Following are the some of the applications where binary trees play an important
role:

• Expression trees are used in compilers.

• Huffman coding trees that are used in data compression algorithms.
• Binary Search Tree (BST), which supports search, insertion and deletion
on a collection of items in O(logn) (average).
• Priority Queue (PQ), which supports search and deletion of minimum (or
maximum) on a collection of items in logarithmic time (in worst
case).

6.6 Binary Tree Traversals

In order to process trees, we need a mechanism for traversing them, and that
forms the subject of this section. The process of visiting all nodes of a tree is
called tree traversal. Each node is processed only once but it may be visited
more than once. As we have already seen in linear data structures (like linked
lists, stacks, queues, etc.), the elements are visited in sequential order. But, in
tree structures there are many different ways.
Tree traversal is like searching the tree, except that in traversal the goal is to
move through the tree in a particular order. In addition, all nodes are processed
in the traversal but searching stops when the required node is found.

Traversal Possibilities
Starting at the root of a binary tree, there are three main steps that can be
performed and the order in which they are performed defines the traversal type.
These steps are: performing an action on the current node (referred to as
“visiting” the node and denoted with “D”), traversing to the left child node
(denoted with “L”), and traversing to the right child node (denoted with “R”).
This process can be easily described through recursion. Based on the above
definition there are 6 possibilities:
1. LDR: Process left subtree, process the current node data and then
process right subtree
2. LRD: Process left subtree, process right subtree and then process the
current node data
3. DLR: Process the current node data, process left subtree and then
process right subtree
4. DRL: Process the current node data, process right subtree and then
process left subtree
5. RDL: Process right subtree, process the current node data and then
process left subtree
6. RLD: Process right subtree, process left subtree and then process the
current node data

Classifying the Traversals

The sequence in which these entities (nodes) are processed defines a particular
traversal method. The classification is based on the order in which current node
is processed. That means, if we are classifying based on current node (D) and if
D comes in the middle then it does not matter whether L is on left side of D or R
is on left side of D.

Similarly, it does not matter whether L is on right side of D or R is on right side

of D. Due to this, the total 6 possibilities are reduced to 3 and these are:

• Preorder (DLR) Traversal

 • Inorder (LDR) Traversal
• Postorder (LRD) Traversal

There is another traversal method which does not depend on the above orders
and it is:

• Level Order Traversal: This method is inspired from Breadth First

Traversal (BFS of Graph algorithms).

Let us use the diagram below for the remaining discussion.

PreOrder Traversal
In preorder traversal, each node is processed before (pre) either of its subtrees.
This is the simplest traversal to understand. However, even though each node is
processed before the subtrees, it still requires that some information must be
maintained while moving down the tree. In the example above, 1 is processed
first, then the left subtree, and this is followed by the right subtree.

Therefore, processing must return to the right subtree after finishing the
processing of the left subtree. To move to the right subtree after processing the
left subtree, we must maintain the root information. The obvious ADT for such
information is a stack. Because of its LIFO structure, it is possible to get the
information about the right subtrees back in the reverse order.
Preorder traversal is defined as follows:

• Visit the root.

• Traverse the left subtree in Preorder.
• Traverse the right subtree in Preorder.

The nodes of tree would be visited in the order: 1 2 4 5 3 6 7

Time Complexity: O(n). Space Complexity: O(n).

Non-Recursive Preorder Traversal

In the recursive version, a stack is required as we need to remember the current
node so that after completing the left subtree we can go to the right subtree. To
simulate the same, first we process the current node and before going to the left
subtree, we store the current node on stack. After completing the left subtree
processing, pop the element and go to its right subtree. Continue this process
until stack is nonempty.
Time Complexity: O(n). Space Complexity: O(n).

InOrder Traversal
In Inorder Traversal the root is visited between the subtrees. Inorder traversal is
defined as follows:

• Traverse the left subtree in Inorder.

• Visit the root.
• Traverse the right subtree in Inorder.

The nodes of tree would be visited in the order: 4 2 5 1 6 3 7

Time Complexity: O(n). Space Complexity: O(n).

Non-Recursive Inorder Traversal

The Non-recursive version of Inorder traversal is similar to Preorder. The only
change is, instead of processing the node before going to left subtree, process it
after popping (which is indicated after completion of left subtree processing).
Time Complexity: O(n). Space Complexity: O(n).

PostOrder Traversal
In postorder traversal, the root is visited after both subtrees. Postorder traversal
is defined as follows:

• Traverse the left subtree in Postorder.

• Traverse the right subtree in Postorder.
• Visit the root.

The nodes of the tree would be visited in the order: 4 5 2 6 7 3 1

Time Complexity: O(n). Space Complexity: O(n).

Non-Recursive Postorder Traversal

In preorder and inorder traversals, after popping the stack element we do not
need to visit the same vertex again. But in postorder traversal, each node is
visited twice. That means, after processing the left subtree we will visit the
current node and after processing the right subtree we will visit the same current
node. But we should be processing the node during the second visit. Here the
problem is how to differentiate whether we are returning from the left subtree or
the right subtree.

We use a previous variable to keep track of the earlier traversed node. Let’s
assume current is the current node that is on top of the stack. When previous is
current’s parent, we are traversing down the tree. In this case, we try to traverse
to current’s left child if available (i.e., push left child to the stack). If it is not
available, we look at current’s right child. If both left and right child do not exist
(ie, current is a leaf node), we print current’s value and pop it off the stack.

If prev is current’s left child, we are traversing up the tree from the left. We look
at current’s right child. If it is available, then traverse down the right child (i.e.,
push right child to the stack); otherwise print current’s value and pop it off the
stack. If previous is current’s right child, we are traversing up the tree from the
right. In this case, we print current’s value and pop it off the stack.
Time Complexity: O(n). Space Complexity: O(n).

Level Order Traversal

Level order traversal is defined as follows:

• Visit the root.

• While traversing level (, keep all the elements at level ( + 1 in queue.
• Go to the next level and visit all the nodes at that level.
 • Repeat this until all levels are completed.

The nodes of the tree are visited in the order: 1 2 3 4 5 6 7

Time Complexity: O(n). Space Complexity: O(n). Since, in the worst case, all
the nodes on the entire last level could be in the queue simultaneously.

Binary Trees: Problems & Solutions

Problem-1 Give an algorithm for finding maximum element in binary tree.
Solution: One simple way of solving this problem is: find the maximum element
in left subtree, find the maximum element in right sub tree, compare them with
root data and select the one which is giving the maximum value. This approach
can be easily implemented with recursion.
Time Complexity: O(n). Space Complexity: O(n).
Problem-2 Give an algorithm for finding the maximum element in binary
tree without recursion.
Solution: Using level order traversal: just observe the element’s data while
deleting.
Time Complexity: O(n). Space Complexity: O(n).
Problem-3 Give an algorithm for searching an element in binary tree.
Solution: Given a binary tree, return true if a node with data is found in the tree.
Recurse down the tree, choose the left or right branch by comparing data with
each node’s data.
Time Complexity: O(n). Space Complexity: O(n).
Problem-4 Give an algorithm for searching an element in binary tree without
recursion.
Solution: We can use level order traversal for solving this problem. The only
change required in level order traversal is, instead of printing the data, we just
need to check whether the root data is equal to the element we want to search.
Time Complexity: O(n). Space Complexity: O(n).
9.1 Introduction
In the real world, many problems are represented in terms of objects and
connections between them. For example, in an airline route map, we might be
interested in questions like: “What’s the fastest way to go from Hyderabad to
New York?” or “What is the cheapest way to go from Hyderabad to New York?”
To answer these questions we need information about connections (airline
routes) between objects (towns). Graphs are data structures used for solving
these kinds of problems.

9.2 Glossary
Graph: A graph is a pair (V, E), where V is a set of nodes, called vertices, and £
is a collection of pairs of vertices, called edges.

• Vertices and edges are positions and store elements

• Definitions that we use:
○ Directed edge:
▪ ordered pair of vertices (u, v)
▪ first vertex u is the origin
▪ second vertex v is the destination
▪ Example: one-way road traffic

○ Undirected edge:
▪ unordered pair of vertices (u, v)
▪ Example: railway lines

○ Directed graph:
▪ all the edges are directed
▪ Example: route network
 ○ Undirected graph:
▪ all the edges are undirected
▪ Example: flight network

• When an edge connects two vertices, the vertices are said to be adjacent
to each other and the edge is incident on both vertices.
• A graph with no cycles is called a tree. A tree is an acyclic connected
graph.

• A self loop is an edge that connects a vertex to itself.

• Two edges are parallel if they connect the same pair of vertices.

• The Degree of a vertex is the number of edges incident on it.

• A subgraph is a subset of a graph’s edges (with associated vertices) that
form a graph.
• A path in a graph is a sequence of adjacent vertices. Simple path is a path
with no repeated vertices. In the graph below, the dotted lines
represent a path from G to E.

• A cycle is a path where the first and last vertices are the same. A simple
cycle is a cycle with no repeated vertices or edges (except the first
and last vertices).
• We say that one vertex is connected to another if there is a path that
contains both of them.
• A graph is connected if there is a path from every vertex to every other
vertex.
• If a graph is not connected then it consists of a set of connected
components.

• A directed acyclic graph [DAG] is a directed graph with no cycles.

• A forest is a disjoint set of trees.
• A spanning tree of a connected graph is a subgraph that contains all of
that graph’s vertices and is a single tree. A spanning forest of a graph
is the union of spanning trees of its connected components.
• A bipartite graph is a graph whose vertices can be divided into two sets
such that all edges connect a vertex in one set with a vertex in the
other set.

• In weighted graphs integers (weights) are assigned to each edge to

represent (distances or costs).
• Graphs with all edges present are called complete graphs.

• Graphs with relatively few edges (generally if it edges < |V| log |V|) are
 called sparse graphs.
• Graphs with relatively few of the possible edges missing are called dense.
• Directed weighted graphs are sometimes called network.
• We will denote the number of vertices in a given graph by |V|, and the
number of edges by |E|. Note that E can range anywhere from 0 to |V|
(|V| – l)/2 (in undirected graph). This is because each node can
connect to every other node.

9.3 Applications of Graphs

• Representing relationships between components in electronic circuits
• Transportation networks: Highway network, Flight network
• Computer networks: Local area network, Internet, Web
• Databases: For representing ER (Entity Relationship) diagrams in
databases, for representing dependency of tables in databases

9.4 Graph Representation

As in other ADTs, to manipulate graphs we need to represent them in some
useful form. Basically, there are three ways of doing this:

• Adjacency Matrix
• Adjacency List
• Adjacency Set

Adjacency Matrix

Graph Declaration for Adjacency Matrix

First, let us look at the components of the graph data structure. To represent
graphs, we need the number of vertices, the number of edges and also their
interconnections. So, the graph can be declared as:
Description
In this method, we use a matrix with size V × V. The values of matrix are boolean.
Let us assume the matrix is Adj. The value Adj[u, v] is set to 1 if thereis an edge
from vertex u to vertex v and 0 otherwise.

In the matrix, each edge is represented by two bits for undirected graphs. That
means, an edge from u to v is represented by 1 value in both Adj[u,v ] and
Adj[u,v]. To save time, we can process only half of this symmetric matrix. Also,
we can assume that there is an “edge” from each vertex to itself. So, Adj[u, u] is
set to 1 for all vertices.

If the graph is a directed graph then we need to mark only one entry in the
adjacency matrix. As an example, consider the directed graph below.

The adjacency matrix for this graph can be given as:

Now, let us concentrate on the implementation. To read a graph, one way is to
first read the vertex names and then read pairs of vertex names (edges). The code
below reads an undirected graph.
The adjacency matrix representation is good if the graphs are dense. The matrix
requires O(V2) bits of storage and O(V2) time for initialization. If the number of
edges is proportional to V2, then there is no problem because V2 steps are
required to read the edges. If the graph is sparse, the initialization of the matrix
dominates the running time of the algorithm as it takes takes O(V2).

Adjacency List

Graph Declaration for Adjacency List

In this representation all the vertices connected to a vertex v are listed on an
adjacency list for that vertex v. This can be easily implemented with linked lists.
That means, for each vertex v we use a linked list and list nodes represents the
connections between v and other vertices to which v has an edge.

The total number of linked lists is equal to the number of vertices in the graph.
The graph ADT can be declared as:

Description
Considering the same example as that of the adjacency matrix, the adjacency list
representation can be given as:

Since vertex A has an edge for B and D, we have added them in the adjacency
list for A. The same is the case with other vertices as well.
Disadvantages of Adjacency Lists

Using adjacency list representation we cannot perform some operations

efficiently. As an example, consider the case of deleting a node. . In adjacency
list representation, it is not enugh if we simply delete a node from the list
representation, if we delete a node from the adjacency list then that is enough.
For each node on the adjacency list of that node specifies another vertex. We
need to search other nodes linked list also for deleting it. This problem can be
solved by linking the two list nodes that correspond to a particular edge and
making the adjacency lists doubly linked. But all these extra links are risky to
process.

Adjacency Set
It is very much similar to adjacency list but instead of using Linked lists, Disjoint
Sets [Union-Find] are used. For more details refer to the Disjoint Sets ADT
chapter.

Comparison of Graph Representations

Directed and undirected graphs are represented with the same structures. For
directed graphs, everything is the same, except that each edge is represented just
once. An edge from x to y is represented by a 1 value in Agj[x][y] in the adjacency
matrix, or by adding y on x’s adjacency list. For weighted graphs, everything is
the same, except fill the adjacency matrix with weights instead ofboolean values.
9.5 Graph Traversals
To solve problems on graphs, we need a mechanism for traversing the graphs.
Graph traversal algorithms are also called graph search algorithms. Like trees
traversal algorithms (Inorder, Preorder, Postorder and Level-Order traversals),
graph search algorithms can be thought of as starting at some source vertex in a
graph and “searching” the graph by going through the edges and marking the
vertices. Now, we will discuss two such algorithms for traversing the graphs.

• Depth First Search [DFS]

• Breadth First Search [BFS]

Depth First Search [DFS]

DFS algorithm works in a manner similar to preorder traversal of the trees. Like
preorder traversal, internally this algorithm also uses stack.

Let us consider the following example. Suppose a person is trapped inside a

maze. To come out from that maze, the person visits each path and each
intersection (in the worst case). Let us say the person uses two colors of paint to
mark the intersections already passed. When discovering a new intersection, it is
marked grey, and he continues to go deeper.

After reaching a “dead end” the person knows that there is no more unexplored
path from the grey intersection, which now is completed, and he marks it with
black. This “dead end” is either an intersection which has already been marked
grey or black, or simply a path that does not lead to an intersection.

The intersections of the maze are the vertices and the paths between the
intersections are the edges of the graph. The process of returning from the “dead
end” is called backtracking. We are trying to go away from the starting vertex
into the graph as deep as possible, until we have to backtrack to the preceding
grey vertex. In DFS algorithm, we encounter the following types of edges.

Tree edge: encounter new vertex

Back edge: from descendent to ancestor
Forward edge: from ancestor to descendent
Cross edge: between a tree or subtrees

For most algorithms boolean classification, unvisited/visited is enough (for three

color implementation refer to problems section). That means, for some problems
we need to use three colors, but for our discussion two colors are enough.

Initially all vertices are marked unvisited (false). The DFS algorithm starts at a
vertex u in the graph. By starting at vertex u it considers the edges from u to
other vertices. If the edge leads to an already visited vertex, then backtrack to
current vertex u. If an edge leads to an unvisited vertex, then go to that vertex
and start processing from that vertex. That means the new vertex becomes the
current vertex. Follow this process until we reach the dead-end. At this point
start backtracking.

The process terminates when backtracking leads back to the start vertex. The
algorithm based on this mechanism is given below: assume Visited[] is a global
array.
As an example, consider the following graph. We can see that sometimes an
edge leads to an already discovered vertex. These edges are called back edges,
and the other edges are called tree edges because deleting the back edges from
the graph generates a tree.

The final generated tree is called the DFS tree and the order in which the vertices
are processed is called DFS numbers of the vertices. In the graph below, the gray
color indicates that the vertex is visited (there is no other significance). We need
to see when the Visited table is updated.
Visited Table Visited Table

1 0 0 0 1 01 01 0 0 1 1 0 0 1 01 01 0 1 0 1

---0
Starting vertex A is
marked visited Vertex B is visited

Visited Table

Recursive call of DFS, vertex

E is visited
Visited Table
From the above diagrams, it can be seen that the DFS traversal creates a tree
(without back edges) and we call such tree a DFS tree. The above algorithm
works even if the given graph has connected components.

The time complexity of DFS is O(V + E), if we use adjacency lists for
representing the graphs. This is because we are starting at a vertex and
processing the adjacent nodes only if they are not visited. Similarly, if an
adjacency matrix is used for a graph representation, then all edges adjacent to a
vertex can’t be found efficiently, and this gives O(V2) complexity.

Applications of DFS
• Topological sorting
• Finding connected components
• Finding articulation points (cut vertices) of the graph
• Finding strongly connected components
• Solving puzzles such as mazes

For algorithms refer to Problems Section.

Breadth First Search [BFS]

The BFS algorithm works similar to level – order traversal of the trees. Like
level – order traversal, BFS also uses queues. In fact, level – order traversal got
inspired from BFS. BFS works level by level. Initially, BFS starts at a given
vertex, which is at level 0. In the first stage it visits all vertices at level 1 (that
means, vertices whose distance is 1 from the start vertex of the graph). In the
second stage, it visits all vertices at the second level. These new vertices are the
ones which are adjacent to level 1 vertices.

BFS continues this process until all the levels of the graph are completed.
Generally queue data structure is used for storing the vertices of a level.

As an example, let us consider the same graph as that of the DFS example. The
BFS traversal can be shown as:
10.1 What is Sorting?
Sorting is an algorithm that arranges the elements of a list in a certain order
[either ascending or descending]. The output is a permutation or reordering of
the input.

10.2 Why is Sorting Necessary?

Sorting is one of the important categories of algorithms in computer science and
a lot of research has gone into this category. Sorting can significantly reduce the
complexity of a problem, and is often used for database algorithms and searches.

10.3 Classification of Sorting Algorithms

Sorting algorithms are generally categorized based on the following parameters.

By Number of Comparisons
In this method, sorting algorithms are classified based on the number of
comparisons. For comparison based sorting algorithms, best case behavior is
O(nlogn) and worst case behavior is O(n2). Comparison-based sorting
algorithms evaluate the elements of the list by key comparison operation and
need at least O(nlogn) comparisons for most inputs.

Later in this chapter we will discuss a few non – comparison (linear) sorting
algorithms like Counting sort, Bucket sort, Radix sort, etc. Linear Sorting
algorithms impose few restrictions on the inputs to improve the complexity.

By Number of Swaps
In this method, sorting algorithms are categorized by the number of swaps (also
called inversions).

By Memory Usage
Some sorting algorithms are “in place” and they need O(1) or O(logn) memory
to create auxiliary locations for sorting the data temporarily.

By Recursion
Sorting algorithms are either recursive [quick sort] or non-recursive [selection
sort, and insertion sort], and there are some algorithms which use both (merge
sort).

By Stability
Sorting algorithm is stable if for all indices i and j such that the key A[i] equals
key A[j], if record R[i] precedes record R[j] in the original file, record R[i]
precedes record R[j] in the sorted list. Few sorting algorithms maintain the
relative order of elements with equal keys (equivalent elements retain their
relative positions even after sorting).

By Adaptability
With a few sorting algorithms, the complexity changes based on presortedness
[quick sort]: presortedness of the input affects the running time. Algorithms that
take this into account are known to be adaptive.

10.4 Other Classifications

Another method of classifying sorting algorithms is:
• Internal Sort
• External Sort

Internal Sort
Sort algorithms that use main memory exclusively during the sort are called
internal sorting algorithms. This kind of algorithm assumes high-speed random
access to all memory.

External Sort
Sorting algorithms that use external memory, such as tape or disk, during the
sort come under this category.

10.5 Bubble Sort

Bubble sort is the simplest sorting algorithm. It works by iterating the input array
from the first element to the last, comparing each pair of elements and swapping
them if needed. Bubble sort continues its iterations until no more swaps are
needed. The algorithm gets its name from the way smaller elements “bubble” to
the top of the list. Generally, insertion sort has better performance than bubble
sort. Some researchers suggest that we should not teach bubble sort because of
its simplicity and high time complexity.

The only significant advantage that bubble sort has over other implementations
is that it can detect whether the input list is already sorted or not.

Implementation

Algorithm takes O(n2) (even in best case). We can improve it by using one extra
flag. No more swaps indicate the completion of sorting. If the list is already
sorted, we can use this flag to skip the remaining passes.
This modified version improves the best case of bubble sort to O(n).

Performance

Worst case complexity : O(n2)

Best case complexity (Improved version) : O(n)

Average case complexity (Basic version) : O(n2)

Worst case space complexity : O(1) auxiliary

10.6 Selection Sort

Selection sort is an in-place sorting algorithm. Selection sort works well for
small files. It is used for sorting the files with very large values and small keys.
This is because selection is made based on keys and swaps are made only when
required.

Advantages
• Easy to implement
• In-place sort (requires no additional storage space)

Disadvantages

• Doesn’t scale well: O(n2)

Algorithm
1. Find the minimum value in the list
2. Swap it with the value in the current position
3. Repeat this process for all the elements until the entire array is sorted

This algorithm is called selection sort since it repeatedly selects the smallest
element.
Implementation

Performance

Worst case complexity : O(n2)

Best case complexity : O(n2)

Average case complexity : O(n2)
Worst case space complexity: O(1) auxiliary

10.7 Insertion Sort

Insertion sort is a simple and efficient comparison sort. In this algorithm, each
iteration removes an element from the input data and inserts it into the correct
position in the list being sorted. The choice of the element being removed from
the input is random and this process is repeated until all input elements have
gone through.

Advantages
 • Simple implementation
• Efficient for small data
• Adaptive: If the input list is presorted [may not be completely] then
insertions sort takes O(n + d), where d is the number of inversions
• Practically more efficient than selection and bubble sorts, even though all
of them have O(n2) worst case complexity
• Stable: Maintains relative order of input data if the keys are same
• In-place: It requires only a constant amount O(1) of additional memory
space
• Online: Insertion sort can sort the list as it receives it

Algorithm
Every repetition of insertion sort removes an element from the input data, and
inserts it into the correct position in the already-sorted list until no inputelements
remain. Sorting is typically done in-place. The resulting array after k iterations
has the property where the first k + 1 entries are sorted.

Each element greater than x is copied to the right as it is compared against x.

Implementation
Example
Given an array: 6 8 1 4 5 3 7 2 and the goal is to put them in ascending order.

Analysis
Worst case analysis

Worst case occurs when for every i the inner loop has to move all elements A[1],
. . . , A[i – 1] (which happens when A[i] = key is smaller than all of them), that
takes Θ(i – 1) time.
Average case analysis

For the average case, the inner loop will insert A[i] in the middle of A[1], . . . ,
A[i – 1]. This takes Θ(i/2) time.

Performance
If every element is greater than or equal to every element to its left, the running
time of insertion sort is Θ(n). This situation occurs if the array starts out already
sorted, and so an already-sorted array is the best case for insertion sort.

Worst case complexity: Θ(n2)

Best case complexity: Θ(n)

Average case complexity: Θ(n2)
Worst case space complexity: O(n2) total, O(1) auxiliary

Comparisons to Other Sorting Algorithms

Insertion sort is one of the elementary sorting algorithms with O(n2) worst-case
time. Insertion sort is used when the data is nearly sorted (due to its adaptiveness)
or when the input size is small (due to its low overhead). For thesereasons and
due to its stability, insertion sort is used as the recursive base case (when the
problem size is small) for higher overhead divide-and-conquer sortingalgorithms,
such as merge sort or quick sort.

Notes:

• Bubble sort takes comparisons and swaps (inversions) in both

average case and in worst case.
 • Selection sort takes comparisons and n swaps.

• Insertion sort takes comparisons and swaps in average case and in

the worst case they are double.
• Insertion sort is almost linear for partially sorted input.
• Selection sort is best suits for elements with bigger values and small keys.

10.8 Shell Sort

Shell sort (also called diminishing increment sort) was invented by Donald Shell.
This sorting algorithm is a generalization of insertion sort. Insertion sort works
efficiently on input that is already almost sorted. Shell sort is also known as n-
gap insertion sort. Instead of comparing only the adjacent pair, shell sort makes
several passes and uses various gaps between adjacent elements (ending with the
gap of 1 or classical insertion sort).

In insertion sort, comparisons are made between the adjacent elements. At most
1 inversion is eliminated for each comparison done with insertion sort. The
variation used in shell sort is to avoid comparing adjacent elements until the last
step of the algorithm. So, the last step of shell sort is effectively the insertion sort
algorithm. It improves insertion sort by allowing the comparison and exchange
of elements that are far away. This is the first algorithm which got less than
quadratic complexity among comparison sort algorithms.

Shellsort is actually a simple extension for insertion sort. The primary difference
is its capability of exchanging elements that are far apart, making it considerably
faster for elements to get to where they should be. For example, if the smallest
element happens to be at the end of an array, with insertion sort it will require
the full array of steps to put this element at the beginning of the array. However,
with shell sort, this element can jump more than one step a time and reach the
proper destination in fewer exchanges.

The basic idea in shellsort is to exchange every hth element in the array. Now
this can be confusing so we’ll talk more about this, h determines how far apart
element exchange can happen, say for example take h as 13, the first element
(index-0) is exchanged with the 14th element (index-13) if necessary (of course).
The second element with the 15th element, and so on. Now if we take has 1, it is
exactly the same as a regular insertion sort.
Shellsort works by starting with big enough (but not larger than the array size) h
so as to allow eligible element exchanges that are far apart. Once a sort is
complete with a particular h, the array can be said as h-sorted. The next step is to
reduce h by a certain sequence, and again perform another complete h-sort. Once
h is 1 and h-sorted, the array is completely sorted. Notice that the last sequence
for ft is 1 so the last sort is always an insertion sort, except by this time the array
is already well-formed and easier to sort.

Shell sort uses a sequence h1,h2, ...,ht called the increment sequence. Any
increment sequence is fine as long as h1 = 1, and some choices are better than
others. Shell sort makes multiple passes through the input list and sorts a number
of equally sized sets using the insertion sort. Shell sort improves the efficiency
of insertion sort by quickly shifting values to their destination.

Implementation

Note that when h == 1, the algorithm makes a pass over the entire list,comparing
adjacent elements, but doing very few element exchanges. For h == 1, shell sort
works just like insertion sort, except the number of inversions that have to be
eliminated is greatly reduced by the previous steps of the algorithm with h > 1.
Analysis
Shell sort is efficient for medium size lists. For bigger lists, the algorithm is not
the best choice. It is the fastest of all O(n2) sorting algorithms.

The disadvantage of Shell sort is that it is a complex algorithm and not nearly as
efficient as the merge, heap, and quick sorts. Shell sort is significantly slower
than the merge, heap, and quick sorts, but is a relatively simple algorithm, which
makes it a good choice for sorting lists of less than 5000 items unless speed is
important. It is also a good choice for repetitive sorting of smaller lists.

The best case in Shell sort is when the array is already sorted in the right order.
The number of comparisons is less. The running time of Shell sort depends on
the choice of increment sequence.

Performance

Worst case complexity depends on gap sequence. Best known:

O(nlog2n)
Best case complexity: O(n)
Average case complexity depends on gap sequence
Worst case space complexity: O(n)

10.9 Merge Sort

Merge sort is an example of the divide and conquer strategy.

Important Notes
• Merging is the process of combining two sorted files to make one bigger
sorted file.
• Selection is the process of dividing a file into two parts: k smallest
elements and n – k largest elements.
• Selection and merging are opposite operations
○ selection splits a list into two lists
 ○ merging joins two files to make one file
• Merge sort is Quick sort’s complement
• Merge sort accesses the data in a sequential manner
• This algorithm is used for sorting a linked list
• Merge sort is insensitive to the initial order of its input
• In Quick sort most of the work is done before the recursive calls. Quick
sort starts with the largest subfile and finishes with the small ones
and as a result it needs stack. Moreover, this algorithm is not stable.
Merge sort divides the list into two parts; then each part is conquered
individually. Merge sort starts with the small subfiles and finishes
with the largest one. As a result it doesn’t need stack. This algorithm
is stable.

10.10 Quicksort
Quick sort is an example of a divide-and-conquer algorithmic technique. It is
also called partition exchange sort. It uses recursive calls for sorting the
elements, and it is one of the famous algorithms among comparison-based
sorting algorithms.

Divide: The array A[low ...high] is partitioned into two non-empty sub arrays
A[low ...q] and A[q + 1... high], such that each element of A[low ... high] is less
than or equal to each element of A[q + 1... high]. The index q is computed as part
of this partitioning procedure.

Conquer: The two sub arrays A[low ...q] and A[q + 1 ...high] are sorted by
recursive calls to Quick sort.

Algorithm
The recursive algorithm consists of four steps:

1) If there are one or no elements in the array to be sorted, return.

2) Pick an element in the array to serve as the “pivot” point. (Usually the
leftmost element in the array is used.)
3) Split the array into two parts – one with elements larger than the pivot
and the other with elements smaller than the pivot.
4) Recursively repeat the algorithm for both halves of the original array.
Implementation
Analysis
Let us assume that T(n) be the complexity of Quick sort and also assume that all
elements are distinct. Recurrence for T(n) depends on two subproblem sizes
which depend on partition element. If pivot is ith smallest element then exactly (i
– 1) items will be in left part and (n – i) in right part. Let us call it as i –split.
Since each element has equal probability of selecting it as pivot the probability
of selecting ith element is .

Best Case: Each partition splits array in halves and gives

T(n) = 2T(n/2) + Θ(n) = Θ(nlogn), [using Divide and Conquer master
theorem]

Worst Case: Each partition gives unbalanced splits and we get

T(n) = T(n – 1) + Θ(n) = Θ(n2)[using Subtraction and Conquer master theorem]

The worst-case occurs when the list is already sorted and last element chosen as
pivot.

Average Case: In the average case of Quick sort, we do not know where the
split happens. For this reason, we take all possible values of split locations, add
all their complexities and divide with n to get the average case complexity.

Multiply both sides by n.

Same formula for n – 1.

Subtract the n – 1 formula from n.

Divide with n(n + 1).

Time Complexity, T(n) = O(nlogn).

Performance

Worst case Complexity: O(n2)

Best case Complexity: O(nlogn)

Average case Complexity: O(nlogn)
Worst case space Complexity: O(1)
Randomized Quick sort

In average-case behavior of Quick sort, we assume that all permutations of the

input numbers are equally likely. However, we cannot always expect it to hold.
We can add randomization to an algorithm in order to reduce the probability of
getting worst case in Quick sort.

There are two ways of adding randomization in Quick sort: either by randomly
placing the input data in the array or by randomly choosing an element in the
input data for pivot. The second choice is easier to analyze and implement. The
change will only be done at the partition algorithm.

In normal Quick sort, pivot element was always the leftmost element in the list
to be sorted. Instead of always using A[low] as pivot, we will use a randomly
chosen element from the subarray A[low..high] in the randomized version of
Quick sort. It is done by exchanging element A[low] with an element chosen at
random from A[low..high]. This ensures that the pivot element is equally likely
to be any of the high – low + 1 elements in the subarray.

Since the pivot element is randomly chosen, we can expect the split of the input
array to be reasonably well balanced on average. This can help in preventing the
worst-case behavior of quick sort which occurs in unbalanced partitioning. Even
though the randomized version improves the worst case complexity, its worst
case complexity is still O(n2). One way to improve Randomized – Quick sort is
to choose the pivot for partitioning more carefully than by picking a random
element from the array. One common approach is to choose the pivot as the
median of a set of 3 elements randomly selected from the array.

10.11 Tree Sort

Tree sort uses a binary search tree. It involves scanning each element of the input
and placing it into its proper position in a binary search tree. This has two phases:

• First phase is creating a binary search tree using the given array elements.
• Second phase is traversing the given binary search tree in inorder, thus
resulting in a sorted array.
Performance
The average number of comparisons for this method is O(nlogn). But in worst
case, the number of comparisons is reduced by O(n2), a case which arises when
the sort tree is skew tree.

10.12 Comparison of Sorting Algorithms

Note: n denotes the number of elements in the input.

10.13 Linear Sorting Algorithms

In earlier sections, we have seen many examples of comparison-based sorting
algorithms. Among them, the best comparison-based sorting has the complexity
O(nlogn). In this section, we will discuss other types of algorithms: Linear
Sorting Algorithms. To improve the time complexity of sorting these algorithms,
we make some assumptions about the input. A few examples of Linear Sorting
Algorithms are:

• Counting Sort
• Bucket Sort
 • Radix Sort

10.14 Counting Sort

Counting sort is not a comparison sort algorithm and gives O(n) complexity for
sorting. To achieve O(n) complexity, counting sort assumes that each of the
elements is an integer in the range 1 to K, for some integer K. When if = O(n),
the counting sort runs in O(n) time. The basic idea of Counting sort is to
determine, for each input element X, the number of elements less than X. This
information can be used to place it directly into its correct position. For example,
if 10 elements are less than X, then X belongs to position 11 in the output.

In the code below, A[0 ..n – 1] is the input array with length n. In Counting sort
we need two more arrays: let us assume array B[0 ..n – 1] contains the sorted
output and the array C[0 ..K – 1] provides temporary storage.
Total Complexity: O(K) + O(n) + O(K) + O(n) = O(n) if K =O(n). Space
Complexity: O(n) if K =O(n).

Note: Counting works well if K =O(n). Otherwise, the complexity will be

greater.

10.15 Bucket Sort (or Bin Sort)

Like Counting sort, Bucket sort also imposes restrictions on the input to improve
the performance. In other words, Bucket sort works well if the input is drawn
from fixed set. Bucket sort is the generalization of Counting Sort. For example,
assume that all the input elements from {0, 1, . . . , K – 1}, i.e., the set of integers
in the interval [0, K – 1]. That means, K is the number of distant elements in the
input. Bucket sort uses K counters. The ith counter keeps track of the number of
occurrences of the ith element. Bucket sort with two buckets is effectively a
version of Quick sort with two buckets.

For bucket sort, the hash function that is used to partition the elements need to be
very good and must produce ordered hash: if i < k then hash(i) < hash(k). Second,
the elements to be sorted must be uniformly distributed.

The aforementioned aside, bucket sort is actually very good considering that
counting sort is reasonably speaking its upper bound. And counting sort is very
fast. The particular distinction for bucket sort is that it uses a hash function to
partition the keys of the input array, so that multiple keys may hash to the same
bucket. Hence each bucket must effectively be a growable list; similar to radix
sort.

In the below code insertionsort is used to sort each bucket. This is to inculcate
that the bucket sort algorithm does not specify which sorting technique to use on
the buckets. A programmer may choose to continuously use bucket sort on each
bucket until the collection is sorted (in the manner of the radix sort program
below). Whichever sorting method is used on the , bucket sort still tends toward
O(n).
Time Complexity: O(n). Space Complexity: O(n).

10.16 Radix Sort

Similar to Counting sort and Bucket sort, this sorting algorithm also assumes
some kind of information about the input elements. Suppose that the input values
to be sorted are from base d. That means all numbers are d-digit numbers.

In Radix sort, first sort the elements based on the last digit [the least significant
digit]. These results are again sorted by second digit [the next to least significant
digit]. Continue this process for all digits until we reach the most significant
digits. Use some stable sort to sort them by last digit. Then stable sort them by
the second least significant digit, then by the third, etc. If we use Counting sort
as the stable sort, the total time is O(nd) ≈ O(n).

Algorithm:

1) Take the least significant digit of each element.

2) Sort the list of elements based on that digit, but keep the order of
elements with the same digit (this is the definition of a stable sort).
3) Repeat the sort with each more significant digit.

The speed of Radix sort depends on the inner basic operations. If the operations
are not efficient enough, Radix sort can be slower than other algorithms such as
Quick sort and Merge sort. These operations include the insert and delete
functions of the sub-lists and the process of isolating the digit we want. If the
numbers are not of equal length then a test is needed to check for additional
digits that need sorting. This can be one of the slowest parts of Radix sort and
also one of the hardest to make efficient.

Since Radix sort depends on the digits or letters, it is less flexible than other
sorts. For every different type of data, Radix sort needs to be rewritten, and if the
sorting order changes, the sort needs to be rewritten again. In short, Radix sort
takes more time to write, and it is very difficult to write a general purpose Radix
sort that can handle all kinds of data.

For many programs that need a fast sort, Radix sort is a good choice. Still, there
are faster sorts, which is one reason why Radix sort is not used as much as some
other sorts.

Time Complexity: O(nd) ≈ O(n), if d is small.

10.17 Topological Sort

Refer to Graph Algorithms Chapter.

10.18 External Sorting

External sorting is a generic term for a class of sorting algorithms that can handle
massive amounts of data. These external sorting algorithms are useful when the
files are too big and cannot fit into main memory.

As with internal sorting algorithms, there are a number of algorithms for external
sorting. One such algorithm is External Mergesort. In practice, these external
sorting algorithms are being supplemented by internal sorts.

Simple External Mergesort

A number of records from each tape are read into main memory, sorted using an
internal sort, and then output to the tape. For the sake of clarity, let us assume
that 900 megabytes of data needs to be sorted using only 100 megabytes of
RAM.

1) Read 100MB of the data into main memory and sort by some
conventional method (let us say Quick sort).
2) Write the sorted data to disk.
3) Repeat steps 1 and 2 until all of the data is sorted in chunks of 100MB.
Now we need to merge them into one single sorted output file.
4) Read the first 10MB of each sorted chunk (call them input buffers) in
main memory (90MB total) and allocate the remaining 10MB for
output buffer.
5) Perform a 9-way Mergesort and store the result in the output buffer. If
the output buffer is full, write it to the final sorted file. If any of the 9
input buffers gets empty, fill it with the next 10MB of its associated
100MB sorted chunk; or if there is no more data in the sorted chunk,
mark it as exhausted and do not use it for merging.
The above algorithm can be generalized by assuming that the amount of data to
be sorted exceeds the available memory by a factor of K. Then, K chunks of data
need to be sorted and a K -way merge has to be completed.

If X is the amount of main memory available, there will be K input buffers and 1
output buffer of size X/(K + 1) each. Depending on various factors (how fast is
the hard drive?) better performance can be achieved if the output buffer is made
larger (for example, twice as large as one input buffer).

Complexity of the 2-way External Merge sort: In each pass we read + write each
page in file. Let us assume that there are n pages in file. That means we need
⌈logn⌉ + 1 number of passes. The total cost is 2n(⌈logn⌉ + 1).

10.19 Sorting: Problems & Solutions

Problem-1 Given an array A[0...n– 1] of n numbers containing the repetition
of some number. Give an algorithm for checking whether there are
repeated elements or not. Assume that we are not allowed to use additional
space (i.e., we can use a few temporary variables, O(1) storage).
Solution: Since we are not allowed to use extra space, one simple way is to scan
the elements one-by-one and for each element check whether that element
appears in the remaining elements. If we find a match we return true.

Each iteration of the inner, j-indexed loop uses O(1) space, and for a fixed value
of i, the j loop executes n – i times. The outer loop executes n – 1 times, so the
entire function uses time proportional to
Time Complexity: O(n2). Space Complexity: O(1).
Problem-2 Can we improve the time complexity of Problem-1?

Solution: Yes, using sorting technique.

Heapsort function takes O(nlogn) time, and requires O(1) space. The scan clearly
takes n – 1 iterations, each iteration using O(1) time. The overall time is O(nlogn
+ n) = O(nlogn).

Time Complexity: O(nlogn). Space Complexity: O(1).

Note: For variations of this problem, refer Searching chapter.

Problem-3 Given an array A[0 ...n – 1], where each element of the array
represents a vote in the election. Assume that each vote is given as an
integer representing the ID of the chosen candidate. Give an algorithm for
determining who wins the election.
Solution: This problem is nothing but finding the element which repeated the
maximum number of times. The solution is similar to the Problem-1 solution:
keep track of counter.
Time Complexity: O(n2). Space Complexity: O(1).

Note: For variations of this problem, refer to Searching chapter.

Problem-4 Can we improve the time complexity of Problem-3? Assume we
don’t have any extra space.

Solution: Yes. The approach is to sort the votes based on candidate ID, then
scan the sorted array and count up which candidate so far has the most votes. We
only have to remember the winner, so we don’t need a clever data structure. We
can use Heapsort as it is an in-place sorting algorithm.
Since Heapsort time complexity is O(nlogn) and in-place, it only uses an
additional O(1) of storage in addition to the input array. The scan of the sorted
array does a constant-time conditional n – 1 times, thus using O(n) time. The
overall time bound is O(nlogn).
Problem-5 Can we further improve the time complexity of Problem-3?
Solution: In the given problem, the number of candidates is less but the number
of votes is significantly large. For this problem we can use counting sort.

Time Complexity: O(n), n is the number of votes (elements) in the array. Space
Complexity: O(k), k is the number of candidates participating in the election.
Problem-6 Given an array A of n elements, each of which is an integer in
 the range [1, n2], how do we sort the array in O(n) time?
Solution: If we subtract each number by 1 then we get the range [0, n2 – 1]. If
we consider all numbers as 2 –digit base n. Each digit ranges from 0 to n2 – 1.
Sort this using radix sort. This uses only two calls to counting sort. Finally, add 1
to all the numbers. Since there are 2 calls, the complexity is O(2n) ≈ O(n).

Problem-7 For Problem-6, what if the range is [1... n3]?

Solution: If we subtract each number by 1 then we get the range [0, n3 – 1].
Considering all numbers as 3-digit base n: each digit ranges from 0 to n3 – 1.
Sort this using radix sort. This uses only three calls to counting sort. Finally, add
1 to all the numbers. Since there are 3 calls, the complexity is O(3n) ≈ O(n).

Problem-8 Given an array with n integers, each of value less than n100, can
it be sorted in linear time?

Solution: Yes. The reasoning is same as in of Problem-6 and Problem-7.

Problem-9 Let A and B be two arrays of n elements each. Given a number
K, give an O(nlogn) time algorithm for determining whether there exists a
∈ A and b ∈ B such that a + b = K.
Solution: Since we need O(nlogn), it gives us a pointer that we need to sort. So,
we will do that.

Note: For variations of this problem, refer to Searching chapter.

Problem-10 Let A,B and C be three arrays of n elements each. Given a
number K, give an O(nlogn) time algorithm for determining whether there
 exists a ∈ A, b ∈ B and c ∈ C such that a + b + c = K.
Solution: Refer to Searching chapter.
Problem-11 Given an array of n elements, can we output in sorted order the
K elements following the median in sorted order in time O(n + KlogK).

Solution: Yes. Find the median and partition the median. With this we can find
all the elements greater than it. Now find the Kth largest element in this set and
partition it; and get all the elements less than it. Output the sorted list of the final
set of elements. Clearly, this operation takes O(n + KlogK) time.
Problem-12 Consider the sorting algorithms: Bubble sort, Insertion sort,
Selection sort, Merge sort, Heap sort, and Quick sort. Which of these are
stable?
Solution: Let us assume that A is the array to be sorted. Also, let us say R and S
have the same key and R appears earlier in the array than S. That means, R is at
A[i] and S is at A[j], with i < j. To show any stable algorithm, in the sorted output
R must precede S.

Bubble sort: Yes. Elements change order only when a smaller record follows a
larger. Since S is not smaller than R it cannot precede it.

Selection sort: No. It divides the array into sorted and unsorted portions and
iteratively finds the minimum values in the unsorted portion. After finding a
minimum x, if the algorithm moves x into the sorted portion of the array by
means of a swap, then the element swapped could be R which then could be
moved behind S. This would invert the positions of R and S, so in general it is
not stable. If swapping is avoided, it could be made stable but the cost in time
would probably be very significant.

Insertion sort: Yes. As presented, when S is to be inserted into sorted subarray

A[1..j – 1], only records larger than S are shifted. Thus R would not be shifted
during S’s insertion and hence would always precede it.

Merge sort: Yes, In the case of records with equal keys, the record in the left
subarray gets preference. Those are the records that came first in the unsorted
array. As a result, they will precede later records with the same key.

Heap sort: No. Suppose i = 1 and R and S happen to be the two records with the
largest keys in the input. Then R will remain in location 1 after the array is
heapified, and will be placed in location n in the first iteration of Heapsort. Thus
S will precede R in the output.

Quick sort: No. The partitioning step can swap the location of records many
times, and thus two records with equal keys could swap position in the final
output.
Problem-13 Consider the same sorting algorithms as that of Problem-12.
Which of them are in-place?
Solution:

Bubble sort: Yes, because only two integers are required.

Insertion sort: Yes, since we need to store two integers and a record.

Selection sort: Yes. This algorithm would likely need space for two integers and
one record.

Merge sort: No. Arrays need to perform the merge. (If the data is in the form of
a linked list, the sorting can be done in-place, but this is a nontrivial
modification.)

Heap sort: Yes, since the heap and partially-sorted array occupy opposite ends
of the input array.

Quicksort: No, since it is recursive and stores O(logn) activation records on the
stack. Modifying it to be non-recursive is feasible but nontrivial.
Problem-14 Among Quick sort, Insertion sort, Selection sort, and Heap sort
algorithms, which one needs the minimum number of swaps?
Solution: Selection sort – it needs n swaps only (refer to theory section).
Problem-15 What is the minimum number of comparisons required todetermine
if an integer appears more than n/2 times in a sorted array of n integers?
Solution: Refer to Searching chapter.
Problem-16 Sort an array of 0’s, 1’s and 2’s: Given an array A[] consisting
of 0’s, 1’s and 2’s, give an algorithm for sorting A[]. The
 algorithm should put all 0’s first, then all 1’s and all 2’s last.
Example: Input = {0,1,1,0,1,2,1,2,0,0,0,1}, Output =
{0,0,0,0,0,1,1,1,1,1,2,2}
Solution: Use Counting sort. Since there are only three elements and the
maximum value is 2, we need a temporary array with 3 elements.

Time Complexity: O(n). Space Complexity: O(1).

Note: For variations of this problem, refer to Searching chapter.

Problem-17 Is there any other way of solving Problem-16?
Solution: Using Quick dort. Since we know that there are only 3 elements, 0,1
and 2 in the array, we can select 1 as a pivot element for Quick sort. Quick sort
finds the correct place for 1 by moving all 0’s to the left of 1 and all 2’s to the
right of 1. For doing this it uses only one scan.

Time Complexity: O(n). Space Complexity: O(1).

Note: For efficient algorithm, refer to Searching chapter.

Problem-18 How do we find the number that appeared the maximumnumber
of times in an array?
Solution: One simple approach is to sort the given array and scan the sorted
array. While scanning, keep track of the elements that occur the maximum
number of times.
Algorithm:
11.1 What is Searching?
In computer science, searching is the process of finding an item with specified
properties from a collection of items. The items may be stored as records in a
database, simple data elements in arrays, text in files, nodes in trees, vertices and
edges in graphs, or they may be elements of other search spaces.

11.2 Why do we need Searching?

Searching is one of the core computer science algorithms. We know that today’s
computers store a lot of information. To retrieve this information proficiently we
need very efficient searching algorithms. There are certain ways of organizing
the data that improves the searching process. That means, if we keep the data in
proper order, it is easy to search the required element. Sorting is one of the
techniques for making the elements ordered. In this chapter we will see different
searching algorithms.

11.3 Types of Searching

Following are the types of searches which we will be discussing in this book.

• Unordered Linear Search

• Sorted/Ordered Linear Search
• Binary Search
• Interpolation search
• Binary Search Trees (operates on trees and refer Trees chapter)
• Symbol Tables and Hashing
• String Searching Algorithms: Tries, Ternary Search and Suffix Trees

11.4 Unordered Linear Search

Let us assume we are given an array where the order of the elements is not
known. That means the elements of the array are not sorted. In this case, to search
for an element we have to scan the complete array and see if the element is there
in the given list or not.

Time complexity: O(n), in the worst case we need to scan the complete array.
Space complexity: O(1).

11.5 Sorted/Ordered Linear Search

If the elements of the array are already sorted, then in many cases we don’t have
to scan the complete array to see if the element is there in the given array or not.
In the algorithm below, it can be seen that, at any point if the value at A[i] is
greater than the data to be searched, then we just return –1 without searching the
remaining array.

Time complexity of this algorithm is O(n).This is because in the worst case we

need to scan the complete array. But in the average case it reduces the
complexity even though the growth rate is the same.

Space complexity: O(1).

Note: For the above algorithm we can make further improvement by

incrementing the index at a faster rate (say, 2). This will reduce the number of
comparisons for searching in the sorted list.

11.6 Binary Search

Let us consider the problem of searching a word in a dictionary. Typically, we
directly go to some approximate page [say, middle page] and start searching
from that point. If the name that we are searching is the same then the search is
complete. If the page is before the selected pages then apply the same process
for the first half; otherwise apply the same process to the second half. Binary
search also works in the same way. The algorithm applying such a strategy is
referred to as binary search algorithm.
Recurrence for binary search is . This is because we
are always considering only half of the input list and throwing out the other half.
Using Divide and Conquer master theorem, we get, T(n) = O(logn).

Time Complexity: O(logn). Space Complexity: O(1) [for iterative algorithm].

11.7 Interpolation Search

Undoubtedly binary search is a great algorithm for searching with average
running time complexity of logn. It always chooses the middle of the remaining
search space, discarding one half or the other, again depending on the
comparison between the key value found at the estimated (middle) position and
the key value sought. The remaining search space is reduced to the part before or
after the estimated position.

In the mathematics, interpolation is a process of constructing new data points

within the range of a discrete set of known data points. In computer science, one
often has a number of data points which represent the values of a function for a
limited number of values of the independent variable. It is often required to
interpolate (i.e. estimate) the value of that function for an intermediate value of
the independent variable.

For example, suppose we have a table like this, which gives some values of an
unknown function f. Interpolation provides a means of estimating the function at
intermediate points, such as x = 55.

x f(x)
1 10
2 20
3 30
4 40
5 50
6 60
7 70

There are many different interpolation methods, and one of the simplest methods
is linear interpolation. Since 55 is midway between 50 and 60, it is reasonable to
take f(55) midway between f(5) = 50 and f(6) = 60, which yields 55.

Linear interpolation takes two data points, say (x1; y2) and (x2, y2), and the
interpolant is given by:

With above inputs, what will happen if we don’t use the constant ½, but another
more accurate constant “K”, that can lead us closer to the searched item.

This algorithm tries to follow the way we search a name in a phone book, or a
word in the dictionary. We, humans, know in advance that in case the name
we’re searching starts with a “m”, like “monk” for instance, we should start
searching near the middle of the phone book. Thus if we’re searching the word
“career” in the dictionary, you know that it should be placed somewhere at the
beginning. This is because we know the order of the letters, we know the interval
(a-z), and somehow we intuitively know that the words are dispersed equally.
These facts are enough to realize that the binary search can be a bad choice.
Indeed the binary search algorithm divides the list in two equal sub-lists, which
is useless if we know in advance that the searched item is somewhere in the
beginning or the end of the list. Yes, we can use also jump search if the item is at
the beginning, but not if it is at the end, in that case this algorithm is not so
effective.

The interpolation search algorithm tries to improve the binary search. The
question is how to find this value? Well, we know bounds of the interval and
looking closer to the image above we can define the following formula.
This constant K is used to narrow down the search space. For binary search, this
constant K is (low + high)/2.

Now we can be sure that we’re closer to the searched value. On average the
interpolation search makes about log (logn) comparisons (if the elements are
uniformly distributed), where n is the number of elements to be searched. In the
worst case (for instance where the numerical values of the keys increase
exponentially) it can make up to O(n) comparisons. In interpolation-sequential
search, interpolation is used to find an item near the one being searched for, then
linear search is used to find the exact item. For this algorithm to give best results,
the dataset should be ordered and uniformly distributed.

11.8 Comparing Basic Searching Algorithms

Note: For discussion on binary search trees refer Trees chapter.

11.9 Symbol Tables and Hashing

Refer to Symbol Tables and Hashing chapters.

11.10 String Searching Algorithms

Refer to String Algorithms chapter.

11.11 Searching: Problems & Solutions

Problem-1 Given an array of n numbers, give an algorithm for checking whether
there are any duplicate elements in the array or no?
Solution: This is one of the simplest problems. One obvious answer to this is
exhaustively searching for duplicates in the array. That means, for each input
element check whether there is any element with the same value. This we can
solve just by using two simple for loops. The code for this solution can be given
as:
Time Complexity: O(n2), for two nested for loops. Space Complexity: O(1).
Problem-2 Can we improve the complexity of Problem-1’s solution?

Solution: Yes. Sort the given array. After sorting, all the elements with equal
values will be adjacent. Now, do another scan on this sorted array and see if there
are elements with the same value and adjacent.

Time Complexity: O(nlogn), for sorting (assuming nlogn sorting algorithm).

Space Complexity: O(1).
Problem-3 Is there any alternative way of solving Problem-1?

Solution: Yes, using hash table. Hash tables are a simple and effective method
used to implement dictionaries. Average time to search for an element is O(1),
while worst-case time is O(n). Refer to Hashing chapter for more details on
hashing algorithms. As an example, consider the array, A = {3,2,1,2,2,3}.

Scan the input array and insert the elements into the hash. For each inserted
element, keep the counter as 1 (assume initially all entires are filled with zeros).
This indicates that the corresponding element has occurred already. For the
given array, the hash table will look like (after inserting the first three elements
3,2 and 1):

Now if we try inserting 2, since the counter value of 2 is already 1, we can say
the element has appeared twice.

Time Complexity: O(n). Space Complexity: O(n).

Problem-4 Can we further improve the complexity of Problem-1 solution?
Solution: Let us assume that the array elements are positive numbers and all the
elements are in the range 0 to n – 1. For each element A[i], go to the array element
whose index is A[i]. That means select A[A[i]] and mark - A[A[i]] (negate the
value at A[A[i]]). Continue this process until we encounter the element whose
value is already negated. If one such element exists then we say duplicate
elements exist in the given array. As an example, consider the array, A
= {3,2,1,2,2,3}.

Initially,

At step-1, negate A[abs(A[0])],

At step-2, negate A[abs(A[l])],

At step-3, negate A[abs(A[2])],

At step-4, negate A[abs(A[3])],

At step-4, observe that A[abs(A[3])] is already negative. That means we have

encountered the same value twice.
Time Complexity: O(n). Since only one scan is required. Space Complexity:
O(1).

Notes:

• This solution does not work if the given array is read only.
• This solution will work only if all the array elements are positive.
• If the elements range is not in 0 to n – 1 then it may give exceptions.
Problem-5 Given an array of n numbers. Give an algorithm for finding the
element which appears the maximum number of times in the array?

Brute Force Solution: One simple solution to this is, for each input element
check whether there is any element with the same value, and for each such
occurrence, increment the counter. Each time, check the current counter with the
max counter and update it if this value is greater than max counter. This we can
solve just by using two simple for loops.

Time Complexity: O(n2), for two nested for loops. Space Complexity: O(1).
Problem-6 Can we improve the complexity of Problem-5 solution?

Solution: Yes. Sort the given array. After sorting, all the elements with equal
values come adjacent. Now, just do another scan on this sorted array and see
which element is appearing the maximum number of times.
Time Complexity: O(nlogn). (for sorting). Space Complexity: O(1).
Problem-7 Is there any other way of solving Problem-5?

Solution: Yes, using hash table. For each element of the input, keep track of
how many times that element appeared in the input. That means the counter
value represents the number of occurrences for that element.

Time Complexity: O(n). Space Complexity: O(n).

Problem-8 or Problem-5, can we improve the time complexity? Assume that
the elements’ range is 1 to n. That means all the elements are within this
range only.

Solution: Yes. We can solve this problem in two scans. We cannot use the
negation technique of Problem-3 for this problem because of the number of
repetitions. In the first scan, instead of negating, add the value n. That means for
each occurrence of an element add the array size to that element. In the second
scan, check the element value by dividing it by n and return the element which
gives the maximum value. The code based on this method is given below.

Notes:

• This solution does not work if the given array is read only.
• This solution will work only if the array elements are positive.
• If the elements range is not in 1 to n then it may give exceptions.
Time Complexity: O(n). Since no nested for loops are required. Space
Complexity: O(1).
Problem-9 Given an array of n numbers, give an algorithm for finding the first
element in the array which is repeated. For example, in the array A =
{3,2,1,2,2,3}, the first repeated number is 3 (not 2). That means, we need
to return the first element among the repeated elements.
Solution: We can use the brute force solution that we used for Problem-1. For
each element, since it checks whether there is a duplicate for that element or not,
whichever element duplicates first will be returned.
Problem-10 For Problem-9, can we use the sorting technique?
Solution: No. For proving the failed case, let us consider the following array.
For example, A = {3, 2, 1, 2, 2, 3}. After sorting we get A = {1,2,2,2,3,3}. In this
sorted array the first repeated element is 2 but the actual answer is 3.
Problem-11 For Problem-9, can we use hashing technique?
Solution: Yes. But the simple hashing technique which we used for Problem-3
will not work. For example, if we consider the input array as A = {3,2,1,2,3},
then the first repeated element is 3, but using our simple hashing technique we
get the answer as 2. This is because 2 is coming twice before 3. Now let us
change the hashing table behavior so that we get the first repeated element. Let
us say, instead of storing 1 value, initially we store the position of the element in
the array. As a result the hash table will look like (after inserting 3,2 and 1):

Now, if we see 2 again, we just negate the current value of 2 in the hash table.
That means, we make its counter value as –2. The negative value in the hash
table indicates that we have seen the same element two times. Similarly, for 3
(the next element in the input) also, we negate the current value of the hash table
and finally the hash table will look like:
After processing the complete input array, scan the hash table and return the
highest negative indexed value from it (i.e., –1 in our case). The highest negative
value indicates that we have seen that element first (among repeated elements)
and also repeating.

What if the element is repeated more than twice? In this case, just skip the
element if the corresponding value i is already negative.
Problem-12 For Problem-9, can we use the technique that we used for Problem-
3 (negation technique)?

Solution: No. As an example of contradiction, for the array A = {3,2,1,2,2,3}

the first repeated element is 3. But with negation technique the result is 2.
Problem-13 Finding the Missing Number: We are given a list of n – 1 integers
and these integers are in the range of 1 to n. There are noduplicates
in the list. One of the integers is missing in the list. Given an algorithm to
find the missing integer. Example: I/P: [1,2,4,6,3,7,8] O/P: 5

Brute Force Solution: One simple solution to this is, for each number in 1 to n,
check whether that number is in the given array or not.
Time Complexity: O(n2). Space Complexity: O(1).
Problem-14 For Problem-13, can we use sorting technique?

Solution: Yes. Sorting the list will give the elements in increasing order and
with another scan we can find the missing number.

Time Complexity: O(nlogn), for sorting. Space Complexity: O(1).

Problem-15 For Problem-13, can we use hashing technique?

Solution: Yes. Scan the input array and insert elements into the hash. For
inserted elements, keep counter as 1 (assume initially all entires are filled with
zeros). This indicates that the corresponding element has occurred already. Now,
scan the hash table and return the element which has counter value zero.

Time Complexity: O(n). Space Complexity: O(n).

Problem-16 For Problem-13, can we improve the complexity?

Solution: Yes. We can use summation formula.

1) Get the sum of numbers, sum = n × (n + l)/2.

2) Subtract all the numbers from sum and you will get the missing number.

Time Complexity: O(n), for scanning the complete array.

Problem-17 In Problem-13, if the sum of the numbers goes beyond the
maximum allowed integer, then there can be integer overflow and we may
not get the correct answer. Can we solve this problem?
Solution:

1) XOR all the array elements, let the result of XOR be X.

2) XOR all numbers from 1 to n, let XOR be Y.
3) XOR of X and Y gives the missing number.
Time Complexity: O(n), for scanning the complete array. Space Complexity:
O(1).
Problem-18 Find the Number Occurring an Odd Number of Times:
Given an array of positive integers, all numbers occur an even number
of times except one number which occurs an odd number of times. Find
the number in O(n) time & constant space. Example : I/P =
[1,2,3,2,3,1,3] O/P
=3
Solution: Do a bitwise XOR of all the elements. We get the number which
has odd occurrences. This is because, A XOR A = 0.

Time Complexity: O(n). Space Complexity: O(1).

Problem-19 Find the two repeating elements in a given array: Given an
array with size, all elements of the array are in range 1 to n and also all
elements occur only once except two numbers which occur twice. Find
those two repeating numbers. For example: if the array is 4,2,4,5,2,3,1
withsize = 7 and n = 5. This input has n + 2 = 7 elements with all
elements occurring once except 2 and 4 which occur twice. So the output
should be4 2.
Solution: One simple way is to scan the complete array for each element of
theinput elements. That means use two loops. In the outer loop, select elements
oneby one and count the number of occurrences of the selected element in the
innerloop. For the code below, assume that PrintRepeatedElements is called
with n +2 to indicate the size.