ALGORITHM

Algorithm is a step-by-step procedure, which defines a set of instructions to be

executed in a certain order to get the desired output. Algorithms are generally created
independent of underlying languages, i.e. an algorithm can be implemented in more
than one programming language.
From the data structure point of view, following are some important categories of
algorithms −
 Search − Algorithm to search an item in a data structure.
 Sort − Algorithm to sort items in a certain order.
 Insert − Algorithm to insert item in a data structure.
 Update − Algorithm to update an existing item in a data structure.
 Delete − Algorithm to delete an existing item from a data structure.
1.7.1 Characteristics of an Algorithm
Not all procedures can be called an algorithm. An algorithm should have the following
characteristics −
An algorithm should have the following characteristics −
 Clear and Unambiguous: Algorithm should be clear and unambiguous. Each
of its steps should be clear in all aspects and must lead to only one meaning.
 Well-Defined Inputs: If an algorithm says to take inputs, it should be well-
defined inputs.
 Well-Defined Outputs: The algorithm must clearly define what output will be
yielded and it should be well-defined as well.
 Finite-ness: The algorithm must be finite, i.e. it should not end up in an infinite
loops or similar.
 Feasible: The algorithm must be simple, generic and practical, such that it can
be executed upon will the available resources. It must not contain some future
technology, or anything.
 Language Independent: The Algorithm designed must be language
independent, i.e. it must be just plain instructions that can be implemented in
any language, and yet the output will be same, as expected.
1.7.2 Advantages and Disadvantages of Algorithm
Advantages of Algorithms:
 It is easy to understand.
 Algorithm is a step-wise representation of a solution to a given problem.
 In Algorithm the problem is broken down into smaller pieces or steps hence, it
is easier for the programmer to convert it into an actual program.
Disadvantages of Algorithms:
  Writing an algorithm takes a long time so it is time-consuming.
 Branching and looping statements are difficult to show in Algorithms.
1.7.3 Different approach to design an algorithm
1. Top-Down Approach: A top-down approach starts with identifying major
components of system or program decomposing them into their lower level
components & iterating until desired level of module complexity is achieved . In this
we start with topmost module & incrementally add modules that is calls.
2. Bottom-Up Approach: A bottom-up approach starts with designing most basic or
primitive component & proceeds to higher level components. Starting from very
bottom, operations that provide layer of abstraction are implemented
1.7.4 How to Write an Algorithm?
There are no well-defined standards for writing algorithms. Rather, it is problem and
resource dependent. Algorithms are never written to support a particular programming
code.
As we know that all programming languages share basic code constructs like loops
(do, for, while), flow-control (if-else), etc. These common constructs can be used to
write an algorithm.
We write algorithms in a step-by-step manner, but it is not always the case. Algorithm
writing is a process and is executed after the problem domain is well-defined. That is,
we should know the problem domain, for which we are designing a solution.
Example
Let's try to learn algorithm-writing by using an example.
Problem − Design an algorithm to add two numbers and display the result.
Step 1 − START
Step 2 − declare three integers a, b & c
Step 3 − define values of a & b
Step 4 − add values of a & b
Step 5 − store output of step 4 to c
Step 6 − print c
Step 7 − STOP
Algorithms tell the programmers how to code the program. Alternatively, the
algorithm can be written as −
Step 1 − START ADD
Step 2 − get values of a & b
Step 3 − c ← a + b
Step 4 − display c
Step 5 − STOP
 In design and analysis of algorithms, usually the second method is used to
describe an algorithm. It makes it easy for the analyst to analyse the algorithm
ignoring all unwanted definitions. He can observe what operations are being
used and how the process is flowing.
 Writing step numbers, is optional.
 We design an algorithm to get a solution of a given problem. A problem can be
solved in more than one ways.

Hence, many solution algorithms can be derived for a given problem. The next step is
to analyse those proposed solution algorithms and implement the best suitable
solution.
1.8 ALGORITHM COMPLEXITY
Suppose X is an algorithm and n is the size of input data, the time and space used by
the algorithm X are the two main factors, which decide the efficiency of X.
 Time Factor − Time is measured by counting the number of key operations
such as comparisons in the sorting algorithm.
 Space Factor − Space is measured by counting the maximum memory space
required by the algorithm.
The complexity of an algorithm f(n) gives the running time and/or the storage space
required by the algorithm in terms of n as the size of input data.
1.8.1 Space Complexity
Space complexity of an algorithm represents the amount of memory space required by
the algorithm in its life cycle. The space required by an algorithm is equal to the sum
of the following two components −
  A fixed part that is a space required to store certain data and variables that are
independent of the size of the problem. For example, simple variables and
constants used, program size, etc.
 A variable part is a space required by variables, whose size depends on the size
of the problem. For example, dynamic memory allocation, recursion stack
space, etc.
Space complexity S(P) of any algorithm P is S(P) = C + SP(I), where C is the fixed
part and S(I) is the variable part of the algorithm, which depends on instance
characteristic I. Following is a simple example that tries to explain the concept −
Algorithm: SUM(A, B)
Step 1 - START
Step 2 - C ← A + B + 10
Step 3 - Stop
Here we have three variables A, B, and C and one constant. Hence S(P) = 1 + 3. Now,
space depends on data types of given variables and constant types and it will be
multiplied accordingly.
1.8.2 Time Complexity
Time complexity of an algorithm represents the amount of time required by the
algorithm to run to completion. Time requirements can be defined as a numerical
function T(n), where T(n) can be measured as the number of steps, provided each step
consumes constant time.

computational time is T(n) = c ∗ n, where c is the time taken for the addition of two
For example, addition of two n-bit integers takes n steps. Consequently, the total

bits. Here, we observe that T(n) grows linearly as the input size increases.
1.9 ALGORITHM ANALYSIS
Efficiency of an algorithm can be analyzed at two different stages, before
implementation and after implementation. They are the following –
 A Priori Analysis or Performance or Asymptotic Analysis − This is a
theoretical analysis of an algorithm. Efficiency of an algorithm is measured by
assuming that all other factors, for example, processor speed, are constant and
have noeffect on the implementation.
 A Posterior Analysis or Performance Measurement − This is an empirical
analysis of an algorithm. The selected algorithm is implemented using
programming language. This is then executed on target computer machine. In
this analysis, actual statistics like running time and space required, are
collected.
We shall learn about a priori algorithm analysis. Algorithm analysis deals with the
execution or running time of various operations involved. The running time of an
operation can be defined as the number of computer instructions executed per
operation.
Analysis of an algorithm is required to determine the amount of resources such as time
and storage necessary to execute the algorithm. Usually, the efficiency or running time
of an algorithm is stated as a function which relates the input length to the time
complexity or space complexity.
Algorithm analysis framework involves finding out the time taken and the memory
space required by a program to execute the program. It also determines how the input
size of a program influences the running time of the program.
In theoretical analysis of algorithms, it is common to estimate their complexity in the
asymptotic sense, i.e., to estimate the complexity function for arbitrarily large input.
BigO notation, Omega notation, and Theta notation are used to estimate the
complexity function for large arbitrary input.
1.9.1 Types of Analysis
The efficiency of some algorithms may vary for inputs of the same size. For such
algorithms, we need to differentiate between the worst case, average case and best
case efficiencies.
1.9.1.1 Best Case Analysis
If an algorithm takes the least amount of time to execute a specific set of input, then it
is called the best case time complexity. The best case efficiency of an algorithm is the
efficiency for the best case input of size n. Because of this input, the algorithm runs
the fastest among all the possible inputs of the same size.
1.9.1.2 Average Case Analysis
If the time complexity of an algorithm for certain sets of inputs are on an average,
then such a time complexity is called average case time complexity. Average case
analysis provides necessary information about an algorithm’s behaviour on a typical
or random input. You must make some assumption about the possible inputs of size n
to analyse the average case efficiency of algorithm.
1.9.1.3 Worst Case Analysis
If an algorithm takes maximum amount of time to execute for a specific set of input,
then it is called the worst case time complexity. The worst case efficiency of an
algorithm is the efficiency for the worst case input of size n. The algorithm runs the
longest among all the possible inputs of the similar size because of this input of size n.
1.10 MATHEMATICAL NOTATION
Algorithms are widely used in various areas of study. We can solve different problems
using the same algorithm. Therefore, all algorithms must follow a standard. The
mathematical notations use symbols or symbolic expressions, which have a precise
semantic meaning.
1.10.1 Asymptotic Notations
A problem may have various algorithmic solutions. In order to choose the best
algorithm for a particular process, you must be able to judge the time taken to run a
particular solution. More accurately, you must be able to judge the time taken to run
two solutions, and choose the better among the two.
To select the best algorithm, it is necessary to check the efficiency of each algorithm.
The efficiency of each algorithm can be checked by computing its time complexity.
The asymptotic notations help to represent the time complexity in a shorthand way. It
can generally be represented as the fastest possible, slowest possible or average
possible The notations such as O (Big-O), Ώ (Omega), and θ (Theta) are called as
asymptotic notations. These are the mathematical notations that are used in three
different cases of time complexity.
1.10.1.1 Big-O Notation
‘O’ is the representation for Big-O notation. Big -O is the method used to express the
upper bound of the running time of an algorithm. It is used to describe the
performance or time complexity of the algorithm. Big-O specifically describes the
worst-case scenario and can be used to describe the execution time required or the
space used by the algorithm.
Table 2.1 gives some names and examples of the common orders used to describe
functions. These orders are ranked from top to bottom.

Big-O notation is generally used to express an ordering property among the functions.
This notation helps in calculating the maximum amount of time taken by an algorithm
to compute a problem. Big-O is defined as:
f(n) ≤ c ∗ g(n)
where, n can be any number of inputs or outputs and f(n) as well as g(n) are two non-
negative functions. These functions are true only if there is a constant c and a non-
negative integer n0 such that, n ≥ n0.
The Big-O can also be denoted as f(n) = O(g(n)), where f(n) and g(n) are two non -
negative functions and f(n) < g(n) if g(n) is multiple of some constant c. The graphical
representation of f(n) = O(g(n)) is shown in figure 2.1, where the running time
increases considerably when n increases.
Example: Consider f(n)=15n3+40n2+2nlog n+2n. As the value of n increases, n3
becomes much larger than n2, nlog n, and n. Hence, it dominates the function f(n) and
we can consider the running time to grow by the order of n3. Therefore, it can be
written as f(n)=O(n3).
The values of n for f(n) and C* g(n) will not be less than n0. Therefore, the values less
than n0 are not considered relevant.

Figure : Big-O Notation f(n) = O(g(n))

Example:
Consider function f(n) = 2(n)+2 and g(n) = n2.
We need to find the constant c such that f(n) ≤ c ∗ g(n).
Let n = 1, then
f(n) = 2(n)+2 = 2(1)+2 = 4
g(n) = n2 = 12 = 1
Here, f(n)>g(n)
Let n = 2, then
f(n) = 2(n)+2 = 2(2)+2 = 6
g(n) = n2 = 22 = 4
Here, f(n)>g(n)
Let n = 3, then
f(n) = 2(n)+2 = 2(3)+2 = 8
 g(n) = n2 = 32 = 9
Here, f(n)<g(n)
Thus, when n is greater than 2, we get f(n)<g(n). In other words, as n becomes larger,
the running time increases considerably. This concludes that the Big-O helps to
determine the ‘upper bound’ of the algorithm’s run-time.
Limitations of Big O Notation
There are certain limitations with the Big O notation of expressing the complexity of
algorithms. These limitations are as follows:
 Many algorithms are simply too hard to analyse mathematically.
 There may not be sufficient information to calculate the behaviour of the algorithm
in the average case.
 Big O analysis only tells us how the algorithm grows with the size of the problem,
not how efficient it is, as it does not consider the programming effort.
 It ignores important constants. For example, if one algorithm takes O(n2 ) time to
execute and the other takes O(100000n2 ) time to execute, then as per Big O, both
algorithm have equal time complexity. In real-time systems, this may be a serious
consideration.
1.10.1.2 Omega Notation
‘Ω’ is the representation for Omega notation. Omega describes the manner in which
an algorithm performs in the best case time complexity. This notation provides the
minimum amount of time taken by an algorithm to compute a problem. Thus, it is
considered that omega gives the "lower bound" of the algorithm's run-time. Omega is
defined as:
f(n) ≥ c ∗ g(n)
Where, n is any number of inputs or outputs and f(n) and g(n) are two non-negative
functions. These functions are true only if there is a constant c and a non-negative
integer n0 such that n>n0.
Omega can also be denoted as f(n) = Ώ (g(n)) where, f of n is equal to Omega of g of
n . The graphical representation of f(n) = Ώ (g(n)) is shown in figure 2.2. The function
f(n) is said to be in Ώ (g(n)), if f(n) is bounded below by some constant multiple of

non-negative integer n0, such that f(n) ≥ c ∗ g(n) for all n ≥n0. Figure 2.2 shows
g(n) for all large values of n, i.e., if there exists some positive constant c and some

Omega notation.
 Figure Omega Notation f(n) = Ώ (g(n))
Let us take an example to understand the Omega notation more clearly.
Example:
Consider function f(n) = 2n2+5 and g(n) = 7n.
We need to find the constant c such that f(n) ≥ c ∗ g(n).
Let n = 0, then
f(n) = 2n2+5 = 2(0)2+5 = 5
g(n) = 7(n) = 7(0) = 0
Here, f(n)>g(n)
Let n = 1, then
f(n) = 2n2+5 = 2(1)2+5 = 7
g(n) = 7(n) = 7(1) = 7
Here, f(n)=g(n)
Let n = 2, then
f(n) = 2n2+5 = 2(2)2+5 = 13
g(n) = 7(n) = 7(2) = 14
Here, f(n) )<g(n)
Thus, for n=1, we get f(n) ≥ c ∗ g(n). This concludes that Omega helps to determine
the "lower bound" of the algorithm's run-time.
1.10.1.3 Theta Notation
'θ' is the representation for Theta notation. Theta notation is used when the upper
bound and lower bound of an algorithm are in the same order of magnitude. Theta can
be defined as:
c1 ∗ g(n) ≤ f(n) ≤ c2 ∗ g(n) for all n>n0
Where, n is any number of inputs or outputs and f(n) and g(n) are two nonnegative
functions. These functions are true only if there are two constants namely, c1, c2, and
a non-negative integer n0.
Theta can also be denoted as f(n) = θ(g(n)) where, f of n is equal to Theta of g of n.
The graphical representation of f(n) = θ(g(n)) is shown in figure 2.3. The function f(n)
is said to be in θ (g(n)) if f(n) is bounded both above and below by some positive
constant multiples of g(n) for all large values of n, i.e., if there exists some positive
constant c1 and c2 and some non-negative integer n0, such that C2g(n)≤f(n)≤ C1g(n)
for all n≥n0.

Figure shows Theta notation.

Let us take an example to understand the Theta notation more clearly.
Example: Consider function f(n) = 4n + 3 and g(n) = 4n for all n ≥ 3; and f(n) = 4n +
3 and g(n) = 5n for all n ≥ 3.
Then the result of the function will be:
Let n = 3
f(n) = 4n + 3 = 4(3)+3 = 15
g(n) = 4n =4(3) = 12 and
f(n) = 4n + 3 = 4(3)+3 = 15
g(n) = 5n =5(3) = 15 and
here, c1 is 4, c2 is 5 and n0 is 3
Thus, from the above equation we get c1 g(n) f(n) c2 g(n). This concludes that Theta
notation depicts the running time between the upper bound and lower bound.
DATA STRUCTURE
Basic Concept:
Data structure is a branch of computer science. The study of data structure helps you
to understand how data is organized and how data flow is managed to increase
efficiency of any process or program. Data structure is the structural representation of
logical relationship between data elements. This means that a data structure organizes
data items based on the relationship between the data elements.
Example:
A house can be identified by the house name, location, number of floors and so on.
These structured set of variables depend on each other to identify the exact house.
Similarly, data structure is a structured set of variables that are linked to each other,
which forms the basic component of a system
1.2.1 Basic Terminology
Data structures are the building blocks of any program or the software. Choosing the
appropriate data structure for a program is the most difficult task for a programmer.
Following terminology is used as far as data structures are concerned
Data: Data can be defined as an elementary value or the collection of values, for
example, student's name and its id are the data about the student.
Group Items: Data items which have subordinate data items are called Group item,
for example, name of a student can have first name and the last name.
Record: Record can be defined as the collection of various data items, for example, if
we talk about the student entity, then its name, address, course and marks can be
grouped together to form the record for the student.
File: A File is a collection of various records of one type of entity, for example, if
there are 60 employees in the class, then there will be 20 records in the related file
where each record contains the data about each employee.
Attribute and Entity: An entity represents the class of certain objects. it contains
various attributes. Each attribute represents the particular property of that entity.
Field: Field is a single elementary unit of information representing the attribute of an
entity.
1.2.2 Need for Data Structure
 It gives different level of organization data.
 It tells how data can be stored and accessed in its elementary level.
 Provide operation on group of data, such as adding an item, looking up highest
priority item.
 Provide a means to manage huge amount of data efficiently.
 Provide fast searching and sorting of data.
1.2.3 Goals of Data Structure
Data structure basically implements two complementary goals.
Correctness: Data structure is designed such that it operates correctly for all kinds of
input, which is based on the domain of interest. In other words, correctness forms the
primary goal of data structure, which always depends on the specific problems that the
data structure is intended to solve.
Efficiency: Data structure also needs to be efficient. It should process the data at high
speed without utilizing much of the computer resources such as memory space. In a
real time state, the efficiency of a data structure is an important factor that determines
the success and failure of the process.
1.2.4 Features of Data Structure
Some of the important features of data structures are:
Robustness: Generally, all computer programmers wish to produce software that
generates correct output for every possible input provided to it, as well as execute
efficiently on all hardware platforms. This kind of robust software must be able to
manage both valid and invalid inputs.
Adaptability: Developing software projects such as word processors, Web browsers
and Internet search engine involves large software systems that work or execute
correctly and efficiently for many years. Moreover, software evolves due to ever
changing market conditions or due to emerging technologies.
Reusability: Reusability and adaptability go hand-in-hand. It is a known fact that the
programmer requires many resources for developing any software, which makes it an
expensive enterprise. However, if the software is developed in a reusable and
adaptable way, then it can be implemented in most of the future applications. Thus, by
implementing quality data structures, it is possible to develop reusable software,
which tends to be cost effective and time saving.
1.3 CLASSIFICATION OF DATA STRUCTURES
A data structure provides a structured set of variables that are associated with each
other in different ways. It forms a basis of programming tool that represents the
relationship between data elements and helps programmers to process the data easily.
Data structure can be classified into two categories:
1 Primitive data structure
2 Non-primitive data structure
1.3.1 Primitive Data Structure
Primitive data structures consist of the numbers and the characters which are built in
programs. These can be manipulated or operated directly by the machine level
instructions. Basic data types such as integer, real, character, and Boolean come under
primitive data structures. These data types are also known as simple data types
because they consist of characters that cannot be divided.
1.3.2 Non-primitive Data Structure
Non-primitive data structures are those that are derived from primitive data structures.
These data structures cannot be operated or manipulated directly by the machine level
instructions. They focus on formation of a set of data elements that is either
homogeneous (same data type) or heterogeneous (different data type). These are
 further divided into linear and non-linear data structure based on the structure and
arrangement of data.
1.3.2.1 Linear Data Structure
A data structure that maintains a linear relationship among its elements is called a
linear data structure. Here, the data is arranged in a linear fashion. But in the memory,
the arrangement may not be sequential. Ex: Arrays, linked lists, stacks, queues.
1.3.2.1 Non-linear Data Structure
Non-linear data structure is a kind of data structure in which data elements are not
arranged in a sequential order. There is a hierarchical relationship between individual
data items. Here, the insertion and deletion of data is not possible in a linear fashion.
Ex: Trees and graphs.
1.5 OPERATIONS ON DATA STRUCTURES
This section discusses the different operations that can be performed on the various
data structures previously mentioned.
Traversing-It means to access each data item exactly once so that it can be processed.
For example, to print the names of all the students in a class.
Searching-It is used to find the location of one or more data items that satisfy the
given constraint. Such a data item may or may not be present in the given collection of
data items. For example, to find the names of all the students who secured 100 marks
in mathematics.
Inserting-It is used to add new data items to the given list of data items. For example,
to add the details of a new student who has recently joined the course.
Deleting- It means to remove (delete) a particular data item from the given collection
of data items. For example, to delete the name of a student who has left the course.
Sorting-Data items can be arranged in some order like ascending order or descending
order depending on the type of application. For example, arranging the names of
students in a class in an alphabetical order, or calculating the top three winners by
arranging the participants’ scores in descending order and then extracting the top
three.
Merging- Lists of two sorted data items can be combined to form a single list of
sorted data items.
PROGRAM PERFORMANCE MEASUREMENT
In computer science, there are numerous algorithms for solving a problem. When
there are several different algorithms to solve a problem, we evaluate the performance
of all those algorithms. Performance evaluation aids in the selection of the best
algorithm from a set of competing algorithms for a given issue. So, we can express
algorithm performance as a practice of producing evaluation judgments regarding
algorithms.
 Factors Determining Algorithm’s Performance
 o Space Complexity
o Time Complexity
 Notation of Performance Measurement
o Big – O (Big-Oh)
o Big – Ω (Omega)
o Big – Θ (Theta)
RECURRENCE RELATIONS
 A recurrence relation (or recurrence) is an equation or inequality that describes
a function in terms of its value on smaller inputs. We saw one in last example.
 There are several methods for solving recurrence relations, including the
substitution method, the recursion tree method and the master method.

SUBSTITUTION METHOD
 The substitution method for solving recurrence relations consists of three
steps:
1. Guess the form of the solution.
2. Verify by induction.
3. Solve for constants.

Let’s take a look at an example. Suppose we want to solve

We will assume that T(1)=Θ(1).

1.Let’s guess that We will assume that

2. Now we need to prove that by induction.

The base case is T(n)=Θ(1) for all n<n where n is a suitable constant.
We’ll now prove the general case.

Our desired form was , so we tried to put the equation in the form desired –
residual: . This is certainly less then our desired form
whenever ≥ (residual), for example, if c≥2 and n≥1.
 3.For 1≤n<n0, we have if we pick c large enough. And so we’ve

shown that But we can do better!

 We shall now prove that .

1. We’ll assume that

2. Now we need to prove this by induction. The base case is the same as before.
Now we attempt to prove the inductive hypothesis.

But there is no c>0 such that . It seems like we’ve hit a dead
end! But we can actually strengthen the inductive hypothesis by subtracting a low-
order term.

Our new inductive hypothesis will be

3.We can pick large enough to handle the initial conditions.

 As you can see, an important part of the substitution method involves making
a good guess. But unfortunately, there is no general way to guess the correct
solutions to recurrences. If a recurrence is similar to one you’ve seen before,
then guessing a similar solution is reasonable.
RECURSION TREE METHOD
 Since the substitution method involves having to guess the solution, it might
often be difficult to effectively use it. We can use recursion trees to devise a
good guess. In a recursion tree, each node represents the cost of a single
subproblem somewhere in the set of recursive function invocations. We sub
the costs within each level of the tree to obtain a set of per-level costs, and
 then we sum all the per-level costs to determine the total cost of all levels of
recursion.
 So let’s use an example to show the recursion tree method! Suppose we

wanted to solve
 Guessing a correct solution might be difficult and expanding the recurrence
might be difficult, so let’s draw a recursion tree.

 The recursion tree will represent the sum of the number of steps the
recurrence has to do on each level. From the original recurrence equation

, so we’ll use the notation in the above to

represent this sum.

 Now let’s expand We know

that and that
. So we can just substitute those
relations in the recurrence tree.

We can keep expanding the tree, and eventually we’ll reach a base case. (We’ll
assume T(1)=Θ(1).)

Now let’s count the cost on each level. We’ll just sum the number of nodes.
  The total cost will be We’ll assume that the

pattern continues and that the total sum is . If we rearrange

the equation, we’ll get . It turns out that

is a constant less than two

(since ). So the total cost

is . (You could indeed see that from the fact

that was the value at the root of the tree.)
 Since the recursion tree method has some ambiguities (such as the in the
diagram), it is a very good tool to come up with a “guess” to later verify
using the substitution method.

ALGORITHMS AS A TECHNOLOGY

Suppose computers were infinitely fast and computer memory was free. Would you
have any reason to study algorithms? The answer is yes, if for no other reason than
that you would still like to demonstrate that your solution method terminates and
does so with the correct answer.

If computers were infinitely fast, any correct method for solving a problem would
do. You would probably want your implementation to be within the bounds of good
software engineering practice (for example, your implementation should be well
designed and documented), but you would most often use whichever method was
the easiest to implement.

Of course, computers may be fast, but they are not infinitely fast. And memory may
be inexpensive, but it is not free. Computing time is therefore a bounded resource,
and so is space in memory. You should use these resources wisely, and algorithms
that are efficient in terms of time or space will help you do so.
Efficiency

Different algorithms devised to solve the same problem often differ dramatically in
their efficiency. These differences can be much more significant than differences
due to hardware and software.

As an example, we will see two algorithms for sorting. The first, known as
insertion sort, takes time roughly equal to to sort n items, where is a
constant that does not depend on n. That is, it takes time roughly proportional to .
The second, merge sort, takes time roughly equal to , where stands for
is another constant that also does not depend on n. Insertion sort
typically has a smaller constant factor than merge sort, so that .We shall see
that the constant factors can have far less of an impact on the running time than the
dependence on the input size n. Let’s write insertion sort’s running time as
and merge sort’s running time as Then we see that where insertion sort has
a factor of n in its running time, merge sort has a factor of lg n, which is much
smaller. (For example, when n D 1000, lg n is approximately 10, and when n equals
one million, lg n is approximately only 20.) Although insertion sort usually runs
faster than merge sort for small input sizes, once the input size n becomes large
enough, merge sort’s advantage of lg n vs. n will more than compensate for the
difference in constant factors. No matter how much smaller is than , there will
always be a crossover point beyond which merge sort is faster.

For a concrete example, let us pit a faster computer (computer A) running

insertion sort against a slower computer (computer B) running merge sort. They
each must sort an array of 10 million numbers. (Although 10 million numbers might
seem like a lot, if the numbers are eight-byte integers, then the input occupies about
80 megabytes, which fits in the memory of even an inexpensive laptop computer
many times over.) Suppose that computer A executes 10 billion instructions per
second (faster than any single sequential computer at the time of this writing) and
computer B executes only 10 million instructions per second, so that computer A is
1000 times faster than computer B in raw computing power. To make the difference
even more dramatic, suppose that the world’s craftiest programmer codes insertion
sort in machine language for computer A, and the resulting code requires
instructions to sort n numbers. Suppose further that just an average programmer
implements merge sort, using a high-level language with an inefficient compiler,
with the resulting code taking 50n lg n instructions. To sort 10 million numbers,
computer A takes

while computer B takes

 By using an algorithm whose running time grows more slowly, even with a
poor compiler, computer B runs more than 17 times faster than computer A! The
advantage of merge sort is even more pronounced when we sort 100 million
numbers: where insertion sort takes more than 23 days, merge sort takes under four
hours. In general, as the problem size increases, so does the relative advantage of
merge sort.