Data Structures & Algorithms

Acknowledgement

Primarily, we would thank God for being able to complete this project with success and like
to express our special thanks of gratitude to our lecture, Miss. Jubailah Begum who gave us
the golden opportunity to do this wonderful Assessment, which also helped us in doing a lot
of Research and we came to know about so many new things we are really thankful to them.

Secondly, we would also like to thank our parents and friends who helped us a lot in
finalizing this Assessment within the limited time frame. And also, we dedicate this
Assessment our lectures who helped us to curry on our education life very well.

1
 Data Structures & Algorithms

Abstract

Abstract Data type (ADT) is a type (or class) for objects whose behaviour is defined by a set
of value and a set of operations.

The definition of ADT only mentions what operations are to be performed but not how these
operations will be implemented. It does not specify how data will be organized in memory
and what algorithms will be used for implementing the operations. It is called “abstract”
because it gives an implementation-independent view. The process of providing only the
essentials and hiding the details is known as abstraction.

2
 Data Structures & Algorithms

Task 1.1

a. Create a Design specification for data structures, explaining the valid

operations that can be carried out on the structures

Information is the summarization of data. Data are raw facts and figures that are processed
into information, such as summaries and totals. Information is the result of processing,
manipulating and organizing data in a way that adds to the knowledge of the receiver. Even
though information and data are often used interchangeably, they are actually very different.

Data is a set of unrelated information and as such is of no use until it is properly evaluated.
Upon evaluation, once there is some significant relation between data, it is converted into
information. Now this data can be used for different purposes. Till data conveys some
information, they are not useful

In computing, data is information that has been translated into a form that is efficient for
movement or processing. Relative to today's computers and transmission media, data is
information converted into binary digital form. It is acceptable for data to be used as a
singular subject or a plural subject. Raw data is a term used to describe data in its most
basic digital format.

P2

Abstract Data Type in Data Structures

The Data Type is basically a type of data that can be used in different computer program. It
signifies the type like integer, float etc, the space like integer will take 4-bytes, character will
take 1-byte of space etc.

The abstract datatype is special kind of datatype, whose behavior is defined by a set of
values and set of operations. The keyword “Abstract” is used as we can use these
datatypes, we can perform different operations. But how those operations are working that is
totally hidden from the user. The ADT is made of with primitive datatypes, but operation
logics are hidden.

Some examples of ADT are Stack, Queue, List etc.

3
 Data Structures & Algorithms

Let us see some operations of those mentioned ADT −

Stack −

 isFull(), This is used to check whether stack is full or not

 isEmpry(), This is used to check whether stack is empty or not
 push(x), This is used to push x into the stack
 pop(), This is used to delete one element from top of the stack
 peek(), This is used to get the top most element of the stack
 size(), this function is used to get number of elements present into the stack

Queue −

 isFull(), This is used to check whether queue is full or not

 isEmpry(), This is used to check whether queue is empty or not
 insert(x), This is used to add x into the queue at the rear end
 delete(), This is used to delete one element from the front end of the queue
 size(), this function is used to get number of elements present into the queue

Concrete and Abstract Data Types

A concrete data type is a data type whose representation is known and relied upon by the
programmers who use the data type.

f you know the representation of a data type and are allowed to rely upon that knowledge,
then the data type is concrete.

If you do not know the representation of a data type and are not allowed to rely upon its
representation, then the data type is abstract.

Concrete Versus Abstract Types

 Concrete data types or structures (CDT's) are direct implementations of a

relatively simple concept.

 Abstract Data Types (ADT's) offer a high level view (and use) of a concept

4
 Data Structures & Algorithms

independent of its implementation.

 Example: Implementing a student record:

¾ CDT: Use a struct with public data and no functions to represent the record

– does not hide anything

¾ ADT: Use a class with private data and public functions to represent the record

– hides structure of the data, etc.

 Concrete data structures are divided into:

¾ contiguous

¾ linked

¾ hybrid

 „ Some fundamental concrete data structures:

¾ arrays

¾ records,

¾ linked lists

¾ tree

Stack Memory Operations

Stack memory is a memory usage mechanism that allows the system memory to be used as
temporary data storage that behaves as a first-in-last-out buffer. One of the essential
elements of stack memory operation is a register called the Stack Pointer. The stack pointer
indicates where the current stack memory location is, and is adjusted automatically each
time a stack operation is carried out.

In the Cortex®-M processors, the Stack Pointer is register R13 in the register bank.
Physically there are two stack pointers in the Cortex-M processors, but only one of them is
used at a time, depending on the current value of the CONTROL register and the state of the
processor

Function call

5
 Data Structures & Algorithms

Updated: 04/26/2017 by Computer Hope

A function call is a request made by a program or script that performs a predetermined

function. In the example below, a batch file clears the screen and then calls another batch
file.

Example :

@echo off

cls

In computer science, a call stack is a stack data structure that stores information about the
active subroutines of a computer program. This kind of stack is also known as an execution
stack, program stack, control stack, run-time stack, or machine stack, and is often shortened
to just "the stack". Although maintenance of the call stack is important for the proper
functioning of most software, the details are normally hidden and automatic in high-level
programming languages. Many computer instruction sets provide special instructions for
manipulating stacks.

P3

Abstract Data Types

This module presents terminology and definitions related to techniques for managing the
tremendous complexity of computer programs. It also presents working definitions for the
fundamental but somewhat slippery terms "data item" and "data structure". We begin with
the basic elements on which data structures are built.

A type is a collection of values. For example, the Boolean type consists of the values true
and false. The integers also form a type. An integer is a simple type because its values
contain no subparts. A bank account record will typically contain several pieces of
information such as name, address, account number, and account balance. Such a record is
an example of an aggregate type or composite type. A data item is a piece of information or
a record whose value is drawn from a type. A data item is said to be a member of a type.

A data type is a type together with a collection of operations to manipulate the type. For
example, an integer variable is a member of the integer data type. Addition is an example of
an operation on the integer data type.

6
 Data Structures & Algorithms

A distinction should be made between the logical concept of a data type and its physical
implementation in a computer program. For example, there are two traditional
implementations for the list data type: the linked list and the array-based list. The list data
type can therefore be implemented using a linked list or an array. But we don't need to know
how the list is implemented when we wish to use a list to help in a more complex design. For
example, a list might be used to help implement a graph data structure.

As another example, the term "array" could refer either to a data type or an implementation.
"Array" is commonly used in computer programming to mean a contiguous block of memory
locations, where each memory location stores one fixed-length data item. By this meaning,
an array is a physical data structure. However, array can also mean a logical data type
composed of a (typically homogeneous) collection of data items, with each data item
identified by an index number. It is possible to implement arrays in many different ways
besides as a block of contiguous memory locations. The sparse matrix refers to a large, two-
dimensional array that stores only a relatively few non-zero values. This is often
implemented with a linked structure, or possibly using a hash table. But it could be
implemented with an interface that uses traditional row and column indices, thus appearing
to the user in the same way that it would if it had been implemented as a block of contiguous
memory locations.

An abstract data type (ADT) is the specification of a data type within some language,
independent of an implementation. The interface for the ADT is defined in terms of a type
and a set of operations on that type. The behavior of each operation is determined by its
inputs and outputs. An ADT does not specify how the data type is implemented. These
implementation details are hidden from the user of the ADT and protected from outside
access, a concept referred to as encapsulation.

A data structure is the implementation for an ADT. In an object-oriented language, an ADT

and its implementation together make up a class. Each operation associated with the ADT is
implemented by a member function or method. The variables that define the space required
by a data item are referred to as data members. An object is an instance of a class, that is,
something that is created and takes up storage during the execution of a computer program.

7
 Data Structures & Algorithms

The term data structure often refers to data stored in a computer's main memory. The related
term file structure often refers to the organization of data on peripheral storage, such as a
disk drive or CD.

P4

Complex Data Structures

Different algorithms require different data structures. Using references in Perl, it is possible
to build very complex data structures.

This section gives a short introduction to some of the possibilities, such as a hash with array
values and a two-dimensional array of hashes. See the recommended reading in Section 2.9
of this chapter for books and sections of the Perl manual that are very helpful.

Perl uses the basic data types of scalar, array, and hash, plus the ability to declare scalar
references to those basic data types, to build more complex structures. For instance, an
array must have scalar elements, but those scalar elements can be references to hashes, in
which case you have effectively created an array of hashes.

Hash with Array Values

A common example of a complex data structure is a hash with array values. Using such a
data structure, you can associate a list of items with each keyword. The following code
shows an example of how to build and manage such a data structure. Assume you have a
set of human genes, and for each human gene, you want to manage an array of organisms
that are known to have closely related genes. Of course, each such array of related
organisms can be a different length:

use Data::Dumper;

8
 Data Structures & Algorithms

%relatedgenes = ( );

$relatedgenes{'stromelysin'} = [

'C.elegans',

'Arabidopsis thaliana'

];

$relatedgenes{'obesity'} = [

'Drosophila',

'Mus musculus'

];

# Now add a new related organism to the entry for 'stromelysin'

push( @{$relatedgenes{'stromelysin'}}, 'Canis' );

print Dumper(\%relatedgenes);

This program prints out the following (the very useful Data::Dumper module is described in
more detail later; try typing perldoc Data::Dumper for the details of this useful way to print
out complex data structures):

$VAR1 = {

'stromelysin' => [

'C.elegans',

'Arabidopsis thaliana',

'Canis'

],

'obesity' => [

9
 Data Structures & Algorithms

'Drosophila',

'Mus musculus'

};

The tricky part of this short program is the push. The first argument to push must be an
array. In the program, this array is @{$relatedgenes{'stromelysin'}}. Examining this array
from the inside out, you can see that it refers to the value of the hash with key stromelysin:
$relatedgenes{'stromelysin'}. You know that the values of this %relatedgenes hash are
references to anonymous arrays. This hash value is contained within a block of curly braces,
which returns the reference to the anonymous array: {$relatedgenes{'stromelysin'}}, and the
block is preceded by an @ sign that dereferences the anonymous array:
@{$relatedgenes{'stromelysin'}}.

Two-Dimensional Array of Hashes

As another example, say you have data from a microarray experiment in which each location
on a plate can be identified by an x and y location; each location is also associated with a
particular gene and has a set of reported measurements. You can implement this particular
data as a two-dimensional array, each entry of which is a (reference to a) hash whose keys
are gene names and whose values are (references to) arrays of the measurements. Here's
how you can initialize one of the entries of that two-dimensional array:

$array[3][4]{'stromelysin'} = [3, 4, 5];

The position on the plate is represented by an entry in the two-dimensional array such as
$array[3][4]. The fact that the entry is a hash is shown by the reference to a particular key
with {'stromelysin'}. That the value for that key is an array is shown by the assignment to that
key $array[3][4]{'stromelysin'} of the anonymous array [3, 4, 5]. To print out the array
associated with the key stromelysin, you have to remember to tell Perl that the value for that
key is an array by surrounding the expression with curly braces preceded by an @ sign
@{$array[3][4]{'stromelysin'}}:

$array[3][4]{'stromelysin'} = [3, 4, 5];

10
 Data Structures & Algorithms

print "The scores for plate position 3, 4 were @{$array[3][4]{'stromelysin'}}

\n";

This prints:

The scores for plate position 3, 4 were 3 4 5

A common Perl trick is to dereference a complex data structure by enclosing the whole thing
in curly braces and preceding it with the correct symbol: $, @, or %. So, take a moment and
reread the last example. Do you see how the following:

$array[3][4]{'stromelysin'}

is the key for a hash? Do you see how the phrase:

@{$array[3][4]{'stromelysin'}}

makes it clear that the value for that hash key is an array? Similarly, if the value for that hash
key was a scalar, you could say:

${$array[3][4]{'stromelysin'}}

and if the value for that hash key was a hash, you could say:

%{$array[3][4]{'stromelysin'}}

2.4.3 Complex Data Structures

References give you a fair amount of flexibility. For example, your data structures can
combine references to different types of data. You can have an anonymous array such as in
the following short program:

$gene = [

# hash of basic information about the gene name, discoverer,

11
 Data Structures & Algorithms

# discovery date and laboratory.

name => 'antiaging',

reference => [ 'G. Mendel', '1865'],

laboratory => [ 'Dept. of Genetics', 'Cornell University', 'USA']

},

# scalar giving priority

'high',

# array of local work history

['Jim', 'Rose', 'Eamon', 'Joe']

];

print "Name is ", ${$gene->[0]}{'name'}, "\n";

print "Priority is ", $gene->[1], "\n";

print "Research center is ", ${${$gene->[0]}{'laboratory'}}[1], "\n";

print "These individuals worked on the gene: ", "@{$gene->[2]}", "\n";

This program produces the output:

12
 Data Structures & Algorithms

Name is antiaging

Priority is high

Research center is Cornell University

These individuals worked on the gene: Jim Rose Eamon Joe

Let's examine this code to understand how it works; it contains most of the points made in
this chapter.

$gene is a pointer to an anonymous array of three elements. Therefore each element of

$gene is referred to by either:

$$gene[0]

$$gene[1]

$$gene[2]

or equivalently (and our choice in this code) by:

$gene->[0]

$gene->[1]

$gene->[2]

To be specific, the first element is a reference to an anonymous hash, the second element is
a scalar string high, and the third element is a reference to an anonymous workgroup array.

The plot thickens when you examine the anonymous hash that is referenced by the first
array element. It has three keys, one of which, name, has a simple scalar value. The other
two keys have values that are references to anonymous arrays of scalar strings.

13
 Data Structures & Algorithms

So, this certainly qualifies as a complex data structure!

When you place any of the elements of the $gene anonymous array within a block of curly
braces, you have a reference that must be dereferenced appropriately. To refer to the entire
hash at the beginning of the array, say:

%{$gene->[0]}

As done with the program code, the scalar value that is the second element of the array is
accessed simply as:

$gene->[1]

The third part of this data structure is an anonymous array, which we can refer to in total as:

@{$gene->[2]}

This is also done in the program code.

Now, let's finish by looking into the first element of the $gene anonymous array. This is a
reference to an anonymous hash. One of the keys of that hash has a simple scalar string
value, which is referenced with:

${$gene->[0]}{name}

as was done in the program code. To make sure we understand this, let's write it out:

${$gene->[0]}{name}

14
 Data Structures & Algorithms

is

$ hashref {name}

is

'antiaging'

{$gene->[0]} is a block containing a reference to an anonymous hash. It is then used as is

typical for a hash reference: it's preceded by a $ and followed by the key name in curly
braces and so resolves to a lookup of the key name in the anonymous hash.

The most intricate dereference in this program is that which digs out the name of the
research center:

${${$gene->[0]}{laboratory} }[1]

is

${$ hashref {laboratory} }[1]

is

$ arrayref [1]

is

'Cornell University'

Here, the {$gene->[0]} is a reference to an anonymous hash. The value for the key
laboratory is retrieved from that anonymous hash; the value is an anonymous array. Finally,
that anonymous array ${$gene->[0]}{laboratory} is enclosed in a block of curly braces,
preceded by a $, and followed by an array index 1 in square brackets, which dereferences
the anonymous array and returns the second element Cornell University.

Note that the last expression can also be written as:

$gene->[0]->{laboratory}->[1]

You see how the use of references within blocks enables you to dereference some rather
deep-nested data structures. I urge you to take the time to understand this example and to
use the resources listed in Section 2.9.

15
 Data Structures & Algorithms

P5

Error Handling

No program or program fragment can be considered complete until appropriate error

handling has been added. Unexpected program failures are a disaster - at the best, they
cause frustration because the program user must repeat minutes or hours of work, but in life-
critical applications, even the most trivial program error, if not processed correctly, has the
potential to kill someone.

If an error is fatal, in the sense that a program cannot sensibly continue, then the program
must be able to "die gracefully". This means that it must

inform its user(s) why it died, and

save as much of the program state as possible.

2.7.1 Defining Errors

The first step in determining how to handle errors is to define precisely what is considered to
be an error. Careful specification of each software component is part of this process. The
pre-conditions of an ADT's methods will specify the states of a system (the input states)
which a method is able to process. The post-conditions of each method should clearly
specify the result of processing each acceptable input state. Thus, if we have a method:

int f( some_class a, int i )

/* PRE-CONDITION: i >= 0 */

/* POST-CONDITION:

if ( i == 0 )

return 0 and a is unaltered

else

return 1 and update a's i-th element by .... */

This specification tells us that i==0 is a meaningless input that f should flag by returning 0
but otherwise ignore.

16
 Data Structures & Algorithms

f is expected to handle correctly all positive values of i.

The behaviour of f is not specified for negative values of i, ie it also tells us that

It is an error for a client to call f with a negative value of i.

Thus, a complete specification will specify

all the acceptable input states, and

the action of a method when presented with each acceptable input state.

By specifying the acceptable input states in pre-conditions, it will also divide responsibility for
errors unambiguously.

The client is responsible for the pre-conditions: it is an error for the client to call the method
with an unacceptable input state, and

The method is responsible for establishing the post-conditions and for reporting errors which
occur in doing so.

2.7.2 Processing errors

Let's look at an error which must be handled by the constructor for any dynamically allocated
object: the system may not be able to allocate enough memory for the object.

A good way to create a disaster is to do this:

X ConsX( .... )

X x = malloc( sizeof(struct t_X) );

if ( x == NULL ) {

printf("Insuff mem\n"); exit( 1 );

else

.....

Not only is the error message so cryptic that it is likely to be little help in locating the cause of
the error (the message should at least be "Insuff mem for X"!), but the program will simply

17
 Data Structures & Algorithms

exit, possibly leaving the system in some unstable, partially updated, state. This approach
has other potential problems:

What if we've built this code into some elaborate GUI program with no provision for
"standard output"? We may not even see the message as the program exits!

We may have used this code in a system, such as an embedded processor (a control
computer), which has no way of processing an output stream of characters at all.

The use of exit assumes the presence of some higher level program, eg a Unix shell, which
will capture and process the error code 1.

As a general rule, I/O is non-portable!

A function like printf will produce error messages on the 'terminal' window of your modern
workstation, but if you are running a GUI program like Netscape, where will the messages
go?

So, the same function may not produce useful diagnostic output for two programs running in
different environments on the same processor! How can we expect it to be useful if we
transport this program to another system altogether, eg a Macintosh or a Windows machine?

Before looking at what we can do in ANSI C, let's look at how some other languages tackle
this problem.

P6

P6 Discuss how asymptotic analysis can be used to assess the effectiveness of an

algorithm.

Asymptotic AnalysisAsymptotic analysis of an algorithm is defining the mathematical framing

its run-timeperformance. Using asymptotic analysis, we can conclude the best case, average
case and worst-case scenario of an algorithm. The big idea that can handles above issues in
analyzingalgorithms. In Asymptotic Analysis, we evaluate the performance of an algorithm in
terms ofinput size but we don’t measure the actual running time. We calculate, how does the
time orspace take by an algorithm increases with the input size.For example, the running
time of one operation is computed asf(n) and may be for anotheroperation it is computed
asg(n2). This means the first operation running time will increaselinearly with the increase
innand the running time of the second operation will increaseexponentially whennincreases.
Similarly, the running time of both operations will be nearly thesame ifnis significantly small.

18
 Data Structures & Algorithms

How it can be used to find effectiveness of an algorithm

They are finding out more effectiveness of an algorithm can be used so we time required
byalgorithm falls under the three types: Worst case maximum time required by an algorithm
and itis mostly or done while analyzing the algorithm.The commonly used notation for
calculating the running time complexity of the algorithm asfollows:„Big O
notation„Bigθnotation„Big notation

P7

Complexity Analysis

An essential aspect to data structures is algorithms. Data structures are implemented using
algorithms. An algorithm is a procedure that you can write as a C function or program, or any
other language. An algorithm states explicitly how the data will be manipulated.

Algorithm Efficiency

Some algorithms are more efficient than others. We would prefer to chose an efficient
algorithm, so it would be nice to have metrics for comparing algorithm efficiency.

The complexity of an algorithm is a function describing the efficiency of the algorithm in

terms of the amount of data the algorithm must process. Usually there are natural units for
the domain and range of this function. There are two main complexity measures of the
efficiency of an algorithm:

Time complexity is a function describing the amount of time an algorithm takes in terms of
the amount of input to the algorithm. "Time" can mean the number of memory accesses
performed, the number of comparisons between integers, the number of times some inner

19
 Data Structures & Algorithms

loop is executed, or some other natural unit related to the amount of real time the algorithm
will take. We try to keep this idea of time separate from "wall clock" time, since many factors
unrelated to the algorithm itself can affect the real time (like the language used, type of
computing hardware, proficiency of the programmer, optimization in the compiler, etc.). It
turns out that, if we chose the units wisely, all of the other stuff doesn't matter and we can
get an independent measure of the efficiency of the algorithm.

Space complexity is a function describing the amount of memory (space) an algorithm

takes in terms of the amount of input to the algorithm. We often speak of "extra" memory
needed, not counting the memory needed to store the input itself. Again, we use natural (but
fixed-length) units to measure this. We can use bytes, but it's easier to use, say, number of
integers used, number of fixed-sized structures, etc. In the end, the function we come up
with will be independent of the actual number of bytes needed to represent the unit. Space
complexity is sometimes ignored because the space used is minimal and/or obvious, but
sometimes it becomes as important an issue as time.

For example, we might say "this algorithm takes n2 time," where n is the number of items in
the input. Or we might say "this algorithm takes constant extra space," because the amount
of extra memory needed doesn't vary with the number of items processed.

For both time and space, we are interested in the asymptotic complexity of the algorithm:
When n (the number of items of input) goes to infinity, what happens to the performance of
the algorithm?

An example: Selection Sort

Suppose we want to put an array of n floating point numbers into ascending numerical order.
This task is called sorting and should be somewhat familiar. One simple algorithm for sorting
is selection sort. You let an index i go from 0 to n-1, exchanging the ith element of the array
with the minimum element from i up to n. Here are the iterations of selection sort carried out
on the sequence {4 3 9 6 1 7 0}:

index 0 1 2 3 4 5 6 comments

--------------------------------------------------------- --------

| 4 3 9 6 1 7 0 initial

20
 Data Structures & Algorithms

i=0 | 0 3 9 6 1 7 4 swap 0,4

i=1 | 0 1 9 6 3 7 4 swap 1,3

i=2 | 0 1 3 6 9 7 4 swap 3, 9

i=3 | 0 1 3 4 9 7 6 swap 6, 4

i=4 | 0 1 3 4 6 7 9 swap 9, 6

i=5 | 0 1 3 4 6 7 9 (done)

Here is a simple implementation in C:

int find_min_index (float [], int, int);

void swap (float [], int, int);

/* selection sort on array v of n floats */

void selection_sort (float v[], int n) {

int i;

/* for i from 0 to n-1, swap v[i] with the minimum

* of the i'th to the n'th array elements

*/

for (i=0; i<n-1; i++)

swap (v, i, find_min_index (v, i, n));

/* find the index of the minimum element of float array v from

* indices start to end

*/

int find_min_index (float v[], int start, int end) {

21
 Data Structures & Algorithms

int i, mini;

mini = start;

for (i=start+1; i<end; i++)

if (v[i] < v[mini]) mini = i;

return mini;

/* swap i'th with j'th elements of float array v */

void swap (float v[], int i, int j) {

float t;

t = v[i];

v[i] = v[j];

v[j] = t;

Now we want to quantify the performance of the algorithm, i.e., the amount of time and
space taken in terms of n. We are mainly interested in how the time and space requirements
change as n grows large; sorting 10 items is trivial for almost any reasonable algorithm you
can think of, but what about 1,000, 10,000, 1,000,000 or more items?

For this example, the amount of space needed is clearly dominated by the memory
consumed by the array, so we don't have to worry about it; if we can store the array, we can
sort it. That is, it takes constant extra space.

So we are mainly interested in the amount of time the algorithm takes. One approach is to
count the number of array accesses made during the execution of the algorithm; since each
array access takes a certain (small) amount of time related to the hardware, this count is
proportional to the time the algorithm takes.

22
 Data Structures & Algorithms

We will end up with a function in terms of n that gives us the number of array accesses for
the algorithm. We'll call this function T(n), for Time.

T(n) is the total number of accesses made from the beginning of selection_sort until the end.
selection_sort itself simply calls swap and find_min_index as i goes from 0 to n-1, so

T(n) = [ time for swap + time for find_min_index (v, i, n)] .

(n-2 because the for loop goes from 0 up to but not including n-1). (Note: for those not
familiar with Sigma notation, that nasty looking formula above just means "the sum, as we let
i go from 0 to n-2, of the time for swap plus the time for find_min_index (v, i, n).) The swap
function makes four accesses to the array, so the function is now

T(n) = [ 4 + time for find_min_index (v, i, n)] .

If we look at find_min_index, we see it does two array accesses for each iteration through
the for loop, and it does the for loop n - i - 1 times:

T(n) = [ 4 + 2 (n - i - 1)] .

With some mathematical manipulation, we can break this up into:

T(n) = 4(n-1) + 2n(n-1) - 2(n-1) - 2 i .

(everything times n-1 because we go from 0 to n-2, i.e., n-1 times). Remembering that the
sum of i as i goes from 0 to n is (n(n+1))/2, then substituting in n-2 and cancelling out the 2's:

T(n) = 4(n-1) + 2n(n-1) - 2(n-1) - ((n-2)(n-1)).

and to make a long story short,

T(n) = n2 + 3n - 4 .

So this function gives us the number of array accesses selection_sort makes for a given
array size, and thus an idea of the amount of time it takes. There are other factors affecting
the performance, for instance the loop overhead, other processes running on the system,
and the fact that access time to memory is not really a constant. But this kind of analysis
gives you a good idea of the amount of time you'll spend waiting, and allows you to compare
this algorithms to other algorithms that have been analyzed in a similar way.

Another algorithm used for sorting is called merge sort. The details are somewhat more
complicated and will be covered later in the course, but for now it's sufficient to state that a

23
 Data Structures & Algorithms

certain C implementation takes Tm(n) = 8n log n memory accesses to sort n elements. Let's
look at a table of T(n) vs. Tm(n):

n T(n) Tm(n)

--- ---- -----

2 6 11

3 14 26

4 24 44

5 36 64

6 50 86

7 66 108

8 84 133

9 104 158

10 126 184

11 150 211

12 176 238

13 204 266

14 234 295

15 266 324

16 300 354

17 336 385

18 374 416

19 414 447

20 456 479

T(n) seems to outperform Tm(n) here, so at first glance one might think selection sort is
better than merge sort. But if we extend the table:

n T(n) Tm(n)

24
 Data Structures & Algorithms

--- ---- -----

20 456 479

21 500 511

22 546 544

23 594 576

24 644 610

25 696 643

26 750 677

27 806 711

28 864 746

29 924 781

30 986 816

we see that merge sort starts to take a little less time than selection sort for larger values of
n. If we extend the table to large values:

n T(n) Tm(n)

--- ---- -----

100 10,296 3,684

1,000 1,002,996 55,262

10,000 100,029,996 736,827

100,000 10,000,299,996 9,210,340

1,000,000 1,000,002,999,996 110,524,084

10,000,000 100,000,029,999,996 1,289,447,652

we see that merge sort does much better than selection sort. To put this in perspective,
recall that a typical memory access is done on the order of nanoseconds, or billionths of a
second. Selection sort on ten million items takes roughly 100 trillion accesses; if each one
takes ten nanoseconds (an optimistic assumption based on 1998 hardware) it will take
1,000,000 seconds, or about 11 and a half days to complete. Merge sort, with a "mere" 1.2
billion accesses, will be done in 12 seconds. For a billion elements, selection sort takes

25
 Data Structures & Algorithms

almost 32,000 years, while merge sort takes about 37 minutes. And, assuming a large
enough RAM size, a trillion elements will take selection sort 300 million years, while merge
sort will take 32 days. Since computer hardware is not resilient to the large asteroids that hit
our planet roughly once every 100 million years causing mass extinctions, selection sort is
not feasible for this task. (Note: you will notice as you study CS that computer scientists like
to put things in astronomical and geological terms when trying to show an approach is the
wrong one. Just humor them.)

Asymptotic Notation

This function we came up with, T(n) = n2 + 3n - 4, describes precisely the number of array
accesses made in the algorithm. In a sense, it is a little too precise; all we really need to say
is n2; the lower order terms contribute almost nothing to the sum when n is large. We would
like a way to justify ignoring those lower order terms and to make comparisons between
algorithms easy. So we use asymptotic notation.

Big O

The most common notation used is "big O" notation. In the above example, we would say n2
+ 3n - 4 = O(n2) (read "big oh of n squared"). This means, intuitively, that the important part
of n2 + 3n - 4 is the n2 part.

Definition: Let f(n) and g(n) be functions, where n is a positive integer. We write f(n) =
O(g(n)) if and only if there exists a real number c and positive integer n0 satisfying 0 <= f(n)
<= cg(n) for all n >= n0. (And we say, "f of n is big oh of g of n." We might also say or write
f(n) is in O(g(n)), because we can think of O as a set of functions all with the same property.
But we won't often do that in Data Structures.)

This means that, for example, that functions like n2 + n, 4n2 - n log n + 12, n2/5 - 100n, n log
n, 50n, and so forth are all O(n2). Every function f(n) bounded above by some constant
multiple g(n) for all values of n greater than a certain value is O(g(n)).

Examples:

Show 3n2 + 4n - 2 = O(n2).

We need to find c and n0 such that:

3n2 + 4n - 2 <= cn2 for all n >= n0 .

Divide both sides by n2, getting:

3 + 4/n - 2/n2 <= c for all n >= n0 .

26
 Data Structures & Algorithms

If we choose n0 equal to 1, then we need a value of c such that:

3 + 4 - 2 <= c

We can set c equal to 6. Now we have:

3n2 + 4n - 2 <= 6n2 for all n >= 1 .

Show n3 != O(n2). Let's assume to the contrary that

n3 = O(n2)

Then there must exist constants c and n0 such that

n3 <= cn2 for all n >= n0.

Dividing by n2, we get:

n <= c for all n >= n0.

But this is not possible; we can never choose a constant c large enough that n will never
exceed it, since n can grow without bound. Thus, the original assumption, that n3 = O(n2),
must be wrong so n3 != O(n2).

Big O gives us a formal way of expressing asymptotic upper bounds, a way of bounding from
above the growth of a function. Knowing where a function falls within the big-O hierarchy
allows us to compare it quickly with other functions and gives us an idea of which algorithm
has the best time performance. And yes, there is also a "little o" we'll see later.

Properties of Big O

The definition of big O is pretty ugly to have to work with all the time, kind of like the "limit"
definition of a derivative in Calculus. Here are some helpful theorems you can use to simplify
big O calculations:

Any kth degree polynomial is O(nk).

a nk = O(nk) for any a > 0.

Big O is transitive. That is, if f(n) = O(g(n)) and g(n) is O(h(n)), then f(n) = O(h(n)).

logan = O(logb n) for any a, b > 1. This practically means that we don't care, asymptotically,
what base we take our logarithms to. (I said asymptotically. In a few cases, it does matter.)

Big O of a sum of functions is big O of the largest function. How do you know which one is
the largest? The one that all the others are big O of. One consequence of this is, if f(n) =
O(h(n)) and g(n) is O(h(n)), then f(n) + g(n) = O(h(n)).

27
 Data Structures & Algorithms

f(n) = O(g(n)) is true if limn->infinityf(n)/g(n) is a constant.

Lower Bounds and Tight Bounds

Big O only gives you an upper bound on a function, i.e., if we ignore constant factors and let
n get big enough, we know some function will never exceed some other function. But this
can give us too much freedom. For instance, the time for selection sort is easily O(n3),
because n2 is O(n3). But we know that O(n2) is a more meaningful upper bound. What we
need is to be able to describe a lower bound, a function that always grows more slowly than
f(n), and a tight bound, a function that grows at about the same rate as f(n). Your book give a
good theoretical introduction to these two concepts; let's look at a different (and probably
easier to understand) way to approach this.

Big Omega is for lower bounds what big O is for upper bounds:

Definition: Let f(n) and g(n) be functions, where n is a positive integer. We write f(n) = (g(n))
if and only if g(n) = O(f(n)). We say "f of n is omega of g of n."

This means g is a lower bound for f; after a certain value of n, and without regard to
multiplicative constants, f will never go below g.

Finally, theta notation combines upper bounds with lower bounds to get tight bounds:

Definition: Let f(n) and g(n) be functions, where n is a positive integer. We write f(n) = (g(n))
if and only if g(n) = O(f(n)). and g(n) = (f(n)). We say "f of n is theta of g of n."

More Properties

The first four properties listed above for big O are also true for Omega and Theta.

Replace O with and "largest" with "smallest" in the fifth property for big O and it remains
true.

f(n) = (g(n)) is true if limn->infinityg(n)/f(n) is a constant.

f(n) = (g(n)) is true if limn->infinityf(n)/g(n) is a non-zero constant.

nk = O((1+) n)) for any positive k and . That is, any polynomial is bound from above by any
exponential. So any algorithm that runs in polynomial time is (eventually, for large enough
value of n) preferable to any algorithm that runs in exponential time.

28
 Data Structures & Algorithms

(log n) = O(n k) for any positive k and . That means a logarithm to any power grows more
slowly than a polynomial (even things like square root, 100th root, etc.) So an algorithm that
runs in logarithmic time is (eventually) preferable to an algorithm that runs in polynomial (or
indeed exponential, from above) time.

29