Design and Analysis of Algorithms

By

Arslan Anjum

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Objective of This Course
Major objective of this course is:
• Design and analysis of modern algorithms
• Different variants
• Accuracy
• Efficiency
• Comparing efficiencies
• Motivation thinking new algorithms
• Advanced designing techniques
• Real world problems will be taken as examples
• To create feelings about usefulness of this course

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Expected Results
On successful completion, students will be able to
• Argue and prove correctness of algorithms
• Derive and solve mathematical models of problems
• Reasoning when an algorithm calls certain approach
• Analyze average and worst-case running times
• Integrating approaches in dynamic and greedy algos.
• Use of graph theory in problems solving
• Several other algorithms such as
• String matching, NP completeness, approximate
algorithms etc.

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Lecture No 1

Introduction

(What, Why and Where Algorithms . . .)

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Today Covered
In this lecture we will cover the following
• What is Algorithm?
• Designing Techniques
• Model of Computation
• Algorithms as a technology
• Algorithms and other technologies
• Importance of algorithms
• Difference in Users and Developers
• Kinds of problems solved by algorithms
• Conclusion

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 What is Algorithm?
• A computer algorithm is a detailed step-by-step method
for solving a problem by using a computer.

• An algorithm is a sequence of unambiguous instructions

for solving a problem in a finite amount of time.

• An Algorithm is well defined computational procedure that

takes some value, or set of values, as input and produces
some value, or set of values as output.

Input output
Algorithm

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 What is Algorithm?

• More generally, an Algorithm is any well defined

computational procedure that takes collection of elements
as input and produces a collection of elements as output.

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Why study algorithms and performance?
• Algorithms help us to understand scalability.

• Performance often draws the line between what is

feasible and what is impossible.

• Algorithmic mathematics provides a language for talking

about program behaviour.

• Performance is the currency of computing.

• The lessons of the program performance generalize to

other computing resources.

• Speed is fun!

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Popular Algorithms, Factors of Dependence
• Most basic and popular algorithms are
– Sorting algorithms
– Searching algorithms
Which algorithm is best?
• Mainly, it depends upon various factors, for
example in case of sorting
– The number of items to be sorted
– The extent to which the items are already sorted
– Possible restrictions on the item values
– The kind of storage device to be used etc.

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 One Problem, Many Algorithms
Problem
• The statement of the problem specifies, in general
terms, the desired input/output relationship.
Algorithm
• The algorithm describes a specific computational
procedure for achieving input/output relationship.
Example
• One might need to sort a sequence of numbers into
non-decreasing order.
Algorithms
• Various algorithms e.g. merge sort, quick sort,
heap sorts etc.
arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms
 Important Designing Techniques
• Brute Force
– Straightforward, naive approach
– Mostly expensive
• Divide-and-Conquer
– Divide into smaller sub-problems
• Decrease-and-Conquer
– Decrease instance size
• Transform-and-Conquer
– Modify problem first and then solve it
• Space and Time Tradeoffs
– Use more space now to save time later

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Some of the Important Designing Techniques
• Greedy Approach
– Locally optimal decisions, can not change once made.
– Efficient
– Easy to implement
– The solution is expected to be optimal
– Every problem may not have greedy solution
• Dynamic programming
– Decompose into sub-problems like divide and conquer
– Sub-problems are dependant
– Record results of smaller sub-problems
– Re-use it for further occurrence
– Mostly reduces complexity exponential to polynomial

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Problem Solving Phases
• Analysis
– How does system work?
– Breaking a system down to known components
– How components (processes) relate to each other
– Breaking a process down to known functions

• Synthesis
– Building tools
– Building functions with supporting tools
– Composing functions to form a process
– How components should be put together?
– Final solution

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Problem Solving Process
• Problem
• Strategy
• Algorithm
– Input
– Output
– Steps
• Analysis
– Correctness
– Time & Space
– Optimality
• Implementation
• Verification
arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms
 Model of Computation (Assumptions)
• Design assumption
– Level of abstraction which meets our requirements
– Neither more nor less
• Analysis independent of the variations in
– Machine
– Operating system
– Programming languages
– Compiler etc.
• Low-level details will not be considered
• Our model will be an abstraction of a standard
generic single-processor machine, called a random
access machine or RAM.
arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms
 Model of Computation (Assumptions)
• A RAM is assumed to be an idealized machine
– Infinitely large random-access memory
– Instructions execute sequentially
• Every instruction is in fact a basic operation on two
values in the machines memory which takes unit time.
• These might be characters or integers.
• Example of basic operations include
– Assigning a value to a variable
– Arithmetic operation (+, - , × , /) on integers
– Performing any comparison e.g. a < b
– Boolean operations
– Accessing an element of an array.

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Model of Computation (Assumptions)
• In theoretical analysis, computational complexity
– Estimated in asymptotic sense
– Estimating for large inputs
• Big O, Omega, Theta etc. notations are used to
compute the complexity
• Asymptotic notations are used because different
implementations of algorithm may differ in efficiency
• Efficiencies of two given algorithm are related
– By a constant multiplicative factor
– Called hidden constant.

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Drawbacks in Model of Computation
First poor assumption
• We assumed that each basic operation takes
constant time, i.e. model allows
– Adding
– Multiplying
– Comparing etc.
two numbers of any length in constant time
• Addition of two numbers takes a unit time!
– Not good because numbers may be arbitrarily
• Addition and multiplication both take unit time!
– Again very bad assumption

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Model of Computation not so Bad
Finally what about Our Model?
• But with all these weaknesses, our model is not so
bad because we have to give the
– Comparison not the absolute analysis of any algorithm.
• Model seems to work well describing computational
power of modern nonparallel machines
Can we do Exact Measure of Efficiency ?
• Exact (not asymptotic) measure of efficiency can be
sometimes computed but it usually requires certain
assumptions concerning implementation
– Time efficiency
– Space efficiency
arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms
 Analysis : Computational Model
• Analysis will be performed with respect to this
computational model for comparison of algorithms
• We will give asymptotic analysis not detailed
comparison i.e. for large inputs
• We will use generic uniprocessor random-access
machine (RAM) in analysis
– All memory equally expensive to access
– No concurrent operations
– All reasonable instructions take unit time,
except, of course, function calls

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms

 Conclusion
• What, Why and Where Algorithms?
• Designing Techniques
• Problem solving Phases and Procedure
• Model of computations
– Major assumptions at design and analysis level
– Merits and demerits, justification of assumptions taken
• We proved that algorithm is a technology
• Compared algorithmic technology with others
• Discussed importance of algorithms
– In almost all areas of computer science and engineering

arsalan.anjum@uskt.edu.pk Design and Analysis of Algorithms