CMP 318 – ANALYSIS & COMPLEXITY OF ALGORITHMS

LECTURE NOTES

BY

CHARLES OKONJI
 OUTLINE
• Overview of Algorithms
• Asymptotic Analysis
• Complexity Classes
• Time and Space Tradeoffs in Algorithms
• Analysis of Recursive Algorithms
• Numerical Algorithms
• Sequential and Binary Search Algorithms
• Sorting Algorithms
• Binary Search Trees
• Hash Tables
• Graphs and its Representation.
 Overview of Algorithms
• Algorithm means ” A set of finite rules or instructions to be followed in calculations or other problem-solving operations ” Or ” A
procedure for solving a mathematical problem in a finite number of steps that frequently involves recursive operations”.
An algorithm is a procedure used for solving a problem or performing a computation. Algorithms act as an exact list of instructions
that conduct specified actions step by step in either hardware- or software-based routines.

• Use of the Algorithms

Some of the key areas where algorithms are used include:
1. Computer Science: Algorithms form the basis of computer programming and are used to solve problems ranging from simple sorting
and searching to complex tasks such as artificial intelligence and machine learning.
2. Mathematics: Algorithms are used to solve mathematical problems, such as finding the optimal solution to a system of linear
equations or finding the shortest path in a graph.
3. Operations Research: Algorithms are used to optimize and make decisions in fields such as transportation, logistics, and resource
allocation.
4. Artificial Intelligence: Algorithms are the foundation of artificial intelligence and machine learning, and are used to develop
intelligent systems that can perform tasks such as image recognition, natural language processing, and decision-making.
5. Data Science: Algorithms are used to analyze, process, and extract insights from large amounts of data in fields such as marketing,
finance, and healthcare.
 Overview of Algorithms
• Why Algorithms?
1. Algorithms are necessary for solving complex problems efficiently and effectively.
2. They help to automate processes and make them more reliable, faster, and easier to perform.
3. Algorithms also enable computers to perform tasks that would be difficult or impossible for humans to do manually.
4. They are used in various fields such as mathematics, computer science, engineering, finance, and many others to optimize processes,
analyze data, make predictions, and provide solutions to problems.

• Characteristics of an Algorithm
Not all written instructions for programming are an algorithm. For some instructions to be an algorithm, it must have the following
characteristics:
o Clear and Unambiguous: The algorithm should be unambiguous. Each of its steps should be clear in all aspects and must lead to only
one meaning.
o Well-Defined Inputs: If an algorithm is to take inputs, then these inputs should be well-defined.
o Well-Defined Outputs: The algorithm must clearly define what output will be yielded and it should be well-defined as well. Every
algorithm should produce at least 1 output.
o Finite-ness: The algorithm must be finite, i.e. it should terminate after a finite time.
o Feasible: The algorithm must be simple, generic, and practical, such that it can be executed with the available resources. It must not
contain some future technology or anything abstract.
o Language Independent: The Algorithm designed must be language-independent, i.e. it must be just plain instructions that can be
implemented in any language, and yet the output will be the same, as expected.
o Effectiveness: An algorithm must be developed by using very basic, simple, and feasible operations so that one can trace it out by
using just paper and pencil.
 Overview of Algorithms

• Properties of Algorithm
o It should terminate after a finite time.
o It should produce at least one output.
o It should take zero or more input.
o It should be deterministic means giving the same output for the same input case.
o Every step in the algorithm must be effective i.e. every step should do some work.

• Types of Algorithms
o Brute Force Algorithm: A straightforward approach that exhaustively tries all possible solutions, suitable for small problem instances
but may become impractical for larger ones due to its high time complexity.
o Recursive Algorithm: A method that breaks a problem into smaller, similar sub-problems and repeatedly applies itself to solve them
until reaching a base case, making it effective for tasks with recursive structures.
o Encryption Algorithm: Utilized to transform data into a secure, unreadable form using cryptographic techniques, ensuring
confidentiality and privacy in digital communications and transactions.
o Backtracking Algorithm: A trial-and-error technique used to explore potential solutions by undoing choices when they lead to an
incorrect outcome, commonly employed in puzzles and optimization problems.
o Searching Algorithm: Designed to find a specific target within a dataset, enabling efficient retrieval of information from sorted or
unsorted collections.
o Sorting Algorithm: Aimed at arranging elements in a specific order, like numerical or alphabetical, to enhance data organization and
retrieval.
o Hashing Algorithm: Converts data into a fixed-size hash value, enabling rapid data access and retrieval in hash tables, commonly used
in databases and password storage.
o Divide and Conquer Algorithm: Breaks a complex problem into smaller subproblems, solves them independently, and then combines
their solutions to address the original problem effectively.
 Overview of Algorithms
o Greedy Algorithm: Makes locally optimal choices at each step in the hope of finding a global optimum, useful for optimization
problems but may not always lead to the best solution.
o Dynamic Programming Algorithm: Stores and reuses intermediate results to avoid redundant computations, enhancing the efficiency
of solving complex problems.
o Randomized Algorithm: Utilizes randomness in its steps to achieve a solution, often used in situations where an approximate or
probabilistic answer suffices.

• How do Algorithms Work?

Algorithms are step-by-step procedures designed to solve specific problems and perform tasks efficiently in the realm of computer science
and mathematics. These powerful sets of instructions form the backbone of modern technology and govern everything from web searches
to artificial intelligence.
Here's how algorithms work:
o Input: Algorithms take input data, which can be in various formats, such as numbers, text, or images.
o Processing: The algorithm processes the input data through a series of logical and mathematical operations, manipulating and
transforming it as needed.
o Output: After the processing is complete, the algorithm produces an output, which could be a result, a decision, or some other
meaningful information.
o Efficiency: A key aspect of algorithms is their efficiency, aiming to accomplish tasks quickly and with minimal resources.
o Optimization: Algorithm designers constantly seek ways to optimize their algorithms, making them faster and more reliable.
o Implementation: Algorithms are implemented in various programming languages, enabling computers to execute them and produce
desired outcomes.
 Overview of Algorithms
• How to Write an Algorithm?
o There are no well-defined standards for writing algorithms. It is, however, a problem that is resource-dependent. Algorithms are never
written with a specific programming language in mind.
o As you all know, basic code constructs such as loops like do, for, while, all programming languages share flow control such as if-else, and
so on. An algorithm can be written using these common constructs.
o Algorithms are typically written in a step-by-step fashion, but this is not always the case. Algorithm writing is a process that occurs after
the problem domain has been well-defined. That is, you must be aware of the problem domain for which you are developing a solution.

Example: Create an algorithm that multiplies two numbers and displays the output.
Step 1 – Start
Step 2 − declare three integers x, y & z
Step 3 − define values of x & y
Step 4 − multiply values of x & y
Step 5 − store result of step 4 to z
Step 6 − print z
Step 7 − Stop

• OR
Step 1 − Start mul
Step 2 − get values of x & y
Step 3 − z ← x * y
Step 4 − display z
Step 5 – Stop

• In Algorithm Design and Analysis, the second method is typically used to describe an algorithm, as it allows the Analyst to analyze the
algorithm while ignoring all unwanted definitions easily. Meaning, they can see which operations are being used and how the process
is progressing. Writing step numbers is optional.
 Overview of Algorithms
• To solve a given problem, you create an algorithm. A problem can be solved in a variety of ways. As a result, many solution algorithms
for a given problem can be derived.

• Thus, to evaluate the proposed solution algorithms and implement the most appropriate solution, the following factors are
considered:
o Modularity: This entails breaking down an algorithm into small-small modules or small-small steps, which is a basic definition of
an algorithm.
o Correctness: An algorithm's correctness is defined as when the given inputs produce the desired output, indicating that the
algorithm was designed correctly. An algorithm's analysis has been completed correctly.
o Maintainability: It means that the algorithm should be designed in a straightforward, structured way so that when you redefine
the algorithm, no significant changes are made to the algorithm.
o Functionality: It takes into account various logical steps to solve a real-world problem.
o Robustness: Robustness refers to an algorithm's ability to define your problem clearly.
o User-friendly: If the algorithm is difficult to understand, the designer will not explain it to the programmer.
o Simplicity: If an algorithm is simple, it is simple to understand.
o Extensibility: Your algorithm should be extensible if another algorithm designer or programmer wants to use it.
 Overview of Algorithms
• Qualities of a Good Algorithm
o Efficiency: A good algorithm should perform its task quickly and use minimal resources.
o Correctness: It must produce the correct and accurate output for all valid inputs.
o Clarity: The algorithm should be easy to understand and comprehend, making it maintainable and modifiable.
o Scalability: It should handle larger data sets and problem sizes without a significant decrease in performance.
o Reliability: The algorithm should consistently deliver correct results under different conditions and environments.
o Optimality: Striving for the most efficient solution within the given problem constraints.
o Robustness: Capable of handling unexpected inputs or errors gracefully without crashing.
o Adaptability: Ideally, it can be applied to a range of related problems with minimal adjustments.
o Simplicity: Keeping the algorithm as simple as possible while meeting its requirements, avoiding unnecessary complexity.