Branch and Bound:

• An algorithm design technique, primarily for solving hard optimization

problems
• Guarantees that the optimal solution will be found
• Does not necessarily guarantee worst case polynomial time complexity
-- But tries to ensure faster time on most instances
• Basic Idea
-- Model the entire solution space as a tree
-- Search for a solution in the tree systematically, eliminating parts of the tree
from the search intelligently
• Important technique for solving many problems for which efficient algorithms
(worst case polynomial time) are not known
Optimization Problems:
• Set of input variables I
• Set of output variables O
-- Values of the output variables define the solution space
• Set of constraints C over the variables
• Set of feasible solutions S
-- Set of solutions that satisfy all the constraints
• Objective function F: S → R
-- Gives a value F(s) for each solution s S
• Optimal solution
-- A solution s S for which F(s) is maximum among all s S (for a
maximization problem) or minimum (for a minimization problem)
Example :
• Many problems:
-- Minimum weight spanning tree, matrix chain multiplication, fractional
knapsack problem, ….
-- All these (above) have known efficient algorithms
Graph Coloring Problem:
-- Input: an undirected graph
-- Output: a color assigned to each vertex
-- Constraint: no two adjacent vertices have the same color
-- Objective function: no of colors used by a feasible solution
-- Goal: Find a coloring that uses the minimum number of colors
-- No efficient algorithm known for general graphs
0-1 Knapsack Problem:
• Given a set of items with weights and
values
paci
ty find the largest subset of items that can be packed into the
knapsack such that the total value gained is maximized.
• Solution format
-- if item i is chosen in the subset, 0 otherwise
• Feasible solution:
• Objective function F:
• Optimal solution: feasible solution with maximum value of
• Solution space size
State Space Tree:
• Represent the solution space as a tree
-- Each edge represents a choice of one
- Level 0 to Level 1 edges show choice of
- Level 1 to Level 2 edges show choice of
- Level to Level edges show choice of
-- Each internal node represents a partial solution
- Partitions the solution space into disjoint subspaces
-- Leaf nodes represent the complete solution (may or may not be
feasible)
-- Models the complete solution being built by choosing one
component at a time
State Space Tree: (Knapsack using Branch and Bound)
State Space Tree: (Example: 0-1 Knapsack)
• Level to Level edges show choice of
• Level to Level edges show choice of
• Level to Level edges show choice of
• Level (root) node partitions the solution space into those that
contain and those that do not contain
• For the subtree which has chosen, Level nodes partitions
the subspace ( present) further into ( and present) and
( present but not present)
• Leaf nodes represent the solutions (the path from root to leaf
shows what items are chosen (edges marked 1 along the path)
State Space Tree: (Example: 0-1 Knapsack)
• Level to Level edges show choice of
• Level to Level edges show choice of
• Level to Level edges show choice of
• Level (root) node partitions the solution space into those that
contain and those that do not contain
• For the subtree which has chosen, Level nodes partitions
the subspace ( present) further into ( and present) and
( present but not present)
• Leaf nodes represent the solutions (the path from root to leaf
shows what items are chosen (edges marked 1 along the path)
Finding the Optimal Solution :
• One possible approach
-- Generate the state space tree, look at all solutions (leaf nodes), find the
feasible solutions, apply objective function on each feasible solution, and
choose the optimal
• But generating the tree is as good as doing brute force search
-- Will take huge space (to store the tree) and huge time (to generate all
nodes)
-- Need to do something better
- To reduce space, do not generate all nodes at once
- To reduce time, do not generate all nodes (how to decide?)
• We will first look at the problem of finding just one feasible solution to
understand a basic technique
Finding One Feasible Solution:
Basic Approach :
--Expand the tree systematically in parts, stop when you find a feasible solution
- Reduces space as whole tree is not generated at one go
- The parts that have been looked at already are not stored
- May reduce time for many instances as feasible solution may be found
without generating the whole tree
- But in worst case, you may still have to generate all the nodes
Finding One Feasible Solution:
How to expand the tree:
• Start with root node
• Generate other nodes in some order
-- DFS, BFS, …..
• Live node: a node which is generated but all children of which has not
been generated
• Dead node: A generated node which is not to be expanded (will see why
later) or whose all children have been generated
• The different node generation orders will pick one live node to expand
(generate its children) at one time
• E-node: The live node being expanded currently
Finding One Feasible Solution:
How to expand the tree:
Example: n-Queens Problem :
• Consider a board
• You have to place queens on the squares so that no two queens
can attack each other, i.e., no two queens should be
-- In the same row
-- In the same column
-- In the same diagonal
One Solution for 8-queens :
• Consider a board
• You have to place queens on the squares so that no two queens
can attack each other, i.e., no two queens should be
-- In the same row
-- In the same column
-- In the same diagonal

Courtesy: Computer Algorithms: E. Horowitz, S. Sahni, and S. Rajasekharan

How to find a solution:
• 4 queen solution space, nodes are numbered as in depth first search

Courtesy: Computer Algorithms: E. Horowitz, S. Sahni, and S. Rajasekharan

Backtracking :
• Systematic search of the state space tree with pruning
-- Pruning done using bounding functions
• Tree generated using DFS order
-- When a child C of the current E-node R is generated, C becomes the new
E-node
-- R will become the E-node again after subtree rooted at C is fully explored
• At every step, apply the bounding function (a predicate) on a node to check
if the subtree rooted at the node needs to be explored
-- Do not generate the subtree rooted at the node if the bounding function
returns false
• Find all feasible solutions (or stop at the first one if only one is needed)
Notes on Backtracking :
• While most definitions of backtracking specify DFS order generation of
state-space tree, other definitions do not restrict the order to DFS
• Backtracking can be applied to problems other than optimization problems
also
-- Just answer yes/no (decision problem)
-- Find one/all feasible solution
-- Some definitions actually restrict backtracking to non-optimization
problems only, with branch and bound for optimization problems
• Forms the basis for state space search for optimization problems also
• Branch and bound uses backtracking with different tree generation order
and augmented bounding function to prune the tree further for finding an
optimal solution
Branch and Bound:
• Use similar methods as backtracking to generate the state space tree
-- But need not be DFS order
• Use the bounding function to prune off parts of the tree
-- Parts that do not contain any feasible solutions as before
-- Current partial solution cannot lead to a feasible solution
-- Parts that may contain feasible solutions but cannot contain an
optimal solution
-- Current partial solution cannot be extended to an optimal
solution
-- Use the value of the solution (objective function applied to the
solution) for deciding this
Tree Generation Orders :
• DFS order (mostly used in backtracking)
• FIFO Branch and Bound/BFS
-- Same as BFS. A E-node node remains an E-node until all its children are
either generated or killed (due to pruning)
-- Implemented the same way as BFS with a queue. Node at head of queue
is the next E-node
• LIFO Branch and Bound/D-Search
-- Similar to BFS in that all nodes of an E-node are generated first
-- Different from BFS in that the generated nodes are placed in a stack
instead of in a queue
Tree of 0-1 Knapsack with LIFO B&B :

Courtesy: Computer Algorithms: E. Horowitz, S. Sahni, and S. Rajasekharan