Algorithms and Data Structures

Lecture notes: Graph Algorithms, Cormen chapters

22 to 25

Lecturer: Michel Toulouse

Hanoi University of Science & Technology

michel.toulouse@soict.hust.edu.vn

3 novembre 2021
Graphs

A graph G = (V , E ) is
I V = set of vertices
I E = set of edges = subset of V × V
Undirected graphs
In an undirected graph :
I Edge (u, v ) = edge (v , u)
I No self-loops
In the graph below, (2, 1) = (1, 2)
There is no edge (self-loop) such as (i, i)
Directed graphs
In a directed graph :
I Edge (u,v) goes from vertex u to vertex v, notated u → v
In the graph below, there is an edge (1 → 2) but no edge (2 → 1),
thus edge (u, v ) 6= edge (v , u)
Connected graphs
A connected graph has a path from every node to every other
I believe the graph in Figure 1 is connected
The graph in Figure 2 is definitely not connected, there is no path
from node 0 to node 1.

Figure – 1
Figure – 2
Weighted graphs

A weighted graph associates weights with the edges

Multigraphs
A multigraph allows multiple edges between a same pair of nodes
The graph below is a weighted multigraph. For example there are 2
edges (A → B).

Some confuse edges (B → C ) and (C → B) for defining a multigraph,

it is not !
Degrees of nodes
The degree of a node v in an undirected graph is the number of adjacent edges
In the undirected graph below, the degree of node 3 is 3 while the degree of node 4
is 1

Directed graphs have in-degree and out-degree

In the directed graph below, the in-degree of node 2 is 2 while its out-degree is 0.
The out-degree of node 1 is 2
Graphs

We will typically express the running time of algorithms on graphs in

terms of |E | and |V | (often dropping the |’s)
I If |E | ≈ |V |2 the graph is dense
I If |E | ≈ |V | the graph is sparse
Representing Graphs : adjacency matrix
When working with dense or sparse graphs, you may want to use
different data structures to represent these graphs
Assume V = {1, 2, . . . , n}

An adjacency matrix representation of a graph is a n × n matrix A :

I A[i, j] = 1 (or weight of edge) if edge (i, j) ∈ E
I A[i, j] = 0 if edge (i, j) 6∈ E

Example :
Graphs : Adjacency Matrix
Example :

A 1 2 3 4
1 a d
2 b
3
4 c
Graphs : Adjacency Matrix
I Using an adjacency matrix, an n × n array is needed to store it, so
it has a O(V 2 ) space requirement
I The maximum amount of storage needed by an adjacency matrix
representation of an undirected graph with 4 vertices is 6 bits in
unweighted graphs, 6 integers for weighted graphs. This is
because
I Undirected graph implies that the matrix is symmetric
I No self-loops, therefore don’t need diagonal

1 2 3 4
1 1 1
2 1
3 1
4
Graphs : Adjacency Matrix

The adjacency matrix is a dense representation

I Usually too much storage for large graphs
I But can be very efficient for small graphs
Most large interesting graphs are sparse
I For this reason the adjacency list is often a more appropriate
respresentation
Representing graphs : adjacency lists

Declare an array of pointers of size n, each entry pointing to a link list

For each vertex v ∈ V , stores the vertices adjacent to v in the link list
pointed to by Adj[v ] (see example below)

Example :
I Adj[1] = {2,3}
I Adj[2] = {3}
I Adj[3] = {}
I Adj[4] = {3}
Variation : can also keep a list of
edges coming into vertex
Graphs : Adjacency List

How much space is required ?

I For directed graphs, # of items in adjacency lists is

P
out-degree(v ) = |E | takes Θ(V + E ) storage
I For undirected graphs, # items in adjcency lists is
P
degree(v ) = 2|E |, also in Θ(V + E ) storage

So : Adjacency lists take O(V + E ) space.

For dense graphs, |E | is closed to |V 2 |, for sparse graphs |E | ≈ V . So

for sparse graphs the space requirement is about O(V ) which is much
better than adjacency matrices
Graph searching : Breadth-First Search

Explore a graph, turning it into a tree

I One vertex at a time
I Expand frontier of explored vertices across the breadth of the
frontier
Builds a tree over the graph
I Pick a source vertex to be the root
I Find its children, then their children, etc.
Breadth-First Search : Algorithm

BFS(G, s)
for each u ∈ G .V − {s}
u.color = WHITE
u.d = ∞ ; /* u.d is distance from source s to u */
u.π = NIL /* u.π is the predecessor of u */
s.color = GRAY
s.d = 0 ; s.π = NIL
Q=∅
Enqueue(Q,s)
while (Q not empty)
u = Dequeue(Q)
for each v ∈ u.adj
if (v.color == WHITE)
v.color = GREY
v.d = u.d + 1
v.π = u
Enqueue(Q, v) ;
u.color = BLACK ;
Breadth-First Search : Algorithm

I White vertices have not been discovered

I All vertices start out white
I Grey vertices are discovered but not fully explored
I They may be adjacent to white vertices
I Black vertices are discovered and fully explored
I They are adjacent only to black and gray vertices
Explore vertices by scanning adjacency list of grey vertices
Breadth-First Search : Example
Breadth-First Search : Example
Breadth-First Search : Example
Breadth-First Search : Example
Breadth-First Search : Example
Breadth-First Search : Example
Breadth-First Search : Example
Breadth-First Search : Example
Breadth-First Search : Example
Breadth-First Search : Example
BFS : time complexity analysis

BFS(G, s)
for each u ∈ G .V − {s} I Initialisation phase takes O(V )
u.color = WHITE I while iterates V times
u.d = ∞ ; u.π = NIL
s.color = GRAY
I In the worst case the inner for loop
s.d = 0 ; s.π = NIL can run V − 1 times, thus in O(V )
Q=∅ I So O(V 2 ) for the while loop in
Enqueue(Q,s) worst case
while (Q not empty)
u = Dequeue(Q)
for each v ∈ u.adj
if (v.color == WHITE)
v.color = GREY
v.d = u.d + 1
v.π = u
Enqueue(Q, v) ;
u.color = BLACK ;
BFS : aggregate analysis
Assuming the inner for loop in the while loop is our basic operation, how many
P
time this for can be executed ? k∈V deg (k) = 2|E |
BFS(G, s)
for each k ∈ G .V − {s} s= {r,w}
k.color = WHITE r= {v,s}
k.d = ∞ ; k.π = NIL w= {s,t,x}
s.color = GRAY v= {y}
s.d = 0 ; s.π = NIL t= {w,x,u}
Q=∅ x= {w,y,t}
Enqueue(Q,s) u= {t,y}
while (Q not empty) y= {x,u}
k = Dequeue(Q)
Iteration 1, k = s, v = {r , w }
for each v ∈ k.adj
Iteration 2 k = r , v = {v , s}
if (v.color == WHITE)
Iteration 3 k = w , v = {s, t, x}
v.color = GREY
Iteration 4 k = v , v = {y }
v.d = k.d + 1
Iteration 5 k = t, v = {w , x, u}
v.π = k
Iteration 6 k = x, v = {w , y , t}
Enqueue(Q, v) ;
Iteration 7 k = y , v = {t, y }
u.color = BLACK ;
Iteration 8 k = u, v = {x, u}
BFS : aggregate analysis
In one iteration k of the while loop, each iteration of the for loop copy in the queue
Q one white node from the adjacency list of node k
BFS(G, s)
for each k ∈ G .V − {s} s= {r,w}
k.color = WHITE r= {v,s}
k.d = ∞ ; k.π = NIL w= {s,t,x}
s.color = GRAY v= {y}
s.d = 0 ; s.π = NIL t= {w,x,u}
Q=∅ x= {w,y,t}
Enqueue(Q,s) u= {t,y}
while (Q not empty) y= {x,u}
k = Dequeue(Q)
Iteration 0 of the while loop Q = {s}
for each v ∈ k.adj
Iteration 1, k = s, Q = {r , w }
if (v.color == WHITE)
Iteration 2 k = r , Q = {w , v }
v.color = GREY
Iteration 3 k = w , Q = {v , t, x}
v.d = k.d + 1
Iteration 4 k = v , Q = {t, x, y }
v.π = k
Iteration 5 k = t, Q = {x, y , u}
Enqueue(Q, v) ;
Iteration 6 k = x, Q = {y , u}
u.color = BLACK ;
Iteration 7 k = y , Q = {u}
Iteration 7 k = u, Q = {}
BFS : aggregate analysis

BFS(G, s)
for each u ∈ G .V − {s} I Actually, each edge is considered at most
u.color = WHITE one time for all the V iterations of the
u.d = ∞ ; u.π = NIL
inner for loop, so rather we use
s.color = GRAY
aggregate analysis :
s.d = 0 ; s.π = NIL
Q=∅ I rather than considering the worst
Enqueue(Q,s) case of the for loop, we consider
while (Q not empty) I the total number of iterations of the
u = Dequeue(Q) for loop over the V iterations of the
for each v ∈ u.adj
while loop
if (v.color == WHITE) I This total number cannot exceed E
v.color = GREY
v.d = u.d + 1 as each edge is considered at most
v.π = u once
Enqueue(Q, v) ; I Total running time : O(V + E )
u.color = BLACK ;
Breadth-First Search & shortest path
In an unweighted, undirected grah, BFS calculates the shortest-path
distance to the source vertex
I Shortest-path distance δ(s, v ) = minimum number of edges from
s to v , or ∞ if v not reachable from s
I BFS builds breadth-first tree, in which paths to root represent
shortest paths in G
I Thus can use BFS to calculate shortest path from one vertex to
the others in the graph in O(V + E ) time
Depth-First Search

Depth-first search is another strategy for traversing a graph

I Explore ”deeper” in the graph whenever possible
I Edges are explored out of the most recently discovered vertex v
that still has unexplored edges
I When all of v’s edges have been explored, backtrack to the vertex
from which v was discovered
Depth-First Search

Vertices initially colored white

Then each vertex is colored gray when discovered

Then each vertex is colored black when its exploration is completed

Depth-First Search : Algorithm

u.d = time node u is first visited ; u.f = time when the exploration of
the node u is completed

DFS(G) DFS Visit(u)

for each vertex u ∈ G.V u.color = GREY ;
u.color = WHITE ; time = time+1 ;
time = 0 ; u.d = time ;
for each vertex u ∈ G.V for each v ∈ u.Adj[]
if (u.color == WHITE) if (v.color == WHITE)
DFS Visit(u) ; DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Example

DFS Visit(u)
u.color = GREY ;
time = time+1 ;
u.d = time ;
for each v ∈ u.Adj[]
if (v.color == WHITE)
DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
Aggregate analysis of DFS
I The two for loops in DFS(G) each takes Θ(V )
I The procedure DFS visit is called exactly once for each vertex
v ∈V
I The for loop in DFS visit is executed |u.adj[]| times
I Since v ∈V |u.adj[]| = Θ(E ), the total cost of DFS visit is Θ(E )
P

I The running time of the DFS algorithm is Θ(V + E )

DFS(G) DFS Visit(u)
for each vertex u ∈ G.V u.color = GREY ;
u.color = WHITE ; time = time+1 ;
time = 0 ; u.d = time ;
for each vertex u ∈ G.V for each v ∈ u.Adj[]
if (u.color == WHITE) if (v.color == WHITE)
DFS Visit(u) ; DFS Visit(v) ;
u.color = BLACK ;
time = time+1 ;
u.f = time ;
DFS : Kinds of edges

DFS introduces an important distinction among edges in the original

graph :
I Tree edge : encounter new (white) vertex
I The tree edges form a spanning forest
DFS : Kinds of edges

DFS introduces an important distinction among edges in the original

graph :
I Tree edge : encounter new (white) vertex, edges of the DFS
I Back edge : from descendant to ancestor in DFS tree
I Encounter a grey vertex (grey to grey)
DFS : Kinds of edges

DFS introduces an important distinction among edges in the original

graph :
I Tree edge : encounter new (white) vertex, edges of the DFS
I Back edge : from descendant to ancestor in DFS tree
I Forward edge : non-tree edges from ancestor to descendant in
DFS tree
DFS : Kinds of edges
DFS introduces an important distinction among edges in the original
graph :
I Tree edge : encounter new (white) vertex, edges of the DFS
I Back edge : from descendant to ancestor in DFS tree
I Forward edge : non-tree edges from ancestor to descendant in
DFS tree
I Cross edge : all other edges between a tree or subtrees
DFS : Kinds of edges

Theorem 22.10 :
If G is undirected, a DFS produces only tree and back edges

See proof of this theorem 22.10 in textbook

DFS : Kinds of edges
Lemma 22.11 :
A directed graph G is acyclic iff a DFS yields no back edges
Proof :
Any back edge combines with tree edges represents a cycle in G

If a cycle in G , then DFS enters the cycle by a white vertex v , eventually

re-visit v as a gray vertex from a vertex u in the cycle, therefore (u, v )
is a back edge
We can run DFS to find whether a graph has a cycle
Directed Acyclic Graphs

A directed acyclic graph or DAG is a directed graph with no directed

cycles :

A directed graph G is acyclic iff a DFS of G yields no back edges

Topological Sort

Topological sort of a DAG is a linear ordering of all vertices in graph G

such that vertex u comes before vertex v if edge (u, v ) ∈ G
Topological Sort
Topological sort of a DAG is a linear ordering of all vertices in graph G
such that vertex u comes before vertex v if edge (u, v ) ∈ G

Topological sort is not necessary unique :

Topological Sort Algorithm

Topological-Sort()
I Run DFS
I When a vertex u is finished (i.e. when u.f is assigned a time
value), output it
Vertices are output in reverse topological order. Time : O(V + E )
Topological Sort

Real-world example : getting dressed

Exercises

Figure – 1

Consider Figure 1 above.

1. Give the adjacency list represention of this directed graph
2. Which node(s) have the largest in-degree value
3. Which node(s) have the largest out-degree value
Exercises

4. Is the graph in Figure 1 a multi-graph ? Explain briefly your

answer.
5. How long does it take to compute the out-degree of every node ?
6. How long does it take to compute the in-degrees ?
7. Give an adjacency-list representation for a complete binary tree on
8 nodes. Give an equivalent adjacency-matrix representation.
Assume that nodes are numbered from 1 to 8 as in a binary heap
8. Run the breadth-first search algorithm on slide 17 on the directed
graph of Figure 1, using node 1 as the source. Draw the resulting
tree or, as indicated in the BFS algo give for each node i its
predecessor π (parent of i in the tree) and its level d in the tree
Exercises

9. The BFS algorithm on slide 17 assumes that the graph of Figure 1

is represented using adjacency lists.
9.1 If in one instance (a) the nodes are always stored in increasing
order in the adjacency lists while in another instances (b) the nodes
are always stored in decreasing order, will the trees be the same in
each instance ?
9.2 Will the depth of each node change if nodes are stored in
increasing versus decreasing orders in the adjacency lists ?
10. Using the DFS algorithm on slide 36, show how depth-first search
works on the graph of Figure 1. Assume that the second for loop
of the algorithm considers the nodes in increasing order order, and
assume that nodes in each adjacency list are stored in increasing
order. Write the u.d and u.f time of each node in the tree.
Exercises

11. Using the graph of Figure 1 and the tree you obtained in the
previous question, identify in Figure 1 the edges that are tree
edges, back edges and forward edges
12. Does the graph in Figure 1 is an acyclic graph ? Explain briefly
your answer.
13. Rewrite the DFS algorithm on slide 36, using a stack to eliminate
recursion
Exercises

14. Show the ordering of vertices produced by TOPOLOGICAL-SORT

when it is run on the dag of the Figure below

374 414
311 373
143 351 352
332
312 344 417
331 333
341 140 413
410
403 142
154 415