0% found this document useful (0 votes)

16 views776 pages

Lecture 10

The document discusses the concept of graphs in the context of data science, defining a graph as a set of vertices and edges that represent relationships between entities. It explains directed and undirected graphs, paths, and reachability, emphasizing how to determine if one vertex is reachable from another. Examples include teacher-course allocations and airline routes to illustrate these concepts.

Uploaded by

bazaarsanskriti

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

16 views776 pages

Lecture 10

Uploaded by

bazaarsanskriti

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

Graphs

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 10
Visualizing relations

Cartesian product A × B
{(a, b) | a ∈ A, b ∈ B}

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 2/6

Visualizing relations

Cartesian product A × B
{(a, b) | a ∈ A, b ∈ B}
A relation is a subset of A × B

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 2/6

Visualizing relations

Cartesian product A × B
{(a, b) | a ∈ A, b ∈ B}
A relation is a subset of A × B
Teachers and courses
T , set of teachers in a college
C , set of courses being offered

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 2/6

Visualizing relations

Cartesian product A × B
{(a, b) | a ∈ A, b ∈ B}
A relation is a subset of A × B
Teachers and courses
T , set of teachers in a college
C , set of courses being offered
A ⊆ T × C describes the allocation of
teachers to courses
A = {(t, c) | (t, c) ∈ T × C , t teaches c}

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 2/6

Visualizing relations

Cartesian product A × B A relation as a graph

{(a, b) | a ∈ A, b ∈ B} •
Sheila
A relation is a subset of A × B • Biology
Aziz •
Teachers and courses • English
T , set of teachers in a college Priya •
C , set of courses being offered • History
A ⊆ T × C describes the allocation of Kumar •
teachers to courses • Maths
A = {(t, c) | (t, c) ∈ T × C , t teaches c} Deb •

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 2/6

Visualizing relations

Cartesian product A × B A relation as a graph

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 2/6

Graphs
Graph: G = (V , E )
A relation as a graph
V is a set of vertices or nodes
One vertex, many vertices Sheila •
E is a set of edges • Biology
E ⊆ V × V — binary relation Aziz •
• English
Priya •
• History
Kumar •
• Maths
Deb •

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 3/6

Graphs
Graph: G = (V , E )
A relation as a graph
V is a set of vertices or nodes
One vertex, many vertices Sheila •
E is a set of edges • Biology
E ⊆ V × V — binary relation Aziz •
• English
Directed graph Priya •
0 0 • History
(v , v ) ∈ E does not imply (v , v ) ∈ E
The teacher-course graph is directed Kumar •
• Maths
Deb •

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 3/6

Graphs
Graph: G = (V , E )
Friendship as a graph
V is a set of vertices or nodes
One vertex, many vertices Sheila •
E is a set of edges • Colin
E ⊆ V × V — binary relation Aziz •
• Feroze
Directed graph Priya •
0 0 •
(v , v ) ∈ E does not imply (v , v ) ∈ E Badri
The teacher-course graph is directed Kumar •
• Radhika
Undirected graph •
Deb
(v , v 0 ) ∈ E iff (v 0 , v ) ∈ E
Effectively (v , v 0 ), (v 0 , v ) are the same edge
Friendship relation
Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 3/6
Paths
Priya needs some help that Radhika can
provide. How will Priya come to know about Friendship as a graph
this?
Sheila •
• Colin
Aziz •
• Feroze
Priya •
• Badri
Kumar •
• Radhika
Deb •

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 4/6

Paths
Priya needs some help that Radhika can
provide. How will Priya come to know about Friendship as a graph
this?
Sheila •
Priya — Aziz — Badri — Radhika • Colin
Aziz •
• Feroze
Priya •
• Badri
Kumar •
• Radhika
Deb •

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 4/6

Paths
Priya needs some help that Radhika can
provide. How will Priya come to know about Friendship as a graph
this?
Sheila •
Priya — Aziz — Badri — Radhika • Colin
Aziz •
Priya — Feroze — Kumar — Radhika
• Feroze
A path is a sequence of vertices v1 , v2 , . . . , vk Priya •
connected by edges • Badri
For 1 ≤ i < k, (vi , vi+1 ) ∈ E Kumar •
• Radhika
Deb •

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 4/6

Reachability

Paths in directed graphs

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 5/6

Reachability

Paths in directed graphs Airline routes

How can I fly from Madurai to Delhi? v0

v1
v2
v3
v4 v6
v5

v8
v7

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 5/6

Reachability

Paths in directed graphs Airline routes

How can I fly from Madurai to Delhi? v0

v1
Find a path from v9 to v0
v2
v3
v4 v6
v5

v8
v7

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 5/6

Reachability

Paths in directed graphs Airline routes

How can I fly from Madurai to Delhi? v0

v1
Find a path from v9 to v0
v2
Vertex v is reachable from vertex u if there is a v3
path from u to v v4 v6
v5

v8
v7

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 5/6

Reachability

Paths in directed graphs Airline routes

How can I fly from Madurai to Delhi? v0

v1
Find a path from v9 to v0
v2
Vertex v is reachable from vertex u if there is a v3
path from u to v v4 v6
Typical questions v5

Is v reachable from u?
v8
v7

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 5/6

Reachability

Paths in directed graphs Airline routes

How can I fly from Madurai to Delhi? v0

v1
Find a path from v9 to v0
v2
Vertex v is reachable from vertex u if there is a v3
path from u to v v4 v6
Typical questions v5

Is v reachable from u?
What is the shortest path from u to v ? v8
v7

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 5/6

Reachability

Paths in directed graphs Airline routes

How can I fly from Madurai to Delhi? v0

v1
Find a path from v9 to v0
v2
Vertex v is reachable from vertex u if there is a v3
path from u to v v4 v6
Typical questions v5

Is v reachable from u?
What is the shortest path from u to v ? v8
v7
What are the vertices reachable from u?
v9

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 5/6

Reachability

Paths in directed graphs Airline routes

How can I fly from Madurai to Delhi? v0

v1
Find a path from v9 to v0
v2
Vertex v is reachable from vertex u if there is a v3
path from u to v v4 v6
Typical questions v5

Is v reachable from u?
What is the shortest path from u to v ? v8
v7
What are the vertices reachable from u?
Is the graph connected? Are all vertices v9
reachable from each other?

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 5/6

Reachability

Paths in directed graphs Airline routes

How can I fly from Madurai to Delhi? v0

v1
Find a path from v9 to v0
v2
Vertex v is reachable from vertex u if there is a v3
path from u to v v4 v6
Typical questions v5

Is v reachable from u?
What is the shortest path from u to v ? v8
v7
What are the vertices reachable from u?
Is the graph connected? Are all vertices v9
reachable from each other?

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 5/6

Reachability

Paths in directed graphs Airline routes

How can I fly from Madurai to Delhi? v0

v1
Find a path from v9 to v0
v2
Vertex v is reachable from vertex u if there is a v3
path from u to v v4 v6
Typical questions v5

Is v reachable from u?
What is the shortest path from u to v ? v8
v7
What are the vertices reachable from u?
Is the graph connected? Are all vertices v9
reachable from each other?

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 5/6

Summary

A graph represents relationships between entities

Entities are vertices/nodes
Relationships are edges

A graph may be directed or undirected

A is a parent of B — directed
A is a friend of B — undirected

Paths are sequences of connected edges

Reachability: is there a path from u to v ?

Madhavan Mukund Graphs Mathematics for Data Science 1, Week 10 6/6

Mathematics for Data Science 1

Week 10
Graphs

Graph G = (V , E ) Airline routes

V — set of vertices v0
E ⊆ V × V — set of edges v1
v2
v3
v4 v6
v5

v8
v7

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 2/8
Graphs

Graph G = (V , E ) Airline routes

V — set of vertices v0
E ⊆ V × V — set of edges v1
v2
A path is a sequence of vertices v1 , v2 , . . . , vk
v3
connected by edges
v4 v6
For 1 ≤ i < k, (vi , vi+1 ) ∈ E v5

v8
v7

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 2/8
Graphs

Graph G = (V , E ) Airline routes

V — set of vertices v0
E ⊆ V × V — set of edges v1
v2
A path is a sequence of vertices v1 , v2 , . . . , vk
v3
connected by edges
v4 v6
For 1 ≤ i < k, (vi , vi+1 ) ∈ E v5
Vertex v is reachable from vertex u if there is a
path from u to v v8
v7

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 2/8
Graphs

Graph G = (V , E ) Airline routes

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 2/8
Map colouring
Assign each state a colour
States that share a border should be
coloured differently

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 3/8
Map colouring
Assign each state a colour
States that share a border should be
coloured differently
How many colours do we need?

Assign colours to nodes so that endpoints

of an edge have different colours

Assign colours to nodes so that endpoints

of an edge have different colours

Assign colours to nodes so that endpoints

of an edge have different colours

Assign colours to nodes so that endpoints

of an edge have different colours

Assign colours to nodes so that endpoints

of an edge have different colours

Assign colours to nodes so that endpoints

of an edge have different colours

Assign colours to nodes so that endpoints

of an edge have different colours
Only need the underlying graph

Assign colours to nodes so that endpoints

of an edge have different colours
Only need the underlying graph
Abstraction: if we distort the graph,
problem is unchanged
Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 3/8
Graph colouring
Graph G = (V , E ), set of colours C

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 4/8
Graph colouring
Graph G = (V , E ), set of colours C
Colouring is a function c : V → C such
that (u, v ) ∈ E ⇒ c(u) 6= c(v )
Given G = (V , E ), what is the smallest
set of colours need to colour G
Four Colour Theorem For graphs derived
from geographical maps, 4 colours suffice

How many classrooms do we need?

Courses and timetable slots

How many classrooms do we need?

Courses and timetable slots
Graph: Edges are overlaps in slots

How many classrooms do we need?

Courses and timetable slots
Graph: Edges are overlaps in slots
Colours are classrooms
Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 4/8
Vertex cover
A hotel wants to install security cameras
All corridors are straight lines
Camera at the intersection of corridors
can monitor all those corridor.

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 5/8
Vertex cover
A hotel wants to install security cameras
All corridors are straight lines
Camera at the intersection of corridors
can monitor all those corridor.

Minimum number of cameras needed?

Represent the floor plan as a graph
V — intersections of corridors
E — corridor segments connecting
intersections

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 5/8
Vertex cover
A hotel wants to install security cameras
All corridors are straight lines
v5
Camera at the intersection of corridors
can monitor all those corridor.
v2
Minimum number of cameras needed?
Represent the floor plan as a graph v0 v4
V — intersections of corridors
v1
E — corridor segments connecting
intersections
v3
Vertex cover
Marking v covers all edges from v
Mark smallest subset of V to cover all
edges
Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 5/8
Vertex cover
A hotel wants to install security cameras
All corridors are straight lines
v5
Camera at the intersection of corridors
can monitor all those corridor.
v2
Minimum number of cameras needed?
Represent the floor plan as a graph v0 v4
V — intersections of corridors
v1
E — corridor segments connecting
intersections
v3
Vertex cover
Marking v covers all edges from v
Mark smallest subset of V to cover all
edges
Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 5/8
Vertex cover
A hotel wants to install security cameras
All corridors are straight lines
v5
Camera at the intersection of corridors
can monitor all those corridor.
v2
Minimum number of cameras needed?
Represent the floor plan as a graph v0 v4
V — intersections of corridors
v1
E — corridor segments connecting
intersections
v3
Vertex cover
Marking v covers all edges from v
Mark smallest subset of V to cover all
edges
Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 5/8
Independent set
A dance school puts up group dances
Each dance has a set of dancers
Sets of dancers may overlap across
dances

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 6/8
Independent set
A dance school puts up group dances
Each dance has a set of dancers
Sets of dancers may overlap across
dances

Organizing a cultural programme

Each dancer performs at most once
Maximum number of dances possible?

Organizing a cultural programme

Each dancer performs at most once
Maximum number of dances possible?

Represent the dances as a graph

V — dances
E — sets of dancers overlap

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 6/8
Independent set
A dance school puts up group dances
Each dance has a set of dancers
v4 v3
Sets of dancers may overlap across
dances

Organizing a cultural programme v8 v7

Each dancer performs at most once
Maximum number of dances possible?
v5 v6
Represent the dances as a graph
V — dances
E — sets of dancers overlap v1 v2
Independent set
Subset of vertices such that no two are
connected by an edge
Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 6/8
Independent set
A dance school puts up group dances
Each dance has a set of dancers
v4 v3
Sets of dancers may overlap across
dances

Organizing a cultural programme v8 v7

Each dancer performs at most once
Maximum number of dances possible?
v5 v6
Represent the dances as a graph
V — dances
E — sets of dancers overlap v1 v2
Independent set
Subset of vertices such that no two are
connected by an edge
Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 6/8
Matching
Class project can be done by one or two
people
If two people, they must be friends

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 7/8
Matching
Class project can be done by one or two
people
If two people, they must be friends

Assume we have a graph describing

friendships

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 7/8
Matching
Class project can be done by one or two
people
If two people, they must be friends

Assume we have a graph describing

friendships
Find a good allocation of groups

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 7/8
Matching
Class project can be done by one or two
people
v2 v4 v5
If two people, they must be friends

Assume we have a graph describing v0

friendships
Find a good allocation of groups v1 v3

Matching
G = (V , E ), an undirected graph
A matching is a subset M ⊆ E of
mutually disjoint edges

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 7/8
Matching
Class project can be done by one or two
people
v2 v4 v5
If two people, they must be friends

Assume we have a graph describing v0

friendships
Find a good allocation of groups v1 v3

Matching
G = (V , E ), an undirected graph
A matching is a subset M ⊆ E of
mutually disjoint edges

Find a maximal matching in G

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 7/8
Matching
Class project can be done by one or two
people
v2 v4 v5
If two people, they must be friends

Assume we have a graph describing v0

friendships
Find a good allocation of groups v1 v3

Matching
G = (V , E ), an undirected graph
A matching is a subset M ⊆ E of
mutually disjoint edges

Find a maximal matching in G

Is there a perfect matching, covering all
vertices?
Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 7/8
Summary

Graphs are useful abstract representations for a wide range of problems

Reachability and connectedness are not the only interesting problems we can solve
on graphs
Graph colouring
Vertex cover
Independent set
Matching
...

Madhavan Mukund More on Graphs Mathematics for Data Science 1, Week 10 8/8
Representing Graphs

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 10
Working with graphs
Graph G = (V , E )
Airline routes
V — set of vertices
E ⊆ V × V — set of edges v0
v1
A path is a sequence of vertices v1 , v2 , . . . , vk v2
connected by edges v3
For 1 ≤ i < k, (vi , vi+1 ) ∈ E v4 v6
v5
Vertex v is reachable from vertex u if there is a
path from u to v
v8
v7

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 2 / 11

Working with graphs
Graph G = (V , E )
Airline routes
V — set of vertices
E ⊆ V × V — set of edges v0
v1
A path is a sequence of vertices v1 , v2 , . . . , vk v2
connected by edges v3
For 1 ≤ i < k, (vi , vi+1 ) ∈ E v4 v6
v5
Vertex v is reachable from vertex u if there is a
path from u to v
v8
Looking at the picture of G , we can “see” that v7
v0 is reachable from v9
v9
How do we represent this picture so that we
can compute reachability?
Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 2 / 11
Adjacency matrix
Let |V | = n
Airline routes
Assume V = {0, 1, . . . , n − 1}
Use a table to map actual vertex “names” to v0
this set v1
v2
v3
v4 v6
v5

v8
v7

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 3 / 11

Adjacency matrix
Let |V | = n
Airline routes
Assume V = {0, 1, . . . , n − 1}
Use a table to map actual vertex “names” to v0
this set v1
v2
Edges are now pairs (i, j), where 0 ≤ i, j < n v3
Usually assume i 6= j, no self loops
v4 v6
v5

v8
v7

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 3 / 11

Adjacency matrix
Adjacency matrix
Airline routes
Rows and columns numbered {0, 1, . . . , n − 1}
A[i, j] = 1 if (i, j) ∈ E v0
v1
v2
v3
v4 v6
v5

v8
v7

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 4 / 11

Adjacency matrix
Adjacency matrix
Airline routes
Rows and columns numbered {0, 1, . . . , n − 1}
A[i, j] = 1 if (i, j) ∈ E v0
v1
0 1 2 3 4 5 6 7 8 9
v2
0 0 1 0 0 1 0 0 0 0 0
v3
1 0 0 1 0 0 0 0 0 0 0
v4 v6
2 1 0 0 0 0 0 0 0 0 0
v5
3 0 0 0 0 1 0 1 0 0 0
4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 0 1 0 0 v8
v7
6 0 0 0 0 0 1 0 0 0 0
7 0 0 0 0 1 0 0 0 1 0
v9
8 0 0 0 0 0 1 0 0 0 1
9 0 0 0 0 0 0 0 0 1 0
Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 4 / 11
Adjacency matrix
Undirected graph
Airline routes, all routes bidirectional
A[i, j] = 1 iff A[j, i] = 1
Symmetric across main diagonal v0
v1
0 1 2 3 4 5 6 7 8 9
0 0 1 1 0 1 0 0 0 0 0 v2
1 1 0 1 0 0 0 0 0 0 0 v3
2 1 1 0 0 0 0 0 0 0 0 v4 v6
3 0 0 0 0 1 1 1 0 0 0 v5
4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 1 1 1 0 v8
6 0 0 0 1 0 1 0 0 0 0 v7
7 0 0 0 0 1 1 0 0 1 0
8 0 0 0 0 0 1 0 1 0 1 v9
9 0 0 0 0 0 0 0 0 1 0
Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 5 / 11
Computing with the adjacency matrix
Neighbours of i — column j with entry 1
Undirected airline routes
Scan row i to identify neighbours of i
0 1 2 3 4 5 6 7 8 9
Neighbours of 6 are 3 and 5
0 0 1 1 0 1 0 0 0 0 0
1 1 0 1 0 0 0 0 0 0 0
2 1 1 0 0 0 0 0 0 0 0
3 0 0 0 0 1 1 1 0 0 0
4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 1 1 1 0
6 0 0 0 1 0 1 0 0 0 0
7 0 0 0 0 1 1 0 0 1 0
8 0 0 0 0 0 1 0 1 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 6 / 11

Computing with the adjacency matrix
Neighbours of i — column j with entry 1
Directed airline routes
Scan row i to identify neighbours of i
0 1 2 3 4 5 6 7 8 9
Neighbours of 6 are 3 and 5
0 0 1 0 0 1 0 0 0 0 0
Directed graph 1 0 0 1 0 0 0 0 0 0 0
Rows represent outgoing edges 2 1 0 0 0 0 0 0 0 0 0
3 0 0 0 0 1 0 1 0 0 0
4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 0 1 0 0
6 0 0 0 0 0 1 0 0 0 0
7 0 0 0 0 1 0 0 0 1 0
8 0 0 0 0 0 1 0 0 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 6 / 11

Computing with the adjacency matrix
Neighbours of i — column j with entry 1
Directed airline routes
Scan row i to identify neighbours of i
0 1 2 3 4 5 6 7 8 9
Neighbours of 6 are 3 and 5
0 0 1 0 0 1 0 0 0 0 0
Directed graph 1 0 0 1 0 0 0 0 0 0 0
Rows represent outgoing edges 2 1 0 0 0 0 0 0 0 0 0
3 0 0 0 0 1 0 1 0 0 0
Columns represent incoming edges
4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 0 1 0 0
6 0 0 0 0 0 1 0 0 0 0
7 0 0 0 0 1 0 0 0 1 0
8 0 0 0 0 0 1 0 0 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 6 / 11

Computing with the adjacency matrix
Neighbours of i — column j with entry 1
Undirected airline routes
Scan row i to identify neighbours of i
0 1 2 3 4 5 6 7 8 9
Neighbours of 6 are 3 and 5
0 0 1 1 0 1 0 0 0 0 0
Directed graph 1 1 0 1 0 0 0 0 0 0 0
Rows represent outgoing edges 2 1 1 0 0 0 0 0 0 0 0
3 0 0 0 0 1 1 1 0 0 0
Columns represent incoming edges
4 1 0 0 1 0 0 0 1 0 0
Degree of a vertex i 5 0 0 0 1 0 0 1 1 1 0
Number of edges incident on i 6 0 0 0 1 0 1 0 0 0 0
degree(6) = 2 7 0 0 0 0 1 1 0 0 1 0
8 0 0 0 0 0 1 0 1 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 6 / 11

Computing with the adjacency matrix
Neighbours of i — column j with entry 1
Directed airline routes
Scan row i to identify neighbours of i
0 1 2 3 4 5 6 7 8 9
Neighbours of 6 are 3 and 5
0 0 1 0 0 1 0 0 0 0 0
Directed graph 1 0 0 1 0 0 0 0 0 0 0
Rows represent outgoing edges 2 1 0 0 0 0 0 0 0 0 0
3 0 0 0 0 1 0 1 0 0 0
Columns represent incoming edges
4 1 0 0 1 0 0 0 1 0 0
Degree of a vertex i 5 0 0 0 1 0 0 0 1 0 0
Number of edges incident on i 6 0 0 0 0 0 1 0 0 0 0
degree(6) = 2 7 0 0 0 0 1 0 0 0 1 0
8 0 0 0 0 0 1 0 0 0 1
For directed graphs, outdegree and
indegree 9 0 0 0 0 0 0 0 0 1 0
indegree(6) = 1, outdegree(6) = 1

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 6 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

0 1 2 3 4 5 6 7 8 9
0 0 1 1 0 1 0 0 0 0 0
1 1 0 1 0 0 0 0 0 0 0
2 1 1 0 0 0 0 0 0 0 0
3 0 0 0 0 1 1 1 0 0 0
4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 1 1 1 0
6 0 0 0 1 0 1 0 0 0 0
7 0 0 0 0 1 1 0 0 1 0
8 0 0 0 0 0 1 0 1 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

0 1 2 3 4 5 6 7 8 9
Mark 9 as reachable
0 0 1 1 0 1 0 0 0 0 0
1 1 0 1 0 0 0 0 0 0 0
2 1 1 0 0 0 0 0 0 0 0
3 0 0 0 0 1 1 1 0 0 0
4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 1 1 1 0
6 0 0 0 1 0 1 0 0 0 0
7 0 0 0 0 1 1 0 0 1 0
8 0 0 0 0 0 1 0 1 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

0 1 2 3 4 5 6 7 8 9
Mark 9 as reachable
0 0 1 1 0 1 0 0 0 0 0
Mark each neighbour of 9 as reachable 1 1 0 1 0 0 0 0 0 0 0
2 1 1 0 0 0 0 0 0 0 0
3 0 0 0 0 1 1 1 0 0 0
4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 1 1 1 0
6 0 0 0 1 0 1 0 0 0 0
7 0 0 0 0 1 1 0 0 1 0
8 0 0 0 0 0 1 0 1 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

0 1 2 3 4 5 6 7 8 9
Mark 9 as reachable
0 0 1 1 0 1 0 0 0 0 0
Mark each neighbour of 9 as reachable 1 1 0 1 0 0 0 0 0 0 0
2 1 1 0 0 0 0 0 0 0 0
Systematically mark neighbours of marked 3 0 0 0 0 1 1 1 0 0 0
vertices 4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 1 1 1 0
6 0 0 0 1 0 1 0 0 0 0
7 0 0 0 0 1 1 0 0 1 0
8 0 0 0 0 0 1 0 1 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

0 1 2 3 4 5 6 7 8 9
Mark 9 as reachable
0 0 1 1 0 1 0 0 0 0 0
Mark each neighbour of 9 as reachable 1 1 0 1 0 0 0 0 0 0 0
2 1 1 0 0 0 0 0 0 0 0
Systematically mark neighbours of marked 3 0 0 0 0 1 1 1 0 0 0
vertices 4 1 0 0 1 0 0 0 1 0 0
Stop when 0 becomes marked 5 0 0 0 1 0 0 1 1 1 0
6 0 0 0 1 0 1 0 0 0 0
7 0 0 0 0 1 1 0 0 1 0
8 0 0 0 0 0 1 0 1 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Is Delhi (0) reachable from Madurai (9)? Undirected airline routes

0 1 2 3 4 5 6 7 8 9
Mark 9 as reachable
0 0 1 1 0 1 0 0 0 0 0
Mark each neighbour of 9 as reachable 1 1 0 1 0 0 0 0 0 0 0
2 1 1 0 0 0 0 0 0 0 0
Systematically mark neighbours of marked 3 0 0 0 0 1 1 1 0 0 0
vertices 4 1 0 0 1 0 0 0 1 0 0
Stop when 0 becomes marked 5 0 0 0 1 0 0 1 1 1 0
6 0 0 0 1 0 1 0 0 0 0
If marking process stops without target 7 0 0 0 0 1 1 0 0 1 0
becoming marked, the target is 8 0 0 0 0 0 1 0 1 0 1
unreachable 9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 7 / 11

Checking reachability

Mark source vertex as reachable Undirected airline routes

0 1 2 3 4 5 6 7 8 9
Systematically mark neighbours of marked
0 0 1 1 0 1 0 0 0 0 0
vertices
1 1 0 1 0 0 0 0 0 0 0
Stop when target becomes marked 2 1 1 0 0 0 0 0 0 0 0
3 0 0 0 0 1 1 1 0 0 0
4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 1 1 1 0
6 0 0 0 1 0 1 0 0 0 0
7 0 0 0 0 1 1 0 0 1 0
8 0 0 0 0 0 1 0 1 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 8 / 11

Checking reachability

Mark source vertex as reachable Undirected airline routes

0 1 2 3 4 5 6 7 8 9
Systematically mark neighbours of marked
0 0 1 1 0 1 0 0 0 0 0
vertices
1 1 0 1 0 0 0 0 0 0 0
Stop when target becomes marked 2 1 1 0 0 0 0 0 0 0 0
3 0 0 0 0 1 1 1 0 0 0
Need a strategy to systematically explore
4 1 0 0 1 0 0 0 1 0 0
marked neighbours
5 0 0 0 1 0 0 1 1 1 0
6 0 0 0 1 0 1 0 0 0 0
7 0 0 0 0 1 1 0 0 1 0
8 0 0 0 0 0 1 0 1 0 1
9 0 0 0 0 0 0 0 0 1 0

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 8 / 11

Checking reachability

Mark source vertex as reachable Undirected airline routes

0 1 2 3 4 5 6 7 8 9
Systematically mark neighbours of marked
0 0 1 1 0 1 0 0 0 0 0
vertices
1 1 0 1 0 0 0 0 0 0 0
Stop when target becomes marked 2 1 1 0 0 0 0 0 0 0 0
3 0 0 0 0 1 1 1 0 0 0
Need a strategy to systematically explore
4 1 0 0 1 0 0 0 1 0 0
marked neighbours
5 0 0 0 1 0 0 1 1 1 0
Two primary strategies 6 0 0 0 1 0 1 0 0 0 0
Breadth first — propagate marks in 7 0 0 0 0 1 1 0 0 1 0
“layers” 8 0 0 0 0 0 1 0 1 0 1
Depth first — explore a path till it dies 9 0 0 0 0 0 0 0 0 1 0
out, then backtrack

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 8 / 11

Adjacency lists
Adjacency matrix has many 0’s
Airline routes
Size is n2 , regardless of number of edges
Undirected graph: |E | ≤ n(n − 1)/2 v0
v1
Directed graph: |E | ≤ n(n − 1)
v2
Typically |E | much less than n2
v3
v4 v6
v5

v8
v7

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 9 / 11

0 {1,4} 5 {3,7}
1 {2} 6 {5} v8
v7
2 {0} 7 {4,8}
3 {4,6} 8 {5,9}
4 {0,3,7} 9 {8} v9

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 9 / 11

Comparing representations
Adjacency list typically requires less space
0 1 2 3 4 5 6 7 8 9
0 0 1 0 0 1 0 0 0 0 0
1 0 0 1 0 0 0 0 0 0 0
2 1 0 0 0 0 0 0 0 0 0
3 0 0 0 0 1 0 1 0 0 0
4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 0 1 0 0
6 0 0 0 0 0 1 0 0 0 0
7 0 0 0 0 1 0 0 0 1 0
8 0 0 0 0 0 1 0 0 0 1
9 0 0 0 0 0 0 0 0 1 0

0 {1,4} 5 {3,7}
1 {2} 6 {5}
2 {0} 7 {4,8}
3 {4,6} 8 {5,9}
4 {0,3,7} 9 {8}

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 10 / 11

0 {1,4} 5 {3,7}
1 {2} 6 {5}
2 {0} 7 {4,8}
3 {4,6} 8 {5,9}
4 {0,3,7} 9 {8}

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 10 / 11

Comparing representations
Adjacency list typically requires less space
0 1 2 3 4 5 6 7 8 9
Is j a neighbour of i? 0 0 1 0 0 1 0 0 0 0 0
1 0 0 1 0 0 0 0 0 0 0
Check if A[i, j] = 1 in adjacency matrix 2 1 0 0 0 0 0 0 0 0 0
Scan all neighbours of i in adjacency list 3 0 0 0 0 1 0 1 0 0 0
4 1 0 0 1 0 0 0 1 0 0
Which are the neighbours of i? 5 0 0 0 1 0 0 0 1 0 0
6 0 0 0 0 0 1 0 0 0 0
Scan all n entries in row i in adjacency 7 0 0 0 0 1 0 0 0 1 0
matrix 8 0 0 0 0 0 1 0 0 0 1
Takes time proportional to (out)degree 9 0 0 0 0 0 0 0 0 1 0
of i in adjacency list
0 {1,4} 5 {3,7}
1 {2} 6 {5}
2 {0} 7 {4,8}
3 {4,6} 8 {5,9}
4 {0,3,7} 9 {8}

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 10 / 11

Comparing representations
Adjacency list typically requires less space
0 1 2 3 4 5 6 7 8 9
Is j a neighbour of i? 0 0 1 0 0 1 0 0 0 0 0
1 0 0 1 0 0 0 0 0 0 0
Check if A[i, j] = 1 in adjacency matrix 2 1 0 0 0 0 0 0 0 0 0
Scan all neighbours of i in adjacency list 3 0 0 0 0 1 0 1 0 0 0
4 1 0 0 1 0 0 0 1 0 0
Which are the neighbours of i? 5 0 0 0 1 0 0 0 1 0 0
6 0 0 0 0 0 1 0 0 0 0
Scan all n entries in row i in adjacency 7 0 0 0 0 1 0 0 0 1 0
matrix 8 0 0 0 0 0 1 0 0 0 1
Takes time proportional to (out)degree 9 0 0 0 0 0 0 0 0 1 0
of i in adjacency list
0 {1,4} 5 {3,7}
Choose representation depending on 1 {2} 6 {5}
requirement 2 {0} 7 {4,8}
3 {4,6} 8 {5,9}
4 {0,3,7} 9 {8}

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 10 / 11

Summary

To operate on graphs, we need to represent them

Adjacency matrix
n × n matrix, A[i, j] = 1 iff (i, j) ∈ E

Adjacency list
For each vertex i, list of neighbours of i

Can systematically explore a graph using these representations

For reachability, propagate marking to all reachable vertices

Madhavan Mukund Representing Graphs Mathematics for Data Science 1, Week 10 11 / 11

Breadth First Search

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 10
Reachability in a graph
Mark source vertex as reachable
v0
Systematically mark neighbours of marked v1
vertices v2
v3
Stop when target becomes marked
v4 v6
v5

v8
v7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 2 / 11
Reachability in a graph
Mark source vertex as reachable 0 1 2 3 4 5 6 7 8 9
0 0 1 0 0 1 0 0 0 0 0
Systematically mark neighbours of marked
1 0 0 1 0 0 0 0 0 0 0
vertices 2 1 0 0 0 0 0 0 0 0 0
3 0 0 0 0 1 0 1 0 0 0
Stop when target becomes marked 4 1 0 0 1 0 0 0 1 0 0
5 0 0 0 1 0 0 0 1 0 0
Choose an appropriate representation
6 0 0 0 0 0 1 0 0 0 0
Adjacency matrix 7 0 0 0 0 1 0 0 0 1 0
Adjacency list 8 0 0 0 0 0 1 0 0 0 1
9 0 0 0 0 0 0 0 0 1 0

0 {1,4} 5 {3,7}
1 {2} 6 {5}
2 {0} 7 {4,8}
3 {4,6} 8 {5,9}
4 {0,3,7} 9 {8}

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 2 / 11
Breadth first search (BFS)

Explore the graph level by level 0

First visit vertices one step away 1
Then two steps away 2
... 3
4 6
5

8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 3 / 11
Breadth first search (BFS)

Explore the graph level by level 0

First visit vertices one step away 1
Then two steps away 2
... 3
4 6
Each visited vertex has to be explored 5
Extend the search to its neighbours
Do this only once for each vertex!
8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 3 / 11
Breadth first search (BFS)

Explore the graph level by level 0

First visit vertices one step away 1
Then two steps away 2
... 3
4 6
Each visited vertex has to be explored 5
Extend the search to its neighbours
Do this only once for each vertex!
8
7
Maintain information about vertices
Which vertices have been visited already 9
Among these, which are yet to be
explored

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 3 / 11
Breadth first search (BFS) . . .
Assume V = {0, 1, . . . , n − 1}
0
1
2
3
4 6
5

8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 4 / 11
Breadth first search (BFS) . . .
Assume V = {0, 1, . . . , n − 1}
0
visited : V → {True, False} tells us 1
whether v ∈ V has been visited 2
Initially, visited(v ) = False for all v ∈ V 3
4 6
Maintain a sequence of visited vertices yet 5
be explored
A queue — first in, first out
Initially empty
8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 4 / 11
Breadth first search (BFS) . . .
Initially
visited(v ) = False for all v ∈ V
0
1
Queue of vertices to be explored is empty
2
3
4 6
5

8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 5 / 11
Breadth first search (BFS) . . .
Initially
visited(v ) = False for all v ∈ V
0
1
Queue of vertices to be explored is empty
2
Start BFS from vertex j 3
Set visited(j) = True 4 6
Add j to the queue 5

8
7

Remove and explore vertex i at head of

queue 8
7
For each edge (i, j), if visited(j) is False,
Set visited(j) to True
9
Append j to the queue

Remove and explore vertex i at head of

queue 8
7
For each edge (i, j), if visited(j) is False,
Set visited(j) to True
9
Append j to the queue

Stop when queue is empty

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 5 / 11
BFS from vertex 7
Visited To explore queue
0 False 0
1 False 1
2 False 2
3 False 3
4 False 4 6
5 False 5
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 False 7 0
1 False 1
Mark 7 and add to queue 2
2 False
3 False 3
4 False 4 6
5 False 5
6 False
7 True 8
8 False 7
9 False
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 False 4 5 8 0
1 False 1
Mark 7 and add to queue 2
2 False
Explore 7, visit {4,5,8} 3
3 False
4 True 4 6
5 True 5
6 False
7 True 8
8 True 7
9 False
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 True 5 8 0 3 0
1 False 1
Mark 7 and add to queue 2
2 False
Explore 7, visit {4,5,8} 3
3 True
4 True Explore 4, visit {0,3} 4 6
5 True 5
6 False
7 True 8
8 True 7
9 False
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 True 8 0 3 6 0
1 False 1
Mark 7 and add to queue 2
2 False
Explore 7, visit {4,5,8} 3
3 True
4 True Explore 4, visit {0,3} 4 6
5 True Explore 5, visit {6} 5
6 True
7 True 8
8 True 7
9 False
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 True 0 3 6 9 0
1 False 1
Mark 7 and add to queue 2
2 False
Explore 7, visit {4,5,8} 3
3 True
4 True Explore 4, visit {0,3} 4 6
5 True Explore 5, visit {6} 5
6 True Explore 8, visit {9}
7 True 8
8 True 7
9 True
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 True 3 6 9 1 2 0
1 True 1
Mark 7 and add to queue 2
2 True
Explore 7, visit {4,5,8} 3
3 True
4 True Explore 4, visit {0,3} 4 6
5 True Explore 5, visit {6} 5
6 True Explore 8, visit {9}
7 True Explore 0, visit {1,2} 8
8 True 7
9 True
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 True 6 9 1 2 0
1 True 1
Mark 7 and add to queue 2
2 True
Explore 7, visit {4,5,8} 3
3 True
4 True Explore 4, visit {0,3} 4 6
5 True Explore 5, visit {6} 5
6 True Explore 8, visit {9}
7 True Explore 0, visit {1,2} 8
8 True 7
Explore 3
9 True
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 True 9 1 2 0
1 True 1
Mark 7 and add to queue 2
2 True
Explore 7, visit {4,5,8} 3
3 True
4 True Explore 4, visit {0,3} 4 6
5 True Explore 5, visit {6} 5
6 True Explore 8, visit {9}
7 True Explore 0, visit {1,2} 8
8 True 7
Explore 3
9 True
Explore 6 9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 True 1 2 0
1 True 1
Mark 7 and add to queue 2
2 True
Explore 7, visit {4,5,8} 3
3 True
4 True Explore 4, visit {0,3} 4 6
5 True Explore 5, visit {6} 5
6 True Explore 8, visit {9}
7 True Explore 0, visit {1,2} 8
8 True 7
Explore 3
9 True
Explore 6 9
Explore 9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 True 2 0
1 True 1
Mark 7 and add to queue 2
2 True
Explore 7, visit {4,5,8} 3
3 True
4 True Explore 4, visit {0,3} 4 6
5 True Explore 5, visit {6} 5
6 True Explore 8, visit {9}
7 True Explore 0, visit {1,2} 8
8 True 7
Explore 3
9 True
Explore 6 9
Explore 9
Explore 1

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
BFS from vertex 7
Visited To explore queue
0 True 0
1 True 1
Mark 7 and add to queue 2
2 True
Explore 7, visit {4,5,8} 3
3 True
4 True Explore 4, visit {0,3} 4 6
5 True Explore 5, visit {6} 5
6 True Explore 8, visit {9}
7 True Explore 0, visit {1,2} 8
8 True 7
Explore 3
9 True
Explore 6 9
Explore 9
Explore 1
Explore 2
Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 6 / 11
Enhancing BFS to record paths

If BFS from i sets visited(j) = True, we 0

know that j is reachable from i 1
2
3
4 6
5

8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 7 / 11
Enhancing BFS to record paths

If BFS from i sets visited(j) = True, we 0

know that j is reachable from i 1
2
How do we recover a path from i to j? 3
4 6
5

8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 7 / 11
Enhancing BFS to record paths

If BFS from i sets visited(j) = True, we 0

know that j is reachable from i 1
2
How do we recover a path from i to j? 3
visited(j) was set to True when exploring 4 6
some vertex k 5

8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 7 / 11
Enhancing BFS to record paths

If BFS from i sets visited(j) = True, we 0

know that j is reachable from i 1
2
How do we recover a path from i to j? 3
visited(j) was set to True when exploring 4 6
some vertex k 5
Record parent(j) = k
8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 7 / 11
Enhancing BFS to record paths

If BFS from i sets visited(j) = True, we 0

know that j is reachable from i 1
2
How do we recover a path from i to j? 3
visited(j) was set to True when exploring 4 6
some vertex k 5
Record parent(j) = k
From j, follow parent links to trace back a 8
7
path to i
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 7 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 False 0
1 False 1
2 False 2
3 False 3
4 False 4 6
5 False 5
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 False 7 0
1 False 1
Mark 7, add to queue 2
2 False
3 False 3
4 False 4 6
5 False 5
6 False
7 True 8
8 False 7
9 False
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 False 4 5 8 0
1 False 1
2 False Mark 7, add to queue 2
3 False Explore 7, visit {4,5,8} 3
4 True 7 4 6
5 True 7 5
6 False
7 True 8
8 True 7 7
9 False
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 5 8 0 3 0
1 False 1
2 False Mark 7, add to queue 2
3 True 4 Explore 7, visit {4,5,8} 3
4 True 7 4 6
Explore 4, visit {0,3}
5 True 7 5
6 False
7 True 8
8 True 7 7
9 False
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 8 0 3 6 0
1 False 1
2 False Mark 7, add to queue 2
3 True 4 Explore 7, visit {4,5,8} 3
4 True 7 4 6
Explore 4, visit {0,3}
5 True 7 5
Explore 5, visit {6}
6 True 5
7 True 8
8 True 7 7
9 False
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 0 3 6 9 0
1 False 1
2 False Mark 7, add to queue 2
3 True 4 Explore 7, visit {4,5,8} 3
4 True 7 4 6
Explore 4, visit {0,3}
5 True 7 5
Explore 5, visit {6}
6 True 5
Explore 8, visit {9}
7 True 8
8 True 7 7
9 True 8
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 3 6 9 1 2 0
1 True 0 1
2 True 0 Mark 7, add to queue 2
3 True 4 Explore 7, visit {4,5,8} 3
4 True 7 4 6
Explore 4, visit {0,3}
5 True 7 5
Explore 5, visit {6}
6 True 5
Explore 8, visit {9}
7 True 8
8 True 7 Explore 0, visit {1,2} 7
9 True 8
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 6 9 1 2 0
1 True 0 1
2 True 0 Mark 7, add to queue 2
3 True 4 Explore 7, visit {4,5,8} 3
4 True 7 4 6
Explore 4, visit {0,3}
5 True 7 5
Explore 5, visit {6}
6 True 5
Explore 8, visit {9}
7 True 8
8 True 7 Explore 0, visit {1,2} 7
9 True 8 Explore 3
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 9 1 2 0
1 True 0 1
2 True 0 Mark 7, add to queue 2
3 True 4 Explore 7, visit {4,5,8} 3
4 True 7 4 6
Explore 4, visit {0,3}
5 True 7 5
Explore 5, visit {6}
6 True 5
Explore 8, visit {9}
7 True 8
8 True 7 Explore 0, visit {1,2} 7
9 True 8 Explore 3
Explore 6 9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 1 2 0
1 True 0 1
2 True 0 Mark 7, add to queue 2
3 True 4 Explore 7, visit {4,5,8} 3
4 True 7 4 6
Explore 4, visit {0,3}
5 True 7 5
Explore 5, visit {6}
6 True 5
Explore 8, visit {9}
7 True 8
8 True 7 Explore 0, visit {1,2} 7
9 True 8 Explore 3
Explore 6 9
Explore 9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 2 0
1 True 0 1
Mark 7, add to queue 2
2 True 0
Explore 7, visit {4,5,8} 3
3 True 4
4 True 7 Explore 4, visit {0,3} 4 6
5 True 7 Explore 5, visit {6} 5
6 True 5 Explore 8, visit {9}
7 True Explore 0, visit {1,2} 8
8 True 7 7
Explore 3
9 True 8
Explore 6 9
Explore 9
Explore 1

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 0
1 True 0 1
Mark 7, add to queue 2
2 True 0
Explore 7, visit {4,5,8} 3
3 True 4
4 True 7 Explore 4, visit {0,3} 4 6
5 True 7 Explore 5, visit {6} 5
6 True 5 Explore 8, visit {9}
7 True Explore 0, visit {1,2} 8
8 True 7 7
Explore 3
9 True 8
Explore 6 9
Explore 9
Explore 1
Explore 2
Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 0
1 True 0 1
Mark 7, add to queue 2
2 True 0
Explore 7, visit {4,5,8} 3
3 True 4
4 True 7 Explore 4, visit {0,3} 4 6
5 True 7 Explore 5, visit {6} 5
6 True 5 Explore 8, visit {9}
7 True Explore 0, visit {1,2} 8
8 True 7 7
Explore 3
9 True 8
Explore 6 9
Path from 7 to 6 is Explore 9
7–5–6 Explore 1
Explore 2
Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
BFS from vertex 7 with parent information
Visited Parent To explore queue
0 True 4 0
1 True 0 1
Mark 7, add to queue 2
2 True 0
Explore 7, visit {4,5,8} 3
3 True 4
4 True 7 Explore 4, visit {0,3} 4 6
5 True 7 Explore 5, visit {6} 5
6 True 5 Explore 8, visit {9}
7 True Explore 0, visit {1,2} 8
8 True 7 7
Explore 3
9 True 8
Explore 6 9
Path from 7 to 2 is Explore 9
7–4–0–2 Explore 1
Explore 2
Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 8 / 11
Enhancing BFS to record distance

BFS explores neighbours level by level 0

1
2
3
4 6
5

8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 9 / 11
Enhancing BFS to record distance

BFS explores neighbours level by level 0

1
By recording the level at which a vertex is
visited, we get its distance from the 2
source vertex 3
4 6
5

8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 9 / 11
Enhancing BFS to record distance

BFS explores neighbours level by level 0

1
By recording the level at which a vertex is
visited, we get its distance from the 2
source vertex 3
4 6
Instead of visited(j), maintain level)(j) 5

8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 9 / 11
Enhancing BFS to record distance

BFS explores neighbours level by level 0

1
By recording the level at which a vertex is
visited, we get its distance from the 2
source vertex 3
4 6
Instead of visited(j), maintain level)(j) 5
Initalize level(j) = −1 for all j
8
7

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 9 / 11
Enhancing BFS to record distance

BFS explores neighbours level by level 0

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 9 / 11
Enhancing BFS to record distance

BFS explores neighbours level by level 0

1
By recording the level at which a vertex is
visited, we get its distance from the 2
source vertex 3
4 6
Instead of visited(j), maintain level)(j) 5
Initalize level(j) = −1 for all j
Set level(i) = 0 for source vertex 8
7
If we visit j from k, set level(j) to
level(k) + 1 9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 9 / 11
Enhancing BFS to record distance

BFS explores neighbours level by level 0

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 9 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 −1 0
1 −1 1
2 −1 2
3 −1 3
4 −1 4 6
5 −1 5
6 −1
7 −1 8
8 −1 7
9 −1
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 −1 7 0
1 −1 1
Mark 7, add to queue 2
2 −1
3 −1 3
4 −1 4 6
5 −1 5
6 −1
7 0 8
8 −1 7
9 −1
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 −1 4 5 8 0
1 −1 1
2 −1 Mark 7, add to queue 2
3 −1 Explore 7, visit {4,5,8} 3
4 1 7 4 6
5 1 7 5
6 −1
7 0 8
8 1 7 7
9 −1
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 2 4 5 8 0 3 0
1 −1 1
2 −1 Mark 7, add to queue 2
3 2 4 Explore 7, visit {4,5,8} 3
4 1 7 4 6
Explore 4, visit {0,3}
5 1 7 5
6 −1
7 0 8
8 1 7 7
9 −1
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 2 4 8 0 3 6 0
1 −1 1
2 −1 Mark 7, add to queue 2
3 2 4 Explore 7, visit {4,5,8} 3
4 1 7 4 6
Explore 4, visit {0,3}
5 1 7 5
Explore 5, visit {6}
6 2 5
7 0 8
8 1 7 7
9 −1
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 2 4 0 3 6 9 0
1 −1 1
2 −1 Mark 7, add to queue 2
3 2 4 Explore 7, visit {4,5,8} 3
4 1 7 4 6
Explore 4, visit {0,3}
5 1 7 5
Explore 5, visit {6}
6 2 5
Explore 8, visit {9}
7 0 8
8 1 7 7
9 2 8
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 2 4 3 6 9 1 2 0
1 3 0 1
2 3 0 Mark 7, add to queue 2
3 2 4 Explore 7, visit {4,5,8} 3
4 1 7 4 6
Explore 4, visit {0,3}
5 1 7 5
Explore 5, visit {6}
6 2 5
Explore 8, visit {9}
7 0 8
8 1 7 Explore 0, visit {1,2} 7
9 2 8
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 2 4 6 9 1 2 0
1 3 0 1
2 3 0 Mark 7, add to queue 2
3 2 4 Explore 7, visit {4,5,8} 3
4 1 7 4 6
Explore 4, visit {0,3}
5 1 7 5
Explore 5, visit {6}
6 2 5
Explore 8, visit {9}
7 0 8
8 1 7 Explore 0, visit {1,2} 7
9 2 8 Explore 3
9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 2 4 9 1 2 0
1 3 0 1
2 3 0 Mark 7, add to queue 2
3 2 4 Explore 7, visit {4,5,8} 3
4 1 7 4 6
Explore 4, visit {0,3}
5 1 7 5
Explore 5, visit {6}
6 2 5
Explore 8, visit {9}
7 0 8
8 1 7 Explore 0, visit {1,2} 7
9 2 8 Explore 3
Explore 6 9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 2 4 1 2 0
1 3 0 1
2 3 0 Mark 7, add to queue 2
3 2 4 Explore 7, visit {4,5,8} 3
4 1 7 4 6
Explore 4, visit {0,3}
5 1 7 5
Explore 5, visit {6}
6 2 5
Explore 8, visit {9}
7 0 8
8 1 7 Explore 0, visit {1,2} 7
9 2 8 Explore 3
Explore 6 9
Explore 9

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 2 4 2 0
1 3 0 1
Mark 7, add to queue 2
2 3 0
Explore 7, visit {4,5,8} 3
3 2 4
4 1 7 Explore 4, visit {0,3} 4 6
5 1 7 Explore 5, visit {6} 5
6 2 5 Explore 8, visit {9}
7 0 Explore 0, visit {1,2} 8
8 1 7 7
Explore 3
9 2 8
Explore 6 9
Explore 9
Explore 1

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
BFS from vertex 7 with parent and distance information
Level Parent To explore queue
0 2 4 0
1 3 0 1
Mark 7, add to queue 2
2 3 0
Explore 7, visit {4,5,8} 3
3 2 4
4 1 7 Explore 4, visit {0,3} 4 6
5 1 7 Explore 5, visit {6} 5
6 2 5 Explore 8, visit {9}
7 0 Explore 0, visit {1,2} 8
8 1 7 7
Explore 3
9 2 8
Explore 6 9
Explore 9
Explore 1
Explore 2
Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 10 / 11
Summary

Breadth first search is a systematic strategy to explore a graph, level by level

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 11 / 11
Summary

Breadth first search is a systematic strategy to explore a graph, level by level

Record which vertices have been visited

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 11 / 11
Summary

Breadth first search is a systematic strategy to explore a graph, level by level

Record which vertices have been visited
Maintain visited but unexplored vertices in a queue

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 11 / 11
Summary

Breadth first search is a systematic strategy to explore a graph, level by level

Record which vertices have been visited
Maintain visited but unexplored vertices in a queue
Add parent information to recover the path to each reachable vertex

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 11 / 11
Summary

Breadth first search is a systematic strategy to explore a graph, level by level

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 11 / 11
Summary

Breadth first search is a systematic strategy to explore a graph, level by level

Record which vertices have been visited
Maintain visited but unexplored vertices in a queue
Add parent information to recover the path to each reachable vertex
Maintain level information to record length of the shortest path, in terms of number
of edges
In general, edges are labelled with a cost (distance, time, ticket price, . . . )

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 11 / 11
Summary

Breadth first search is a systematic strategy to explore a graph, level by level

Madhavan Mukund Breadth First Search Mathematics for Data Science 1, Week 10 11 / 11
Depth First Search

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 10
Depth first search (DFS)

Start from i, visit an unexplored neighbour 0

j 1
2
3
4 6
5

8
7

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 2/5
Depth first search (DFS)

Start from i, visit an unexplored neighbour 0

j 1
Suspend the exploration of i and explore j 2
instead 3
4 6
5

8
7

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 2/5
Depth first search (DFS)

Start from i, visit an unexplored neighbour 0

j 1
Suspend the exploration of i and explore j 2
instead 3
4 6
Continue till you reach a vertex with no 5
unexplored neighbours

8
7

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 2/5
Depth first search (DFS)

Start from i, visit an unexplored neighbour 0

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 2/5
Depth first search (DFS)

Start from i, visit an unexplored neighbour 0

j 1
Suspend the exploration of i and explore j 2
instead 3
4 6
Continue till you reach a vertex with no 5
unexplored neighbours
Backtrack to nearest suspended vertex 8
that still has an unexplored neighbour 7
Suspended vertices are stored in a stack
9
Last in, first out
Most recently suspended is checked first

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 2/5
DFS from vertex 4
Visited Stack of suspended vertices
0 False 0
1 False 1
2 False 2
3 False 3
4 False 4 6
5 False 5
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 False 0
1 False 1
Mark 4,
2 False 2
3 False 3
4 True 4 6
5 False 5
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 0
1 False 1
Mark 4, Suspend 4, explore 0
2 False 2
3 False 3
4 True 4 6
5 False 5
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 0 0
1 True 1
Mark 4, Suspend 4, explore 0
2 False 2
suspend 0, explore 1 3
3 False
4 True 4 6
5 False 5
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 0 1 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 False
Suspend 1, explore 2 4 6
4 True
5 False 5
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 0 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 False
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 5
5 False
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 False
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 5
5 False
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 False
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 False
Suspend 4, explore 3
6 False
7 False 8
8 False 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 3 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 True
Suspend 4, explore 3
6 False
7 False Suspend 3, explore 5
8
8 False 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 3 5 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 True
Suspend 4, explore 3
6 True
7 False Suspend 3, explore 5
8
8 False Suspend 5, explore 6 7
9 False
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 3 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 True
Suspend 4, explore 3
6 True
7 False Suspend 3, explore 5
8
8 False Suspend 5, explore 6 7
9 False Backtrack to 5,
9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 3 5 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 True
Suspend 4, explore 3
6 True
7 True Suspend 3, explore 5
8
8 False Suspend 5, explore 6 7
9 False Backtrack to 5, suspend 5,
explore 7 9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 3 5 7 8 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 True
Suspend 4, explore 3
6 True
7 True Suspend 3, explore 5
8
8 True Suspend 5, explore 6 7
9 True Backtrack to 5, suspend 5,
explore 7 9
Suspend 7, explore 8
Suspend 8, explore 9

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 3 5 7 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 True
Suspend 4, explore 3
6 True
7 True Suspend 3, explore 5
8
8 True Suspend 5, explore 6 7
9 True Backtrack to 5, suspend 5,
explore 7 9
Suspend 7, explore 8
Suspend 8, explore 9
Backtrack to 8,
Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 3 5 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 True
Suspend 4, explore 3
6 True
7 True Suspend 3, explore 5
8
8 True Suspend 5, explore 6 7
9 True Backtrack to 5, suspend 5,
explore 7 9
Suspend 7, explore 8
Suspend 8, explore 9
Backtrack to 8, 7,
Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 3 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 True
Suspend 4, explore 3
6 True
7 True Suspend 3, explore 5
8
8 True Suspend 5, explore 6 7
9 True Backtrack to 5, suspend 5,
explore 7 9
Suspend 7, explore 8
Suspend 8, explore 9
Backtrack to 8, 7, 5,
Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 4 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 True
Suspend 4, explore 3
6 True
7 True Suspend 3, explore 5
8
8 True Suspend 5, explore 6 7
9 True Backtrack to 5, suspend 5,
explore 7 9
Suspend 7, explore 8
Suspend 8, explore 9
Backtrack to 8, 7, 5, 3,
Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
DFS from vertex 4
Visited Stack of suspended vertices
0 True 0
1 True 1
Mark 4, Suspend 4, explore 0
2 True 2
suspend 0, explore 1 3
3 True
Suspend 1, explore 2 4 6
4 True
Backtrack to 1, 0, 4 5
5 True
Suspend 4, explore 3
6 True
7 True Suspend 3, explore 5
8
8 True Suspend 5, explore 6 7
9 True Backtrack to 5, suspend 5,
explore 7 9
Suspend 7, explore 8
Suspend 8, explore 9
Backtrack to 8, 7, 5, 3, 4
Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 3/5
Depth first search (DFS)
Paths discovered by BFS are not shortest
paths, unlike BFS 0
1
2
3
4 6
5

8
7

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 4/5
Depth first search (DFS)
Paths discovered by BFS are not shortest
paths, unlike BFS 0
1
Useful features can be found by recording
2
the order in which DFS visits vertices
3
DFS numbering — maintain a counter 4 6
Increment and record counter value each 5
time you start and finish exploring a
vertex
8
7

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 4/5
Summary

DFS is another systematic strategy to explore a graph

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 5/5
Summary

DFS is another systematic strategy to explore a graph

DFS uses a stack to suspend exploration and move to unexplored neighbours

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 5/5
Summary

DFS is another systematic strategy to explore a graph

DFS uses a stack to suspend exploration and move to unexplored neighbours
DFS numbering can be used to discover many facts about graphs

Madhavan Mukund Depth First Search Mathematics for Data Science 1, Week 10 5/5
Complexity of BFS and DFS

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 10
BFS and DFS
Breadth first search
Explore graph level by level
Keep track of
visited : V → {True,False}
Queue of unexplored vertices
BFS from vertex j
Set visited(j) = True
Add j to the queue
Explore vertex i at head of queue
For edge (i, j), if visited(j) is False,
Set visited(j) to True
Append j to the queue

Stop when queue is empty

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 2/6
BFS and DFS
Breadth first search Depth first search
Explore graph level by level Start from i, visit an unexplored
neighbour j
Keep track of
visited : V → {True,False} Suspend the exploration of i and
Queue of unexplored vertices explore j instead

BFS from vertex j Continue till you reach a vertex with no

unexplored neighbours
Set visited(j) = True
Add j to the queue Backtrack to nearest suspended vertex
that still has an unexplored neighbour
Explore vertex i at head of queue
For edge (i, j), if visited(j) is False, Keep track of
Set visited(j) to True visited : V → {True,False}
Append j to the queue Stack of suspended vertices
Stop when queue is empty
Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 2/6
Complexity BFS and DFS

G = (V , E )
|V | = n
|E | = m
If G is connected, m can vary from
n − 1 to n(n − 1)/2

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 3/6
Complexity BFS and DFS

G = (V , E )
|V | = n
|E | = m
If G is connected, m can vary from
n − 1 to n(n − 1)/2
In both BFS and DFS, each reachable
vertex is visited exactly once
Visit and explore at most n vertices

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 3/6
Complexity BFS and DFS

G = (V , E )
|V | = n
|E | = m
If G is connected, m can vary from
n − 1 to n(n − 1)/2
In both BFS and DFS, each reachable
vertex is visited exactly once
Visit and explore at most n vertices
Each visited vertex is explored once
Check all outgoing edges
How long does this take?

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 3/6
Complexity BFS and DFS

G = (V , E ) Adjacency matrix
|V | = n To explore i, scan row i
|E | = m Look up n entries, regardless of number
If G is connected, m can vary from of actual edges from i
n − 1 to n(n − 1)/2 Degree of i
In both BFS and DFS, each reachable Overall, n2 steps
vertex is visited exactly once
Adjacency list
Visit and explore at most n vertices
To explore i, scan list of neighbours of i
Each visited vertex is explored once
Time to explore i is degree of i
Check all outgoing edges
How long does this take? Degree varies across vertices
Estimate overall time?
Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 3/6
Calculating with degrees

Adjacency list
To explore i, scan list of neighbours of i
Time to explore i is degree of i
Total time is the sum of the degrees

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 4/6
Calculating with degrees

Adjacency list
To explore i, scan list of neighbours of i
Time to explore i is degree of i
Total time is the sum of the degrees
Sum of degrees
Each edge (i, j) contributes to degree of
both i and j
Sum of degrees is 2m

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 4/6
Calculating with degrees

Adjacency list BFS/DFS with adjacency list

To explore i, scan list of neighbours of i n steps to visit each vertex
Time to explore i is degree of i 2m steps (sum of degrees) to explore all
vertices
Total time is the sum of the degrees
Sum of degrees Overall time proportional to n + m

Each edge (i, j) contributes to degree of

both i and j
Sum of degrees is 2m

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 4/6
Calculating with degrees

Adjacency list BFS/DFS with adjacency list

Each edge (i, j) contributes to degree of BFS/DFS with adjacency matrix

both i and j n steps to visit each vertex
Sum of degrees is 2m n steps to explore each vertex
Overall time proportional to n2

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 4/6
Calculating with degrees

Adjacency list BFS/DFS with adjacency list

Each edge (i, j) contributes to degree of BFS/DFS with adjacency matrix

both i and j n steps to visit each vertex
Sum of degrees is 2m n steps to explore each vertex
Overall time proportional to n2
If m is proportional to n, big saving by using
adjacency list representation
Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 4/6
More about degrees

Degree of a vertex i is

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is
Number of 1’s in row i of adjacency
matrix

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is
Number of 1’s in row i of adjacency
matrix
Number of 1’s in column i of
adjacency matrix

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is
Number of 1’s in row i of adjacency
matrix
Number of 1’s in column i of
adjacency matrix
Length of adjacency list for i

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is
Number of 1’s in row i of adjacency
matrix
Number of 1’s in column i of
adjacency matrix
Length of adjacency list for i
Sum of degrees is 2m

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is
Number of 1’s in row i of adjacency
matrix
Number of 1’s in column i of
adjacency matrix
Length of adjacency list for i
Sum of degrees is 2m
Sum is an even number

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is
Number of 1’s in row i of adjacency
matrix
Number of 1’s in column i of
adjacency matrix
Length of adjacency list for i
Sum of degrees is 2m
Sum is an even number
For each vertex with odd degree, must
be another vertex of odd degree to
make the sum even

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is Degree of a vertex between 0 and n − 1

Number of 1’s in row i of adjacency Degree 0 — disconnected vertex
matrix Degree n − 1 — connected to all
Number of 1’s in column i of other vertices
adjacency matrix
Length of adjacency list for i
Sum of degrees is 2m
Sum is an even number
For each vertex with odd degree, must
be another vertex of odd degree to
make the sum even
Number of vertices of odd degree is
even

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is Degree of a vertex between 0 and n − 1

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is Degree of a vertex between 0 and n − 1

Number of 1’s in row i of adjacency Degree 0 — disconnected vertex
matrix Degree n − 1 — connected to all
Number of 1’s in column i of other vertices
adjacency matrix
Complete graph — every vertex has
Length of adjacency list for i
degree n − 1
Sum of degrees is 2m
If all degrees are bounded by k, at most
Sum is an even number kn/2 edges
For each vertex with odd degree, must
be another vertex of odd degree to
make the sum even
Number of vertices of odd degree is
even

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is Degree of a vertex between 0 and n − 1

Number of 1’s in row i of adjacency Degree 0 — disconnected vertex
matrix Degree n − 1 — connected to all
Number of 1’s in column i of other vertices
adjacency matrix
Complete graph — every vertex has
Length of adjacency list for i
degree n − 1
Sum of degrees is 2m
If all degrees are bounded by k, at most
Sum is an even number kn/2 edges
For each vertex with odd degree, must
be another vertex of odd degree to For directed graphs, indegree and
make the sum even outdegree
Number of vertices of odd degree is
even

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
More about degrees

Degree of a vertex i is Degree of a vertex between 0 and n − 1

Number of 1’s in row i of adjacency Degree 0 — disconnected vertex
matrix Degree n − 1 — connected to all
Number of 1’s in column i of other vertices
adjacency matrix
Complete graph — every vertex has
Length of adjacency list for i
degree n − 1
Sum of degrees is 2m
If all degrees are bounded by k, at most
Sum is an even number kn/2 edges
For each vertex with odd degree, must
be another vertex of odd degree to For directed graphs, indegree and
make the sum even outdegree
Number of vertices of odd degree is Sum of indegrees = m = Sum of
even outdegrees
Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 5/6
Summary

BFS and DFS with adjacency matrix — time proportional to n2

BFS and DFS with adjacency list — time proportional to n + m
Exploring vertices examines each edge twice, sum of the degrees, 2m steps

For a connected graph, m varies from n − 1 to n(n − 1)/2

Considerable saving with small m
All degrees bounded by k, at most kn/2 edges

Madhavan Mukund Complexity of BFS and DFS Mathematics for Data Science 1, Week 10 6/6
Directed Acyclic Graphs (DAGs)

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 11
Directed cycles

In a directed graph, a cycle must follow same

direction 1 0 2
0 → 2 → 3 → 0 is a cycle
0 → 5 → 1 ← 0 is not
4 5 3

7 6

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 2/7
Directed cycles

In a directed graph, a cycle must follow same

direction 1 0 2
0 → 2 → 3 → 0 is a cycle
0 → 5 → 1 ← 0 is not
4 5 3
DFS reveals different types of non-tree edges
Forward edges
Back edges 7 6
Cross edges

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 2/7
Directed cycles

In a directed graph, a cycle must follow same

direction 1 0 2
0 → 2 → 3 → 0 is a cycle
0 → 5 → 1 ← 0 is not
4 5 3
DFS reveals different types of non-tree edges
Forward edges
Back edges 7 6
Cross edges

Only back edges generate cycles

Classify non-tree edges using DFS numbering

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 2/7
Directed cycles

In a directed graph, a cycle must follow same

direction 1 0 2
0 → 2 → 3 → 0 is a cycle
0 → 5 → 1 ← 0 is not
4 5 3
DFS reveals different types of non-tree edges
Forward edges
Back edges 7 6
Cross edges

Only back edges generate cycles

Classify non-tree edges using DFS numbering

Why bother about directed cycles?

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 2/7
Tasks and dependencies

Startup moving into new office space

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 3/7
Tasks and dependencies

Startup moving into new office space

Major tasks for completing the interiors
Lay floor tiles
Plaster the walls
Paint the walls
Lay conduits (pipes) for electrical wires
Do electrical wiring
Install electrical fittings
Lay telecom conduits
Do phone and network cabling

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 3/7
Tasks and dependencies

Startup moving into new office space Constraints on the sequence

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 3/7
Tasks and dependencies

Startup moving into new office space Constraints on the sequence

Major tasks for completing the interiors Lay conduits before tiles and plastering

Lay floor tiles

Plaster the walls
Paint the walls
Lay conduits (pipes) for electrical wires
Do electrical wiring
Install electrical fittings
Lay telecom conduits
Do phone and network cabling

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 3/7
Tasks and dependencies

Startup moving into new office space Constraints on the sequence

Major tasks for completing the interiors Lay conduits before tiles and plastering
Lay tiles, plaster wall before painting
Lay floor tiles
Plaster the walls
Paint the walls
Lay conduits (pipes) for electrical wires
Do electrical wiring
Install electrical fittings
Lay telecom conduits
Do phone and network cabling

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 3/7
Tasks and dependencies

Startup moving into new office space Constraints on the sequence

Major tasks for completing the interiors Lay conduits before tiles and plastering
Lay tiles, plaster wall before painting
Lay floor tiles
Finish painting before any
Plaster the walls
cabling/wiring work
Paint the walls
Lay conduits (pipes) for electrical wires
Do electrical wiring
Install electrical fittings
Lay telecom conduits
Do phone and network cabling

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 3/7
Tasks and dependencies

Startup moving into new office space Constraints on the sequence

Major tasks for completing the interiors Lay conduits before tiles and plastering
Lay tiles, plaster wall before painting
Lay floor tiles
Finish painting before any
Plaster the walls
cabling/wiring work
Paint the walls
Electrical wiring before installing
Lay conduits (pipes) for electrical wires fittings
Do electrical wiring
Install electrical fittings
Lay telecom conduits
Do phone and network cabling

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 3/7
Tasks and dependencies

Startup moving into new office space Constraints on the sequence

Major tasks for completing the interiors Lay conduits before tiles and plastering
Lay tiles, plaster wall before painting
Lay floor tiles
Finish painting before any
Plaster the walls
cabling/wiring work
Paint the walls
Electrical wiring before installing
Lay conduits (pipes) for electrical wires fittings
Do electrical wiring
Represent constraints as a directed
Install electrical fittings graph
Lay telecom conduits
Vertices are tasks
Do phone and network cabling
Edge (t, u) if task t has to be
completed before task u

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 3/7
Tasks and dependencies
Constraints on the sequence
Lay conduits before tiles and
plastering
Lay tiles, plaster wall before
painting
Finish painting before any
cabling/wiring work
Electrical wiring before installing
fittings

Represent constraints as a directed

graph
Vertices are tasks
Edge (t, u) if task t has to be
completed before task u
Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 4/7
Tasks and dependencies
Constraints on the sequence
Conduits (E) Conduits (T)
Lay conduits before tiles and
plastering
Lay tiles, plaster wall before
painting Tiling Plastering
Finish painting before any
cabling/wiring work
Painting
Electrical wiring before installing
fittings

Represent constraints as a directed Wiring (E) Cabling (T)

graph
Vertices are tasks
Edge (t, u) if task t has to be Fittings (E)
completed before task u
Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 4/7
Tasks and dependencies
Constraints on the sequence
Conduits (E) Conduits (T)
Lay conduits before tiles and
plastering
Lay tiles, plaster wall before
painting Tiling Plastering
Finish painting before any
cabling/wiring work
Painting
Electrical wiring before installing
fittings

Represent constraints as a directed Wiring (E) Cabling (T)

Schedule the tasks respecting the Conduits (E) Conduits (T)

dependencies

Tiling Plastering

Painting

Wiring (E) Cabling (T)

Fittings (E)

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 5/7
Typical questions

Schedule the tasks respecting the Conduits (E) Conduits (T)

dependencies
Conduits (E) – Conduits (T) –
Tiling – Plastering – Painting – Tiling Plastering
Wiring (E) – Cabling (T) –
Fittings (E)
Painting

Wiring (E) Cabling (T)

Fittings (E)

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 5/7
Typical questions

Schedule the tasks respecting the Conduits (E) Conduits (T)

dependencies
Conduits (E) – Conduits (T) –
Tiling – Plastering – Painting – Tiling Plastering
Wiring (E) – Cabling (T) –
Fittings (E)
Conduits (T) – Conduits (E) – Painting
Plastering – Tiling – Painting –
Wiring (E) – Fittings (E) –
Cabling (T)
Wiring (E) Cabling (T)
...

Fittings (E)

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 5/7
Typical questions

Schedule the tasks respecting the Conduits (E) Conduits (T)

How long will the work take?

Fittings (E)

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 5/7
Directed Acyclic Graphs

Formally, we have a directed acyclic Conduits (E) Conduits (T)

graph (DAG)

G = (V , E ), a directed graph without Tiling Plastering

directed cycles

Painting

Wiring (E) Cabling (T)

Fittings (E)

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 6/7
Directed Acyclic Graphs

Formally, we have a directed acyclic Conduits (E) Conduits (T)

graph (DAG)

G = (V , E ), a directed graph without Tiling Plastering

directed cycles
Find a schedule
Painting
Enumerate V = {0, 1, . . . , n − 1}
such that for any (i, j) ∈ E , i
appears before j
Wiring (E) Cabling (T)

Fittings (E)

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 6/7
Directed Acyclic Graphs

Formally, we have a directed acyclic Conduits (E) Conduits (T)

graph (DAG)

G = (V , E ), a directed graph without Tiling Plastering

directed cycles
Find a schedule
Painting
Enumerate V = {0, 1, . . . , n − 1}
such that for any (i, j) ∈ E , i
appears before j
Topological sorting Wiring (E) Cabling (T)

Fittings (E)

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 6/7
Directed Acyclic Graphs

Formally, we have a directed acyclic Conduits (E) Conduits (T)

graph (DAG)

G = (V , E ), a directed graph without Tiling Plastering

directed cycles
Find a schedule
Painting
Enumerate V = {0, 1, . . . , n − 1}
such that for any (i, j) ∈ E , i
appears before j
Topological sorting Wiring (E) Cabling (T)

How long with the work take?

Find the longest path in the DAG Fittings (E)

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 6/7
Summary

Directed acyclic graphs are a natural way to represent dependencies

Arise in many contexts
Pre-requisites between courses for completing a degree
Recipe for cooking
Construction projects
...

Problems to be solved on DAGS

Topological sorting
Longest paths

Madhavan Mukund Directed Acyclic Graphs (DAGs) Mathematics for Data Science 1, Week 11 7/7
Topological Sorting

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 11
Directed Acyclic Graphs

G = (V , E ), a directed graph without

directed cycles 0 1

2
4

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 2/8
Directed Acyclic Graphs

G = (V , E ), a directed graph without

directed cycles 0 1
Topological sorting
Enumerate V = {0, 1, . . . , n − 1}
such that for any (i, j) ∈ E , i
2
4
appears before j
3

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 2/8
Directed Acyclic Graphs

G = (V , E ), a directed graph without

directed cycles 0 1
Topological sorting
Enumerate V = {0, 1, . . . , n − 1}
such that for any (i, j) ∈ E , i
2
4
appears before j

Represents a feasible schedule 3

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 2/8
Topological Sort

A graph with directed cycles cannot

be sorted topologically 0 1

2
4

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 3/8
Topological Sort

A graph with directed cycles cannot

be sorted topologically 0 1
Path i ; j means i must be listed
before j
2
4

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 3/8
Topological Sort

A graph with directed cycles cannot

be sorted topologically 0 1
Path i ; j means i must be listed
before j
2
Cycle ⇒ vertices i, j such that there 4
are paths i ; j and j ; i
3

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 3/8
Topological Sort

A graph with directed cycles cannot

be sorted topologically 0 1
Path i ; j means i must be listed
before j
2
Cycle ⇒ vertices i, j such that there 4
are paths i ; j and j ; i
3
i must appear before j, and j must
appear before i, impossible!
5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 3/8
Topological Sort

A graph with directed cycles cannot

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 3/8
How to topologically sort a DAG?

Strategy
0 1

2
4

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 4/8
How to topologically sort a DAG?

Strategy
First list vertices with no 0 1
dependencies

2
4

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 4/8
How to topologically sort a DAG?

Strategy
First list vertices with no 0 1
dependencies
As we proceed, list vertices whose
2
dependencies have already been listed
4
...
3

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 4/8
How to topologically sort a DAG?

Strategy
First list vertices with no 0 1
dependencies
As we proceed, list vertices whose
2
dependencies have already been listed
4
...
3
Questions

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 4/8
How to topologically sort a DAG?

Strategy
First list vertices with no 0 1
dependencies
As we proceed, list vertices whose
2
dependencies have already been listed
4
...
3
Questions
Why will there be a starting vertex 5 6 7
with no dependencies?

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 4/8
How to topologically sort a DAG?

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 4/8
Algorithm for topological sort
A vertex with no dependencies has no
incoming edges, indegree(v ) = 0
0 1

2
4

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 5/8
Algorithm for topological sort
A vertex with no dependencies has no
incoming edges, indegree(v ) = 0
0 1
Claim
Every DAG has a vertex with indegree 0
2
Start with any vertex with 4
indegree > 0
3
Follow edge back to one of its
predecessors
5 6 7
Repeat so long as indegree > 0
If we repeat n times, we must have a
cycle, which is impossible in a DAG
Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 5/8
Topological sort algorithm

Fact
Every DAG has a vertex with indegree 0 0 1

2
4

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 6/8
Topological sort algorithm

Fact
Every DAG has a vertex with indegree 0 0 1

List out a vertex j with indegree = 0

2
4

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 6/8
Topological sort algorithm

Fact
Every DAG has a vertex with indegree 0 0 1

List out a vertex j with indegree = 0

Delete j and all edges from j 2
4

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 6/8
Topological sort algorithm

Fact
Every DAG has a vertex with indegree 0 0 1

List out a vertex j with indegree = 0

Delete j and all edges from j 2
4
What remains is again a DAG!
3

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 6/8
Topological sort algorithm

Fact
Every DAG has a vertex with indegree 0 0 1

List out a vertex j with indegree = 0

Delete j and all edges from j 2
4
What remains is again a DAG!
3
Can find another vertex with
indegree = 0 to list and eliminate
5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 6/8
Topological sort algorithm

Fact
Every DAG has a vertex with indegree 0 0 1

List out a vertex j with indegree = 0

Delete j and all edges from j 2
4
What remains is again a DAG!
3
Can find another vertex with
indegree = 0 to list and eliminate
5 6 7
Repeat till all vertices are listed

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 6/8
Topological sort algorithm
Compute indegree of each vertex

0 1

2
4
3

5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency 0 0
matrix 0 1

2
2 1
4
1
3
2 1 4
5 6 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency 0
matrix 0
List a vertex with indegree 0 and
remove it from the DAG 2
2 1
4
1
3
2 1 4
5 6 7

Topologically sorted sequence

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG 1
2 1
Update indegrees 4
1
Can find another vertex with 3
indegree = 0 to list and eliminate 2 1 3
5 6 7

Topologically sorted sequence

1, 0,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG 0
2 0
Update indegrees 4
0
Can find another vertex with 3
indegree = 0 to list and eliminate 2 1 3
5 6 7

Topologically sorted sequence

1, 0,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG 0
2 0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 2 1 3
5 6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG 0
2 0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 1 1 2
5 6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 1 1 2
5 6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 0 1 2
5 6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 1 2
6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 0 2
6 7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 2
7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5, 6,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
0
Update indegrees 4
Can find another vertex with
indegree = 0 to list and eliminate 1
7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5, 6,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
Update indegrees
Can find another vertex with
indegree = 0 to list and eliminate 1
7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5, 6, 4,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
Update indegrees
Can find another vertex with
indegree = 0 to list and eliminate 0
7
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5, 6, 4,

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex Indegree
Scan each column of the adjacency
matrix

List a vertex with indegree 0 and

remove it from the DAG
Update indegrees
Can find another vertex with
indegree = 0 to list and eliminate
Repeat till all vertices are listed

Topologically sorted sequence

1, 0, 3, 2, 5, 6, 4, 7

Topologically sorted sequence

1, 0, 3, 2, 5, 6, 4, 7

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Topological sort algorithm
Compute indegree of each vertex
Scan each column of the adjacency
matrix 0 1
List a vertex with indegree 0 and
remove it from the DAG
2
Update indegrees 4
Can find another vertex with 3
indegree = 0 to list and eliminate
5 6 7
Repeat till all vertices are listed
Using adjacency lists? Topologically sorted sequence
Scan each list i → [j1 , j2 , . . . , jk ] 1, 0, 3, 2, 5, 6, 4, 7
Increment indegree(j` ) for each j`
Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 7/8
Summary

Directed acyclic graphs are a natural way to represent dependencies

Topological sort gives a feasible schedule that represents dependencies
At least one vertex with no dependencies, indegree 0
Eliminating such a vertex retains DAG structure
Repeat the process till all vertices are listed

More than one topological sort is possible

Choice of which vertex with indegree 0 to list next

Madhavan Mukund Topological Sorting Mathematics for Data Science 1, Week 11 8/8
Longest Paths in DAGs

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 11
Directed Acyclic Graphs

G = (V , E ), a directed graph without

directed cycles
0 1

2
4

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 2/7
Directed Acyclic Graphs

G = (V , E ), a directed graph without

directed cycles
0 1
Topological sorting
Enumerate V = {0, 1, . . . , n − 1}
such that for any (i, j) ∈ E , i
2
appears before j 4
Feasible schedule
3

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 2/7
Directed Acyclic Graphs

G = (V , E ), a directed graph without

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 2/7
Directed Acyclic Graphs

G = (V , E ), a directed graph without

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 2/7
Directed Acyclic Graphs

G = (V , E ), a directed graph without

directed cycles
0 1
Topological sorting
Enumerate V = {0, 1, . . . , n − 1}
such that for any (i, j) ∈ E , i
2
appears before j 4
Feasible schedule
3
Imagine the DAG represents
prerequisites between courses
5 6 7
Each course takes a semester
Minimum number of semesters to
complete the programme?
Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 2/7
Directed Acyclic Graphs

G = (V , E ), a directed graph without

Find the longest path in a DAG

0 1

2
4

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 3/7
Longest Path

Find the longest path in a DAG

0 1
If indegree(i) = 0,
longest-path-to(i) = 0

2
4

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 3/7
Longest Path

Find the longest path in a DAG

0 1
If indegree(i) = 0,
longest-path-to(i) = 0
If indegree(i) > 0, longest path to i is 2
1 more than longest path to its 4
incoming neighbours
3
longest-path-to(i) =
1 + max{longest-path-to(j) | (j, i) ∈ E }
5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 3/7
Longest Path

longest-path-to(i) =
1 + max{longest-path-to(j) | (j, i) ∈ E } 0 1

2
4

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 4/7
Longest Path

longest-path-to(i) =
1 + max{longest-path-to(j) | (j, i) ∈ E } 0 1
To compute longest-path-to(i), need
longest-path-to(k), for each incoming
neighbour k 2
4

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 4/7
Longest Path

longest-path-to(i) =
1 + max{longest-path-to(j) | (j, i) ∈ E } 0 1
To compute longest-path-to(i), need
longest-path-to(k), for each incoming
neighbour k 2
4
If graph is topologically sorted, k is
listed before i 3

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 4/7
Longest Path

Hence compute longest-path-to() in

topological order 5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 4/7
Longest Path

Let i0 , i1 , . . . , in−1 be a topological

ordering of V 0 1

2
4

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 5/7
Longest Path

Let i0 , i1 , . . . , in−1 be a topological

ordering of V 0 1
All neighbours of ik appear before it
in this list
2
4

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 5/7
Longest Path

Let i0 , i1 , . . . , in−1 be a topological

ordering of V 0 1
All neighbours of ik appear before it
in this list
2
From left to right, compute 4
longest-path-to(ik ) as
1 + max{longest-path-to(ij ) | (ij , ik ) ∈ E } 3

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 5/7
Longest Path

Let i0 , i1 , . . . , in−1 be a topological

ordering of V 0 1
All neighbours of ik appear before it
in this list
2
From left to right, compute 4
longest-path-to(ik ) as
1 + max{longest-path-to(ij ) | (ij , ik ) ∈ E } 3

Overlap this computation with

topological sorting 5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 5/7
Longest path algorithm
Compute indegree of each vertex

0 1

2
4
3

5 6 7

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 6/7
Longest path algorithm
Compute indegree of each vertex Indegree, Longest path
Scan each column of the adjacency 0 0 0 0
matrix 0 1
Initialize textlongest − path − to to 0
for all vertices 2
2 1
0 4
0 1 0
3
2 1 4
5 6 7
0 0 0

Topological order
Longest path to

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 6/7
Longest path algorithm
Compute indegree of each vertex Indegree, Longest path
Scan each column of the adjacency 0 0
matrix 0
Initialize textlongest − path − to to 0
for all vertices 2
2 1
List a vertex with indegree 0 and 0 4
remove it from the DAG 0 1 0
3
2 1 4
5 6 7
0 0 0

Topological order 1
Longest path to 0

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 6/7
Longest path algorithm
Compute indegree of each vertex Indegree, Longest path
Scan each column of the adjacency 0 0
matrix 0
Initialize textlongest − path − to to 0
for all vertices 1
2 1
List a vertex with indegree 0 and 1 4
remove it from the DAG 0 1 0
3
Update indegrees, longest path 2 1 3
5 6 7
0 0 1

Topological order 1
Longest path to 0

Initialize textlongest − path − to to 0

for all vertices 1
2 1
List a vertex with indegree 0 and 1 4
remove it from the DAG 0 1 0
3
Update indegrees, longest path 2 1 3
5 6 7
Repeat till all vertices are listed 0 0 1

Topological order 1 0
Longest path to 0 0

Initialize textlongest − path − to to 0

for all vertices 0
2 0
List a vertex with indegree 0 and 1 4
remove it from the DAG 1 0 1
3
Update indegrees, longest path 2 1 3
5 6 7
Repeat till all vertices are listed 0 0 1

Topological order 1 0
Longest path to 0 0

Initialize textlongest − path − to to 0

for all vertices 0
2 0
List a vertex with indegree 0 and 1 4
remove it from the DAG 1

Update indegrees, longest path 2 1 3

5 6 7
Repeat till all vertices are listed 0 0 1

Topological order 1 0 3
Longest path to 0 0 1

Initialize textlongest − path − to to 0

for all vertices 0
2 0
List a vertex with indegree 0 and 1 4
remove it from the DAG 1

Update indegrees, longest path 1 1 2

5 6 7
Repeat till all vertices are listed 2 0 2

Topological order 1 0 3
Longest path to 0 0 1

Initialize textlongest − path − to to 0

for all vertices
0
List a vertex with indegree 0 and 4
remove it from the DAG 1

Update indegrees, longest path 1 1 2

5 6 7
Repeat till all vertices are listed 2 0 2

Topological order 1 0 3 2
Longest path to 0 0 1 1

Initialize textlongest − path − to to 0

for all vertices
0
List a vertex with indegree 0 and 4
remove it from the DAG 1

Update indegrees, longest path 0 1 2

5 6 7
Repeat till all vertices are listed 2 0 2

Topological order 1 0 3 2
Longest path to 0 0 1 1

Initialize textlongest − path − to to 0

for all vertices
0
List a vertex with indegree 0 and 4
remove it from the DAG 1

Update indegrees, longest path 1 2

6 7
Repeat till all vertices are listed 0 2

Topological order 1 0 3 2 5
Longest path to 0 0 1 1 2

Initialize textlongest − path − to to 0

for all vertices
0
List a vertex with indegree 0 and 4
remove it from the DAG 1

Update indegrees, longest path 0 2

6 7
Repeat till all vertices are listed 3 2

Topological order 1 0 3 2 5
Longest path to 0 0 1 1 2

Initialize textlongest − path − to to 0

for all vertices
0
List a vertex with indegree 0 and 4
remove it from the DAG 1

Update indegrees, longest path 2

7
Repeat till all vertices are listed 2

Topological order 1 0 3 2 5 6
Longest path to 0 0 1 1 2 3

Initialize textlongest − path − to to 0

for all vertices
0
List a vertex with indegree 0 and 4
remove it from the DAG 1

Update indegrees, longest path 1

7
Repeat till all vertices are listed 4

Topological order 1 0 3 2 5 6
Longest path to 0 0 1 1 2 3

Initialize textlongest − path − to to 0

for all vertices
List a vertex with indegree 0 and
remove it from the DAG
Update indegrees, longest path 1
7
Repeat till all vertices are listed 4

Topological order 1 0 3 2 5 6 4
Longest path to 0 0 1 1 2 3 1

Initialize textlongest − path − to to 0

for all vertices
List a vertex with indegree 0 and
remove it from the DAG
Update indegrees, longest path 0
7
Repeat till all vertices are listed 4

Topological order 1 0 3 2 5 6 4
Longest path to 0 0 1 1 2 3 1

Initialize textlongest − path − to to 0

for all vertices
List a vertex with indegree 0 and
remove it from the DAG
Update indegrees, longest path
Repeat till all vertices are listed

Topological order 1 0 3 2 5 6 4 7
Longest path to 0 0 1 1 2 3 1 4

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 6/7
Longest path algorithm
Compute indegree of each vertex
Scan each column of the adjacency
matrix 0 1
Initialize textlongest − path − to to 0
for all vertices
2
List a vertex with indegree 0 and 4
remove it from the DAG 3
Update indegrees, longest path
5 6 7
Repeat till all vertices are listed

Topological order 1 0 3 2 5 6 4 7
Longest path to 0 0 1 1 2 3 1 4

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 6/7
Summary

Directed acyclic graphs are a natural way to represent dependencies

Topological sort gives a feasible schedule that represents dependencies
In parallel with topological sort, we can compute the longest path

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 7/7
Summary

Directed acyclic graphs are a natural way to represent dependencies

Topological sort gives a feasible schedule that represents dependencies
In parallel with topological sort, we can compute the longest path
Notion of longest path makes sense even for graphs with cycles
No repeated vertices in a path, so path has at most n − 1 edges

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 7/7
Summary

Directed acyclic graphs are a natural way to represent dependencies

However, computing longest paths in arbitrary graphs is much harder than for DAGs
No better strategy known than exhaustively enumerating paths

Madhavan Mukund Longest Paths in DAGs Mathematics for Data Science 1, Week 11 7/7
Transitive Closure

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 11
Transitive closure of a relation

Let R ⊆ S × S be a relation on a set S

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 2/8
Transitive closure of a relation

Let R ⊆ S × S be a relation on a set S

For instance, S is a set of people, and (p, q) ∈ R if p is a parent of q

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 2/8
Transitive closure of a relation

Let R ⊆ S × S be a relation on a set S

For instance, S is a set of people, and (p, q) ∈ R if p is a parent of q
Can compute the ancestor relation from the parent relation

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 2/8
Transitive closure of a relation

Let R ⊆ S × S be a relation on a set S

For instance, S is a set of people, and (p, q) ∈ R if p is a parent of q
Can compute the ancestor relation from the parent relation
p is an ancestor of q if we can find a sequence of people r0 , r1 , . . . , rn such that
p = r0
For each i ∈ {0, 1, . . . , n − 1}, (ri , ri+1 ) ∈ R
q = rn

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 2/8
Transitive closure of a relation

Let R ⊆ S × S be a relation on a set S

This is called the transitive closure of R, written R +

R + ⊆ S × S is also a relation
R + is derived from R ⊆ S × S

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 2/8
Computing transitive closure

Represent R ⊆ S × S as a (directed) graph

G = (V , E ) v0
V =S
v1
v2
(u, v ) ∈ E if and only if (u, v ) ∈ R
v3
v4 v6
v5

v8
v7

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 3/8
Computing transitive closure

Represent R ⊆ S × S as a (directed) graph

G = (V , E ) v0
V =S
v1
v2
(u, v ) ∈ E if and only if (u, v ) ∈ R
v3
(u, v ) ∈ R+ if and only if there is a path from v4 v6
u to v in the graph v5

v8
v7

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 3/8
Computing transitive closure

Represent R ⊆ S × S as a (directed) graph

G = (V , E ) v0
V =S
v1
v2
(u, v ) ∈ E if and only if (u, v ) ∈ R
v3
(u, v ) ∈ R+ if and only if there is a path from v4 v6
u to v in the graph v5
We know how to compute reachability in
graphs v8
v7
BFS, DFS

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 3/8
Computing transitive closure

Represent R ⊆ S × S as a (directed) graph

Perform BFS/DFS from all verties to compute v9

R+

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 3/8
Using the adjacency matrix
Another strategy

v0
v1
v2
v3
v4 v6
v5

v8
v7

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 4/8
Using the adjacency matrix
Another strategy
Consider the adjacency matrix A for G v0
0 1 2 3 4 5 6 7 8 9 v1
0 0 1 0 0 1 0 0 0 0 0
v2
1 0 0 1 0 0 0 0 0 0 0
2 1 0 0 0 0 0 0 0 0 0 v3
3 0 0 0 0 1 0 1 0 0 0 v4 v6
4 1 0 0 1 0 0 0 1 0 0 v5
5 0 0 0 1 0 0 0 1 0 0
6 0 0 0 0 0 1 0 0 0 0
7 0 0 0 0 1 0 0 0 1 0 v8
8 0 0 0 0 0 1 0 0 0 1 v7
9 0 0 0 0 0 0 0 0 1 0
v9

A[i, j] = 1 — path of length 1 from i to j v9

Want A+ [i, j] = 1 — path of length ≥ 1 from i to j

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 4/8
Paths of length 2

Path of length 2 from i to j passes through intermediate k

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 5/8
Paths of length 2

Path of length 2 from i to j passes through intermediate k

A2 [i, j] = 1 if there is some k such that A[i, k] = 1 and A[k, j] = 1

A A2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
0 0 1 0 0 1 0 0 0 0 0 0 1 0 1 1 0 0 0 1 0 0
1 0 0 1 0 0 0 0 0 0 0 1
2 1 0 0 0 0 0 0 0 0 0 2
3 0 0 0 0 1 0 1 0 0 0 3
4 1 0 0 1 0 0 0 1 0 0 4
5 0 0 0 1 0 0 0 1 0 0 5
6 0 0 0 0 0 1 0 0 0 0 6
7 0 0 0 0 1 0 0 0 1 0 7
8 0 0 0 0 0 1 0 0 0 1 8
9 0 0 0 0 0 0 0 0 1 0 9

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 5/8
Paths of length 2

Path of length 2 from i to j passes through intermediate k

A2 [i, j] = 1 if there is some k such that A[i, k] = 1 and A[k, j] = 1

A A2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
0 0 1 0 0 1 0 0 0 0 0 0 1 0 1 1 0 0 0 1 0 0
1 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0
2 1 0 0 0 0 0 0 0 0 0 2
3 0 0 0 0 1 0 1 0 0 0 3
4 1 0 0 1 0 0 0 1 0 0 4
5 0 0 0 1 0 0 0 1 0 0 5
6 0 0 0 0 0 1 0 0 0 0 6
7 0 0 0 0 1 0 0 0 1 0 7
8 0 0 0 0 0 1 0 0 0 1 8
9 0 0 0 0 0 0 0 0 1 0 9

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 5/8
Paths of length 2

Path of length 2 from i to j passes through intermediate k

A2 [i, j] = 1 if there is some k such that A[i, k] = 1 and A[k, j] = 1

A A2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
0 0 1 0 0 1 0 0 0 0 0 0 1 0 1 1 0 0 0 1 0 0
1 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0
2 1 0 0 0 0 0 0 0 0 0 2 0 1 0 1 0 0 0 0 0 0
3 0 0 0 0 1 0 1 0 0 0 3
4 1 0 0 1 0 0 0 1 0 0 4
5 0 0 0 1 0 0 0 1 0 0 5
6 0 0 0 0 0 1 0 0 0 0 6
7 0 0 0 0 1 0 0 0 1 0 7
8 0 0 0 0 0 1 0 0 0 1 8
9 0 0 0 0 0 0 0 0 1 0 9

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 5/8
Paths of length 2

Path of length 2 from i to j passes through intermediate k

A2 [i, j] = 1 if there is some k such that A[i, k] = 1 and A[k, j] = 1

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 5/8
Paths of length 2

Path of length 2 from i to j passes through intermediate k

A2 [i, j] = 1 if there is some k such that A[i, k] = 1 and A[k, j] = 1

A A2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
0 0 1 0 0 1 0 0 0 0 0 0 1 0 1 1 0 0 0 1 0 0
1 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0
2 1 0 0 0 0 0 0 0 0 0 2 0 1 0 1 0 0 0 0 0 0
3 0 0 0 0 1 0 1 0 0 0 3 1 0 0 0 0 1 0 1 0 0
4 1 0 0 1 0 0 0 1 0 0 4 0 1 0 0 0 0 1 0 1 0
5 0 0 0 1 0 0 0 1 0 0 5 0 0 0 0 1 0 1 0 1 0
6 0 0 0 0 0 1 0 0 0 0 6 0 0 0 1 0 0 0 1 0 0
7 0 0 0 0 1 0 0 0 1 0 7 0 0 0 0 0 1 0 0 0 1
8 0 0 0 0 0 1 0 0 0 1 8 0 0 1 0 0 0 0 1 0 0
9 0 0 0 0 0 0 0 0 1 0 9 0 0 0 0 0 1 0 0 0 0

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 5/8
Paths of length 3 and beyond

Extend path of length 2 from i to k by path of length 1 from k to j

A3 [i, j] = 1 if there is some k such that A2 [i, k] = 1 and A[k, j] = 1

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 6/8
Paths of length 3 and beyond

Extend path of length 2 from i to k by path of length 1 from k to j

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 6/8
Paths of length 3 and beyond

Extend path of length 2 from i to k by path of length 1 from k to j

A3 [i, j] = 1 if there is some k such that A2 [i, k] = 1 and A[k, j] = 1
Extend path of length 3 from i to k by path of length 1 from k to j
A4 [i, j] = 1 if there is some k such that A3 [i, k] = 1 and A[k, j] = 1
...
Extend path of length ` from i to k by path of length 1 from k to j
A`+1 [i, j] = 1 if there is some k such that A` [i, k] = 1 and A[k, j] = 1

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 6/8
Paths of length 3 and beyond

Extend path of length 2 from i to k by path of length 1 from k to j

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 6/8
Paths of length 3 and beyond

Extend path of length 2 from i to k by path of length 1 from k to j

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 6/8
Back to transitive closure

(i, j) ∈ R + if there is a path from i to j in G

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 7/8
Back to transitive closure

(i, j) ∈ R + if there is a path from i to j in G

Length of path is bounded by n − 1

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 7/8
Back to transitive closure

(i, j) ∈ R + if there is a path from i to j in G

Length of path is bounded by n − 1
Combine information in A = A1 , A2 , . . . , An−1 about paths of length 1 to n − 1

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 7/8
Back to transitive closure

(i, j) ∈ R + if there is a path from i to j in G

Length of path is bounded by n − 1
Combine information in A = A1 , A2 , . . . , An−1 about paths of length 1 to n − 1
A+ [i, j] = max{A` [i, j] | 1 ≤ ` < n}

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 7/8
Back to transitive closure

(i, j) ∈ R + if there is a path from i to j in G

Length of path is bounded by n − 1
Combine information in A = A1 , A2 , . . . , An−1 about paths of length 1 to n − 1
A+ [i, j] = max{A` [i, j] | 1 ≤ ` < n}
Each A` [i, j] is either 0 or 1

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 7/8
Back to transitive closure

(i, j) ∈ R + if there is a path from i to j in G

Length of path is bounded by n − 1
Combine information in A = A1 , A2 , . . . , An−1 about paths of length 1 to n − 1
A+ [i, j] = max{A` [i, j] | 1 ≤ ` < n}
Each A` [i, j] is either 0 or 1
max{A` [i, j] | 1 ≤ ` < n} is 1 if and only if A` [i, j] = 1 for some 1 ≤ ` < n

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 7/8
Back to transitive closure

(i, j) ∈ R + if there is a path from i to j in G

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 7/8
Back to transitive closure

(i, j) ∈ R + if there is a path from i to j in G

This calculation can be described directly using matrix multiplication

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 7/8
Back to transitive closure

(i, j) ∈ R + if there is a path from i to j in G

This calculation can be described directly using matrix multiplication

A2 = A × A, A3 = A2 × A, . . . , A`+1 = A` × A

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 7/8
Summary

The transitive closure R+ of a relation R connects pairs of elements related by a

sequence of intermediate elements

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 8/8
Summary

The transitive closure R+ of a relation R connects pairs of elements related by a

sequence of intermediate elements
A typical example is the ancestor relation derived from the parent relation

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 8/8
Summary

The transitive closure R+ of a relation R connects pairs of elements related by a

sequence of intermediate elements
A typical example is the ancestor relation derived from the parent relation
If we represent a relation as a graph, transitive closure corresponds to reachability

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 8/8
Summary

The transitive closure R+ of a relation R connects pairs of elements related by a

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 8/8
Summary

The transitive closure R+ of a relation R connects pairs of elements related by a

sequence of intermediate elements
A typical example is the ancestor relation derived from the parent relation
If we represent a relation as a graph, transitive closure corresponds to reachability
Reachability between all pairs of vertices can be checked using repeated BFS/DFS
starting from each vertex
Alternatively, we can perform repeated matrix multiplication on the adjacency
matrix A, observing that the length of a path is at most n − 1

Madhavan Mukund Transitive Closure Mathematics for Data Science 1, Week 11 8/8
Matrix Multiplication

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 11
Matrices
A matrix is a two dimensional table
r × c matrix — r rows, c columns

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 2/9
Matrices
A matrix is a two dimensional table
r × c matrix — r rows, c columns
Assume rows are numbered {0, 1, . . . , r −1}, columns are numbered {0, 1, . . . , c−1}

Example: Freight traffic by rail between major cities: Bangalore, Chennai, Delhi,
Hyderabad, Kolkata, Mumbai

Number the cities:

0-Bangalore, 1-Chennai, 2-Delhi,
3-Hyderabad, 4-Kolkata,
5-Mumbai

Example: Freight traffic by rail between major cities: Bangalore, Chennai, Delhi,
Hyderabad, Kolkata, Mumbai

Number the cities: 0 1 2 3 4 5

0-Bangalore, 1-Chennai, 2-Delhi, 0 0 694 828 384 247 479
3-Hyderabad, 4-Kolkata, 1 642 0 919 575 402 673
5-Mumbai 2 768 734 0 231 595 540
3 731 606 156 0 351 804
Represent freight volume for a
4 825 607 316 490 0 998
month as a 6 × 6 matrix
5 196 580 339 588 394 0
Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 2/9
Adding matrices

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 3/9
Adding matrices

Suppose we have freight volumes for first and second half of financial year.
April–September October–March
0 1 2 3 4 5 0 1 2 3 4 5
0 0 694 828 384 247 479 0 0 851 626 280 399 365
1 642 0 919 575 402 673 1 544 0 479 269 432 933
2 768 734 0 231 595 540 2 867 804 0 681 326 398
3 731 606 156 0 351 804 3 727 976 418 0 667 294
4 825 607 316 490 0 998 4 894 390 247 547 0 314
5 196 580 339 588 394 0 5 914 147 574 859 524 0
How do we compute the freight volumes for the entire financial year?

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 3/9
Adding matrices

For a matrix M, M[i, j] is the entry in row

i, column j

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 4/9
Adding matrices
0 1 2 3 4 5
0 0 694 828 384 247 479
For a matrix M, M[i, j] is the entry in row 1 642 0 919 575 402 673
i, column j A 2 768 734 0 231 595 540
3 731 606 156 0 351 804
4 825 607 316 490 0 998
Let A and B represent the volumes in the 5 196 580 339 588 394 0
two half-years 0 1 2 3 4 5
0 0 851 626 280 399 365
Let C represent the annual volume 1 544 0 479 269 432 933
B 2 867 804 0 681 326 398
For each i,j, C [i, j] = A[i, j] + B[i, j] 3 727 976 418 0 667 294
4 894 390 247 547 0 314
5 914 147 574 859 524 0
0 1 2 3 4 5
0 0 1545 1454 664 646 844
1 1186 0 1398 844 834 1606
C 2 1635 1538 0 912 921 938
3 1458 1582 574 0 1018 1098
4 1719 997 563 1037 0 1312
5 1110 727 913 1447 918 0

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 4/9
Adding matrices
0 1 2 3 4 5
0 0 694 828 384 247 479
For a matrix M, M[i, j] is the entry in row 1 642 0 919 575 402 673
i, column j A 2 768 734 0 231 595 540
3 731 606 156 0 351 804
4 825 607 316 490 0 998
Let A and B represent the volumes in the 5 196 580 339 588 394 0
two half-years 0 1 2 3 4 5
0 0 851 626 280 399 365
Let C represent the annual volume 1 544 0 479 269 432 933
B 2 867 804 0 681 326 398
For each i,j, C [i, j] = A[i, j] + B[i, j] 3 727 976 418 0 667 294
4 894 390 247 547 0 314
5 914 147 574 859 524 0
More concisely, C = A + B — matrix
0 1 2 3 4 5
addition 0 0 1545 1454 664 646 844
1 1186 0 1398 844 834 1606
C 2 1635 1538 0 912 921 938
3 1458 1582 574 0 1018 1098
4 1719 997 563 1037 0 1312
5 1110 727 913 1447 918 0

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 4/9
Multiplying matrices
Can we multiply matrices the same way?

0 1 2 3
0 0 1 2 8
B 1 5 0 4 2
2 6 0 0 1
3 2 6 4 0

0 1 2 3
0 0 6 16 24
C 1 20 0 36 14
2 36 0 0 1
3 2 0 20 0

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 5/9
Multiplying matrices
0 1 2 3
Can we multiply matrices the same way? 0 0 6 8 3
A 1 4 0 9 7
For each i,j, C [i, j] = A[i, j] × B[i, j]
2 6 3 0 1
3 1 0 5 0
This turns out to be not very useful
Instead, we compute C [i, j] in a more 0 1 2 3
0 0 1 2 8
complicated way
B 1 5 0 4 2
Assume r = c = n. Let 2 6 0 0 1
3 2 6 4 0
C [i, j] = A[i, 0] · B[0, j]+
A[i, 1] · B[1, j]+
···+
A[i, n − 1] · B[n − 1, j]
For instance,
C [1, 3] = 4 · 8 + 0 · 2 + 9 · 1 + 7 · 0 = 41
Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 5/9
Multiplying matrices
0 1 2 3
Can we multiply matrices the same way? 0 0 6 8 3
A 1 4 0 9 7
For each i,j, C [i, j] = A[i, j] × B[i, j]
2 6 3 0 1
3 1 0 5 0
This turns out to be not very useful
Instead, we compute C [i, j] in a more 0 1 2 3
0 0 1 2 8
complicated way
B 1 5 0 4 2
Assume r = c = n. Let 2 6 0 0 1
3 2 6 4 0
C [i, j] = A[i, 0] · B[0, j]+
A[i, 1] · B[1, j]+ 0 1 2 3
···+ 0 84 18 36 20
C 1 68 46 36 41
A[i, n − 1] · B[n − 1, j]
2 17 12 28 54
For instance, 3 30 1 2 13
C [1, 3] = 4 · 8 + 0 · 2 + 9 · 1 + 7 · 0 = 41
Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 5/9
Multiplying matrices
0 1 2 3
Matrix product: C = A × B 0 0 6 8 3
A 1 4 0 9 7
2 6 3 0 1
3 1 0 5 0

0 1 2 3
0 0 1 2 8
B 1 5 0 4 2
2 6 0 0 1
3 2 6 4 0

0 1 2 3
0 84 18 36 20
C 1 68 46 36 41
2 17 12 28 54
3 30 1 2 13

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 6/9
Multiplying matrices
0 1 2 3
Matrix product: C = A × B 0 0 6 8 3
A 1 4 0 9 7
For r = c = n,
2 6 3 0 1
C [i, j] = A[i, 0] · B[0, j]+ 3 1 0 5 0
A[i, 1] · B[1, j]+
···+ 0 1 2 3
0 0 1 2 8
A[i, n − 1] · B[n − 1, j] B 1 5 0 4 2
n−1
2 6 0 0 1
X 3 2 6 4 0
More concisely, C [i, j] = A[i, k] · B[k, j]
k=0 0 1 2 3
Don’t require both A and B to be n × n 0 84 18 36 20
C 1 68 46 36 41
Each row entry of A must have a matching 2 17 12 28 54
column entry in B 3 30 1 2 13
If A is m × n and B is n × p, A × B is m × p
Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 6/9
Multiplying matrices
Matrix product: C = A × B 0 1 2 3
0 0 6 8 3
For r = c = n, A 1 4 0 9 7
2 6 3 0 1
C [i, j] = A[i, 0] · B[0, j]+
A[i, 1] · B[1, j]+ 0 1 2
···+ 0 0 1 2
A[i, n − 1] · B[n − 1, j] B 1 5 0 4
2 6 0 0
n−1
X 3 2 6 4
More concisely, C [i, j] = A[i, k] · B[k, j]
k=0
0 1 2
0 84 18 36
Don’t require both A and B to be n × n C 1 68 46 36
2 17 12 28
Each row entry of A must have a matching
column entry in B
If A is m × n and B is n × p, A × B is m × p
Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 6/9
Transitive closure
0 1 2 3 4 5
A is an adjacency matrix 0 0 1 0 0 1 0
A[i, j] = 1 if and only if there is a 1 0 0 1 0 0 0
direct edge (path of length 1) from i 2 1 0 0 0 0 0
to j 3 0 0 0 0 1 0
4 1 0 0 1 0 0
5 0 0 0 1 0 0

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 7/9
Transitive closure
0 1 2 3 4 5
A is an adjacency matrix 0 0 1 0 0 1 0
A[i, j] = 1 if and only if there is a 1 0 0 1 0 0 0
direct edge (path of length 1) from i 2 1 0 0 0 0 0
to j 3 0 0 0 0 1 0
4 1 0 0 1 0 0
A2 represents paths of length 2 5 0 0 0 1 0 0
A2 [i, j] = 1 if, for some k, A[i, k] = 1
and A[k, j] = 1

Algebra of boolean values

True is 1, False is 0

Algebra of boolean values

True is 1, False is 0
Logical or is represented by +:
0 + 0 = 0, 0 + 1 = 1 + 0 = 1 + 1 = 1

Algebra of boolean values

True is 1, False is 0
Logical or is represented by +:
0 + 0 = 0, 0 + 1 = 1 + 0 = 1 + 1 = 1
Logical and is represented by ×:
1 × 1 = 1, 0 × 0 = 0 × 1 = 1 × 0 = 0
Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 7/9
Transitive closure
0 1 2 3 4 5
A is an adjacency matrix 0 0 1 0 0 1 0
A[i, j] = 1 if and only if there is a 1 0 0 1 0 0 0
direct edge (path of length 1) from i 2 1 0 0 0 0 0
to j 3 0 0 0 0 1 0
4 1 0 0 1 0 0
A2 represents paths of length 2 5 0 0 0 1 0 0
A2 [i, j] = 1 if, for some k, A[i, k] = 1
and A[k, j] = 1 A2 [i, j] = (A[i, 0] and A[0, j]) or
(A[i, 1] and A[1, j]) or
Algebra of boolean values
···
True is 1, False is 0 (A[i, n−1] and A[n−1, j])
Logical or is represented by +:
0 + 0 = 0, 0 + 1 = 1 + 0 = 1 + 1 = 1
Logical and is represented by ×:
1 × 1 = 1, 0 × 0 = 0 × 1 = 1 × 0 = 0
Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 7/9
Transitive closure
0 1 2 3 4 5
A is an adjacency matrix 0 0 1 0 0 1 0
A[i, j] = 1 if and only if there is a 1 0 0 1 0 0 0
direct edge (path of length 1) from i 2 1 0 0 0 0 0
to j 3 0 0 0 0 1 0
4 1 0 0 1 0 0
A2 represents paths of length 2 5 0 0 0 1 0 0
A2 [i, j] = 1 if, for some k, A[i, k] = 1
and A[k, j] = 1 A2 [i, j] = (A[i, 0] and A[0, j]) or
(A[i, 1] and A[1, j]) or
Algebra of boolean values
···
True is 1, False is 0 (A[i, n−1] and A[n−1, j])
Logical or is represented by +:
0 + 0 = 0, 0 + 1 = 1 + 0 = 1 + 1 = 1
A2 [i, j] = A[i, 0] × A[0, j]+
A[i, 1] × A[1, j]+
Logical and is represented by ×:
···+
1 × 1 = 1, 0 × 0 = 0 × 1 = 1 × 0 = 0
A[i, n−1] × A[n−1, j]
Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 7/9
Transitive closure

A2 [i, j] = (A[i, 0] and A[0, j]) or

(A[i, 1] and A[1, j]) or
···
(A[i, n−1] and A[n−1, j])
A2 [i, j] = A[i, 0] × A[0, j]+
A[i, 1] × A[1, j]+
···+
A[i, n−1] × A[n−1, j]

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 8/9
Transitive closure

A2 [i, j] = (A[i, 0] and A[0, j]) or

(A[i, 1] and A[1, j]) or
···
(A[i, n−1] and A[n−1, j])
A2 [i, j] = A[i, 0] × A[0, j]+
A[i, 1] × A[1, j]+
···+
A[i, n−1] × A[n−1, j]
n−1
X
2
So A [i, j] = A[i, k] × A[k, j]
k=0

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 8/9
Transitive closure

A2 [i, j] = (A[i, 0] and A[0, j]) or

(A[i, 1] and A[1, j]) or
···
(A[i, n−1] and A[n−1, j])
A2 [i, j] = A[i, 0] × A[0, j]+
A[i, 1] × A[1, j]+
···+
A[i, n−1] × A[n−1, j]
n−1
X
2
So A [i, j] = A[i, k] × A[k, j]
k=0

In other words, A2 = A × A

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 8/9
Transitive closure

A2 [i, j] = (A[i, 0] and A[0, j]) or Likewise, A3 [i, j] = 1 if, for some k,
(A[i, 1] and A[1, j]) or A2 [i, k] = 1 and A[k, j] = 1
··· Paths of length 3
(A[i, n−1] and A[n−1, j])
A2 [i, j] = A[i, 0] × A[0, j]+
A[i, 1] × A[1, j]+
···+
A[i, n−1] × A[n−1, j]
n−1
X
2
So A [i, j] = A[i, k] × A[k, j]
k=0

In other words, A2 = A × A

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 8/9
Transitive closure

A2 [i, j] = (A[i, 0] and A[0, j]) or Likewise, A3 [i, j] = 1 if, for some k,
(A[i, 1] and A[1, j]) or A2 [i, k] = 1 and A[k, j] = 1
··· Paths of length 3
(A[i, n−1] and A[n−1, j])
A3 = A2 × A
A2 [i, j] = A[i, 0] × A[0, j]+
A[i, 1] × A[1, j]+
···+
A[i, n−1] × A[n−1, j]
n−1
X
2
So A [i, j] = A[i, k] × A[k, j]
k=0

In other words, A2 = A × A

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 8/9
Transitive closure

In other words, A2 = A × A

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 8/9
Transitive closure

In other words, A2 = A × A

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 8/9
Transitive closure

A2 [i, j] = (A[i, 0] and A[0, j]) or Likewise, A3 [i, j] = 1 if, for some k,
(A[i, 1] and A[1, j]) or A2 [i, k] = 1 and A[k, j] = 1
··· Paths of length 3
(A[i, n−1] and A[n−1, j])
A3 = A2 × A = A × A2
A2 [i, j] = A[i, 0] × A[0, j]+
A[i, 1] × A[1, j]+ In general, A`+1 = A` × A = A × A`
···+ Paths of length `+1
A[i, n−1] × A[n−1, j]
Finally, A+ = A + A2 + · · · + An−1
n−1
X A+ [i, j] = 1 if i,j connected by
So A2 [i, j] = A[i, k] × A[k, j]
k=0
path of length 1 or
path of length 2 or
In other words, A2 = A × A
· · · or
path of length n−1
Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 8/9
Summary
Matrix addition is defined elementwise
C [i, j] = A[i, j] + B[i, j]
A, B, C have same dimensions r × c

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 9/9
Summary
Matrix addition is defined elementwise
C [i, j] = A[i, j] + B[i, j]
A, B, C have same dimensions r × c

Matrix multiplication is a more complicated operation

Let A be m × n and B be n × p
n−1
X
C [i, j] = A[i, k] × B[k, j]
k=0
C has dimensions m × p

Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 9/9
Summary
Matrix addition is defined elementwise
C [i, j] = A[i, j] + B[i, j]
A, B, C have same dimensions r × c

Matrix multiplication is a more complicated operation

Let A be m × n and B be n × p
n−1
X
C [i, j] = A[i, k] × B[k, j]
k=0
C has dimensions m × p

Using Boolean algebra, describe transitive closure using matrix multiplication

A, adjacency matrix, paths of length 1
A`+1 = A` × A, paths of length `
Transitive closure, A+ = A + A2 + · · · An−1
Madhavan Mukund Matrix Multiplication Mathematics for Data Science 1, Week 11 9/9
Shortest Paths in Weighted Graphs

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 12
Shortest paths

Recall that BFS explores a graph level 2 3

by level
0 1 4 5

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 2/6
Shortest paths

Recall that BFS explores a graph level 2 3

by level
BFS computes shortest path, in terms 0 1 4 5
of number of edges, to every reachable
vertex 6

G = (V , E ), W : E → R

G = (V , E ), W : E → R
Adjacency matrix: record the weight
wherever there is an edge, 0 if no edge

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 2/6
Shortest paths
70
Recall that BFS explores a graph level 2 3
by level 80 6
50
BFS computes shortest path, in terms 0 1 4 5
10 20
of number of edges, to every reachable 5 10
vertex 6
May assign values to edges
0 1 2 3 4 5 6
Cost, time, distance, . . . 0 0 10 80 0 0 0 0
Weighted graph 1 10 0 6 0 20 0 0
2 80 6 0 70 0 0 0
G = (V , E ), W : E → R 3 0 0 70 0 0 0 0
4 0 20 0 0 0 50 5
Adjacency matrix: record the weight 5 0 0 0 0 50 0 10
wherever there is an edge, 0 if no edge 6 0 0 0 0 5 10 0

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 2/6
Shortest paths in weighted graphs

BFS computes shortest path, in terms 70

2 3
of number of edges, to every reachable
80 6
vertex
50
0 1 4 5
10 20
5 10
6

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 3/6
Shortest paths in weighted graphs

BFS computes shortest path, in terms 70

2 3
of number of edges, to every reachable
80 6
vertex
50
In a weighted graph, add up the weights 0 1 4 5
10 20
along a path 5 10
6

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 3/6
Shortest paths in weighted graphs

BFS computes shortest path, in terms 70

2 3
of number of edges, to every reachable
80 6
vertex
50
In a weighted graph, add up the weights 0 1 4 5
10 20
along a path 5 10
Weighted shortest path need not have 6
minimum number of edges
Shortest path from 0 to 2 is via 1

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 3/6
Shortest path problems

Single source shortest paths

Find shortest paths from a fixed vertex
to every other vertex

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 4/6
Shortest path problems

Single source shortest paths

Find shortest paths from a fixed vertex
to every other vertex
Transport finished product from factory
(single source) to all retail outlets

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 4/6
Shortest path problems

Single source shortest paths

Find shortest paths from a fixed vertex
to every other vertex
Transport finished product from factory
(single source) to all retail outlets
Courier company delivers items from
distribution centre (single source) to
addressees

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 4/6
Shortest path problems

Single source shortest paths All pairs shortest paths

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 4/6
Shortest path problems

Single source shortest paths All pairs shortest paths

Find shortest paths from a fixed vertex Find shortest paths between every pair
to every other vertex of vertices i and j
Transport finished product from factory
(single source) to all retail outlets
Courier company delivers items from
distribution centre (single source) to
addressees

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 4/6
Shortest path problems

Single source shortest paths All pairs shortest paths

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 4/6
Negative edge weights

Negative edge weights

Can negative edge weights be

meaningful?

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 5/6
Negative edge weights

Negative edge weights

Can negative edge weights be

meaningful?
Taxi driver trying to head home at the
end of the day

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 5/6
Negative edge weights

Negative edge weights

Can negative edge weights be

meaningful?
Taxi driver trying to head home at the
end of the day
Roads with few customers, drive
empty (positive weight)

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 5/6
Negative edge weights

Negative edge weights

Can negative edge weights be

meaningful?
Taxi driver trying to head home at the
end of the day
Roads with few customers, drive
empty (positive weight)
Roads with many customers, make
profit (negative weight)

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 5/6
Negative edge weights

Negative edge weights

Can negative edge weights be

meaningful?
Taxi driver trying to head home at the
end of the day
Roads with few customers, drive
empty (positive weight)
Roads with many customers, make
profit (negative weight)
Find a route toward home that
minimizes the cost

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 5/6
Negative edge weights

Negative edge weights Negative cycles

Can negative edge weights be A negative cycle is one whose weight is

meaningful? negative
Sum of the weights of edges that
Taxi driver trying to head home at the make up the cycle
end of the day
Roads with few customers, drive
empty (positive weight)
Roads with many customers, make
profit (negative weight)
Find a route toward home that
minimizes the cost

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 5/6
Negative edge weights

Negative edge weights Negative cycles

Can negative edge weights be A negative cycle is one whose weight is

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 5/6
Negative edge weights

Negative edge weights Negative cycles

Can negative edge weights be A negative cycle is one whose weight is

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 5/6
Negative edge weights

Negative edge weights Negative cycles

Can negative edge weights be A negative cycle is one whose weight is

meaningful? negative
Sum of the weights of edges that
Taxi driver trying to head home at the make up the cycle
end of the day
Roads with few customers, drive By repeatedly traversing a negative
empty (positive weight) cycle, total cost keeps decreasing
Roads with many customers, make If a graph has a negative cycle, shortest
profit (negative weight) paths are not defined
Find a route toward home that
minimizes the cost Without negative cycles, we can
compute shortest paths even if some
weights are negative
Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 5/6
Summary

In a weighted graph, each edge has a cost

Entries in adjacency matrix capture edge weights

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 6/6
Summary

In a weighted graph, each edge has a cost

Entries in adjacency matrix capture edge weights

Length of a path is the sum of the weights

Shortest path in a weighted graph need not be minimum in terms of number of edges

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 6/6
Summary

In a weighted graph, each edge has a cost

Entries in adjacency matrix capture edge weights

Length of a path is the sum of the weights

Shortest path in a weighted graph need not be minimum in terms of number of edges

Different shortest path problems

Single source — from one designated vertex to all others
All-pairs — between every pair of vertices

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 6/6
Summary

In a weighted graph, each edge has a cost

Entries in adjacency matrix capture edge weights

Length of a path is the sum of the weights

Shortest path in a weighted graph need not be minimum in terms of number of edges

Different shortest path problems

Single source — from one designated vertex to all others
All-pairs — between every pair of vertices

Negative edge weights

Should not have negative cycles
Without negative cycles, shortest paths still well defined

Madhavan Mukund Shortest Paths in Weighted Graphs Mathematics for Data Science 1, Week 12 6/6
Single Source Shortest Paths

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 12
Single source shortest paths

Weighted graph: 70
2 3
G = (V , E ) 80 6
W :E →R 50
0 1 4 5
10 20
5 10
6

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 2/7
Single source shortest paths

Weighted graph: 70
2 3
G = (V , E ) 80 6
W :E →R 50
0 1 4 5
Single source shortest paths 10 20
5 10
Find shortest paths from a fixed vertex
to every other vertex
6

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 2/7
Single source shortest paths

Weighted graph: 70
2 3
G = (V , E ) 80 6
W :E →R 50
0 1 4 5
Single source shortest paths 10 20
5 10
Find shortest paths from a fixed vertex
to every other vertex
6

Assume, for now, that edge weights are

all non-negative

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 2/7
Single source shortest paths

Compute shortest paths from 0 to all

70
other vertices 2 3
80 6
50
0 1 4 5
10 20
5 10
6

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 3/7
Single source shortest paths

Compute shortest paths from 0 to all

70
other vertices 2 3
80 6
Imagine vertices are oil depots, edges 50
are pipelines 0 1 4 5
10 20
5 10
6

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 3/7
Single source shortest paths

Compute shortest paths from 0 to all

70
other vertices 2 3
80 6
Imagine vertices are oil depots, edges 50
are pipelines 0 1 4 5
10 20
Set fire to oil depot at vertex 0 5 10
6

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 3/7
Single source shortest paths

Compute shortest paths from 0 to all

70
other vertices 2 3
80 6
Imagine vertices are oil depots, edges 50
are pipelines 0 1 4 5
10 20
Set fire to oil depot at vertex 0 5 10
6
Fire travels at uniform speed along each
pipeline

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 3/7
Single source shortest paths

Compute shortest paths from 0 to all

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 3/7
Single source shortest paths

Compute shortest paths from 0 to all

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 3/7
Single source shortest paths

Compute shortest paths from 0 to all

70
other vertices 2 3
80 6
Imagine vertices are oil depots, edges 50
are pipelines 0 1 4 5
10 20
Set fire to oil depot at vertex 0 5 10
6
Fire travels at uniform speed along each
pipeline
First oil depot to catch fire after 0 is
nearest vertex
Next oil depot is second nearest vertex
...
Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 3/7
Single source shortest paths

Set fire to oil depot at

vertex 0 70
2 3
Fire travels at uniform
speed along each 80 6
pipeline
50
0 1 4 5
First oil depot to catch 10 20
fire after 0 is nearest
vertex 5 10
6
Next oil depot is
second nearest vertex
...

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 4/7
Single source shortest paths

Set fire to oil depot at

vertex 0 70
2 3
Fire travels at uniform
speed along each 80 6
pipeline
50
0 1 4 5
First oil depot to catch 10 20
fire after 0 is nearest t=0
vertex 5 10
6
Next oil depot is
second nearest vertex
...

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 4/7
Single source shortest paths

Set fire to oil depot at

vertex 0 70
2 3
Fire travels at uniform
speed along each 80 6
pipeline
50
0 1 4 5
First oil depot to catch 10 20
fire after 0 is nearest t=0 t = 10
vertex 5 10
6
Next oil depot is
second nearest vertex
...

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 4/7
Single source shortest paths

Set fire to oil depot at

t = 16
vertex 0 70
2 3
Fire travels at uniform
speed along each 80 6
pipeline
50
0 1 4 5
First oil depot to catch 10 20
fire after 0 is nearest t=0 t = 10
vertex 5 10
6
Next oil depot is
second nearest vertex
...

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 4/7
Single source shortest paths

Set fire to oil depot at

t = 16
vertex 0 70
2 3
Fire travels at uniform
speed along each 80 6
t = 30
pipeline
50
0 1 4 5
First oil depot to catch 10 20
fire after 0 is nearest t=0 t = 10
vertex 5 10
6
Next oil depot is
second nearest vertex
...

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 4/7
Single source shortest paths

Set fire to oil depot at

...

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 4/7
Single source shortest paths

Set fire to oil depot at

t = 16
vertex 0 70
2 3
Fire travels at uniform
speed along each 80 6
t = 30 t = 45
pipeline
50
0 1 4 5
First oil depot to catch 10 20
fire after 0 is nearest t=0 t = 10
vertex 5 10
6
Next oil depot is
second nearest vertex t = 35

...

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 4/7
Single source shortest paths

Set fire to oil depot at

t = 16 t = 86
vertex 0 70
2 3
Fire travels at uniform
speed along each 80 6
t = 30 t = 45
pipeline
50
0 1 4 5
First oil depot to catch 10 20
fire after 0 is nearest t=0 t = 10
vertex 5 10
6
Next oil depot is
second nearest vertex t = 35

...

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 4/7
Single source shortest paths

Compute expected
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
5 10
6

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected ∞ ∞
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

∞ ∞
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours ∞ ∞
5 10
6
∞

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected ∞ ∞
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

∞ ∞
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours ∞
0
5 10
6
∞

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected ∞
80
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

∞ ∞
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
5 10
6
∞

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected ∞
80
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

∞ ∞
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
5 10
6
∞

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected ∞
16
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

30 ∞
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
5 10
6
∞

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected ∞
16
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

30 ∞
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
5 10
6
∞

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected
16 86
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

30 ∞
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
5 10
6
∞

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected
16 86
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

30 ∞
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
5 10
6
∞

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected
16 86
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

30 80
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
5 10
6
35

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected
16 86
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

30 80
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
5 10
6
35

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected
16 86
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

30 45
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
5 10
6
35

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected
16 86
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

30 45
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
5 10
6
35

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Single source shortest paths

Compute expected
16 86
burn time for each 70
vertex 2 3

Each time a new vertex 80 6

30 45
burns, update the
50
expected burn times of 0 1 4 5
10 20
its neighbours
0 10
Algorithm due to 5 10
Edsger W Dikjstra 6
35

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 5/7
Dijkstra’s algorithm: Proof of correctness
Each new shortest path we discover extends an
earlier one

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 6/7
Dijkstra’s algorithm: Proof of correctness
Each new shortest path we discover extends an
earlier one
Burnt vertices
By induction, assume we have found shortest
paths to all vertices already burnt
s
y
Next vertex to burn is v, via x x

This argument breaks down if edge (w,v) can

have negative weight
Can’t use Dijkstra’s algorithm with negative
edge weights
Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 6/7
Summary

Dijkstra’s algorithm computes single source

shortest paths
Use fire analogy
Keep track of expected burn times for each
vertex
Update burn times of neighbours each time a
vertex burns

Correctness requires edge weights to be

non-negative

Madhavan Mukund Single Source Shortest Paths Mathematics for Data Science 1, Week 12 7/7
Single Source Shortest Paths with Negative Weights

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 12
Dijkstra’s algorithm
Recall the burning pipeline analogy

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 2/1
Dijkstra’s algorithm
Recall the burning pipeline analogy
We keep track of the following
The vertices that have been burnt
The expected burn time of vertices

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 2/1
Dijkstra’s algorithm
Recall the burning pipeline analogy Initialization (assume source vertex 0)

We keep track of the following B(i) = False, for 0 ≤ i < n

The vertices that have been burnt UB = {k | B(k) = False}
(
The expected burn time of vertices 0, if i = 0
EBT (i) =
Initially ∞, otherwise
No vertex is burnt
Expected burn time of source vertex is 0
Expected burn time of rest is ∞

We keep track of the following B(i) = False, for 0 ≤ i < n

The vertices that have been burnt UB = {k | B(k) = False}
(
The expected burn time of vertices 0, if i = 0
EBT (i) =
Initially ∞, otherwise
No vertex is burnt Update, if UB 6= ∅
Expected burn time of source vertex is 0 Let j ∈ UB such that
Expected burn time of rest is ∞ EBT (j) ≤ EBT (k) for all k ∈ UB
While there are vertices yet to burn Update B(j) = True, UB = UB \ {j}
Pick unburnt vertex with minimum
For each (j, k) ∈ E such that k ∈ UB,
expected burn time, mark it as burnt
EBT (k) = min( EBT (k),
Update the expected burn time of its
EBT (j) + W (j, k) )
neighbours
Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 2/1
Correctness requires non-negative edge weights

Each new shortest path we discover extends an

earlier one Burnt vertices
By induction, assume we have found shortest
s
paths to all vertices already burnt y
x

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 3/1
Correctness requires non-negative edge weights

Each new shortest path we discover extends an

earlier one Burnt vertices
By induction, assume we have found shortest
s
paths to all vertices already burnt y
x
Next vertex to burn is v, via x
Cannot find a shorter path later from y to v
via w v
Burn time of w ≥ burn time of v w
Edge from w to v has weight ≥ 0

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 3/1
Correctness requires non-negative edge weights

Each new shortest path we discover extends an

This argument breaks down if edge (w,v) can

have negative weight

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 3/1
Extending to negative edge weights

The difficulty with negative edge weights is

that we stop updating the burn time once a Burnt vertices
vertex is burnt
s
y
x

v
w

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 4/1
Extending to negative edge weights

The difficulty with negative edge weights is

that we stop updating the burn time once a Burnt vertices
vertex is burnt
s
What if we allow updates even after a vertex is y
x
burnt?

v
w

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 4/1
Extending to negative edge weights

The difficulty with negative edge weights is

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 4/1
Extending to negative edge weights

The difficulty with negative edge weights is

that we stop updating the burn time once a Burnt vertices
vertex is burnt
s
What if we allow updates even after a vertex is y
x
burnt?
Recall, negative edge weights are allowed, but
no negative cycles
v
Going around a cycle can only add to the w
length

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 4/1
Extending to negative edge weights

The difficulty with negative edge weights is

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 4/1
Extending to negative edge weights
Suppose minimum weight path from 0
to k is
w1 2w 3 w w`−1 ` w
0 −→ j1 −→ j2 −→ · · · −−−→ j`−1 −→ k
Need not be minimum in terms of
number of edges

Every prefix of this path must itself be a

minimum weight path
w1
0 −→ j1
w1 2w
0 −→ j1 −→ j2
...
w1 2w 3 w w`−1
0 −→ j1 −→ j2 −→ · · · −−−→ j`−1

Every prefix of this path must itself be a

minimum weight path
w1
0 −→ j1
w1 2w
0 −→ j1 −→ j2
...
w1 2w 3 w w`−1
0 −→ j1 −→ j2 −→ · · · −−−→ j`−1

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 5/1
Extending to negative edge weights
Suppose minimum weight path from 0 Once we discover shortest path to j`−1 ,
to k is next update will fix shortest path to k
w1 2 w 3 w w`−1 ` w
0 −→ j1 −→ j2 −→ · · · −−−→ j`−1 −→ k Repeatedly update shortest distance to
Need not be minimum in terms of each vertex based on shortest distance
number of edges to its neighbours
Every prefix of this path must itself be a Update cannot push this distance
below actual shortest distance
minimum weight path
w1
0 −→ j1
w1 2w
0 −→ j1 −→ j2
...
w1 2w 3 w w`−1
0 −→ j1 −→ j2 −→ · · · −−−→ j`−1

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 5/1
Extending to negative edge weights
Suppose minimum weight path from 0 Once we discover shortest path to j`−1 ,
to k is next update will fix shortest path to k
w1 2 w 3 w w`−1 ` w
0 −→ j1 −→ j2 −→ · · · −−−→ j`−1 −→ k Repeatedly update shortest distance to
Need not be minimum in terms of each vertex based on shortest distance
number of edges to its neighbours
Every prefix of this path must itself be a Update cannot push this distance
below actual shortest distance
minimum weight path
w1 After ` updates, all shortest paths using
0 −→ j1
w1
0 −→ 2
j1 −→
w
j2
≤ ` edges have stabilized
... Minimum weight path to any node has
w1 2w 3 w w`−1 at most n−1 edges
0 −→ j1 −→ j2 −→ · · · −−−→ j`−1
After n−1 updates, all shortest paths
have stabilized

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 5/1
Bellman-Ford Algorithm
Initialization (source vertex 0)
D(j) : minimum distance known so
far to vertex j
(
0, if j = 0
D(j) =
∞, otherwise

Repeat n−1 times

For each vertex j ∈ {0, 1, . . . , n−1},
for each edge (j, k) ∈ E ,
D(k) = min( D(k),
D(j) + W (j, k) )

Repeat n−1 times

For each vertex j ∈ {0, 1, . . . , n−1},
for each edge (j, k) ∈ E ,
D(k) = min( D(k),
D(j) + W (j, k) )

Works for directed and undirected graphs

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 6/1
Bellman-Ford Algorithm
Initialization (source vertex 0)
D(j) : minimum distance known so 10
0 1
far to vertex j 8 1
(
0, if j = 0 7 2
D(j) = −4
∞, otherwise 2
1 1
Repeat n−1 times −2
6 3
For each vertex j ∈ {0, 1, . . . , n−1},
for each edge (j, k) ∈ E , −1 3
5 4
D(k) = min( D(k), −1
D(j) + W (j, k) )

Works for directed and undirected graphs

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 6/1
Bellman-Ford Algorithm

v D(v ) 10
0 1
0 8 1
1
2 7 2
3 −4
1 2 1
4
5 −2
6 6 3
7 −1 3
5 4
−1

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

v D(v ) 10
0 1
0 0 8 1
1 ∞
2 ∞ 7 2
3 ∞ −4
1 2 1
4 ∞
5 ∞ −2
6 ∞ 6 3
7 ∞ −1 3
5 4
Initialize D(0) = 0 −1

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

v D(v ) 10
0 1
0 0 0 8 1
1 ∞ 10
2 ∞ ∞ 7 2
3 ∞ ∞ −4
1 2 1
4 ∞ ∞
5 ∞ ∞ −2
6 ∞ ∞ 6 3
7 ∞ 8 −1 3
5 4
Initialize D(0) = 0 −1

For each (j, k) ∈ E , update

D(k) = min( D(k),
D(j) + W (j, k) )
Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

v D(v ) 10
0 1
0 0 0 0 8 1
1 ∞ 10 10
2 ∞ ∞ ∞ 7 2
3 ∞ ∞ ∞ −4
1 2 1
4 ∞ ∞ ∞
5 ∞ ∞ 12 −2
6 ∞ ∞ 9 6 3
7 ∞ 8 8 −1 3
5 4
Initialize D(0) = 0 −1

For each (j, k) ∈ E , update

D(k) = min( D(k),
D(j) + W (j, k) )
Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

v D(v ) 10
0 1
0 0 0 0 0 8 1
1 ∞ 10 10 5
2 ∞ ∞ ∞ 10 7 2
3 ∞ ∞ ∞ ∞ −4
1 2 1
4 ∞ ∞ ∞ ∞
5 ∞ ∞ 12 8 −2
6 ∞ ∞ 9 9 6 3
7 ∞ 8 8 8 −1 3
5 4
Initialize D(0) = 0 −1

For each (j, k) ∈ E , update

D(k) = min( D(k),
D(j) + W (j, k) )
Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

v D(v ) 10
0 1
0 0 0 0 0 0 8 1
1 ∞ 10 10 5 5
2 ∞ ∞ ∞ 10 6 7 2
3 ∞ ∞ ∞ ∞ 11 −4
1 2 1
4 ∞ ∞ ∞ ∞ ∞
5 ∞ ∞ 12 8 7 −2
6 ∞ ∞ 9 9 9 6 3
7 ∞ 8 8 8 8 −1 3
5 4
Initialize D(0) = 0 −1

For each (j, k) ∈ E , update

D(k) = min( D(k),
D(j) + W (j, k) )
Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

v D(v ) 10
0 1
0 0 0 0 0 0 0 8 1
1 ∞ 10 10 5 5 5
2 ∞ ∞ ∞ 10 6 5 7 2
3 ∞ ∞ ∞ ∞ 11 7 −4
1 2 1
4 ∞ ∞ ∞ ∞ ∞ 14
5 ∞ ∞ 12 8 7 7 −2
6 ∞ ∞ 9 9 9 9 6 3
7 ∞ 8 8 8 8 8 −1 3
5 4
Initialize D(0) = 0 −1

For each (j, k) ∈ E , update

D(k) = min( D(k),
D(j) + W (j, k) )
Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

v D(v ) 10
0 1
0 0 0 0 0 0 0 0 8 1
1 ∞ 10 10 5 5 5 5
2 ∞ ∞ ∞ 10 6 5 5 7 2
3 ∞ ∞ ∞ ∞ 11 7 6 −4
1 2 1
4 ∞ ∞ ∞ ∞ ∞ 14 10
5 ∞ ∞ 12 8 7 7 7 −2
6 ∞ ∞ 9 9 9 9 9 6 3
7 ∞ 8 8 8 8 8 8 −1 3
5 4
Initialize D(0) = 0 −1

For each (j, k) ∈ E , update

D(k) = min( D(k),
D(j) + W (j, k) )
Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

v D(v ) 10
0 1
0 0 0 0 0 0 0 0 0 8 1
1 ∞ 10 10 5 5 5 5 5
2 ∞ ∞ ∞ 10 6 5 5 5 7 2
3 ∞ ∞ ∞ ∞ 11 7 6 6 −4
1 2 1
4 ∞ ∞ ∞ ∞ ∞ 14 10 9
5 ∞ ∞ 12 8 7 7 7 7 −2
6 ∞ ∞ 9 9 9 9 9 9 6 3
7 ∞ 8 8 8 8 8 8 8 −1 3
5 4
Initialize D(0) = 0 −1

For each (j, k) ∈ E , update

D(k) = min( D(k),
D(j) + W (j, k) )
Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

v D(v ) 10
0 1
0 0 0 0 0 0 0 0 0 8 1
1 ∞ 10 10 5 5 5 5 5
2 ∞ ∞ ∞ 10 6 5 5 5 7 2
3 ∞ ∞ ∞ ∞ 11 7 6 6 −4
1 2 1
4 ∞ ∞ ∞ ∞ ∞ 14 10 9
5 ∞ ∞ 12 8 7 7 7 7 −2
6 ∞ ∞ 9 9 9 9 9 9 6 3
7 ∞ 8 8 8 8 8 8 8 −1 3
5 4
What if there was a negative cycle? −1

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

Distance would continue to decrease

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Bellman-Ford Algorithm

Distance would continue to decrease

Check if update n reduces any D(v )
Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 7/1
Summary

Dijkstra’s algorithm assumes non-negative edge weights

Final distance is frozen each time a vertex “burns”
Should not encounter a shorter route discovered later

Without negative cycles, every shortest route is a path

Every prefix of a shortest path is also a shortest path
Iteratively find shortest paths of length 1, 2, . . . , n−1
Update distance to each vertex with every iteration — Bellman-Ford algorithm
If Bellman-Ford algorithm does not converge after n − 1 iterations, there is a
negative cycle

Madhavan Mukund Single Source Shortest Paths with Negative Weights Mathematics for Data Science 1, Week 12 8/1
All-Pairs Shortest Paths

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 12
Shortest paths in weighted graphs

Two types of shortest path problems of interest

Single source shortest paths All pairs shortest paths

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 2/9
Shortest paths in weighted graphs

Two types of shortest path problems of interest

Single source shortest paths All pairs shortest paths

Find shortest paths from a fixed vertex Find shortest paths between every pair
to every other vertex of vertices i and j
Transport finished product from factory Optimal airline, railway, road routes
(single source) to all retail outlets between cities
Courier company delivers items from
distribution centre (single source) to
addressees
Dijkstra’s algorithm (non-negative
weights), Bellman-Ford algorithm
(allows negative weights)
Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 2/9
Shortest paths in weighted graphs

Two types of shortest path problems of interest

Single source shortest paths All pairs shortest paths

Find shortest paths from a fixed vertex Find shortest paths between every pair
to every other vertex of vertices i and j
Transport finished product from factory Optimal airline, railway, road routes
(single source) to all retail outlets between cities
Courier company delivers items from Run Dijkstra or Bellman-Ford from each
distribution centre (single source) to vertex
addressees
Dijkstra’s algorithm (non-negative
weights), Bellman-Ford algorithm
(allows negative weights)
Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 2/9
Shortest paths in weighted graphs

Two types of shortest path problems of interest

Single source shortest paths All pairs shortest paths

Find shortest paths from a fixed vertex Find shortest paths between every pair
to every other vertex of vertices i and j
Transport finished product from factory Optimal airline, railway, road routes
(single source) to all retail outlets between cities
Courier company delivers items from Run Dijkstra or Bellman-Ford from each
distribution centre (single source) to vertex
addressees
Is there is another way?
Dijkstra’s algorithm (non-negative
weights), Bellman-Ford algorithm
(allows negative weights)
Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 2/9
Transitive closure
Recall transitive closure algorithm

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 3/9
Transitive closure
Recall transitive closure algorithm
Adjacency matrix A represents paths of
length 1

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 3/9
Transitive closure
Recall transitive closure algorithm
Adjacency matrix A represents paths of
length 1
Matrix multiplication, A2 = A × A
A2 [i, j] = 1 if there is a path of length
2 from i to j
For some k, A[i, k] = A[k, j] = 1

In general, A`+1 = A` × A,
A`+1 [i, j] = 1 if there is a path of
length `+1 from i to j
For some k, A` [i, k] = 1, A[k, j] = 1

A+ = A + A2 + · · · + An−1
Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 3/9
Transitive closure
Recall transitive closure algorithm An alternative approach

Adjacency matrix A represents paths of

length 1
Matrix multiplication, A2 = A × A
A2 [i, j] = 1 if there is a path of length
2 from i to j
For some k, A[i, k] = A[k, j] = 1

In general, A`+1 = A` × A,
A`+1 [i, j] = 1 if there is a path of
length `+1 from i to j
For some k, A` [i, k] = 1, A[k, j] = 1

A+ = A + A2 + · · · + An−1
Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 3/9
Transitive closure
Recall transitive closure algorithm An alternative approach
B k [i, j] = 1 if there is path from i to j
Adjacency matrix A represents paths of
via vertices {0, 1, . . . , k−1}
length 1
Constraint applies only to intermediate
Matrix multiplication, A2 =A×A vertices i to j
2
A [i, j] = 1 if there is a path of length B 0 [i, j] = 1 if there is a direct edge
2 from i to j B0 = A
For some k, A[i, k] = A[k, j] = 1
B k+1 [i, j] = 1 if
In general, A`+1 = A` × A, B k [i, j] = 1 — can already reach j
A`+1 [i, j] = 1 if there is a path of from i via {0, 1, . . . , k−1}
length `+1 from i to j B k [i, k] = 1 and B k [k, j] = 1 — use
For some k, A` [i, k] = 1, A[k, j] = 1 {0, 1, . . . , k−1} to go from i to k and
then from k to j
A+ = A + A2 + · · · + An−1
Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 3/9
Warshall’s Algorithm
The algorithm on the right also Computing transitive closure
computes transitive closure — B k [i, j] = 1 if there is path from i to j
Warshall’s algorithm via vertices {0, 1, . . . , k−1}
B 0 [i, j] = A[i, j]
Direct edges, no intermediate vertices

B k+1 [i, j] = 1 if
B k [i, j] = 1, or
B k [i, k] = 1 and B k [k, j] = 1

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 4/9
Warshall’s Algorithm
The algorithm on the right also Computing transitive closure
computes transitive closure — B k [i, j] = 1 if there is path from i to j
Warshall’s algorithm via vertices {0, 1, . . . , k−1}
B n [i, j] = 1 if there is some path from i B 0 [i, j] = A[i, j]
to j with intermediate vertices in Direct edges, no intermediate vertices
{0, 1, . . . , n−1}
B k+1 [i, j] = 1 if
B k [i, j] = 1, or
B k [i, k] = 1 and B k [k, j] = 1

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 4/9
Warshall’s Algorithm
The algorithm on the right also Computing transitive closure
computes transitive closure — B k [i, j] = 1 if there is path from i to j
Warshall’s algorithm via vertices {0, 1, . . . , k−1}
B n [i, j] = 1 if there is some path from i B 0 [i, j] = A[i, j]
to j with intermediate vertices in Direct edges, no intermediate vertices
{0, 1, . . . , n−1}
B k+1 [i, j] = 1 if
Bn = A+
B k [i, j] = 1, or
B k [i, k] = 1 and B k [k, j] = 1

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 4/9
Warshall’s Algorithm
The algorithm on the right also Computing transitive closure
computes transitive closure — B k [i, j] = 1 if there is path from i to j
Warshall’s algorithm via vertices {0, 1, . . . , k−1}
B n [i, j] = 1 if there is some path from i B 0 [i, j] = A[i, j]
to j with intermediate vertices in Direct edges, no intermediate vertices
{0, 1, . . . , n−1}
B k+1 [i, j] = 1 if
Bn = A+
B k [i, j] = 1, or
We adapt Warshall’s algorithm to B k [i, k] = 1 and B k [k, j] = 1
compute all-pairs shortest paths

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 4/9
Floyd-Warshall Algorithm
Let SP k [i, j] be the length of the shortest path from i to j via vertices
{0, 1, . . . , k−1}
SP 0 [i, j] = W [i, j]
No intermediate vertices, shortest path is weight of direct edge
Assume W [i, j] = ∞ if (i, j) ∈
/E

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 5/9
Floyd-Warshall Algorithm
Let SP k [i, j] be the length of the shortest path from i to j via vertices
{0, 1, . . . , k−1}
SP 0 [i, j] = W [i, j]
No intermediate vertices, shortest path is weight of direct edge
Assume W [i, j] = ∞ if (i, j) ∈
/E

SP k+1 [i, j] is the minimum of

SP k [i, j]
Shortest path using only {0, 1, . . . , k−1}
SP k [i, k] + SP k [k, j]
Combine shortest path from i to k and k to j

SP k+1 [i, j] is the minimum of

SP k [i, j]
Shortest path using only {0, 1, . . . , k−1}
SP k [i, k] + SP k [k, j]
Combine shortest path from i to k and k to j

SP n [i, j] = 1 is the length of the shortest path overall from i to j

Intermediate vertices lie in {0, 1, . . . , n−1}
Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 5/9
Floyd-Warshall Algorithm
SP 0 0 1 2 3 4 5 6 7
0 ∞ 10 ∞ ∞ ∞ ∞ ∞ 8 10
1 ∞ ∞ ∞ ∞ ∞ 2 ∞ ∞ 0 1
2 ∞ 1 ∞ 1 ∞ ∞ ∞ ∞
8 1
3 ∞ ∞ ∞ ∞ 3 ∞ ∞ ∞ 7 2
4 ∞ ∞ ∞ ∞ ∞ −1 ∞ ∞
5 ∞ ∞ −2 ∞ ∞ ∞ ∞ ∞
−4
1 2 1
6 ∞ −4 ∞ ∞ ∞ −1 ∞ ∞
7 ∞ ∞ ∞ ∞ ∞ ∞ 1 ∞
−2
6 3
−1 3
5 4
−1

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 6/9
Floyd-Warshall Algorithm
SP 0 0 1 2 3 4 5 6 7
0 ∞ 10 ∞ ∞ ∞ ∞ ∞ 8 10
1 ∞ ∞ ∞ ∞ ∞ 2 ∞ ∞ 0 1
2 ∞ 1 ∞ 1 ∞ ∞ ∞ ∞
8 1
3 ∞ ∞ ∞ ∞ 3 ∞ ∞ ∞ 7 2
4 ∞ ∞ ∞ ∞ ∞ −1 ∞ ∞
5 ∞ ∞ −2 ∞ ∞ ∞ ∞ ∞
−4
1 2 1
6 ∞ −4 ∞ ∞ ∞ −1 ∞ ∞
7 ∞ ∞ ∞ ∞ ∞ ∞ 1 ∞
−2
SP 1 0 1 2 3 4 5 6 7 6 3
0 ∞ 10 ∞ ∞ ∞ ∞ ∞ 8
−1 3
1 ∞ ∞ ∞ ∞ ∞ 2 ∞ ∞ 5 4
2 ∞ 1 ∞ 1 ∞ ∞ ∞ ∞ −1
3 ∞ ∞ ∞ ∞ 3 ∞ ∞ ∞
4 ∞ ∞ ∞ ∞ ∞ −1 ∞ ∞
5 ∞ ∞ −2 ∞ ∞ ∞ ∞ ∞
6 ∞ −4 ∞ ∞ ∞ −1 ∞ ∞
7 ∞ ∞ ∞ ∞ ∞ ∞ 1 ∞
Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 6/9
Floyd-Warshall Algorithm
SP 1 0 1 2 3 4 5 6 7
0 ∞ 10 ∞ ∞ ∞ ∞ ∞ 8 10
1 ∞ ∞ ∞ ∞ ∞ 2 ∞ ∞ 0 1
2 ∞ 1 ∞ 1 ∞ ∞ ∞ ∞
8 1
3 ∞ ∞ ∞ ∞ 3 ∞ ∞ ∞ 7 2
4 ∞ ∞ ∞ ∞ ∞ −1 ∞ ∞
5 ∞ ∞ −2 ∞ ∞ ∞ ∞ ∞
−4
1 2 1
6 ∞ −4 ∞ ∞ ∞ −1 ∞ ∞
7 ∞ ∞ ∞ ∞ ∞ ∞ 1 ∞
−2
6 3
−1 3
5 4
−1

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 7/9
Floyd-Warshall Algorithm
SP 1 0 1 2 3 4 5 6 7
0 ∞ 10 ∞ ∞ ∞ ∞ ∞ 8 10
1 ∞ ∞ ∞ ∞ ∞ 2 ∞ ∞ 0 1
2 ∞ 1 ∞ 1 ∞ ∞ ∞ ∞
8 1
3 ∞ ∞ ∞ ∞ 3 ∞ ∞ ∞ 7 2
4 ∞ ∞ ∞ ∞ ∞ −1 ∞ ∞
5 ∞ ∞ −2 ∞ ∞ ∞ ∞ ∞
−4
1 2 1
6 ∞ −4 ∞ ∞ ∞ −1 ∞ ∞
7 ∞ ∞ ∞ ∞ ∞ ∞ 1 ∞
−2
SP 2 0 1 2 3 4 5 6 7 6 3
0 ∞ 10 ∞ ∞ ∞ 12 ∞ 8
−1 3
1 ∞ ∞ ∞ ∞ ∞ 2 ∞ ∞ 5 4
2 ∞ 1 ∞ 1 ∞ 3 ∞ ∞ −1
3 ∞ ∞ ∞ ∞ 3 ∞ ∞ ∞
4 ∞ ∞ ∞ ∞ ∞ −1 ∞ ∞
5 ∞ ∞ −2 ∞ ∞ ∞ ∞ ∞
6 ∞ −4 ∞ ∞ ∞ −2 ∞ ∞
7 ∞ ∞ ∞ ∞ ∞ ∞ 1 ∞
Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 7/9
Floyd-Warshall Algorithm
SP 2 0 1 2 3 4 5 6 7
0 ∞ 10 ∞ ∞ ∞ 12 ∞ 8 10
1 ∞ ∞ ∞ ∞ ∞ 2 ∞ ∞ 0 1
2 ∞ 1 ∞ 1 ∞ 3 ∞ ∞
8 1
3 ∞ ∞ ∞ ∞ 3 ∞ ∞ ∞ 7 2
4 ∞ ∞ ∞ ∞ ∞ −1 ∞ ∞
5 ∞ ∞ −2 ∞ ∞ ∞ ∞ ∞
−4
1 2 1
6 ∞ −4 ∞ ∞ ∞ −2 ∞ ∞
7 ∞ ∞ ∞ ∞ ∞ ∞ 1 ∞
−2
6 3
−1 3
5 4
−1

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 8/9
Floyd-Warshall Algorithm
SP 2 0 1 2 3 4 5 6 7
0 ∞ 10 ∞ ∞ ∞ 12 ∞ 8 10
1 ∞ ∞ ∞ ∞ ∞ 2 ∞ ∞ 0 1
2 ∞ 1 ∞ 1 ∞ 3 ∞ ∞
8 1
3 ∞ ∞ ∞ ∞ 3 ∞ ∞ ∞ 7 2
4 ∞ ∞ ∞ ∞ ∞ −1 ∞ ∞
5 ∞ ∞ −2 ∞ ∞ ∞ ∞ ∞
−4
1 2 1
6 ∞ −4 ∞ ∞ ∞ −2 ∞ ∞
7 ∞ ∞ ∞ ∞ ∞ ∞ 1 ∞
−2
SP 3 0 1 2 3 4 5 6 7 6 3
0 ∞ 10 ∞ ∞ ∞ 12 ∞ 8
−1 3
1 ∞ ∞ ∞ ∞ ∞ 2 ∞ ∞ 5 4
2 ∞ 1 ∞ 1 ∞ 3 ∞ ∞ −1
3 ∞ ∞ ∞ ∞ 3 ∞ ∞ ∞
4 ∞ ∞ ∞ ∞ ∞ −1 ∞ ∞
5 ∞ −1 −2 −1 ∞ 1 ∞ ∞
6 ∞ −4 ∞ ∞ ∞ −2 ∞ ∞
7 ∞ ∞ ∞ ∞ ∞ ∞ 1 ∞
Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 8/9
Summary

Warshall’s algorithm is an alternative way to compute transitive closure

B k [i, j] = 1 if we can reach j from i using vertices in {0, 1, . . . , k−1}

Adapt Warshall’s algorithm to compute all pairs shortest paths

SP k [i, j] is the length of the shorest path from i to j using vertices in {0, 1, . . . , k−1}
SP n [i, j] is the length of the overall shorest path
Floyd-Warshall algorithm

Works with negative edge weights, assuming no negative cycles

Madhavan Mukund All-Pairs Shortest Paths Mathematics for Data Science 1, Week 12 9/9
Minimum Cost Spanning Trees

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 12
Examples

Roads
District hit by cyclone, roads are
damaged
Government sets to work to restore
roads
Priority is to ensure that all parts of the
district can be reached
What set of roads should be restored
first?

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 2/7
Examples

Roads Fibre optic cables

District hit by cyclone, roads are Internet service provider has a network
damaged of fibre optic cables
Government sets to work to restore Wants to ensure redundancy against
roads cable faults
Priority is to ensure that all parts of the Lay secondary cables in parallel to first
district can be reached
What is the minimum number of cables
What set of roads should be restored to be doubled up so that entire network
first? is connected via redundant links?

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 2/7
Spanning trees

Retain a minimal set of edges so that 3 4

graph remains connected

0 1 2

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 3/7
Spanning trees

Retain a minimal set of edges so that 3 4

graph remains connected
Recall that a minimally connected
graph is a tree 0 1 2
Adding an edge to a tree creates a
loop
Removing an edge disconnects the
graph

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 3/7
Spanning trees

Retain a minimal set of edges so that 3 4

graph remains connected
Recall that a minimally connected
graph is a tree 0 1 2
Adding an edge to a tree creates a
loop
Removing an edge disconnects the
graph

Want a tree that connects all the

vertices — spanning tree

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 3/7
Spanning trees

Retain a minimal set of edges so that 3 4

graph remains connected
Recall that a minimally connected
graph is a tree 0 1 2
Adding an edge to a tree creates a
loop
Removing an edge disconnects the
graph

Want a tree that connects all the

vertices — spanning tree
More than one spanning tree, in general

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 3/7
Spanning trees with costs

Restoring a road or laying a fibre optic 70

3 4
cable has a cost
18 6 8
10 20
0 1 2

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 4/7
Spanning trees with costs

Restoring a road or laying a fibre optic 70

3 4
cable has a cost
18 6 8
Minimum cost spanning tree
10 20
Add the cost of all the edges in the 0 1 2
tree
Among the different spanning trees,
choose one with minimum cost

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 4/7
Spanning trees with costs

Restoring a road or laying a fibre optic 70

3 4
cable has a cost
18 6 8
Minimum cost spanning tree
10 20
Add the cost of all the edges in the 0 1 2
tree
Among the different spanning trees,
choose one with minimum cost

Example

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 4/7
Spanning trees with costs

Restoring a road or laying a fibre optic 70

3 4
cable has a cost
18 6 8
Minimum cost spanning tree
10 20
Add the cost of all the edges in the 0 1 2
tree
Among the different spanning trees,
choose one with minimum cost

Example
Spanning tree, Cost is 114 — not
minimum cost spanning tree

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 4/7
Spanning trees with costs

Restoring a road or laying a fibre optic 70

3 4
cable has a cost
18 6 8
Minimum cost spanning tree
10 20
Add the cost of all the edges in the 0 1 2
tree
Among the different spanning trees,
choose one with minimum cost

Example
Spanning tree, Cost is 114 — not
minimum cost spanning tree
Anonther spanning tree, Cost is 44 —
minimum cost spanning tree

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 4/7
Some facts about trees

Definition A tree is a connected acylic graph.

Fact 1
A tree on n vertices has exactly n − 1 edges

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 5/7
Some facts about trees

Definition A tree is a connected acylic graph.

Fact 1
A tree on n vertices has exactly n − 1 edges

Initially, one single component

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 5/7
Some facts about trees

Definition A tree is a connected acylic graph.

Fact 1
A tree on n vertices has exactly n − 1 edges

Initially, one single component

Deleting edge (i, j) must split
component
Otherwise, there is still a path from i
to j, combine with (i, j) to form cycle

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 5/7
Some facts about trees

Definition A tree is a connected acylic graph.

Fact 1
A tree on n vertices has exactly n − 1 edges

Initially, one single component

Deleting edge (i, j) must split
component
Otherwise, there is still a path from i
to j, combine with (i, j) to form cycle

Each edge deletion creates one more

component

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 5/7
Some facts about trees

Definition A tree is a connected acylic graph.

Fact 1
A tree on n vertices has exactly n − 1 edges

Initially, one single component

Deleting edge (i, j) must split
component
Otherwise, there is still a path from i
to j, combine with (i, j) to form cycle

Each edge deletion creates one more

component
Deleting n − 1 edges creates n
components, each an isolated vertex
Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 5/7
Some facts about trees

Definition A tree is a connected acylic graph.

Fact 1 Fact 2
A tree on n vertices has exactly n − 1 edges Adding an edge to a tree must create a
cycle.
Initially, one single component
Deleting edge (i, j) must split
component
Otherwise, there is still a path from i
to j, combine with (i, j) to form cycle

Each edge deletion creates one more

component
Deleting n − 1 edges creates n
components, each an isolated vertex
Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 5/7
Some facts about trees

Definition A tree is a connected acylic graph.

Each edge deletion creates one more

component
Deleting n − 1 edges creates n
components, each an isolated vertex
Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 5/7
Some facts about trees

Definition A tree is a connected acylic graph.

Each edge deletion creates one more

component
Deleting n − 1 edges creates n
components, each an isolated vertex
Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 5/7
Some facts about trees

Definition A tree is a connected acylic graph.

Fact 1 Fact 2
A tree on n vertices has exactly n − 1 edges Adding an edge to a tree must create a
cycle.
Initially, one single component
Suppose we add an edge (i, j)
Deleting edge (i, j) must split
component Tree is connected, so there is already a
Otherwise, there is still a path from i path from i to j
to j, combine with (i, j) to form cycle
The new edge (i, j) combined with this
Each edge deletion creates one more path from i to j forms a cycle
component
Deleting n − 1 edges creates n
components, each an isolated vertex
Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 5/7
Some facts about trees

Definition A tree is a connected acylic graph.

Fact 3
In a tree, every pair of vertices is connected
by a unique path.

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 6/7
Some facts about trees

Definition A tree is a connected acylic graph.

Fact 3
In a tree, every pair of vertices is connected
by a unique path.

If there are two paths from i to j, there

must be a cycle

•
•
•
• j
i •
•

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 6/7
Some facts about trees

Definition A tree is a connected acylic graph.

Fact 3 Observation
In a tree, every pair of vertices is connected Any two of the following facts about a graph
by a unique path. G implies the third

If there are two paths from i to j, there G is connected

must be a cycle
G is acyclic
•
• G has n − 1 edges
•
• j
i •
•

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 6/7
Building minimum cost spanning trees

We will use these facts about trees to build minimum cost spanning trees

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 7/7
Building minimum cost spanning trees

We will use these facts about trees to build minimum cost spanning trees
Two natural strategies

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 7/7
Building minimum cost spanning trees

We will use these facts about trees to build minimum cost spanning trees
Two natural strategies
Start with the smallest edge and “grow” a tree
Prim’s algorithm

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 7/7
Building minimum cost spanning trees

We will use these facts about trees to build minimum cost spanning trees
Two natural strategies
Start with the smallest edge and “grow” a tree
Prim’s algorithm

Scan the edges in ascending order of weight to connect components without

forming cycles
Kruskal’s algorithm

Madhavan Mukund Minimum Cost Spanning Trees Mathematics for Data Science 1, Week 12 7/7
Minimum Cost Spanning Trees:
Prim’s Algorithm

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 12
Minimum cost spanning tree (MCST)
Weighted undirected graph,
G = (V , E ), W : E → R
G assumed to be connected

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 2/6
Minimum cost spanning tree (MCST)
Weighted undirected graph,
G = (V , E ), W : E → R
G assumed to be connected

Find a minimum cost spanning tree

Tree connecting all vertices in V

Find a minimum cost spanning tree

Tree connecting all vertices in V

Strategy
Incrementally grow the minimum cost
spanning tree
Start with a smallest weight edge
overall
Extend the current tree by adding the
smallest edge from the tree to a vertex
not yet in the tree

G assumed to be connected 70
3 4
Find a minimum cost spanning tree 18 6 8
Tree connecting all vertices in V 10 20
0 1 2
Strategy
Incrementally grow the minimum cost
spanning tree
Start with a smallest weight edge
overall
Extend the current tree by adding the
smallest edge from the tree to a vertex
not yet in the tree

G assumed to be connected 70
3 4
Find a minimum cost spanning tree 18 6 8
Tree connecting all vertices in V 10 20
0 1 2
Strategy
Incrementally grow the minimum cost
Start with smallest edge, (1, 3)
spanning tree
Start with a smallest weight edge
overall
Extend the current tree by adding the
smallest edge from the tree to a vertex
not yet in the tree

G assumed to be connected 70
3 4
Find a minimum cost spanning tree 18 6 8
Tree connecting all vertices in V 10 20
0 1 2
Strategy
Incrementally grow the minimum cost
Start with smallest edge, (1, 3)
spanning tree
Extend the tree with (1, 0)
Start with a smallest weight edge
overall
Extend the current tree by adding the
smallest edge from the tree to a vertex
not yet in the tree

G assumed to be connected 70
3 4
Find a minimum cost spanning tree 18 6 8
Tree connecting all vertices in V 10 20
0 1 2
Strategy
Incrementally grow the minimum cost
Start with smallest edge, (1, 3)
spanning tree
Extend the tree with (1, 0)
Start with a smallest weight edge
overall Can’t add (0, 3), forms a cycle
Extend the current tree by adding the
smallest edge from the tree to a vertex
not yet in the tree

G assumed to be connected 70
3 4
Find a minimum cost spanning tree 18 6 8
Tree connecting all vertices in V 10 20
0 1 2
Strategy
Incrementally grow the minimum cost
Start with smallest edge, (1, 3)
spanning tree
Extend the tree with (1, 0)
Start with a smallest weight edge
overall Can’t add (0, 3), forms a cycle
Extend the current tree by adding the Instead, extend the tree with (1, 2)
smallest edge from the tree to a vertex
not yet in the tree

G assumed to be connected 70
3 4
Find a minimum cost spanning tree 18 6 8
Tree connecting all vertices in V 10 20
0 1 2
Strategy
Incrementally grow the minimum cost
Start with smallest edge, (1, 3)
spanning tree
Extend the tree with (1, 0)
Start with a smallest weight edge
overall Can’t add (0, 3), forms a cycle
Extend the current tree by adding the Instead, extend the tree with (1, 2)
smallest edge from the tree to a vertex Extend the tree with (2, 4)
not yet in the tree

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 2/6
Prim’s algorithm

G = (V , E ), W : E → R

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 3/6
Prim’s algorithm

G = (V , E ), W : E → R
Incrementally build an MCST
TV ⊆ V : tree vertices, already added to MCST
TE ⊆ E : tree edges, already added to MCST

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 3/6
Prim’s algorithm

G = (V , E ), W : E → R
Incrementally build an MCST
TV ⊆ V : tree vertices, already added to MCST
TE ⊆ E : tree edges, already added to MCST

Initially, TV = TE = ∅

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 3/6
Prim’s algorithm

G = (V , E ), W : E → R
Incrementally build an MCST
TV ⊆ V : tree vertices, already added to MCST
TE ⊆ E : tree edges, already added to MCST

Initially, TV = TE = ∅
Choose minimum weight edge e = (i, j)
Set TV = {i, j}, TE = {e} MCST

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 3/6
Prim’s algorithm

G = (V , E ), W : E → R
Incrementally build an MCST
TV ⊆ V : tree vertices, already added to MCST
TE ⊆ E : tree edges, already added to MCST

Initially, TV = TE = ∅
Choose minimum weight edge e = (i, j)
Set TV = {i, j}, TE = {e} MCST

Repeat n − 2 times
Choose minimum weight edge f = (u, v ) such that
u ∈ TV , v ∈
/ TV
Add v to TV , f to TE
Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 3/6
Prim’s algorithm

G = (V , E ), W : E → R Example

Incrementally build an MCST 70

3 4
TV ⊆ V : tree vertices, already added to MCST 18 6 8
TE ⊆ E : tree edges, already added to MCST
10 20
0 1 2
Initially, TV = TE = ∅
Choose minimum weight edge e = (i, j) TV = ∅
Set TV = {i, j}, TE = {e} MCST TE = ∅
Repeat n − 2 times
Choose minimum weight edge f = (u, v ) such that
u ∈ TV , v ∈
/ TV
Add v to TV , f to TE
Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 3/6
Prim’s algorithm

G = (V , E ), W : E → R Example

Incrementally build an MCST 70

3 4
TV ⊆ V : tree vertices, already added to MCST 18 6 8
TE ⊆ E : tree edges, already added to MCST
10 20
0 1 2
Initially, TV = TE = ∅
Choose minimum weight edge e = (i, j) TV = {1, 3}
Set TV = {i, j}, TE = {e} MCST TE = {(1, 3)}
Repeat n − 2 times
Choose minimum weight edge f = (u, v ) such that
u ∈ TV , v ∈
/ TV
Add v to TV , f to TE
Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 3/6
Prim’s algorithm

G = (V , E ), W : E → R Example

Incrementally build an MCST 70

3 4
TV ⊆ V : tree vertices, already added to MCST 18 6 8
TE ⊆ E : tree edges, already added to MCST
10 20
0 1 2
Initially, TV = TE = ∅
Choose minimum weight edge e = (i, j) TV = {1, 3, 0}
Set TV = {i, j}, TE = {e} MCST TE = {(1, 3), (1, 0)}
Repeat n − 2 times
Choose minimum weight edge f = (u, v ) such that
u ∈ TV , v ∈
/ TV
Add v to TV , f to TE
Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 3/6
Prim’s algorithm

G = (V , E ), W : E → R Example

Incrementally build an MCST 70

3 4
TV ⊆ V : tree vertices, already added to MCST 18 6 8
TE ⊆ E : tree edges, already added to MCST
10 20
0 1 2
Initially, TV = TE = ∅
Choose minimum weight edge e = (i, j) TV = {1, 3, 0, 2}
Set TV = {i, j}, TE = {e} MCST TE = {(1, 3), (1, 0),
Repeat n − 2 times (1, 2)}
Choose minimum weight edge f = (u, v ) such that
u ∈ TV , v ∈
/ TV
Add v to TV , f to TE
Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 3/6
Prim’s algorithm

G = (V , E ), W : E → R Example

Incrementally build an MCST 70

3 4
TV ⊆ V : tree vertices, already added to MCST 18 6 8
TE ⊆ E : tree edges, already added to MCST
10 20
0 1 2
Initially, TV = TE = ∅
Choose minimum weight edge e = (i, j) TV = {1, 3, 0, 2, 4}
Set TV = {i, j}, TE = {e} MCST TE = {(1, 3), (1, 0),
Repeat n − 2 times (1, 2), (2, 4)}
Choose minimum weight edge f = (u, v ) such that
u ∈ TV , v ∈
/ TV
Add v to TV , f to TE
Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 3/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
Every MCST must include e

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 4/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
Every MCST must include e

Assume for now, all edge weights distinct

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 4/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
e w
Every MCST must include e u

Assume for now, all edge weights distinct

Let T be an MCST, e ∈
/T

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 4/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
e w
Every MCST must include e u

Assume for now, all edge weights distinct

Let T be an MCST, e ∈
/T
T contains a path p from u to
W

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 4/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
e w
Every MCST must include e u

Assume for now, all edge weights distinct

Let T be an MCST, e ∈
/T
T contains a path p from u to
W
p starts in U, ends in W
U

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 4/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
e w
Every MCST must include e u

Assume for now, all edge weights distinct

f w0
Let T be an MCST, e ∈
/T u0
T contains a path p from u to
W
p starts in U, ends in W
Let f = (u 0 , w 0 ) be the first edge on p
crossing from U to W U

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 4/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
e w
Every MCST must include e u

Assume for now, all edge weights distinct

w0
Let T be an MCST, e ∈
/T u0
T contains a path p from u to
W
p starts in U, ends in W
Let f = (u 0 , w 0 ) be the first edge on p
crossing from U to W U
Drop f , add e to get a cheaper spanning tree
Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 4/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
e w
Every MCST must include e u

Assume for now, all edge weights distinct

w0
What if two edges have the same weight? u0
W

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 4/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
e w
Every MCST must include e u

Assume for now, all edge weights distinct

w0
What if two edges have the same weight? u0
Assign each edge a unique index from 0 to
W
m−1
U

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 4/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
e w
Every MCST must include e u

Assume for now, all edge weights distinct

w0
What if two edges have the same weight? u0
Assign each edge a unique index from 0 to
W
m−1
Define (e, i) < (f , j) if W (e) < W (j) or U
W (e) = W (j) and i < j

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 4/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
Every MCST must include e

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 5/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
Every MCST must include e

In Prim’s algorithm, TV and W = V \ TV

partition V

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 5/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
Every MCST must include e

In Prim’s algorithm, TV and W = V \ TV

partition V
Algorithm picks smallest edge connecting TV
and W , which must belong to every MCST

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 5/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

In fact, for any v ∈ V , {v } and
Let V be partitioned into two non-empty sets V \ {v } form a partition
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with
u ∈ U, w ∈ W
Every MCST must include e

In Prim’s algorithm, TV and W = V \ TV

partition V
Algorithm picks smallest edge connecting TV
and W , which must belong to every MCST

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 5/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

In Prim’s algorithm, TV and W = V \ TV

partition V
Algorithm picks smallest edge connecting TV
and W , which must belong to every MCST

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 5/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

In fact, for any v ∈ V , {v } and
Let V be partitioned into two non-empty sets V \ {v } form a partition
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with The smallest weight edge leaving
u ∈ U, w ∈ W any vertex must belong to every
MCST
Every MCST must include e
We started with overall
minimum cost edge
In Prim’s algorithm, TV and W = V \ TV
partition V
Algorithm picks smallest edge connecting TV
and W , which must belong to every MCST

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 5/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

In fact, for any v ∈ V , {v } and
Let V be partitioned into two non-empty sets V \ {v } form a partition
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with The smallest weight edge leaving
u ∈ U, w ∈ W any vertex must belong to every
MCST
Every MCST must include e
We started with overall
minimum cost edge
In Prim’s algorithm, TV and W = V \ TV
partition V Instead, can start at any vertex
v , with TV = {v } and TE = ∅
Algorithm picks smallest edge connecting TV
and W , which must belong to every MCST

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 5/6
Correctness of Prim’s algorithm

Minimum Separator Lemma

In fact, for any v ∈ V , {v } and
Let V be partitioned into two non-empty sets V \ {v } form a partition
U and W = V \ U
Let e = (u, w ) be the minimum cost edge with The smallest weight edge leaving
u ∈ U, w ∈ W any vertex must belong to every
MCST
Every MCST must include e
We started with overall
minimum cost edge
In Prim’s algorithm, TV and W = V \ TV
partition V Instead, can start at any vertex
v , with TV = {v } and TE = ∅
Algorithm picks smallest edge connecting TV
and W , which must belong to every MCST First iteration will pick minimum
cost edge from v

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 5/6
Summary

Prim’s algorithm grows an MCST starting with

any vertex
At each step, connect one more vertex to the
tree using minimum cost edge from inside the
tree to outside the tree
Correctness follows from Minimum Separator
Lemma

Madhavan Mukund Minimum Cost Spanning Trees:Prim’s Algorithm Mathematics for Data Science 1, Week 12 6/6
Minimum Cost Spanning Trees:
Kruskal’s Algorithm

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Mathematics for Data Science 1

Week 12
Minimum cost spanning tree (MCST)
Weighted undirected graph,
G = (V , E ), W : E → R
G assumed to be connected

Find a minimum cost spanning tree

Tree connecting all vertices in V

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 2/5
Minimum cost spanning tree (MCST)
Weighted undirected graph,
G = (V , E ), W : E → R
G assumed to be connected

Find a minimum cost spanning tree

Tree connecting all vertices in V

Strategy 2
Start with n components, each a
single vertex
Process edges in ascending order of
cost
Include edge if it does not create a
cycle

G assumed to be connected 70
3 4
Find a minimum cost spanning tree 18 6 8
Tree connecting all vertices in V 10 20
0 1 2
Strategy 2
Start with n components, each a
single vertex
Process edges in ascending order of
cost
Include edge if it does not create a
cycle

G assumed to be connected 70
3 4
Find a minimum cost spanning tree 18 6 8
Tree connecting all vertices in V 10 20
0 1 2
Strategy 2
Start with n components, each a
Start with smallest edge, (1, 3)
single vertex
Add next smallest edge, (2, 4)
Process edges in ascending order of
cost
Include edge if it does not create a
cycle

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 2/5
Kruskal’s algorithm

G = (V , E ), W : E → R

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 3/5
Kruskal’s algorithm

G = (V , E ), W : E → R
Let E = {e0 , e1 , . . . , em−1 } be
edges sorted in ascending order
by weight

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 3/5
Kruskal’s algorithm

G = (V , E ), W : E → R
Let E = {e0 , e1 , . . . , em−1 } be
edges sorted in ascending order
by weight
Let TE ⊆ E be the set of tree
edges already added to MCST

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 3/5
Kruskal’s algorithm

G = (V , E ), W : E → R
Let E = {e0 , e1 , . . . , em−1 } be
edges sorted in ascending order
by weight
Let TE ⊆ E be the set of tree
edges already added to MCST
Initially, TE = ∅

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 3/5
Kruskal’s algorithm

G = (V , E ), W : E → R
Let E = {e0 , e1 , . . . , em−1 } be
edges sorted in ascending order
by weight
Let TE ⊆ E be the set of tree
edges already added to MCST
Initially, TE = ∅
Scan E from e0 to em−1
If adding ei to TE creates a
loop, skip it
Otherwise, add ei to TE

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 3/5
Kruskal’s algorithm

G = (V , E ), W : E → R Example
70
Let E = {e0 , e1 , . . . , em−1 } be 2 3
edges sorted in ascending order 18 6
by weight 10
0 1 4 5
10 20
Let TE ⊆ E be the set of tree 10 5
edges already added to MCST 6
Initially, TE = ∅ Sort E as
Scan E from e0 to em−1 {(5, 6), (1, 2), (0, 1), (4, 5), (4, 6), (0, 2), (1, 4), (2, 3)}
If adding ei to TE creates a
loop, skip it Set TE = ∅
Otherwise, add ei to TE

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 3/5
Kruskal’s algorithm

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 3/5
Correctness of Krusksal’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets U and W = V \ U
Let e = (u, w ) be the minimum cost edge with u ∈ U, w ∈ W
Every MCST must include e

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 4/5
Correctness of Krusksal’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets U and W = V \ U
Let e = (u, w ) be the minimum cost edge with u ∈ U, w ∈ W
Every MCST must include e

Edges in TE partition vertices into connected components

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 4/5
Correctness of Krusksal’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets U and W = V \ U
Let e = (u, w ) be the minimum cost edge with u ∈ U, w ∈ W
Every MCST must include e

Edges in TE partition vertices into connected components

Initially each vertex is a separate component

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 4/5
Correctness of Krusksal’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets U and W = V \ U
Let e = (u, w ) be the minimum cost edge with u ∈ U, w ∈ W
Every MCST must include e

Edges in TE partition vertices into connected components

Initially each vertex is a separate component
Adding e = (u, w ) merges components of u and w

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 4/5
Correctness of Krusksal’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets U and W = V \ U
Let e = (u, w ) be the minimum cost edge with u ∈ U, w ∈ W
Every MCST must include e

Edges in TE partition vertices into connected components

Initially each vertex is a separate component
Adding e = (u, w ) merges components of u and w
If u and w are in the same component, e forms a cycle and is discarded

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 4/5
Correctness of Krusksal’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets U and W = V \ U
Let e = (u, w ) be the minimum cost edge with u ∈ U, w ∈ W
Every MCST must include e

Edges in TE partition vertices into connected components

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 4/5
Correctness of Krusksal’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets U and W = V \ U
Let e = (u, w ) be the minimum cost edge with u ∈ U, w ∈ W
Every MCST must include e

Edges in TE partition vertices into connected components

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 4/5
Correctness of Krusksal’s algorithm

Minimum Separator Lemma

Let V be partitioned into two non-empty sets U and W = V \ U
Let e = (u, w ) be the minimum cost edge with u ∈ U, w ∈ W
Every MCST must include e

Edges in TE partition vertices into connected components

Initially each vertex is a separate component
Adding e = (u, w ) merges components of u and w
If u and w are in the same component, e forms a cycle and is discarded
Otherwise, let U be component of u, W be V \ U
U, W form a partition of V with u ∈ U and w ∈ W
Since we are scanning edges in ascending order of cost, e is minimum cost edge
connecting U and W , so it must be part of any MCST
Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 4/5
Summary

Kruskal’s algorithm builds an MCST bottom up

Start with n components, each an isolated vertex
Scan edges in ascending order of cost
Whenever an edge merges disjoint components, add it to the MCST

Correctness follows from Minimum Separator Lemma

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 5/5
Summary

Kruskal’s algorithm builds an MCST bottom up

Start with n components, each an isolated vertex
Scan edges in ascending order of cost
Whenever an edge merges disjoint components, add it to the MCST

Correctness follows from Minimum Separator Lemma

If edge weights repeat, MCST is not unique

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 5/5
Summary

Kruskal’s algorithm builds an MCST bottom up

Start with n components, each an isolated vertex
Scan edges in ascending order of cost
Whenever an edge merges disjoint components, add it to the MCST

Correctness follows from Minimum Separator Lemma

If edge weights repeat, MCST is not unique
“Choose minimum cost edge” will allow choices
Consider a triangle on 3 vertices with all edges equal

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 5/5
Summary

Kruskal’s algorithm builds an MCST bottom up

Start with n components, each an isolated vertex
Scan edges in ascending order of cost
Whenever an edge merges disjoint components, add it to the MCST

Correctness follows from Minimum Separator Lemma

If edge weights repeat, MCST is not unique
“Choose minimum cost edge” will allow choices
Consider a triangle on 3 vertices with all edges equal

Different choices lead to different spanning trees

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 5/5
Summary

Kruskal’s algorithm builds an MCST bottom up

Start with n components, each an isolated vertex
Scan edges in ascending order of cost
Whenever an edge merges disjoint components, add it to the MCST

Correctness follows from Minimum Separator Lemma

If edge weights repeat, MCST is not unique
“Choose minimum cost edge” will allow choices
Consider a triangle on 3 vertices with all edges equal

Different choices lead to different spanning trees

In general, there may be a very large number of minimum cost spanning trees

Madhavan Mukund Minimum Cost Spanning Trees:Kruskal’s Algorithm Mathematics for Data Science 1, Week 12 5/5

Graph Theory for Computer Scientists
No ratings yet
Graph Theory for Computer Scientists
12 pages
Understanding Graph Theory Concepts
100% (2)
Understanding Graph Theory Concepts
108 pages
Lecture 11
No ratings yet
Lecture 11
32 pages
Introduction to Graphs Overview
No ratings yet
Introduction to Graphs Overview
69 pages
Lecture 1
No ratings yet
Lecture 1
51 pages
Lecture Notes
No ratings yet
Lecture Notes
91 pages
Lec08 Pre
No ratings yet
Lec08 Pre
63 pages
DSA Lec11 Graph
No ratings yet
DSA Lec11 Graph
54 pages
Unit IV: Graphs (Refer T-1 and R-6)
No ratings yet
Unit IV: Graphs (Refer T-1 and R-6)
27 pages
GRAPH THEORY Presentation July192k16
No ratings yet
GRAPH THEORY Presentation July192k16
40 pages
Graphs
No ratings yet
Graphs
136 pages
Graph Traversals: BFS and DFS Explained
No ratings yet
Graph Traversals: BFS and DFS Explained
39 pages
Screenshot 2025-05-01 at 6.31.51 PM
No ratings yet
Screenshot 2025-05-01 at 6.31.51 PM
91 pages
Lecture 5: Basics of Graph Theory: 1 Definitions
No ratings yet
Lecture 5: Basics of Graph Theory: 1 Definitions
11 pages
Lecture 10 - Graph
No ratings yet
Lecture 10 - Graph
35 pages
Graph Theory: Concepts and Applications
No ratings yet
Graph Theory: Concepts and Applications
62 pages
BBM201 Lecture11
No ratings yet
BBM201 Lecture11
72 pages
Mathematics of Graphs
No ratings yet
Mathematics of Graphs
19 pages
Data Structure Algo Graph Classroom - Discussion Notes
No ratings yet
Data Structure Algo Graph Classroom - Discussion Notes
10 pages
APPLICATION HL Oxford CH 15
No ratings yet
APPLICATION HL Oxford CH 15
23 pages
Graphs and Relations Ann
No ratings yet
Graphs and Relations Ann
62 pages
Seminar 4 - Introduction To Graphs
No ratings yet
Seminar 4 - Introduction To Graphs
69 pages
Introduction to Graph Theory Concepts
No ratings yet
Introduction to Graph Theory Concepts
48 pages
Chapter 9
No ratings yet
Chapter 9
19 pages
Introduction To Networks: 15.053 March 22, 2007
No ratings yet
Introduction To Networks: 15.053 March 22, 2007
60 pages
Contemporary Mathematics Revised Edition
No ratings yet
Contemporary Mathematics Revised Edition
272 pages
Module9 Graphs
No ratings yet
Module9 Graphs
35 pages
Module 9 - Graphs
No ratings yet
Module 9 - Graphs
9 pages
20ma302 DM Unit-3 Notes
No ratings yet
20ma302 DM Unit-3 Notes
73 pages
Chapter 9 Graphs
No ratings yet
Chapter 9 Graphs
75 pages
CENG 205 Data Structures: Graphs
No ratings yet
CENG 205 Data Structures: Graphs
72 pages
Data Structures and Algorithms: (CS210/ESO207/ESO211)
No ratings yet
Data Structures and Algorithms: (CS210/ESO207/ESO211)
21 pages
Introduction To Graphs (Part-I)
No ratings yet
Introduction To Graphs (Part-I)
42 pages
Graph Theory Basics and Applications
No ratings yet
Graph Theory Basics and Applications
15 pages
Graph Worksheets
No ratings yet
Graph Worksheets
21 pages
Graph 1
No ratings yet
Graph 1
16 pages
Examsexpert - In: Graphs, Hashing, Sorting, Files
No ratings yet
Examsexpert - In: Graphs, Hashing, Sorting, Files
24 pages
CH 00
No ratings yet
CH 00
7 pages
Graph Theory Basics
No ratings yet
Graph Theory Basics
23 pages
Module 5 - Chapter 9 - Graphs and Algorithms - MSDS 6203 Data Systems and Algorithms N2A
No ratings yet
Module 5 - Chapter 9 - Graphs and Algorithms - MSDS 6203 Data Systems and Algorithms N2A
12 pages
Graph Theory: Dr. Safaa O. Al-Mamory
No ratings yet
Graph Theory: Dr. Safaa O. Al-Mamory
34 pages
Mathematics of Graph
No ratings yet
Mathematics of Graph
46 pages
Graph Theory Basics for Beginners
No ratings yet
Graph Theory Basics for Beginners
89 pages
Euler Trail in Graphs Explained
No ratings yet
Euler Trail in Graphs Explained
80 pages
Lec13 14 Graphs and Its Applications 18042023 091053am
No ratings yet
Lec13 14 Graphs and Its Applications 18042023 091053am
128 pages
Graphs: "One Graph Is Worth A Thousand Logs."
No ratings yet
Graphs: "One Graph Is Worth A Thousand Logs."
52 pages
Lecture 8
No ratings yet
Lecture 8
25 pages
Understanding Graphs and Their Applications
No ratings yet
Understanding Graphs and Their Applications
66 pages
Graph Theory: Circuits and Paths Explained
No ratings yet
Graph Theory: Circuits and Paths Explained
18 pages
Graph Theory Basics & Applications
No ratings yet
Graph Theory Basics & Applications
113 pages
Graph Theory - Shortest Path
No ratings yet
Graph Theory - Shortest Path
53 pages
LECTURE NOTES 1 - Introduction To Graphs To Post
No ratings yet
LECTURE NOTES 1 - Introduction To Graphs To Post
67 pages
Analysis and Design of Algoritms WK 10-12
No ratings yet
Analysis and Design of Algoritms WK 10-12
89 pages
Graphs, Hashing, Sorting, Files: Definitions: Graph, Vertices, Edges
No ratings yet
Graphs, Hashing, Sorting, Files: Definitions: Graph, Vertices, Edges
24 pages
Graph Theory
No ratings yet
Graph Theory
38 pages
Lecture07 Graph2
No ratings yet
Lecture07 Graph2
41 pages
Threads in Operating Systems
No ratings yet
Threads in Operating Systems
54 pages
Scott Britton - 63 Life Hacks
100% (2)
Scott Britton - 63 Life Hacks
82 pages
Chaudhary Devi Lal University
No ratings yet
Chaudhary Devi Lal University
1 page
Health and Safety Policy
No ratings yet
Health and Safety Policy
2 pages
Securities Market: Dr. Rana Singh 9811828987
No ratings yet
Securities Market: Dr. Rana Singh 9811828987
52 pages
Ebury: Cryptocurrency Theft Analysis
No ratings yet
Ebury: Cryptocurrency Theft Analysis
47 pages
URA Data (Jan 2010)
No ratings yet
URA Data (Jan 2010)
4 pages
Dyeing & Finishing List
No ratings yet
Dyeing & Finishing List
2 pages
CNG Station Locations in Gujarat
100% (1)
CNG Station Locations in Gujarat
9 pages
AG Cross Rivers V AG Fed (2012)
No ratings yet
AG Cross Rivers V AG Fed (2012)
134 pages
Valuation Methods of M&A
No ratings yet
Valuation Methods of M&A
17 pages
Constitutional Law Case References List
No ratings yet
Constitutional Law Case References List
10 pages
Machine Learning & Neural Networks Guide
No ratings yet
Machine Learning & Neural Networks Guide
8 pages
User-Generated Content Builds Brand Trust
No ratings yet
User-Generated Content Builds Brand Trust
6 pages
Writing Efficient CCodeforthe Lattice Mico 8 Microcontroller
No ratings yet
Writing Efficient CCodeforthe Lattice Mico 8 Microcontroller
12 pages
Electrical Meter Installation Guide
No ratings yet
Electrical Meter Installation Guide
3 pages
Banking Fraud: Detection & Prevention
No ratings yet
Banking Fraud: Detection & Prevention
10 pages
Apple ID Payment Methods by Country
No ratings yet
Apple ID Payment Methods by Country
1 page
Official Writeup (ENG)
No ratings yet
Official Writeup (ENG)
72 pages
Sports Operations Research Guide
No ratings yet
Sports Operations Research Guide
12 pages
Extended Binary Coded Decimal Interchange Code PDF
No ratings yet
Extended Binary Coded Decimal Interchange Code PDF
2 pages
Agile Web Development With AngularJS
No ratings yet
Agile Web Development With AngularJS
65 pages
Geo12 - India - 5 - Land Resources and Agriculture
No ratings yet
Geo12 - India - 5 - Land Resources and Agriculture
20 pages
Memory Management Notes OS
No ratings yet
Memory Management Notes OS
3 pages
MBBS Exam Revaluation Dispute
No ratings yet
MBBS Exam Revaluation Dispute
32 pages
Mensuration & Linear Equations
No ratings yet
Mensuration & Linear Equations
5 pages
Casr 105
100% (1)
Casr 105
11 pages
2006 Isuzu NPR Engine Diagnostics
No ratings yet
2006 Isuzu NPR Engine Diagnostics
19 pages
A Review of Flood Disaster and Disaster Management in Malaysia
No ratings yet
A Review of Flood Disaster and Disaster Management in Malaysia
9 pages
HC900 Redundant Model Guide
No ratings yet
HC900 Redundant Model Guide
5 pages