EECE 5639 Computer Vision I

Lecture 1
Edge detection, Corner
Hw2 has been poste
Next Clas
Descriptors, Model Fitting

1
s

 Modern Edge Detection

P. Dollar and C. Zitnick

“Structured Forests for Fast Edge Detection,” ICCV 2013.

2
:

Let’s look at Canny’s output

Task: Find the cows in the image

3
Canny Edges

Image Gradients + NMS Canny’s Edges

4
Canny Edges

Image Gradients + NMS Canny’s Edges

5
Canny Edges

Image Gradients + NMS Canny’s Edges

Many distractors, missing edges!

6
 Can we do better?
Imagine … that someone goes and ANNOTATES which edges matter

7
 Annotate
Some people have done that!

The Berkeley Segmentation Dataset and Benchmark

by D. Martin, C. Fowlkes, D. Tal and J. Malik

8
 Using Annotations
How can we use annotations to improve (task oriented) edge detection?

We can use Machine Learning!

9
 Very over-simpli ed explanation
Supervised Classi cation Problem:

Samples are represented by “features”

Class 1

Class -1

We have “training” labeled data from 2 classes.

10
fi
fi
 Very over-simpli ed explanation
Supervised Classi cation Problem:

Class 1
?

Class -1

We have “training” labeled data from 2 classes.

Given a new sample, we want to assign to it a label.
11
fi
fi
 Very over-simpli ed explanation
Supervised Classi cation Problem:

class boundary

Class 1
?

Class -1

We have “training” labeled data from 2 classes.

Given a new sample, we want to assign to it a label.
12
fi
fi
 Very over-simpli ed explanation
Supervised Classi cation Problem

class boundary

Class 1
?

Class -1

We have “training” labeled data from 2 classes.

Given a new sample, we want to assign to it a label.
13
fi
fi
 Supervised Classi cation
There are many strategies to learn classi ers
Nearest neighbor
Support Vector Machin
Classi cation Tree
Classi cation Forest
Neural networks (deep learning
Classi ers provide
a class “label” for new sample
a “score” of how well the label ts the new sampl

14
fi
fi
fi
s

fi
)

fi
fi
e

Training an Edge Detector

15
Training an Edge Detector

Labeled Samples!

16
 Recipe for Training an Edge Detector
• Extract LOTS of image patche
• Assign labels to the samples using annotation
• Represent each patch using features (measurements) of the patc
• Simplest possibility would be to vectorize the patch:

17
s

 Recipe for Training an Edge Detector

• Extract LOTS of image patche
• Assign labels to the samples using annotation
• Represent each patch using features (measurements) of the patc
• Simplest possibility would be to vectorize the patc
• Better: extract meaningful features such as gradients, color histogram, etc

18
s

 Recipe for Training an Edge Detector

• Extract LOTS of image patche
• Assign labels to the samples using annotation
• Represent each patch using features (measurements) of the patc
• Simplest possibility would be to vectorize the patc
• Better: extract meaningful features such as gradients, color histogram, etc
• Train the classi er

19
fi
s

Using the Classi er

20
fi
Using the Classi er

21
fi
Using the Classi er

score

22
fi
Canny vs Structured Edge Detector

23
Canny vs Structured Edge Detector

24
Canny vs Structured Edge Detector

25
Canny vs Structured Edge Detector

26
Canny vs Structured Edge Detector

27
Evaluating Results

28
Evaluation: Recall & Precision

29
Evaluation: Recall & Precision

(green + red)

30
Evaluation

31
Point Features
Corners

 Corners are useful

Which city?

33
 Corners are useful
Which city?

34
 Corners are useful
Which city?

35
Corners are useful
Where can I nd this pattern?

36
fi
 Corners are useful

Textureless patches are nearly impossible to localize.

37
 Corners are useful

Patches with large contrast are easier to nd.

38
fi
 Corners are useful

Patches with large contrast are easier to nd, but edges cannot be localized.

39
fi
 Corners are useful

Corners are easier!!

40
 Detection of Corner Features
Need two strong edges
Example:

Create the following matrix:

Either Ex or Ey but not both are large in a neighborhood of

corner

If min(λ1,λ2) > T
There is a corner!

41

 Detection of Corner Features

What if the corner is not aligned with the image coordinate
system?

But this is also true

for a slanted edge!

Both, Ex or Ey are large in the neighborhood of the corner

42
 Detection of Corner Features
Solution:
“Rotate” the corner to align it with the image coordinate system!

43

 Detection of Corner Features

How do we do this rotation
Since C is symmetric, it can be diagonalized;
the diagonalization is done by the rotation we need!

44
?

 Harris Corner Detector

The Harris corner detector gives a mathematical approach for
determining the amount of changes when we move in all directions a
small window.

45
 Harris Corner Detector
Change of intensity for the shift [u,v]:

Change
Window In
Function Intensity

Window w(x,y):

46

First Order Approximation of f(x,y):

47
Harris Corner Detector

48
Harris Corner Detector

Rewrite as a Matrix:

49
Harris Detector

M is computed from the

gradient components

50
 Harris Detector

Intensity change in shifting window eigenvalue analysis:

51
Classi cation via Eigenvalues

52
fi
Corner Response Measure

53
Corner Response Measure

R = det M - k trace2(M)

54
Corner Response Example

R score; Gradient computed with Sobel mask

Window Gaussian, sigma=1

55

Corner Response Example: Edges

R < - 10000

56
Corner Response Example: Corners

R > 10000

57
Corner Response Example:

-10000 < R < 10000

Neither edges nor corners

58

 Harris Corner Detection Algorithm

Compute the Image Gradient
Ix = Gx,s * I and Iy = Gy,s *
Compute products of derivatives at each pixe
I2x = Ix . Ix and I2y = Iy. Iy and Ixy = Ix . I
Compute the sums of the products at each pixel using a window
averaging
S2x = Gs’ * I2x S2y = Gs’ * I2y Sxy = Gs’ * Ix
De ne the Matrix at each pixel M = [S2x Sxy ; Sxy S2y]
Compute the response R = det M - k trace(M)2
Threshold
Compute Nonmax suppression

59
fi
: