EECE 5639 Computer Vision I

Lecture
Filtering, Noise Cleaning, Separable Filter
Next Clas
Edges, Corners, Model Fitting

1
8

 Probability
Independent events A, B and C P(ABC) = P(A)P(B)P(C)

Bayes Theorem P(A | B)P(B) = P(B | A)P(A)

First and Second Moments

(mean)

Variance: Var(X) = E {(X − E[X])2} = E[X 2] − E 2[X]

Std. Dev:

2
:

 Mean Estimation from Samples

Given a set of N samples from a distribution, we can
estimate the mean of the distribution by:

3
 Variance Estimation from Samples
Given a set of N samples from a distribution, we can
estimate the variance of the distribution by:

4
… back to images
Dealing with Noise
Image patch Noisy surface

We want something

more like this!

Bob Collins 6
 How can we reduce noise?
Image acquisition noise due to light uctuations and sensor
noise can be reduced by acquiring a sequence of images
and averaging them

WHY?

7
.

fl
 Measuring Noise
Noise Amount: SNR = σs/ σn
Noise Estimation:
Given a sequence of images I0,I1, … IN-1

Smoothing Filters
 Linear Spatial Filtering: CORRELATION
Alternative notation:

Where zi are the values of the input image under the mask

10
Correlation Example

11
 Smoothing Spatial Filters
They are used for blurring and noise reduction

Blurring is performed as pre-processing to remove small

detail or bridge curve gap

Noise reduction can be done by linear or nonlinear

ltering

12
fi
s

Linear Smoothing Filters

They are simply averaging lters: they compute the
average of the lters under the mask

They reduce sharp transitions

They are low pass lters.

13
fi
fi
fi
.

Average Filter
Mask with positive entries, that sum 1
Replaces each pixel with an average of its neighborhood
If all weights are equal, it is called a BOX lter.

1 1 1
1 1 1
1/9
1 1 1

14
.

fi
 Example:

X X X X X X
10 11 10 0 0 1
9 10 11 1 0 1 X 10 X
O X X
I 10 9 10 0 2 1
X X
11 10 9 10 9 11
F X X
9 10 11 9 99 11
X X X X X X
10 9 9 11 10 10 1 1 1
1 1
1/9 1
1 1 1

1/9.(10x1 + 11x1 + 10x1 + 9x1 + 10x1 + 11x1 + 10x1 + 9x1 + 10x1) =

1/9.( 90) = 10
15
 Example:

X X X X X X
10 11 10 0 0 1
9 10 11 1 0 1 X 10 7 4 1 X
O X X
I 10 9 10 0 2 1
X X
11 10 9 10 9 11
F X X
9 10 11 9 99 11
X X X X X X
10 9 9 11 10 10 1 1 1
1 1
1/9 1
1 1 1

1/9.(10x1 + 0x1 + 0x1 + 11x1 + 1x1 + 0x1 + 10x1 + 0x1 + 2x1) =

1/9.( 34) = 3.7778
16
 Example:

X X X X X X
10 11 10 0 0 1
9 10 11 1 0 1 X 10 7 4 1 X
O X X
I 10 9 10 0 2 1
X X
11 10 9 10 9 11
F X 20 X
9 10 11 9 99 11
X X X X X X
10 9 9 11 10 10 1 1 1
1 1
1/9 1
1 1 1

1/9.(10x1 + 9x1 + 11x1 + 9x1 + 99x1 + 11x1 + 11x1 + 10x1 + 10x1) =

1/9.( 180) = 20
17
 Example:

X X X X X X
10 11 10 0 0 1
9 10 11 1 0 1 X 10 7 4 1 X
O X X
I 10 9 10 0 2 1
X 18 X
11 10 9 10 9 11
F X 20 X
9 10 11 9 99 11
X X X X X X
10 9 9 11 10 10 1 1 1
1 1
1/9 1
1 1 1

1/9.(10x1 + 0x1 + 2x1 + 9x1 + 10x1 + 9x1 + 11x1 + 9x1 + 99x1) =

1/9.( 159) = 17.6667
18
Does it reduce noise?
Intuitively, takes out small variations
Consider the Image pixel values under the mask and the
corresponding output at the center:

Is the output better? How?

19
.

Does it reduce noise?

Assume that the noise in the image is uncorrelated, zero
mean, with stdev sigma
The expected value of a pixel before ltering is:

20
.

fi
Does it reduce noise?
Assume that the noise in the image is uncorrelated, zero
mean, with stdev sigma
The expected value of a pixel after ltering is:

21
.

fi
Does it reduce noise?
Assume that the noise in the image is uncorrelated, zero
mean, with stdev sigma
The variance of a pixel before ltering is:

22
.

fi
Does it reduce noise?
Assume that the noise in the image is uncorrelated, zero
mean, with stdev sigma
The variance of the pixel after ltering is:

23
.

fi
How big should the mask be?
The bigger the mask
more neighbors contribute
smaller noise variance of the output
bigger noise spread
more blurring
more expensive to compute.

24
.

Weighted Average Filter

Gives more weight at the central pixel and less weights
to the neighbors
The farther away the neighbors, the smaller the weight
Less blurring of edges

25
.

Gaussian Filter
A particular case of weighted averaging
The coef cients are a 2D Gaussian.

(0,0) is the center of the mask

σ determines how fast the weights decay

K is s.t. the sum of the coef cients is 1

26
fi
fi
:

Frequency Domain Interpretation

F
S

Gaussian lter is the only one that has the same shape
in the space and frequency domains.

There are no secondary lobes – I.e. a truly low-pass lter

27
fi
How big should the mask be?
The std. dev of the Gaussian σ determines the amount
of smoothing
The samples should adequately represent a Gaussian
For a 98.76% of the area, we nee
m = 5σ (space domain)
5.(1/σ) ≤ 2π (frequency domain) ⇒ σ ≥ 0.796, m ≥5

28
.

29
Effects of increasing mask size

30
 Ef cient Implementation
Both, the BOX lter and the Gaussian lter are
separable
First convolve each row with a 1D lte
Then convolve each column with a 1D lter.

31
fi
:

fi
fi
r

fi
fi
 Separable Filters

For a KxK lter: K2 operations/pixel

32
fi
 Separable Filters

∑∑ ∑∑ ∑ ∑
w(x, y)p(x, y) = w(x)w(y)p(x, y) = w(x) w(y)p(x, y)
x y x y x y

33
 Separable Filters

For a 1xK lter: K operations/pixel

34
fi
 Separable Filters

For a Kx1 lter: K operations/pixel

35
fi
Gaussian Filter is Separable

36
Separable Filters

37
 Separable Filters
How do we know a lter is separable
By inspectio
Looking at the analytic form of i
Looking at its Singular Value Decomposition (SVD)

F is separable if only one singular value is non-zero.

38
n

fi
t

 Cascading
Convolving twice with a Gaussian Kernel (σ), is the same as convolving once
with a Gaussian Kernel (√2σ)

39
 Ef cient Implementation: Integral Image
If an image is going to be repeatedly convolved with
different box lters a pre-computed summed area table
can save computations for future use.

40
fi
fi
 Ef cient Implementation: Integral Image
If an image is going to be repeatedly convolved with
different box lters a pre-computed summed area table
can save computations for future use.

28 = 19 +17 - 11 + 3
41
fi
fi
 Ef cient Implementation: Integral Image
If an image is going to be repeatedly convolved with
different box lters a pre-computed summed area table
can save computations for future use.

42
fi
fi
Limitations of averaging
Signal frequencies shared with noise are lost, resulting in
blurring
Impulsive noise is diffused but not removed
It spreads pixel values, resulting in blurring.

43
.

 Non-linear Filtering
Replace each pixel with the MEDIAN value of all the pixels in
the neighborhood.

44
 Example:

10 11 10 0 0 1 X X X X X X
9 10 11 1 0 1 X 10 X
I 10 9 10 0 2 1 O X X
11 10 9 10 9 11 X X
9 10 11 9 99 11 X X
10 9 9 11 10 10 X X X X X X

median

10,11,10,9,10,11,10,9, sort
10 9,9,10,10,10,10,10,11,11

45
 Example:

10 11 10 0 0 1 X X X X X X
9 10 11 1 0 1 X 10 10 1 1 X
I 10 9 10 0 2 1 O X X
11 10 9 10 9 11 X X
9 10 11 9 99 11 X X
10 9 9 11 10 10 X X X X X X

median

11,10,0,10,11,1,9,10 sort
,0 0,0,1,9,10,10,10,11,11

46
 Example:

X X X X X X
10 11 10 0 0 1
9 10 11 1 0 1 X 10 X
O X X
I 10 9 10 0 2 1
X 9 X
11 10 9 10 9 11
9 10 11 9 99 11 X 10 X
10 9 9 11 10 10 X X X X X X

median

10,9,11,9,99,11,11,10, sort
10 9,9,10,10,10,11,11,11,99

47
 Median Filter Properties
Non-linea
Does not spread the nois
Can remove spike nois
Expensive to run

48
r

Median Filter

49