Digital Image Processing

Chapter 5: Image Restoration

A Model of the Image
Degradation/Restoration Process
 Degradation
⚫ Degradation function H
⚫ Additive noise  ( x, y )
⚫ Spatial domain

g ( x, y ) = h ( x, y ) * f ( x, y ) +  ( x, y )
⚫ Frequency domain

G (u , v) = H (u , v) F (u , v) + N (u , v)
 Restoration

g ( x, y )  Restoratio n Filter  fˆ ( x, y )
Noise Models
 Sources of noise
⚫ Image acquisition, digitization,
transmission
 White noise
⚫ The Fourier spectrum of noise is
constant
 Assuming
⚫ Noise is independent of spatial
coordinates
⚫ Noise is uncorrelated with respect to
the image itself
 Gaussian noise
⚫ The PDF of a Gaussian random variable,
z,
1 − ( z −  ) 2 / 2 2
p( z ) = e
2 
⚫ Mean: 
⚫ Standard deviation: 
⚫ Variance:  2
⚫ 70% of its values will be in the range

(  −  ), ( +  )
⚫ 95% of its values will be in the range

(  − 2 ), (  + 2 )
 Rayleigh noise
⚫ The PDF of Rayleigh noise,
2
 ( z − a )e −( z − a ) 2 / b
for z  a
p( z ) =  b
 0 for z  a
⚫ Mean:  = a+  b/4
b( 4 −  )
⚫ Variance:  =2

4
 Erlang (Gamma) noise
⚫ The PDF of Erlang noise, a0 , b
is a positive integer,
 a b z b −1 − a z
 e for z  0
p( z ) =  (b − 1)!
 0 for z  0
b
⚫ Mean: =
a
b
⚫ Variance:  = 2
2

a
 Exponential noise
⚫ The PDF of exponential noise, a0 ,

ae− a z for z  0
p( z ) = 
 0 for z  0
1
⚫ Mean: =
a
1
⚫ Variance:  = 2
2

a
 Uniform noise
⚫ The PDF of uniform noise,

 1
 if a  z  b
p( z ) =  b − a
 0 otherwise
a+b
⚫ Mean: =
2
⚫ Variance:  2 =
(b − a ) 2

12
 Impulse (salt-and-pepper) noise
⚫ The PDF of (bipolar) impulse noise,

 Pa for z = a

p( z ) =  Pb for z = b
0
 otherwise
⚫ b  a : gray-level b will appear as a
light dot, while level a will appear like
a dark dot
⚫ Unipolar: either Pa or Pb is zero
⚫ Usually, for an 8-bit image, a =0
(black) and b =0 (white)
 Modeling
⚫ Gaussian
 Electronic circuit noise, sensor noise due
to poor illumination and/or high
temperature
⚫ Rayleigh
 Range imaging
⚫ Exponential and gamma
 Laser imaging
⚫ Impulse
 Quick transients, such as faulty switching
⚫ Uniform
 Least descriptive
 Basis for numerous random number
generators
 Periodic noise
⚫ Arises typically from electrical or
electromechanical interference
⚫ Reduced significantly via frequency
domain filtering
 Estimation of noise parameters
⚫ Inspection of the Fourier spectrum
⚫ Small patches of reasonably constant
gray level
 For example, 150*20 vertical strips
 Calculate  ,  , a , b from
 =  zi p ( zi )
zi S

 =  ( zi −  ) p ( zi )
2 2

zi S
Restoration in the Presence of Noise
Only-Spatial Filtering

 Degradation
⚫ Spatial domain
g ( x, y ) = f ( x, y ) +  ( x, y )
⚫ Frequency domain

G (u , v) = F (u , v) + N (u , v)
 Mean filters
⚫ Arithmetic mean filter

ˆf ( x, y ) = 1
 g ( s, t )
mn ( s ,t )S xy
⚫ Geometric mean filter
1
  mn
fˆ ( x, y ) =   g ( s, t )
( s ,t )S xy 
⚫ Harmonic mean filter
 Works well for salt noise, but fails fpr
pepper noise

mn
fˆ ( x, y ) =
1

( s ,t )S xy g ( s, t )
⚫ Contraharmonic mean filter
 Q0 : eliminates pepper noise
 Q0 : eliminates salt noise

 g ( s ,
( s ,t )S xy
t ) Q +1

fˆ ( x, y ) =
 g ( s ,
( s ,t )S xy
t ) Q
 Usage
⚫ Arithmetic and geometric mean filters:
suited for Gaussian or uniform noise
⚫ Contraharmonic filters: suited for
impulse noise
 Order-statistics filters
⚫ Median filter
 Effective in the presence of both bipolar
and unipolar impulse noise

fˆ ( x, y ) = median{g ( s, t )}
( s ,t )S xy
⚫ Max and min filters
 max filters reduce pepper noise

fˆ ( x, y ) = max {g ( s, t )}
( s ,t )S xy

 min filters salt noise

fˆ ( x, y ) = min {g ( s, t )}
( s ,t )S xy
⚫ Midpoint filter
 Works best for randomly distributed noise,
like Gaussian or uniform noise

ˆf ( x, y ) = 1  max {g ( s, t )} + min {g ( s, t )}

2 ( s ,t )S xy ( s ,t )S xy 
⚫ Alpha-trimmed mean filter
 Delete the d/2 lowest and the d/2 highest
gray-level values
 Useful in situations involving multiple
types of noise, such as a combination of
salt-and-pepper and Gaussian noise

1
fˆ ( x, y ) =
mn − d
 g ( s, t )
( s ,t )S xy
r
 Adaptive, local noise reduction filter
⚫ If  2 is zero, return simply the value
of g ( x, y )
⚫ If    L , return a value close to
2 2

g ( x, y )
⚫ If 
 =  L , return the arithmetic
2 2

mean value mL

ˆf ( x, y ) = g ( x, y ) −   g ( x, y ) − m 
2

L 2 L
 Adaptive median filter
⚫ z min = minimum gray level value in
S xy
⚫ z max = maximum gray level value in
S xy
⚫ z med = median of gray levels in S xy
⚫ z xy = gray level at coordinates ( x, y )
⚫ S max = maximum allowed size of S xy
⚫ Algorithm:
⚫ Level A: A1= z med − z min
⚫ A2= z med − z max
⚫ If A1>0 AND A2<0, Go to
⚫ level B
⚫ Else increase the window size
⚫ If window size  S max
⚫ repeat level A
⚫ Else output z med
⚫ Level B: B1= z xy − z min
⚫ B2= z xy − z max
⚫ If B1>0 AND B2<0, output z xy
⚫ Else output z med
⚫ Purposes of the algorithm
 Remove salt-and-pepper (impulse) noise
 Provide smoothing

 Reduce distortion, such as excessive

thinning or thickening of object
boundaries
 Periodic Noise Reduction by Frequency
Domain Filtering

 Bandreject filters
⚫ Ideal bandreject filter
 W
1 if D(u, v)  D 0 −
2

 W W
H (u, v) = 0 if D 0 −  D(u, v)  D 0 +
 2 2
1 W
if D(u, v)  D 0 +

 2

D(u, v) = (u − M / 2) + (v − N / 2)
2

2 1/ 2
⚫ Butterworth bandreject filter of order n
1
H (u, v) = 2n
 D(u, v)W 
1+  2 2
 D (u, v) − D0 
⚫ Gaussian bandreject filter
2
1  D 2 ( u ,v ) − D02 
−  
2  D ( u ,v )W 
H (u, v) = 1 − e
 Bandpass filters

H bp (u, v) = 1 − H br (u, v)
  Notch filters
⚫ Ideal notch reject filter

0 if D1 (u, v)  D0 or D 2 (u, v)  D0
H (u, v) = 
1 otherwise


D1 (u, v) = (u − M / 2 − u0 ) + (v − N / 2 − v0 )
2

2 1/ 2

D (u, v) = (u − M / 2 + u )
2 0
2
+ (v − N / 2 + v ) 
0
2 1/ 2
 Butterworth notch reject filter of
order n
1
H (u, v) = n
 D 2

1+  0

 D1 (u, v) D2 (u , v) 
 Gaussian notch reject filter

1  D1 ( u ,v ) D2 ( u ,v ) 
−  2 
2  
H (u, v) = 1 − e
D0
 Notch pass filter

H np (u, v) = 1 − H nr (u, v)
 Optimum notch filtering
⚫ Interference noise pattern

N (u , v) = H (u , v)G (u , v)
⚫ Interference noise pattern in the spatial
domain

 ( x, y ) =  {H (u , v)G (u , v)}
−1

⚫ Subtract from g ( x, y ) a weighted

portion of  ( x, y ) to obtain an
estimate of f ( x, y )
fˆ ( x, y ) = g ( x, y ) − w( x, y ) ( x, y )
⚫ Minimize the local variance of fˆ ( x, y )
⚫ The detailed steps are listed in Page
251
⚫ Result

g ( x, y ) ( x, y ) − g ( x, y ) ( x, y )
w( x, y ) =
 2 ( x, y ) −  2 ( x, y )
Linear, Position-Invariant Degradations

 Input-output relationship

g ( x, y ) = H [ f ( x, y )] +  ( x, y )

 ( x, y ) = 0

g ( x, y ) = H [ f ( x, y )]
 H is linear if
H [af1 ( x, y ) + bf 2 ( x, y )]
= aH [ f1 ( x, y )] + bH [ f 2 ( x, y )]

⚫ Additivity

H [ f1 ( x, y ) + f 2 ( x, y )]
= H [ f1 ( x, y )] + H [ f 2 ( x, y )]
⚫ Homogeneity

H [af1 ( x, y )] = aH [ f1 ( x, y )]
⚫ Position (or space) invariant

H [ f ( x −  , y −  )] = g ( x −  , y −  )]
 ⚫ In terms of a continuous impulse
function
 
f ( x, y ) =   f ( ,  ) ( x −  , y −  )d d
− −

g ( x, y ) = H [ f ( x, y )]

= H   f ( ,  ) ( x −  , y −  )d d 
 

 − − 
g ( x, y )
H  f ( ,  ) ( x −  , y −  )d d
 
= 
− −

f ( ,  ) H  ( x −  , y −  )d d
 
= 
− −
 
=  f ( ,  ) h( x,  , y,  )d d
− −
⚫ Impulse response of H

h( x,  , y,  ) = H [ ( x −  , y −  )]
⚫ In optics, the impulse becomes a point
of light
⚫ Point spread function (PSF)

h ( x,  , y ,  )
⚫ All physical optical systems blur
(spread) a point of light to some
degree
 ⚫ Superposition (or Fredholm) integral of
the first kind

g ( x, y ) =
 
 
− −
f ( ,  )h( x,  , y,  )d d
 ⚫ If H is position invariant

H [ ( x −  , y −  )] = h( x −  , y −  )

⚫ Convolution integral

g ( x, y ) =
 
 − −
f ( ,  )h( x −  , y −  )d d
  In the presence of additive noise

g ( x, y ) =
 
 
− −
f ( ,  )h( x,  , y,  )d d +  ( x, y )
⚫ If H is position invariant
g ( x, y ) =
 
 
− −
f ( ,  )h( x −  , y −  )d d +  ( x, y )
⚫ If H is position invariant

g ( x, y ) = h ( x, y )  f ( x, y ) +  ( x, y )
G (u , v) = H (u , v) F (u , v) + N (u , v)
⚫ Restoration approach
 Image deconvolution
 Deconvolution filter
Estimating the Degradation Function

 Estimation by image observation

⚫ In order to reduce the effect of noise in
our observation, we would look for
areas of strong signal content

Gs (u, v)
H s (u, v) =
Fˆs (u, v)
 Estimation by experimentation
⚫ Obtain the impulse response of the
degradation by imaging an impulse
(small dot of light) using the same
system settings
G (u, v)
H (u, v) =
A
⚫ Observed image G (u , v )
⚫ The strength of the impulse A
 Estimation by modeling
⚫ Hufnagel and Stanley
⚫ Physical characteristic of atmospheric
turbulence

5
−k (u 2 +v 2 ) 6
H (u, v) = e
⚫ Image motion
T
g ( x, y ) =  f [ x − x0 (t ), y − y0 (t )]dt
0
  
G (u , v) =   g ( x, y )e − j 2 ( u x + v y )
dx dy
− −

=
  T
 f [ x − x (t ), y − y (t )]dt 
−  −     
0 0
0

− j 2 ( u x + v y )
e dx dy

=     f [ x − x0 (t ), y − y0 (t )]
T  

0  − −
e − j 2 ( u x + v y )

dx dy dt
 T
G (u , v) =  F (u , v)e − j 2 [ u x0 ( t ) + v y0 ( t )]
dt
0
T
= F (u , v)  e − j 2 [ u x0 ( t ) + v y0 ( t )]
dt
0

= F (u , v) H (u , v)
⚫ Where
T
H (u, v) =  e − j 2 [ u x0 ( t ) + v y0 ( t )]
dt
0
⚫ If x0 (t ) = at / T and y0 (t ) = 0
T
H (u, v) =  e − j 2 [ u x0 ( t )]
dt
0
T
= e − j 2 [ u at / T ]
dt
0

T − j ua
= sin( ua)e
 ua
⚫ If x0 (t ) = at / T and y0 (t ) = bt / T

H (u , v)
T − j ( ua + vb)
= sin[ (ua + vb)]e
 (ua + vb)
Inverse Filtering

 Direct inverse filtering

⚫ Limiting the analysis to frequencies
near the origin

ˆ G (u , v)
F (u , v) =
H (u , v)
N (u , v)
= F (u , v) +
H (u , v)
Minimum Mean Square Error (Wiener)
Filtering

 Minimize
2 ˆ
e = E{( f − f ) }
2

 Terms
⚫ H (u , v ) = degradation function
⚫

⚫ H  (u , v) = complex conjugate of H (u , v )
⚫
2
⚫ H (u , v ) = H  (u , v) H (u , v)
⚫
 S (u, v) = N (u, v) = power spectrum
2
⚫
of the noise
S f (u, v) = F (u, v) = power spectrum
2
⚫
of the undegraded image
 Wiener filter
 H * (u , v ) S (u , v ) 
Fˆ (u , v) =  G (u , v)
f

 S f (u , v) H (u , v) + S (u , v) 
2

 H * (u , v) 
= G (u , v)
 H (u , v) + S (u , v) / S f (u , v) 
2

 1 H (u , v)
2

= G (u , v)
 H (u , v) H (u , v) + S (u , v) / S f (u , v) 
2
⚫ White noise

 1 H ( u , v )
2

Fˆ (u , v) =  G (u , v)
 H (u , v) H (u , v) + K 
2
Constrained Least Squares Filtering

 Vector-matrix form
g ( x, y ) = h ( x, y ) * f ( x, y ) +  ( x, y )

⚫ g = Hf + η

⚫ g , f , η : MN  1
⚫ H : MN  MN
 Minimize

 
M −1 N −1
C =   f ( x, y ) 2 2

x =0 y =0

⚫ Subject to
2
ˆ
g − Hf = η
2
 The solution
 H * (u , v ) 
Fˆ (u , v) =  G (u , v)
 H (u , v) +  P (u , v) 
2 2

⚫ Where P (u , v ) is the Fourier transform

of the function
 0 −1 0 
P( x, y ) = − 1 4 − 1
 0 − 1 0 
 Computing  by iteration

r = g − Hfˆ
⚫ Adjust  so that
2 2
r = η a
⚫ Computation
M −1 N −1
r =  r 2 ( x, y )
2

x =0 y =0
M −1 N −1
1
m =
MN
 ( x, y)
x =0 y =0
2

  ( x, y) − m 
M −1 N −1
1
 =
2

MN x =0 y =0

η = MN [ 2 + m2 ]
2
⚫ Algorithm
⚫ 1: Specify an initial value of
⚫ 2: Compute
2 2
⚫ 3: Stop if r = η  a is satisfied;
otherwise return to Step 2 after
increasing  if r  η − a or
2 2

decreasing 
2 2
⚫ if r  η +. a
Geometric Mean FIlter

Fˆ (u , v) =
1−
 
 
 H * (u , v)   H * (u , v) 
    G (u , v)
2
 H (u , v)    S (u , v ) 


H (u , v) +  
2

 
 S ( u , v ) 
 
 f 
Geometric Transformations

 Spatial transformations
x ' = r ( x, y )
y ' = s ( x, y )

⚫ Tiepoints
⚫ Bilinear equations

x' = r ( x, y ) = c1 x + c2 y + c3 xy + c4
y ' = s ( x, y ) = c5 x + c6 y + c7 xy + c8
 Gray-level interpolation
v( x' , y ' ) = ax'+by '+cx ' y '+ d