Shanghai Jiao Tong University

Instructor: Xu Zhao

Computer Vision Class No.: AU7005

Spring 2023

Xu Zhao @ Shanghai Jiao Tong university

Lecture 4: Light and Shading

 Contents

❖ Physical models
❖ What makes a pixel take its brightness values?

❖ Inference from shading

❖ Photometric Stereo: Shape from Multiple Shaded

Images

Slides provided courtesy of Forsyth and Ponce

 Basic model

❖ Light arrives at a surface

❖ from a light source
❖ from another surface
❖ It is reflected into the camera
❖ It arrives at a sensor at the back of the camera
❖ and a record is made 、

❖ this could be a linear or a non-linear function of the amount

of light

 Effectes in camera
❖ Film: It’s non-linear and record is made by chemical processes
in the film.
❖ CCD: linear, with non-linearities made from electronics to mimic
film
❖ Camera response: linearly for most of the situations

Icamera(x) = k Ipatch(X)
Write X for a point in space that projects to x in the image,
Ipatch(X) for the intensity of the surface patch at X, and
Icamera(x) for the camera response at x. k is some constant
to be determined by calibration
7

 Reflection at a surface
❖ Many effects when light strikes a surface, could be:
❖ absorbed; transmitted; reflected; scattered
❖ eg. some people can see arteries, veins under their skin
because light is transmitted through skin, reflected at blood
vessel, transmitted out
❖ Simplify, assume that
❖ surfaces don’t fluoresce
❖ surfaces don’t emit light (i.e. are cool)
❖ all the light leaving a point is due to that arriving at that
point

 The important reflection modes

❖ Specular reflection (mirror like)
❖ Pure mirror: incoming, outgoing directions
and normal are coplanar. incoming,
outgoing angles to normal are equal
❖ Most specular surfaces: some light leaves
the surface along directions near to the
specular direction as well
❖ Diffuse reflection
❖ Light leaves in equal amounts in each
direction. So surface looks equally bright
from each viewing direction
9

D. Forsyth
 Specularities
❖ Mirrors are bright: reflect most incoming light
❖ Most specular surfaces aren’t pure mirrors
❖ eg plastics; rough or brushed metal surfaces; lacquers
❖ The only significant specular reflection is the light source
❖ Result:  small, bright patches on specular surfaces
❖ Specularities
❖ Move when the light source moves
❖ Move when the viewing direction moves
❖ Shape, motion depend on local geometry of the surface
❖ Specular albedo
❖ percentage of incoming light that is specularly reflected

11

 D. Forsyth
Flickr, by piratejohnny
 Diffuse reflection
❖ Light leaves the surface evenly in all directions. eg. cotton
cloth, carpets, matte paper, matte paints, etc.
❖ Described by one parameter:   Albedo
❖ percentage of light arriving that leaves
❖ range 0-1, practical range is smaller (0.05-0.9)
❖ Light leaving is (Albedo)x(Light arriving)
❖ Ambiguity:  A surface could be dark because
❖ It reflects a small percentage of the light arriving
❖ There isn’t very much light arriving

13

 How much light arrives?

❖ Assume source is far away
❖ So light travels in parallel rays

❖ (Light arriving) proportional to (number of rays striking

surface)
❖ Surface A below receives less light than surface B

❖ (number of rays striking surface) proportional to cos theta,

where theta is angle between normal and direction of travel
❖ Shadows: If point can’t see the light source, it is in shadow

14

 • E.g. the sun

• energy travels in parallel rays

Sources-point source at infinity

• energy density received is proportional to cos theta

• E.g.
• energy
the sun
Write:
• p fortravels
• albedoin parallel rays
❖ Energy• travels
• energy inreceived
S fordensity
source parallel
vector rays
is proportional to cos theta
❖ Energy• density
N for normalreceived is proportional to cos theta
• p for albedo
•
Write:I for image intensity

•
• S for source vector
I(x) =normal(x)S · N(x)
• N for
• I for image intensity
- I(x) - image intensity
- S for source vector
I(x) = (x)S · N(x)
- N for normal
A B
- ρ is albedo

16

 Diffuse+Specular model
❖ Most surfaces can be modeled as diffuse+specular
❖ surface parameters: 
❖ diffuse albedo
❖ specular albedo ❖ Seldom known, hard to measure,
❖ phong parameter ❖ usually not important
❖ This justifies the following strategy for many analyses
❖ Find and remove specularities，which are small, and bright
❖ Ignore the missing points, and treat the rest of the image as
diffuse

17

 Shadows
❖ Most shadows aren’t dark
❖ because shadow points get light from other surfaces, not just light source
❖ Area sources
❖ Large, bright areas
❖ eg diffuser boxes, the sky
❖ Yield smooth, blurry shadows
❖ Points that can see the whole source are brighter
❖ Points that can see only part of the source are darker (penumbra)
❖ Points that can see no part of the source are darkest (umbra)
❖ Other surfaces behave like area sources
❖ Smooth, blurry shadows are common (and sometimes too faint to see)

18

 Shadows cast
Shadows cast by
by a a point
point sourcesource
❖ A point can’t see the source is in shadow
❖ • A point that can’t see the source is in shadow
For point sources, the geometry is simple
• For point sources, the geometry is simple

21

Photometric stereo: shape from multiple shaded images

❖ Assumption:
❖ Orthographic projection: (x, y, f(x, y))- Monge patch
❖ Lambertain surface (diffuse model)
❖ Fixed camera and surface

❖ Photometric stereo: recover a representation of the Monge

patch from image data

22

x - a point on the surface

The brightness of x:
B(x) = ρ(x)N(x) ⋅ S1 V1 = kS1
Assume that the response of the camera is linear in the
surface radiosity, and so at pixel (x, y):
I(x, y) = kB(x) = kB(x, y) = kρ(x, y)N(x, y) ⋅ S1 = g(x, y) ⋅ v1

g(x, y) = ρ(x, y)N(x, y) v1 = kS1

g(x, y) - surface v1 - property of illumination

and camera

We want to reconstruct g
• If we have n sources, for each of which vi is known
and we stack each of these vi into a matrix

⎛V ⎞ 1
T

⎜ T⎟ • image measurements at per image point

⎜ V 2 ⎟
V=
⎜ ... ⎟ i(x, y) = {I1 (x, y), I 2 (x, y),..., I n (x, y)} T

⎜ T⎟
⎝ Vn ⎠
• Now we have a linear system:

i(x, y) = Vg(x, y)
Albedo:

ρ (x, y) = g(x, y)

Surface normal
g(x, y)
N(x, y) =
g(x, y)

The surface is (x,y,f(x,y)), so the normal as a function

of (x,y) is T
1 ⎧∂ f ∂ f ⎫
N(x, y) = ⎨ , ,1⎬
∂f 2
∂ f ⎩ ∂x ∂y ⎭
2
1+ +
∂x ∂y

To recover the depth map, we need to determine

f(x,y) from N. Unit normal at some point (x,y) is
(a(x,y), b(x,y), c(x,y))
∂ f a(x, y) ∂2 f ∂2 f
= =
∂x ∂y ∂y ∂x
∂x cb(x, y)
⎛ a(x, y) ⎞ ⎛ b(x, y) ⎞
∂ f b(x, y) ∂⎜ ⎟ ∂⎜
= ⎝ c(x, y) ⎠ ⎝ c(x, y) ⎟⎠
∂y c(x, y) − =0
∂y ∂x

recover f(x,y) by summing the change along some path

⎛∂f ∂f ⎞
∫ C ⎜⎝ ∂x ∂y ⎟⎠
f (x, y) = ! , ⋅ dl + c