Photo processing with cs1media

Otfried Cheong, Edited by JinYeong Bak∗

February 29, 2016

1 First steps with cs1media h rows of pixels (numbered 0 to h-1 from top to
bottom) and w columns (numbered 0 to w-1 from
Suppose you have a photo taken with your digital
left to right).
camera on your computer, and upload it in elice.
The cs1media module represents each pixel as
You can load this photo using:
an RGB color, which is a triple of red, green, and
from cs1media import * blue light intensities. Each intensity is an integer
img = load_picture("photos/[Link]") between 0 (no light) and 255 (full light). So red is
(255, 0, 0), and yellow is (255, 255, 0).
Many colors are predefined and can be accessed
You have now created a picture object and as-
like this:
signed the name img to it. You can get some in-
teresting information about the picture using the >>> [Link]
methods size() (which returns a tuple consisting (255, 255, 0)
of the width and height of the picture in pixels) and >>> [Link]
title() (which typically returns the filename). If (210, 105, 30)
you call the method show(), then Python will start
a program to display the picture:1 To get a list of all predefined colors, use
help(Color) when you have imported the
>>> type(img) cs1media module.
<class '[Link]'>
>>> [Link]()
(500, 375) 3 Color modifications
>>> [Link]() Looking at our photograph, it is rather dark. Let’s
'[Link]' make it lighter by increasing the light intensity of
>>> [Link]() each pixel.

def make_lighter(img, factor):

w, h = [Link]()
for y in range(h):
for x in range(w):
r, g, b = [Link](x, y)
r = int(factor * r)
g = int(factor * g)
b = int(factor * b)
if r > 255: r = 255
if g > 255: g = 255
if b > 255: b = 255
2 Digital images [Link](x, y, (r, g, b))
Digital images are stored as a rectangular matrix
of tiny squares called pixels (picture elements). If img = load_picture("photos/[Link]")
the image has width w and height h, then there are make_lighter(img, 1.5)
∗ He updates this note for Python 3 and elice platform.
[Link]()
Email: [Link]@[Link]
1 ’>>>’ is the primary prompt in Python interactive
To save a copy of the modified image, call
mode. The elice platform do not support the interactive
mode. To see the result, you use print function with each img.save_as("geowi_lighter.jpg")
code or run it in your own computer.

1
 When called as make_lighter(img, 0.5), the
function actually makes the picture darker. def mirror(img):
With the same principle, we can create various w, h = [Link]()
image effects. For instance, we can create the sen- for y in range(0, h):
sation of a sunset by turning an image redder (by for x in range(w / 2):
reducing the blue and green light intensity). We pl = [Link](x, y)
can create a negative image if we replace (r,g,b) pr = [Link](w - x - 1, y)
by (255-r,255-g,255-b): [Link](x, y, pr)
[Link](w - x - 1, y, pl)

Instead of reflecting the entire photo, we can take

To create a black-and-white image, we have to
the left half, reflect it, and use it as the new right
compute the “brightness” or luminance of each
half:
pixel. The total amount of light of a pixel seems to
be r + g + b, but in fact this gives an unnatural
result. The reason is that the human eye perceives
blue to be much darker than green when the inten-
sity of the light is the same. A good conversion to
luminance is given by the following function:

def luminance(p):
r, g, b = p
return int(0.299*r + 0.587*g + 0.114*b)

Once we have the luminance v of a pixel, we can

set it to the graytone pixel (v,v,v).
Even though it’s called “black and white”, we
have actually created graytone images. We can cre-
ate images really using only black and white if we
threshold the image: if the luminance is larger than
some threshold, we set the pixel to white, otherwise It is interesting to apply this technique to photos
to black: of human faces, because faces are symmetric, but
not perfectly so, and the symmetrized version looks
interestingly different from the real face.

5 Pixelation

4 Reflections So far we modified every pixel of the photo inde-

pendently. Let’s see what we can do if we look at
When you look in a mirror, you see yourself mir- several pixels together. For instance, we can take
rored. We can achieve this effect: the average color for a group of pixels:

2
 6 Blurring
def blocks(img, step):
w, h = [Link]() If instead we compute a new color for each pixel by
norm = step * step taking an average of a block around it, we can blur
for y in range(0, h - step, step): pictures. This is useful on pictures where pixels
for x in range(0, w - step, step): are visible because they have been magnified too
# compute average much, etc. To implement this, we need two picture
r, g, b = 0, 0, 0 objects to work with, because we need the original
for x0 in range(step): pixel values to compute each new pixel:
for y0 in range(step):
r0,g0,b0 = [Link](x+x0, y+y0) def blur(img, radius):
r, g, b = r+r0, g+g0, b+b0 w, h = [Link]()
r, g, b = r/norm, g/norm, b/norm result = create_picture(w,h,[Link])
# set block norm = (2 * radius + 1)**2
for x0 in range(step): for y in range(radius, h-radius):
for y0 in range(step): for x in range(radius, w-radius):
[Link](x+x0, y+y0, (r, g, b)) # compute average
r, g, b = 0, 0, 0
for x0 in range(-radius,radius+1):
for y0 in range(-radius,radius+1):
r0, g0, b0 = [Link](x+x0, y+y0)
r, g, b = r+r0, g+g0, b+b0
r, g, b = r/norm, g/norm, b/norm
[Link](x, y, (r, g, b))
return result

This technique is often used to make part of

an image unrecognizable, for instance car licence
plates or faces on Google street view. To do that,
we have to restrict the pixelation to a smaller rect-
angle of the photo:
def blocks_rect(img, step, x1,y1, x2,y2):
w, h = [Link]()
for y in range(x1, x2 - step, step): 7 Edge detection
for x in range(y1, y2 - step, step):
# same as before The human visual system works by detecting
boundaries. Even though natural objects like hu-
man faces have no sharp boundaries, we can easily
recognize a child’s line drawing of a face. When
teaching a computer to recognize objects—an area
called computer vision—one of the first steps is to
recognize edges in the image.
A simple edge detection algorithm is the follow-
ing: we compare the luminances of a pixel and the
pixels to its left and above. If there is a big enough
difference in both directions, then we have found an
“edge” pixel. We can visualize this easily by draw-
ing edge pixels black, and all other pixels white:

3
def edge_detect(img, threshold): def interpolate(t, c1, c2):
w, h = [Link]() r1, g1, b1 = c1
r = create_picture(w, h) r2, g2, b2 = c2
for y in range(1, h-1): r = int((1-t) * r1 + t * r2)
for x in range(1, w-1): g = int((1-t) * g1 + t * g2)
pl = luminance([Link](x-1,y)) b = int((1-t) * b1 + t * b2)
pu = luminance([Link](x,y-1)) return (r, g, b)
p = luminance([Link](x,y))
if (abs(pl - p) > threshold and def gradient(img, c1, c2):
abs(pu - p) > threshold): w, h = [Link]()
[Link](x, y, [Link]) for y in range(h):
else: t = float(y) / (h-1) # 0 .. 1
[Link](x, y, [Link]) p = interpolate(t, c1, c2)
return r for x in range(w):
[Link](x, y, p)

img = create_picture(100, 100)

gradient(img, [Link], [Link])

8 Artificial images
9 Cropping
So far we have modified pixel values based on the
value of the pixels. We can also create entirely arti- We often need to transform photos in various ways.
ficial images by setting pixels to whatever value we Cropping, for instance, means to cut out the part
like. Let’s first put our geese behind prison bars: of the photo we are really interested in:

def prison(img, step):

w, h = [Link]()
for y in range(0, h, step):
for x in range(w):
[Link](x, y, (0, 0, 0))

def crop(img, x1, y1, x2, y2):

w1 = x2 - x1
h1 = y2 - y1
r = create_picture(w1, h1)
for y in range(h1):
for x in range(w1):
[Link](x, y, [Link](x1+x, y1+y))
return r

img = load_picture("photos/[Link]")
r = crop(img, 67, 160, 237, 305)
And now an entirely artificial image:

4
 Sometimes we are given a photo with a frame 10 Rotations
(like our statue above). We can leave it to the com-
If we have taken a photo while holding our camera
puter to find the right coordinates for cropping—all
sideways, then we need to rotate the final photo:
we need to do is remove the part of the photo that
is similar in color to the frame color. def rotate(img):
We first need a function to compare two colors: # swap w and h:
def dist(c1, c2): h, w = [Link]()
r1, g1, b1 = c1 r = create_picture(w, h)
r2, g2, b2 = c2 for y in range(h):
return [Link]((r1-r2)**2 + for x in range(w):
(g1-g2)**2 + [Link](x, y, [Link](h - y - 1, x))
(b1-b2)**2) return r

The following function will then find the right x1

coordinate for cropping:
def auto_left(img, key, tr):
w, h = [Link]()
x = 0
while x < w:
for y in range(h):
if dist([Link](x, y), key) > tr:
return x
x += 1 11 Scaling
return None Very often we want to have a photo in a different
resultion than what we have taken. We create a
We do exactly the same for the top, bottom, and new picture with the right size, and then need to
right side of the photo, and can finally crop like compute the color of each pixel of the scaled image
this: by looking at the pixels of the original image.
def autocrop(img, threshold): Here is a version that is similar to the pixelation
# use top-left color as key idea we used above: we take the average color for
key = [Link](0, 0) a block of pixels from the original image.
x1 = auto_left(img, key, threshold)
x2 = auto_right(img, key, threshold) def scale_down(img, factor):
y1 = auto_up(img, key, threshold) w, h = [Link]()
y2 = auto_down(img, key, threshold) w /= factor
return crop(img, x1, y1, x2, y2) h /= factor
result = create_picture(w, h)
It turns out our statue actually has two frames, a norm = factor ** 2
black outer frame and a whitish inner frame: for y in range(h):
for x in range(w):
r1 = load_picture("photos/[Link]") # compute average
r2 = autocrop(r1, 60) r, g, b = 0, 0, 0
r3 = autocrop(r2, 200) x1 = x * factor
y1 = y * factor
for x0 in range(factor):
for y0 in range(factor):
r0,g0,b0 = [Link](x1+x0,y1+y0)
r, g, b = r+r0, g+g0, b+b0
r, g, b = r/norm, g/norm, b/norm
[Link](x, y, (r, g, b))
return result

This function can only scale down by an integer

factor. To scale down by other factors or to scale
up needs a different way to decide the color of each

5
pixel of the scaled image, possibly using interpola-
tion to create a pleasing result. def chroma(img, key, threshold):
w, h = [Link]()
for y in range(h):
12 Collage
for x in range(w):
A collage is a picture created by using (pieces of) p = [Link](x, y)
existing pictures. For instance, let’s start with this if dist(p, key) < threshold:
photo: [Link](x, y, [Link])

We’ll take our statue above and paste it in at a

suitable location:
def paste(canvas, img, x1, y1):
Now all we need is an improved version of the
w, h = [Link]()
paste function that will skip background pixels
for y in range(h):
when copying an image onto the canvas. We need
for x in range(w):
to pass the color key which we used for the back-
[Link](x1+x,y1+y, [Link](x,y))
ground to the function:
def collage1(): def chroma_paste(canvas,img,x1,y1,key):
trees = load_picture("[Link]") w, h = [Link]()
r1 = load_picture("[Link]") for y in range(h):
r2 = scale_down(r1, 2) for x in range(w):
mirror(r2) p = [Link](x, y)
paste(trees, r2, 100, 20) if p != key:
[Link]() [Link](x1 + x, y1 + y, p)

This works, but the result is not too exciting: Let’s try this on our example:
def collage2():
trees = load_picture("[Link]")
r1 = load_picture("[Link]")
r2 = scale_down(r1, 2)
mirror(r2)
chroma_paste(trees, r2, 100, 20,
[Link])
[Link]()

13 Chromakey
The background of the statue doesn’t look nat-
ural. Can we get rid of this background? In
this case, this is actually very easy, as the color
of the background is very different from the color
of the statue. We use the chromakey technique,
used in the entertainment industry ([Link]
[Link]/wiki/Chromakey). Let’s first try
it by marking the statue’s background in yellow:

6
14 Concatenating photos
Let’s make a comic strip out of a list of given pic-
tures. We first have to compute the size of the
result—its width is the sum of the width of the
given pictures, its height is the maximum height of
any picture. We then create a new picture of the
correct size, and and copy the images at the right
location one by one:

def concat(imglist):
# compute height and width
w = 0
h = 0
for img in imglist:
w1, h1 = [Link]()
w += w1
h = max(h, h1)
r = create_picture(w, h, [Link])
x0 = 0
for img in imglist:
w1, h1 = [Link]()
for y in range(h1):
for x in range(w1):
[Link](x0 + x, y, [Link](x, y))
x0 += w1
return r

Here is how we can use this:

def make_strip():
r1 = load_picture("[Link]")
r2 = load_picture("[Link]")
r3 = load_picture("[Link]")
return concat([r1, r2, r3])