CMP 366.

3 Multimedia System

Chapter 3
Images and Graphics

3.1 Basic Images Concept

An image is a spatial representation of an object, a two dimensional or three dimensional
scenes or another image.
Abstractly, an image is a continuous function defining a rectangular region of a plane.
Intensity image: proportional to radiant energy received by a sensor/detector.
Range image: line of sight distance from sensor position.
An image can be thought of as a function with resulting values of the light intensity at
each point over a planar region.

Digital Image Representation

For computer representation, function (e.g. intensity) must be sampled at discrete
intervals.
Sampling quantizes the intensity values into discrete intervals.
Point at which an image is sampled are called picture elements or pixels.
Resolution specifies the distance between points accuracy.
A digital image is represented by a matrix of numeric values each representing a
quantized intensity value.
A digital image is a numeric representation (normal binary) of two dimensional images.
When I is a two-dimensional matrix, then I (r, c) is the intensity value at the position
corresponding to row r and column c of the matrix.
Intensity value can be represented by bits for black and white images (binary valued
images), 8 bits for monochrome imagery to encode color or grayscale levels, 24 bit
(RGB).
Image Formats
There are different kinds of image formats in the literature. We shall consider the image
format that comes out of an image frame grabber, i.e., the captured image format, and the
format when images are stored, i.e., the stored image format.

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 1
CMP 366.3 Multimedia System

(i) Captured Image Format

(ii) Stored Image Format

Captured Image Format: -

The image format is specified by two main parameters: spatial resolution, which is
specified as pixels x pixels (e.g. 640x480) and color encoding, which is specified by bits
per pixel. Both parameter values depend on hardware and software for input/output of
images.

Stored Image Format: -

When we store an image, we are storing a two-dimensional array of values, in which each
value represents the data associated with a pixel in the image. For a bitmap, this value is a
binary digit.
A bitmap is a simple information matrix describing the individual dots that are the
smallest elements of resolution on a computer screen or other display or printing device.
Image file format include:
GIF (Graphic Interchange Format)
X11 bitmap
Postscript
JPEG (Joint Picture Expert Group)
TIFF (Tagged Image File Format) etc.
There are many file formats used to store bitmaps and vectored drawing. Following is a
list of few image file formats.

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 2
CMP 366.3 Multimedia System

Graphics Format
Graphic image formats are specified through graphics primitives and their attributes.
Graphic primitive – line, rectangle, circle, ellipses, specification 2D and 3D
objects.
Graphic attribute – line style, line width, color.
Graphics formats represent a higher level of image representation, i.e., they are not
represented by a pixel matrix initially.
PHIGS (Programmer’s Hierarchical Interactive Graphics)
GKS (Graphical Kernel System)

3.2 Computer Image Processing

Image processing is any form of signal processing for which the input is an image, such
as a photograph or video frame; the output of image processing may be either an image or
a set of characteristics or parameters related to the image.
Image processing usually refers to digital image processing, but optional and analog
image processing also are possible.
Computer graphics concern the pictorial synthesis of real or imaginary objects from their
computer-based models.
The related field of image processing treats the converse process: the analysis of scenes,
or the reconstruction of model from pictures of 2D or 3D objects.

Image Synthesis
Image synthesis is an integral part of all computer user interfaces is indispensable for
visualizing 2D, 3D and higher dimensional objects. Areas as diverse as education,
science, engineering, medicine, advertising and entertainment all rely on graphics.

Dynamic in Graphics
Graphics are not confined to static pictures. Picture can be dynamically varied; for
example, a user can control animation by adjusting the speed, portion of the total scene
inn view, amount of detail shown, etc.

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 3
CMP 366.3 Multimedia System

Motion Dynamic:
With motion dynamic, objects can be moved and enabled with respect to a stationary
observer.

Update Dynamic:
Update dynamic is the actual change of the shape, color, or other properties of the objects
being viewed.

The Framework of Interactive Graphics System

Image can be generated by video digitizer cards that capture NTSC (PAL) analog signals
and create a digital image.
Graphical images are generated using interactive graphics systems.
The high-level conceptual framework of almost any interactive graphics system consists
of three software components: an application model, an application program and a
graphics system, and a hardware component: graphics hardware.

Application Model:
The application model represents the data or objects to be picture on the screen; it is
stored in an application database. The model is an application-specific and is created
independency of any particular display system.

Application Program:
The application program handles user input. It produces views by sending to the third
component, the graphics system, a series of graphics output commands that contain both
a detailed geometric description of what is to be viewed and the attributes describing how
the objects should appear.

Graphics System:
The graphics system is responsible for actually producing the picture from the detailed
descriptions and for passing the user’s input to the application program for processing.

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 4
CMP 366.3 Multimedia System

The graphics system is an intermediary component between the application program and
the display hardware.

Graphics Hardware:
At the hardware level, a computer receives input from interaction devices and output
images to display devices.
Input:
Current input technology provide us with the ubiquitous mouse, the data tablet and
transparent, touch sensitive panel mounted on the screen.
The other graphics input are track-balls, space-balls or the data glove.
Track-ball can be made to sense rotation about the vertical axis in addition to the about
two horizontal axes.
A space-ball is a rigid sphere containing strain gauges. The user pushes or pulls the
sphere in any direction, providing 3D translation and orientation.
The data glove records hand position and orientation as well as finger movements. It is a
glove covered with small, lightweight sensors.

Output: Raster Display

Most common type of graphic monitors using raster scan display type CRT
Point plotting device
Based on TV technology
Electron beam is swept across the screen, one row at a time from top to bottom,
starting at the upper left corner of the display
Process is repeated until the entire screen is covered, and the beam is then
returned to the upper left corner to start a new scan
Beam intensity is turned on and off to create a pattern of illuminated spots
Pictures are dynamically stored in a piece of memory known as frame buffer or
refresh buffer
This buffer holds the set of intensity values all the screen points (pixels)
Requirement to control the intensity of the screen positions:
Simple black and white system:

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 5
CMP 366.3 Multimedia System

- 1 bit per pixel (bitmap)

Color system:
- 24 bits/pixel (maximum no. of color representation, pixmap)
Frame buffer or refresh buffer (storage) requirements:
Large storage
e.g. 24 bits/pel, screen resolution of
1024x1024 requires 3mb of RAM
Refresh rate: 60 to 80 frames per second

Figure 3.1 Raster Scan

Advantages of Raster Scan Display:
Capable of presenting bright pictures
Unaffected by picture complexity
Suitable for showing dynamic motion
Lower cost
Ability to display areas filled with solid colors or patterns

Disadvantages of Raster Scan Display:

Requires large amount of memory (RAM)
Produced “stair stepped” appearance of diagonal lines on the image (known as
aliasing effect)
True line cannot be represented exactly due to the discretization of the display
surface (discrete nature of pixel representation)

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 6
 CMP 366.3 Multimedia System

Image Analysis
Image analysis is concerned with techniques for extracting descriptions from images that
are necessary for high-level scene analysis methods.
Image analysis techniques include computation of perceived brightness and color, partial
or complete recovery of three-dimensional data in the scene, location of discontinuities
corresponding to objects in the scene and characterization of the properties of uniform
regions in the image.
Image processing includes image enhancement, pattern detection and recognition and
scene analysis and computer vision.
Image enhancement deals with improving image quality by eliminating noise or by
enhancing contrast.
Pattern detection and recognition deal with detecting and clarifying standard patterns and
finding distortions from these patterns.
Scene analysis and computer vision deal with recognizing and reconstructing 3D models
of a scene from several 2D images.

Image Recognition Steps

Intermediate Image

Source of Image Formatting Observed

Image
Conditioning
Conditioned Digital image
data structure
Image
Labeling
Labeled
Image
Grouping
Grouped
Extracting Image
Logical data
structure
Extracted
Synchronization Targeted Image Matching Image

Figure 3.2 Image Recognition Steps

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 7
CMP 366.3 Multimedia System

Formatting
Capturing an image from a camera and bringing it into a digital form. It means that we
will have a digital representation of an image in the form of pixels.

Conditioning
In image, there is usually uninteresting object introduced during digitize as noise. In
conditioning, interesting objects are highlighted by suppressing or analyzing
uninteresting in systematic or patterned variations. Conditioning is typically applied
uniformly and is context-independent.

Labeling
The informative pattern has structure as a spatial arrangement of events, each spatial
event being a set of connected pixels. Labeling determines in what kinds of spatial events
each pixel participates.
E.g. edge detection technique
Edge detection technique determines continuous adjacent pairs which differ in intensity
or color. Another labeling operation must occur after edge detection, namely
thresholding.
Thresholding specifies which edges should be accepted and which should not; the
thresholding operation filters only the significant edges from the image and labels them.

Grouping
It can turn edges into line by determining edges belongs to same spatial event. A
grouping operation, where edges are grouped into lines, is called line filtering. The
grouping operation involves a change of logical data structure.

Extracting
Generating list of properties from set of pixel in spatial event. Extraction can also
measure topological or spatial relationship between two or more grouping.

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 8
CMP 366.3 Multimedia System

Matching
After the completion of the extracting operation, the events occurring on the image have
been identified and measured but the events in and of themselves have no meaning.
It is the matching operation that determines the interpretation of some related set of
image events, associating these events with some given three dimensional object or two-
dimensional shape.
The classic example is template matching, which compares the examined pattern with
stored models (templates) of known patterns and chooses the best match.

Image Transmission
Image transmission takes into account transmission of digital images through computer
networks. There are several requirements on the networks when images are transmitted:
The network must accommodate bursty data transport because image transmission
is bursty (The burst is caused by the large size of the image).
Image transmission requires reliable transport.
Time-dependence is not a dominant characteristic of the image in contrast to
audio/video transmission.

Image size depends on the image representation format used for transmission. There are
several possibilities:
Raw Image Data Transmission
The image is generated through a video digitizer and transmitted in its digital format.
Size = Spatial_resolution x Pixel_quantization
For example, the transmission of an image with a resolution of 640 x 480 pixels and pixel
quantization of 8 bit per pixel requires transmission of 307,200 bytes through the
network.

Compressed Image Data Transmission

The image is generated through a video digitizer and compressed before transmission.
The reduction of image size depends on the compression method and compression rate.
JPEG (Joint Picture Expert Group) & MPEG (Motion Picture Expert Group)

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 9
CMP 366.3 Multimedia System

Symbolic Image Data Transmission

The image is represented through symbolic data representation as image primitives (e.g.
2D or 3D geometric representation), attributes and other control information.

3.3 Image Enhancement

Enhancement is the process an image so that the result is more suitable than the original
image for a specific application.
Enhancement approaches:
Spatial domain:
Spatial domain techniques are techniques that operate directly on pixels.
Frequency domain:
Frequency domain techniques are based on modifying the Fourier transform of an
image.

g(x,y)=T[f(x,y)]
f(x,y): input image, g(x,y): processed image
T: an operator

Figure 3.3: Background of Spatial Domain

Spatial Domain: Point Processing

s=T(r)
r: gray-level at (x,y) in original image f(x,y)
s: gray-level at (x,y) in processed image g(x,y)
T is called gray-level transformation or mapping

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 10
CMP 366.3 Multimedia System

Figure 3.4: Spatial Domain: Point Processing

Contrast Stretching: to get an image with higher contrast than the original image.
The gray levels below m are darkened and the levels above m are brightened.

Figure 3.5: Contrast Stretching

Notes by: Parash Mani Bhandari GCES, Lamachaur, Pokhara

Page 11