Chapter 3

Data Representations
Graphic/Image Data Representation
An image could be described as two-dimensional array of points where every point is
allocated its own color. Every such single point is called pixel, short form of picture
element. Image is a collection of these points that are colored in such a way that they
produce meaningful information/data.

Pixel (picture element) contains the color or hue and relative brightness of that point in
the image. The number of pixels in the image determines the resolution of the image.

 A digital image consists of many picture elements, called pixels.

 The number of pixels determines the quality of the imageimage resolution.
 Higher resolution always yields better quality.
 Bitmap resolution most graphics applications let you create bitmaps up to 300
dots per inch (dpi). Such high resolution is useful for print media, but on the
screen most of the information is lost, since monitors usually display around 72 to
96 dpi.
 A bit-map representation stores the graphic/image data in the same manner that
the computer monitor contents are stored in video memory.
 Most graphic/image formats incorporate compression because of the large size of
the data.


Pixel
Fig 1 pixels

Types of images

There are two basic forms of computer graphics: bit-maps and vector graphics.
The kind you use determines the tools you choose. Bitmap formats are the ones used for
digital photographs. Vector formats are used only for line drawings.

Bit-map images (also called Raster Graphics)

They are formed from pixels—a matrix of dots with different colors. Bitmap images are
defined by their dimension in pixels as well as by the number of colors they represent.
For example, a 640X480 image contains 640 pixels and 480 pixels in horizontal and
vertical direction respectively. If you enlarge a small area of a bit-mapped image, you can
clearly see the pixels that are used to create it (to check this open a picture in flash and
change the magnification to 800 by going into View->magnification->800.).

Each of the small pixels can be a shade of gray or a color. Using 24-bit color, each pixel
can be set to any one of 16 million colors. All digital photographs and paintings are
bitmapped, and any other kind of image can be saved or exported into a bitmap format. In
fact, when you print any kind of image on a laser or ink-jet printer, it is first converted by
either the computer or printer into a bitmap form so it can be printed with the dots the
printer uses.

To edit or modify bitmapped images you use a paint program. Bitmap images are widely
used but they suffer from a few unavoidable problems. They must be printed or displayed
at a size determined by the number of pixels in the image. Bitmap images also have large
file sizes that are determined by the image’s dimensions in pixels and its color depth. To
reduce this problem, some graphic formats such as GIF and JPEG are used to store
images in compressed format.

Vector graphics
They are really just a list of graphical objects such as lines, rectangles, ellipses, arcs, or
curves—called primitives. Draw programs, also called vector graphics programs, are used
to create and edit these vector graphics. These programs store the primitives as a set of
numerical coordinates and mathematical formulas that specify their shape and position in
the image. This format is widely used by computer-aided design programs to create
detailed engineering and design drawings. It is also used in multimedia when 3D
animation is desired. Draw programs have a number of advantages over paint-type
programs.

These include:
 Precise control over lines and colors.
 Ability to skew and rotate objects to see them from different angles or add
perspective.
 Ability to scale objects to any size to fit the available space. Vector graphics always
print at the best resolution of the printer you use, no matter what size you make them.
 Color blends and shadings can be easily changed.
 Text can be wrapped around objects.

Monochrome/Bit-Map Images

 Each pixel is stored as a single bit (0 or 1)

 The value of the bit indicates whether it is light or dark
 A 640 x 480 monochrome image requires 37.5 KB of storage.
 Dithering is often used for displaying monochrome images
 Fig 3 Monochrome bit-map image

Gray-scale Images
 Each pixel is usually stored as a byte (value between 0 to 255)
 This value indicates the degree of brightness of that point. This brightness goes
from black to white
 A 640 x 480 grayscale image requires over 300 KB of storage.

Fig 4 Gray-scale bit-map image

8-bit Color Images

 One byte for each pixel
 Supports 256 out of the millions possible, acceptable color quality
 Requires Color Look-Up Tables (LUTs)
 A 640 x 480 8-bit color image requires 307.2 KB of storage (the same as 8-bit
greyscale)
 Examples: GIF
Fig 5 8-bit Color Image

24-bit Color Images

 Each pixel is represented by three bytes (e.g., RGB)
 Supports 256 x 256 x 256 possible combined colors (16,777,216)
 A 640 x 480 24-bit color image would require 921.6 KB of storage
 Most 24-bit images are 32-bit images,
o the extra byte of data for each pixel is used to store an alpha value
representing special effect information

Image Resolution

Image resolution refers to the spacing of pixels in an image and is measured in pixels per
inch, ppi, sometimes called dots per inch, dpi. The higher the resolution, the more pixels
in the image. A printed image that has a low resolution may look pixelated or made up of
small squares, with jagged edges and without smoothness.
Image size refers to the physical dimensions of an image. Because the number of pixels
in an image is fixed, increasing the size of an image decreases its resolution and
decreasing its size increases its resolution.

Popular File Formats

Choosing the right file type for your image to save in is of vital importance. If you are,
for example, creating image for web pages, then it should load fast. So such images
should be small size. The other criteria to choose file type is taking into consideration the
quality of the image that is possible using the chosen file type. You should also be
concerned about the portability of the image.
To choose file type:
 resulting size of the imagelarge file size or small
 quality of image possible by the file type
 portability of file across different platforms
The most common formats used on internet are the GIF, JPG, and PNG.
Standard System Independent Formats

GIF
 Graphics Interchange Format (GIF) devised CompuServe, initially for transmitting
graphical images over phone lines via modems.
 Uses the Lempel-Ziv Welch algorithm (a form of Huffman Coding), modified slightly
for image scan line packets (line grouping of pixels).
 LZW compression was patented technology by the UNISYS Corp.
 Limited to only 8-bit (256) color images, suitable for images with few distinctive
colors (e.g., graphics drawing)
 Supports one-dimensional interlacing (downloading gradually in web browsers.
Interlaced images appear gradually while they are downloading. They display at a
low blurry resolution first and then transition to full resolution by the time the
download is complete.)
 Supports animation—multiple pictures per file (animated GIF)
 GIF format has long been the most popular on the Internet, mainly because of its
small size
 GIFs allow single-bit transparency, which means when you are creating your image,
you can specify one colour to be transparent. This allows background colours to show
through the image.

PNG
 stands for Portable Network Graphics
 It is intended as a replacement for GIF in the WWW and image editing tools.
 GIF uses LZW compression which is patented by Unisys. All use of GIF may have to pay
royalties to Unisys due to the patent.
 PNG uses unpatented zip technology for compression
 One version of PNG, PNG-8, is similar to the GIF format. It can be saved with a maximum of
256 colours and supports 1-bit transparency. Filesizes when saved in a capable image editor
like FireWorks will be noticeably smaller than the GIF counterpart, as PNGs save their colour
data more efficiently.
 PNG-24 is another version of PNG, with 24-bit colour support, allowing ranges of colour to a
high colour JPG. However, PNG-24 is in no way a replacement format for JPG, because it is
a loss-less compression format which results in large filesize.
 Provides transparency using alpha value
 Supports interlacing
 PNG can be animated through the MNG extension of the format, but browser support is less
for this format.

JPEG/JPG
 A standard for photographic image compression
 created by the Joint Photographic Experts Group
 Intended for encoding and compression of photographs and similar images
 Takes advantage of limitations in the human vision system to achieve high rates of
compression
 Uses complex lossy compression which allows user to set the desired level of quality
(compression). A compression setting of about 60% will result in the optimum
balance of quality and filesize.
 Though JPGs can be interlaced, they do not support animation and transparency
unlike GIF

TIFF
 Tagged Image File Format (TIFF), stores many different types of images (e.g.,
monochrome, grayscale, 8-bit & 24-bit RGB, etc.)
 Uses tags, keywords defining the characteristics of the image that is included in the
file. For example, a picture 320 by 240 pixels would include a 'width' tag followed by
the number '320' and a 'depth' tag followed by the number '240'.
 Developed by the Aldus Corp. in the 1980’s and later supported by the Microsoft
 TIFF is a lossless format (when not utilizing the new JPEG tag which allows for
JPEG compression)
 It does not provide any major advantages over JPEG and is not as user-controllable.
 Do not use TIFF for web images. They produce big files, and more importantly, most
web browsers will not display TIFFs.

System Dependent Formats

Microsoft Windows: BMP
 A system standard graphics file format for Microsoft Windows
 Used in Many PC Graphics programs
 It is capable of storing 24-bit bitmap images

Macintosh: PAINT and PICT

 PAINT was originally used in MacPaint program, initially only for 1-bit
monochrome images.
 PICT is a file format that was developed by Apple Computer in 1984 as the native
format for Macintosh graphics.
 The PICT format is a meta-format that can be used for both bitmap images and vector
images though it was originally used in MacDraw (a vector based drawing program)
for storing structured graphics
 Still an underlying Mac format (although PDF on OS X)

X-windows: XBM
 Primary graphics format for the X Window system
 Supports 24-bit colour bitmap
 Many public domain graphic editors, e.g., xv
 Used in X Windows for storing icons, pixmaps, backdrops, etc.

Digital Audio and MIDI

What is Sound?

Sound is produced by a rapid variation in the average density or pressure of air molecules
above and below the current atmospheric pressure. We perceive sound as these pressure
fluctuations cause our eardrums to vibrate. These usually minute changes in atmospheric
pressure are referred to as sound pressure and the fluctuations in pressure as sound
waves. Sound waves are produced by a vibrating body, be it a guitar string, loudspeaker
cone or jet engine. The vibrating sound source causes a disturbance to the surrounding air
molecules, causing them bounce off each other with a force proportional to the
disturbance. The back and forth oscillation of pressure produces a sound waves.

Source — Generates Sound

 Air Pressure changes
 Electrical —Microphone produces electric signal
 Acoustic — Direct Pressure Variations

Destination — Receives Sound

 Electrical — Loud Speaker
 Ears — Responds to pressure hear sound

How to Record and Play Digital Audio

In order to play digital audio (i.e WAVE file), you need a card with a Digital To Analog
Converter (DAC) circuitry on it. Most sound cards have both an ADC (Analog to Digital
Converter) and a DAC so that the card can both record and play digital audio. This DAC
is attached to the Line Out jack of your audio card, and converts the digital audio values
back into the original analog audio. This analog audio can then be routed to a mixer, or
speakers, or headphones so that you can hear the recreation of what was originally
recorded. Playback process is almost an exact reverse of the recording process.

First, to record digital audio, you need a card which has an Analog to Digital Converter
(ADC) circuitry. The ADC is attached to the Line In (and Mic In) jack of your audio card,
and converts the incoming analog audio to a digital signal. Your computer software can
store the digitized audio on your hard drive, visually display on the computer's monitor,
mathematically manipulate in order to add effects, or process the sound, etc. While the
incoming analog audio is being recorded, the ADC is creates many digital values in its
conversion to a digital audio representation of what is being recorded. These values must
be stored for later playback.

Digitizing Sound
 Microphone produces analog signal
 Computers understands only discrete(digital) entities
This creates a need to convert Analog audio to Digital audio — specialized hardware
This is also known as Sampling.

Common Audio Formats

There are two basic types of audio files: the traditional discrete audio file, that you can
save to a hard drive or other digital storage medium, and the streaming audio file that you
listen to as it downloads in real time from a network/internet server to your computer.
Discrete Audio File Formats
Common discrete audio file formats include WAV, AIF, AU and MP3. A fifth format,
called MIDI is actually not a file format for storing digital audio, but a system of
instructions for creating electronic music.

WAV
The WAV format is the standard audio file format for Microsoft Windows applications,
and is the default file type produced when conducting digital recording within Windows.
It supports a variety of bit resolutions, sample rates, and channels of audio. This format is
very popular upon IBM PC (clone) platforms, and is widely used as a basic format for
saving and modifying digital audio data.

AIF/AIFF
The Audio Interchange File Format (AIFF) is the standard audio format employed by
computers using the Apple Macintosh operating system. Like the WAV format, it
supports a variety of bit resolutions, sample rates, and channels of audio and is widely
used in software programs used to create and modify digital audio.

AU
The AU file format is a compressed audio file format developed by Sun Microsystems
and popular in the Unix world. It is also the standard audio file format for the Java
programming language. Only supports 8-bit depth thus cannot provide CD-quality sound.

MP3
MP3 stands for Motion Picture Experts Group, Audio Layer 3 Compression. MP3 files
provide near-CD-quality sound but are only about 1/10th as large as a standard audio CD
file. Because MP3 files are small, they can easily be transferred across the Internet and
played on any multimedia computer with MP3 player software.

MIDI/MID
MIDI (Musical Instrument Digital Interface), is not a file format for storing or
transmitting recorded sounds, but rather a set of instructions used to play electronic music
on devices such as synthesizers. MIDI files are very small compared to recorded audio
file formats. However, the quality and range of MIDI tones is limited.

Streaming Audio File Formats

Streaming is a network technique for transferring data from a server to client in a format
that can be continuously read and processed by the client computer. Using this method,
the client computer can start playing the initial elements of large time-based audio or
video files before the entire file is downloaded. As the Internet grows, streaming
technologies are becoming an increasingly important way to deliver time-based audio and
video data.

For streaming to work, the client side has to receive the data and continuously ‘feed’ it to
the ‘player’ application. If the client receives the data more quickly than required, it has
to temporarily store or ‘buffer’ the excess for later play. On the other hand, if the data
doesn't arrive quickly enough, the audio or video presentation will be interrupted.
There are three primary streaming formats that support audio files: RealNetwork's
RealAudio (RA, RM), Microsoft’s Advanced Streaming Format (ASF) and its audio
subset called Windows Media Audio 7 (WMA) and Apple’s QuickTime 4.0+ (MOV).

RA/RM
For audio data on the Internet, the de facto standard is RealNetwork's RealAudio (.RA)
compressed streaming audio format. These files require a RealPlayer program or browser
plug-in. The latest versions of RealNetworks’ server and player software can handle
multiple encodings of a single file, allowing the quality of transmission to vary with the
available bandwidth. Webcast radio broadcast of both talk and music frequently uses
RealAudio. Streaming audio can also be provided in conjunction with video as a
combined RealMedia (RM) file.

ASF
Microsoft’s Advanced Streaming Format (ASF) is similar to designed to RealNetwork's
RealMedia format, in that it provides a common definition for internet streaming media
and can accommodate not only synchronized audio, but also video and other multimedia
elements, all while supporting multiple bandwidths within a single media file. Also like
RealNetwork's RealMedia format, Microsoft’s ASF requires a program or browser plug-
in.

The pure audio file format used in Windows Media Technologies is Windows Media
Audio 7 (WMA files). Like MP3 files, WMA audio files use sophisticated audio
compression to reduce file size. Unlike MP3 files, however, WMA files can function as
either discrete or streaming data and can provide a security mechanism to prevent
unauthorized use.

MOV
Apple QuickTime movies (MOV files) can be created without a video channel and used
as a sound-only format. Since version 4.0, Quicktime provides true streaming capability.
QuickTime also accepts different audio sample rates, bit depths, and offers full
functionality in both Windows as well as the Mac OS.

Popular audio file formats are:

o au (Unix)
o aiff (MAC)
o wav (PC)
o mp3

MIDI
MIDI stands for Musical Instrument Digital Interface.

Definition of MIDI:
 MIDI is a protocol that enables computer, synthesizers, keyboards, and other musical
device to communicate with each other. This protocol is a language that allows
interworking between instruments from different manufacturers by providing a link
that is capable of transmitting and receiving digital data. MIDI transmits only
commands, it does not transmit an audio signal.
 It was created in 1982.

Figure 8 MIDI and Computer connection

Components of a MIDI System

Synthesizer:
• It is a sound generator (various pitch, loudness, tone color).
• A good (musician’s) synthesizer often has a microprocessor, keyboard, control panels,
memory, etc.
Sequencer:
 It can be a stand-alone unit or a software program for a personal computer. (It used to
be a storage server for MIDI data. Nowadays it is more a software music editor on the
computer.)
 It has one or more MIDI INs and MIDI OUTs.

Basic MIDI Concepts

Track:
 Track in sequencer is used to organize the recordings.
 Tracks can be turned on or off on recording or playing back.
Channel:
 MIDI channels are used to separate information in a MIDI system.
 There are 16 MIDI channels in one cable.
 Channel numbers are coded into each MIDI message.
Timbre:
 The quality of the sound, e.g., flute sound, cello sound, etc.
 Multitimbral – capable of playing many different sounds at the same time (e.g.,
piano, brass, drums, etc.)
Pitch:
 The Musical note that the instrument plays
Voice:
 Voice is the portion of the synthesizer that produces sound.
 Synthesizers can have many (12, 20, 24, 36, etc.) voices.
 Each voice works independently and simultaneously to produce sounds of
 Different timbre and pitch.
Patch:
 The control settings that define a particular timbre.

Hardware Aspects of MIDI

MIDI connectors:
Three 5-pin ports found on the back of every MIDI unit
MIDI IN: the connector via which the device receives all MIDI data.
MIDI OUT: the connector through which the device transmits all the MIDI data it
generates itself.
MIDI THROUGH: the connector by which the device echoes the data receives from
MIDI IN.
(See picture 8 for diagrammatical view)

MIDI Messages
MIDI messages are used by MIDI devices to communicate with each other.
MIDI messages are very low bandwidth:
• Note On Command
– Which Key is pressed
– Which MIDI Channel (what sound to play)
– 3 Hexadecimal Numbers
• Note Off Command Similar
• Other command (program change) configure sounds to be played.

Advantages:
 Because MIDI is a digital signal, it's very easy to interface electronic instruments to
computers, and then do manipulations on the MIDI data on the computer with
software. For example, software can store MIDI messages to the computer's disk
drive. Also, the software can playback MIDI messages upon all 16 channels with the
same rhythms as the human who originally caused the instrument(s) to generate those
messages.

How is MIDI file Different from a WAV or MP3 Files?

A MIDI file stores MIDI messages. These messages are commands that tell a musical
device what to do in order to make music. For example, there is a MIDI message that
tells a device to play a particular note. There is another MIDI message that tells a device
to change its current "sound" to a particular patch or instrument. Etc.

The MIDI file also stores timestamps, and other information that a sequencer needs to
play some "musical performance" by transmitting all of the MIDI messages in the file to
all MIDI devices. In other words, a MIDI file contains hundreds (to thousands) of
instructions that tell one or more sound modules (either external ones connected to your
sequencer's MIDI Out, or sound modules built into your computer's sound card) how to
reproduce every single, individual note and nuance of a musical performance.

A WAVE and MP3 files store a digital audio waveform. This data is played back by a
device with a Digital to Analog Converter (DAC) such as computer sound card's DAC.
There are no timestamps, or other information concerning musical rhythms or tempo
stored in a WAVE or MP3 files. There is only digital audio data.