Chapter 3

Multimedia Data Representation

 
This chapter introduces:
how best to represent the graphics and image data since
it is of crucial importance in the study of multimedia.

The specifics of file formats for storing such

images are also discussed

1
Graphics and Image Data Representations
Image is two-dimensional array of points where every
point is allocated its own color.
Every such single point is called pixel.
Pixel:- picture element contains the color or hue .
short form of picture element.
 Image is a collection of these points that are
colored and produce meaningful information/data.
The number of pixels in the image determines the
resolution of the image.
Higher resolution always yields better quality.
monitors usually display around 72 to 96 dpi(dots
per inch).

  2
 Image Types
There are two basic forms of computer graphics:
Bit-maps and Vector graphics.
Bit-maps:- Bitmap formats are the ones used
for digital photographs.
Bit-map images (also called Raster Graphics)
They are formed from pixels—a matrix of dots
with different colors.
Bitmap images are defined by their dimension
in pixels as well as by the number of colors they
represent. For example, a 640X480 image
contains 640 pixels and 480 pixels in horizontal
and vertical direction
3
 Images types…
Vector graphics :-They are really just a list of
graphical objects such as lines, rectangles,
ellipses, arcs, or curves—called primitives.
Draw programs are also called vector
graphics programs, are used to create and
edit these vector graphics.
 These programs store the primitives as a set of
numerical coordinates and mathematical formulas that
specify their shape and position in the image.
This format is widely used by computer-aided design
programs to create detailed engineering and design
drawings. It is also used in multimedia when 3D
animation is desired.
4
 Types of Bitmap Images
 
Monochrome/Bit-Map Images
 Each pixel is stored as a single bit (0 or 1)
 The value of the bit indicates whether it is light or dark
 A 640 x 480 monochrome image requires 37.5 KB of
storage.
 Dithering is often used for displaying monochrome
images

5
 Gray-scale Images
 Each pixel is usually stored as a byte (value between 0 to
255)
 This value indicates the degree of brightness of that point.
This brightness goes from black to white
 A 640 x 480 grayscale image requires over 300 KB of
storage.

Fig 2 Gray-scale bit-map image

 8-bit Color Images
 One byte for each pixel
 Supports 256 out of the millions possible, acceptable
color quality.
 Requires Color Look-Up Tables (CLUTs)
 A 640 x 480 8-bit color image requires 307.2 KB of
storage (the same as 8-bit grayscale)
 Examples: GIF
 8-bit color images store only the index of the actual pixel
color instead of the color itself.

Figure 3. 8-bit color image 7

 24-Bit Color Images
 Ina color 24-bit image, each pixel is represented by three
bytes, usually representing RGB.

 Since each value is in the range 0–255, this format supports

256×256×256, or a total of 16,777,216, possible combined
colors; which increases storage size.

a 640 × 480 24-bit color image would require 921.6 kB of

storage. (without any compression applied)
 Compression is used to decrease the image size by simply
grouping pixels effectively

8
 Higher Bit-Depth Images
 In some fields such as medicine (security cameras, satellite
imaging) more accurate images are required to see the patient’s
liver, for example.

 To get such images, special cameras that view more than just 3
colors (RGB) are used.

 Such images are called multispectral (more than three colors) or

hyperspectral (224 colors for satellite imaging).

9
Image Resolution
 

 Image resolution refers to the spacing of pixels

in an image and is measured in pixels per inch,
ppi, sometimes called dots per inch, dpi.
 The higher the resolution, the more pixels in the
image.
 A printed image that has a low resolution may
look pixilated or made up of small squares, with
jagged edges and without smoothness.
 Image size refers to the physical dimensions of
an image.

10
Popular File Formats

Choosing the right file type for your image to

save in is of vital importance in creating image
for web pages
Resulting size of the image large file size or
small.
quality of image possible by the file type.
portability of file across different platforms
The most common formats used on internet are the
GIF, JPG, and PNG.

11
Standard System Independent Formats
 

GIF
Graphics Interchange Format (GIF) devised by
CompuServe initially for transmitting graphical
images over phone lines via modems.
Uses the Lempel-Ziv Welch algorithm (a form of
Huffman Coding), modified slightly for image scan
line packets (line grouping of pixels).
LZW compression was patented technology by the
UNISYS Corp.
Limited to only 8-bit (256) color images, suitable for
images with few distinctive colors (e.g., graphics
drawing).

12
Standard System Independent Formats
PNG
stands for Portable Network Graphics
It is intended as a replacement for GIF in the WWW
and image editing tools.
GIF uses LZW compression which is patented by
Unisys.
PNG uses unpatented zip technology for compression
One version of PNG, PNG-8, is similar to the GIF
format.
PNG-24 is another version of PNG, with 24-bit color
support, allowing ranges of color to a high color JPG.

13
Standard System Independent Formats
JPEG/JPG
A standard for photographic image compression
 created by the Joint Photographic Experts Group
 Intended for encoding and compression of photographs
and similar images
 Takes advantage of limitations in the human vision
system to achieve high rates of compression
 Uses complex lossy compression which allows user to
set the desired level of quality (compression).
 A compression setting of about 60% will result in the
optimum balance of quality and file size.
 Though JPGs can be interlaced, they do not support
animation and transparency unlike GIF
14
Standard System Independent Formats
TIFF
 Tagged Image File Format (TIFF), stores many different types of
images (e.g., monochrome, grayscale, 8-bit & 24-bit RGB, etc.)
 Uses tags, keywords defining the characteristics of the image that is
included in the file.
 Developed by the Aldus Corp. in the 1980’s and later supported by
the Microsoft
 TIFF is a lossless format (when not utilizing the new JPEG tag
which allows for JPEG compression)
 It does not provide any major advantages over JPEG and is not as
user-controllable.
 Do not use TIFF for web images. They produce big files, and more
importantly, most web browsers will not display TIFFs.

15
Digital Audio and MIDI
What is Sound?
 Sound is produced by a rapid variation in the average density or
pressure of air molecules above and below the current atmospheric
pressure.
 We perceive sound as these pressure fluctuations cause our
eardrums to vibrate.
 Source — Generates Sound
 Air Pressure changes
 Electrical —Microphone produces electric signal
 Acoustic — Direct Pressure Variations
 Destination — Receives Sound
 Electrical — Loud Speaker
 Ears — Responds to pressure hear sound

16
How to Record and Play Digital Audio
 

 First, to record digital audio, you need a card with a

Digital To Analog Converter (DAC) circuitry on it.
 The ADC is attached to the Line In (and Mic In) jack of
your audio card, and converts the incoming analog audio
to a digital signal.
Digitizing Sound
 Microphone produces analog signal
 Computers understands only discrete(digital) entities
 This creates a need to convert Analog audio to Digital audio —
specialized hardware
 This is also known as Sampling.

17
Common Audio Formats
There are two basic types of audio files:
1. traditional discrete audio file:
 the that you can save to a hard drive or other
digital storage medium, and the
 It include WAV, AIF, AU and MP3.
 A fifth format, called MIDI is actually not a file
format for storing digital audio, but a system of
instructions for creating electronic music.

18
 Discrete Audio File Formats
WAV
 The WAV format is the standard audio file format for
Microsoft Windows applications.
 It supports a variety of bit resolutions, sample rates, and
channels of audio.
 This format is very popular upon IBM PC (clone)
platforms, and is widely used as a basic format for saving
and modifying digital audio data.
MP3
MP3 stands for Motion Picture Experts Group,
Audio Layer 3 Compression.
19
Discrete Audio File Formats
MIDI/MID
MIDI (Musical Instrument Digital
Interface)
a set of instructions used to play
electronic music on devices such as
synthesizers.
MIDI files are very small compared
to recorded audio file formats.
However, the quality and range of
MIDI tones is limited. 20
2. Streaming Audio File Formats
 Streaming is a network technique for
transferring data from a server to client in
a format that can be continuously read and
processed by the client computer.
 While playing the file you download the
file.
 For streaming to work, the client side has
to receive the data and continuously ‘feed’
it to the ‘player’ application.

21
Streaming Audio File Formats
RA/RM
 Real Network's RealAudio
(RA)compressed streaming audio format.
 require a RealPlayer program or browser
plug-in.
 Webcast radio broadcast of both talk and
music frequently uses RealAudio.

22
 Streaming Audio File Formats
ASF
 Microsoft’s Advanced Streaming
Format(ASF)
 it provides a common definition for internet
streaming media and can accommodate not
only synchronized audio, but also video and
other multimedia elements.
 Microsoft’s ASF requires a program or
browser plug-in.

23
Streaming Audio File Formats
WMA
 The pure audio file format used in Windows
Media Technologies is Windows Media Audio 7
(WMA files).
 Like MP3 files, WMA audio files use
sophisticated audio compression to reduce file
size.
 Unlike MP3 files, however, WMA files can
function as either discrete or streaming data and
can provide a security mechanism to prevent
unauthorized use.
24
Streaming Audio File Formats
MOV
 Apple QuickTime movies (MOV files) can be
created without a video channel and used as a
sound-only format.
 QuickTime provides true streaming capability.
 QuickTime also accepts Windows as well as the
Mac OS.
 Popular audio file formats example:
 au (Unix)
 aiff (MAC)
 wav (PC)

25
MIDI
MIDI stands for Musical Instrument Digital
Interface.
Definition: MIDI is a protocol that enables
computer, synthesizers, keyboards, and
other musical device to communicate with
each other.
allows interworking between instruments
from different manufacturers by providing
a link that is capable of transmitting and
receiving digital data.

26
MIDI

27
MIDI

Figure 5 MIDI and Computer connection

28
Components of a MIDI System
Synthesizer:
 It
is a sound generator (various pitch, loudness,
tone color).
 A good (musician’s) synthesizer often has a
microprocessor, keyboard, control panels and
memory.
Sequencer:
 It can be a stand-alone unit or a software
program for a personal computer.
 It used to be a storage server for MIDI data.
 It has one or more MIDI INs and MIDI OUTs
29
Basic MIDI Concepts
Track:
 Track in sequencer is used to organize the recordings.
 Tracks can be turned on or off on recording or playing
back.
Channel:
 MIDI channels are used to separate information in a
MIDI system.
 There are 16 MIDI channels in one cable.
 Channel numbers are coded into each MIDI message.
Timbre:
 The quality of the sound, e.g., flute sound, cello sound,
etc.

30
Basic MIDI Concepts
Pitch:
 The Musical note that the instrument plays.
Voice:
 Voice is the portion of the synthesizer that produces
sound.
 Synthesizers can have many (12, 20, 24, 36, etc.) voices.
 Each voice works independently and simultaneously to
produce sounds of different timbre and pitch.
Patch:
 The control settings that define a particular timbre.

31
Hardware Aspects of MIDI
 MIDI connectors: Three 5-pin ports found on
the back of every MIDI unit
 MIDI IN: the connector via which the device
receives all MIDI data.
 MIDI OUT: the connector through which the
device transmits all the MIDI data it generates
itself.
 MIDI THROUGH: the connector by which the
device echoes the data receives from MIDI IN.

32
Hardware Aspects of MIDI
MIDI Messages
 MIDI messages are used by MIDI devices to
communicate with each other.
 MIDI messages are very low bandwidth:
Advantages:
Because MIDI is a digital signal, it's
very easy to interface electronic
instruments to computers, and then do
manipulations on the MIDI data on the
computer with software.
33