Multimedia Systems

INTRODUCTION

Prepared by: Abdisa K. FCSE

 What is Multimedia

 Multimedia refers to the use of a combination of different forms of

content, such as text, graphics, audio, video, and animations, to
convey information or create an interactive and engaging user
experience.
 It encompasses a wide range of content and applications, including websites,
video games, virtual reality experiences, presentations, animations, and more.
 Multimedia is the field concerned with the computer controlled
 integration of text, graphics, drawings, still and moving images (Video),
animation, audio, and any other media
 where every type of information can be represented, stored,
transmitted and processed digitally
Cont.
 Multimedia is Focus on multimedia computing and communications
systems. It covers such topics as hardware and software for media
compression, media storage and transport, workstation support, data
modeling, and abstractions to embed multimedia in applications programs
(IEEE).
 Multimedia is designed to engage multiple senses and provide a richer user
experience by combining various media types.
 The main goal of multimedia is to convey information or entertain through
a combination of media elements.
 It allows for more engaging and dynamic content that can be tailored to specific
audiences and purposes.
 Multimedia and Computer Science
 Multimedia and computer science are closely intertwined fields, with
computer science playing a significant role in the development, processing,
and management of multimedia content.
 Multimedia Systems: Computer scientists develop the underlying systems and
technologies that enable the creation, storage, retrieval, and transmission of
multimedia content.
 This includes designing algorithms and data structures for efficient multimedia
processing, developing multimedia compression techniques, and designing multimedia
databases and retrieval systems
 Multimedia Compression: Compression algorithms are essential for reducing
the size of multimedia files without significant loss of quality.
 Cont.
 Multimedia Networking: Computer scientists study and develop networking protocols
and technologies that facilitate the efficient transmission of multimedia data over
networks.
 Human-Computer Interaction (HCI): HCI focuses on designing user interfaces and
systems that enable effective interaction between humans and multimedia applications.
Computer scientists contribute to HCI research by developing innovative user interface
technologies, designing intuitive navigation and interaction techniques, and studying user
behavior and preferences in multimedia environments.
 Computer Graphics: Computer graphics is a subfield of computer science that deals with
the creation, manipulation, and rendering of visual content. It encompasses areas such as
2D and 3D graphics,
 Multimedia Analytics and Machine Learning: With the proliferation of multimedia data,
computer scientists apply machine learning and data analytics techniques to extract
useful information from multimedia content.
 Multimedia Application:
 A Multimedia Application is an application which uses a
collection of multiple media sources including text,
images, drawings (graphics), animation, video, sound
including speech, and interactivity.
 Examples of Multimedia Applications include:
 World Wide Web Computer Games,
 Multimedia Authoring, e.g. Virtual reality
Digital video editing and production
Adobe/Macromedia Director
systems ,
 Hypermedia courseware
Multimedia Database systems
 Video-on-demand,
 Interactive TV ,
Components of multimedia:

Multimedia involves multiple modalities/ components of text, audio,

images, drawing, animation, and video. Examples of how these
modalities are put to use:
1. SKYPE: Video teleconferencing
2. <AVATAR>: “Augmented” reality: placing real-appearing computer
graphics and video objects into scenes.
3. OPEN UNIVERSITY: Distributed lectures for higher education.
4. Youtube/Facebook: Searching in (very) large video and image databases
for target visual objects.
5. Remote Surgery: Tele-medicine.
6. Virtual Lab: Co-operative work environments.
.
Cont.
7. audio cues for where video-conference participants are located.
8. Building searchable features into new video, and enabling very
high- to very low-bit-rate use of new, scalable multimedia products.
9. Making multimedia components editable.
10. Building “inverse-Hollywood” applications that can recreate the
process by which a video was made.
11. Using voice-recognition to build an interactive environment, say
a kitchen-wall web browse
History of Multimedia:
 Newspaper: perhaps the first mass communication medium, uses
text, graphics, and images.
 Motion pictures: conceived of in 1830's in order to observe motion
too rapid for perception by the human eye.
 Wireless radio transmission: Guglielmo Marconi, at Pontecchio,
Italy, in 1895.
 Television: the new medium for the 20th century, established video as
a commonly available medium and has since changed the world of
mass communications.
 Cont.

 The connection between computers and ideas about multimedia covers what is
actually only a short period:
 1945 - Vannevar Bush wrote a landmark article describing what amounts to a
hypermedia system called Memex.
 1960 - Ted Nelson coined the term hypertext.
 •1967 - Nicholas Negroponte formed the Architecture Machine Group.
 •1968 - Douglas Engelbart demonstrated the On-Line System (NLS), another very
early hypertext program.
 •1969 - Nelson and van Dam at Brown University created an early hyper text editor
called FRESS.
 •1976 - The MIT Architecture Machine Group proposed a project entitled Multiple
Media - resulted in the Aspen Movie Map, the first hypermedia videodisk, in 1978.
 1985 - Negroponte and Wiesner co-founded the MIT Media Lab.
 •1989 - Tim Berners-Lee proposed the World Wide Web
Cont.

 1990 - Kristina Hooper Woolsey headed the Apple Multimedia Lab.

 •1991 - MPEG-1 was approved as an international standard for digital video
- led to the newer standards, MPEG-2, MPEG-4, and further MPEGs in the
1990s.
 •1991 - The introduction of PDAs in 1991 began a new period in the use of
computers in multimedia.
 •1992 - JPEG was accepted as the international standard for digital image
compression | led to the new JPEG2000 standard.
 •1992 - The first MBone audio multicast on the Net was made.
 •1993 - The University of Illinois National Center for Supercomputing
Applications produced NCSA Mosaic - the first full-fledged browser.
 1994 - Jim Clark and Marc Andreessen created the Netscape program.
Cont.
 •1995 - The JAVA language was created for platform-
independent application development.
 •1996 - DVD video was introduced; high quality full-length
movies were distributed on a single disk.
 •1998 - XML 1.0 was announced as a W3C Recommendation.
 •1998 - Hand-held MP3 devices first made inroads into
consumerist tastes in the fall of 1998, with the introduction of
devices holding 32MB of flash memory.
 •2000 - WWW size was estimated at over 1 billion pages.
Multimedia System Definition :

 A multimedia system refers to a computer-based system or

platform that integrates different forms of media, such as text,
images, audio, video, and interactive elements, to create and
deliver rich, interactive, and engaging experiences to users.
 It involves the processing, storage, retrieval, and presentation
of multimedia content.
 A Multimedia System is a system capable of processing
multimedia data and applications.
 A Multimedia System is characterised by the
processing,storage, generation, manipulation and rendition of
Multimedia information.
Cont.
 Characteristics of a Multimedia System
 A Multimedia system has four basic characteristics:
 Multimedia systems must be computer controlled.
 Multimedia systems are integrated.
 The information they handle must be represented digitally.
 The interface to the final presentation of media is usually interactive
Challenges for Multimedia Systems

 Distributed Networks
 Temporal relationship between data
 Render different data at same time — continuously.
 Sequencing within the media
 playing frames in correct order/time frame in video
 Synchronisation — inter-media scheduling
 E.g. Video and Audio — Lip synchronisation is clearly important
for humans to watch playback of video and audio and even animation
and audio.
Key Issues for Multimedia Systems

The key issues multimedia systems need to deal with here are:
 How to represent and store temporal information.

 How to strictly maintain the temporal relationships on play

back/retrieval
 What process are involved in the above.

 Data has to represented digitally — Analog–Digital

Conversion, Sampling etc.

 Large Data Requirements — bandwidth, storage, compression


Components of a Multimedia System

Now let us consider the Components (Hardware and Software) required for a
multimedia system:
 Capture devices — Video Camera, Video Recorder, Audio Microphone,

Keyboards, mice, graphics tablets, 3D input devices, tactile sensors, VR

devices.
 Storage Devices — Hard disks, CD-ROMs, Jaz/Zip drives, DVD, etc

 Communication Networks — Ethernet, Token Ring, FDDI, ATM,

Intranets, Internets.
 Computer Systems — Multimedia Desktop machines, Workstations,

MPEG/VIDEO/DSP Hardware
 Display Devices — CD-quality speakers, HDTV,SVGA, Hi-Res monitors,

Colour printers etc.

Multimedia, Hypermedia and Hypertext,
 Multimedia, hypermedia, and hypertext are related concepts that involve
the integration of various media elements and interactivity to enhance
information presentation and navigation.
Multimedia:
 Multimedia refers to the integration of different media elements, such as

text, images, audio, video, and animations, into a cohesive presentation.

 The goal of multimedia is to engage and communicate with users through

multiple sensory channels.

 It encompasses the use of various media types to create a rich and

immersive user experience, allowing for effective communication and

presentation of information.
Cont.
Hypermedia:
 Hypermedia goes beyond multimedia by incorporating interactivity

and nonlinearity.
 It allows users to navigate through information in a non-sequential

manner, accessing related content through hyperlinks or interactive

elements.
 Hypermedia systems enable users to explore content based on their

interests, preferences, and information needs.

 This can include multimedia elements, such as images and videos,

along with text-based information.

Cont.
Hypertext:
 Hypertext specifically refers to the linking of text-based content through

hyperlinks.
 It allows users to access related information by clicking on hyperlinked

words or phrases within a document or system.

 Hypertext provides a non-linear structure to text, enabling users to navigate

between different sections, pages, or documents based on their interests or

information requirements.
 The concept of hypertext is fundamental to the World Wide Web, where

hyperlinks connect web pages, enabling users to browse and access

information across the internet.
Cont.
 A hypertext system: meant to be read nonlinearly, by following
links that point to other parts of the document, or to other documents
Cont.
 HyperMedia: not constrained to be text-based, can include other
media, e.g., graphics, images, and especially the continuous media |
sound and video.
 The World Wide Web (WWW) | the best example of a hypermedia
application.
Cont.
 Multimedia: means that computer information can be represented through
audio, graphics, images, video, and animation in addition to traditional media.
 World wide web:
 Commonly used hypermedia application.
 Its popular due to the amount of information of web servers, the capacity to
post that information,easy to navigate.
 WWW is maintained and developed by the World wide web consortium
(W3C) although by the internet Engineering Task Force(IETC).
 The W3C has listed the following goals
 Universal access of web resources (by everyone every- where).
 Effectiveness of navigating available information.
 Responsible use of posted material
HTTP(HyperTextTransfer Protocol)
 HTTP: a protocol that was originally designed for transmitting hypermedia, but can
also support the transmission of any file type
 HTTP is a stateless request/response protocol: no information carried over for the
next request.
 The basic request format:
 Method
 URI Version
 Additional-Headers:
 Message-body
 The URI (Uniform Resource Identifier): an identifier for the resource accessed, e.g.
the host name, always preceded by the token “http://”.
HTML (HyperText Markup Language)

 HTML: a language for publishing Hypermedia on the World Wide Web

 1. HTML uses ASCII, it is portable to all different (possibly binary
incompatible) computer hardware.
 2. The current version of HTML is version 4.01.
 3. The next generation of HTML is XHTML | a reformulation of HTML
using XML.
 HTML uses tags to describe document elements:
 <token params> | defining a starting point,
 </token> | the ending point of the element.
 {Some elements have no ending tags.:
XML(ExtensibleMarkupLanguage)

 XML: a markup language for the WWW in which there is modularity of data,
structure and view so that user or application can be able to define the tags
(structure).
 Example of using XML to retrieve stock information from a database according to a
user query:
 First use a global Document Type Definition (DTD) that is already defined.
 The server side script will abide by the DTD rules to generate an XML document
according to the query using data from your database.
 Finally send user the XML Style Sheet (XSL) depending on the type of device used to
display the information
 The following XML related specifications are also standardized:
 XML Protocol:used to exchange XML information between processes.
 XML Schema:a more structured and powerful language for defining XML data types (tags).
 XSL: basically CSS for XML.
SMIL: synchronized Multimedia Integration Language
 a particular application of XML (globally predefined DTD) that allows for
specification of interaction among any media types and user input, in a
temporally scripted manner
 Purpose of SMIL: it is also desirable to be able to publish multimedia
presentations using a markup language.
 A multimedia markup language needs to enable scheduling and synchronization
of different multimedia elements, and define their interactivity with the user.
 The W3C established a Working Group in 1997 to come up with specifications
for a multimedia synchronization language
 Designed to:
 For all skill levels of WWW authors
 Schedule audio, video, text, and graphics files across a timeline
 Cont.
 No need to master development tools or complex programming languages.
 HTML-like need a text editor only
 Links to media — media not embedded in SMIL file
 Drawbacks of SMIL:
 Good Points:
 A powerful tool for creating synchronized multimedia presentations on the web
 Deals with low bandwidth connections.
 Bad Points:
 Meant to work with linear presentations
 Several types of media can be synchronized to one timeline.
 Does not work well with non-linear presentations
 Ability to skip around in the timeline is buggy.
 For slideshow style mixed media presentations it the best the web has to offer.
 SMIL supports:
SMIL supports:
 The W3C recommended SMIL in June 1998
 Quicktime 4.0 supports SMIL (1999)
 Not universally supported across the Web.
 No Web browser directly support SMIL
 RealPlayer G2 supports SMIL
 Many other SMIL-compliant players, authoring tools, and servers available.
 Running SMIL Applications
 For this course there are basically three ways to run SMIL applications (two
use the a Java Applet) so there are basically two SMIL supported mediums:
 Quicktime — supported since Quicktime Version 4.0.
 RealPlayer G2 — integrated SMIL support
 Web Browser — use the SOJA SMIL applet viewer with html wrapper
 Applet Viewer — use the SOJA SMIL applet viewer with html wrapper