MultimediaMultimedia SystemMultimedia (MM) ArchitecturesMM Activity TypesMM TopologiesInteractivity in MM PresentationPresentation ComponentsPresenting MaterialMM User InterfacesTelevision Style UI CharacteristicsComputer Style UI CharacteristicsElements of Aural StylePlanning Process - 1Planning Process - 2Media for Publishing ApplicationProblems Facing MM DesignersCompositionMetaphorsSynchronization ProblemsSynchronization TechniquesWhat is need to integrate MM into generic cut and paste environments?Database Integration - 1Database Integration - 201/13/19 1MultimediaCIS 577Bruce R. MaximUM-Dearborn01/13/19 2Multimedia System•Refers to computer system capable of integrating several media into a single display.•Analog Multimedia (requires conversion)–Video tape –Audio tape –Movies•Digital Multimedia–Has been transformed to that it may be manipulated as computer data –Interfaces allow multimedia components to be plugged into the computer systemMultimedia (MM) Architectures•MM systems are built of components called activities •Activities are contained in a series of actions •Activities have single start points •Activities can have multiple end points01/13/19 3MM Activity Types•Presenting (sequential) •Controlling (branching or selection)01/13/19 4MM Topologies•Linear presentation (repeating is allowed) •Data driven engine (control interpreted from list) •Hierarchical menus •Information retrieval (menu with search engine) •Hypermedia (search engine plus display engine) •Simulation (no generic control structures possible)01/13/19 5Interactivity in MM Presentation•User controlled interactive multimedia is the norm.•Canned presentations (e.g. automatic slide show) •Live speaker (e.g. power point under speaker control) •End user interactive presentation01/13/19 6Presentation Components•Getting audience attention (e.g. sound or splash screen) •Setting the mood (e.g. humor to put audience at ease) •Presenting material •Answering questions (e.g. online help) •Concluding presentation (e.g. summaries or review)01/13/19 7Presenting Material•Key is deciding who is telling the story •Subordinate all other MM elements to the story teller •Consistent audio track is good substitute for a talking head •If audio is not used then reserve a portion of screen for lead-ins and transitions •Once story teller is chosen, map out story using script or story board01/13/19 8MM User Interfaces•Choose metaphors familiar to users •Consistency of controls within an application •Simplicity to avoid intimidating user •User control over behavior •Users need immediate feedback •Windowing environment (don't use too many windows at once and don't provide windows functionality which is not needed)01/13/19 9Television Style UI Characteristics•Full-screen •No windowing •Proportionally spaced fonts •Photographic quality images •Audio is on all the time01/13/19 10Computer Style UI Characteristics•Mostly graphics or text •Windows have standard controls •Menu bars or control panels on screen at all times01/13/19 11Elements of Aural Style•Mono or stereo •Looping short segments saves space, if not too repetitious •Voice is cheap to produce and disk space is not too great •Music can be expensive (copyright and disk space)01/13/19 12Planning Process - 1•Defining who user are •Write overview of application •List material to be included as a first content draft •Make top level flow chart for application •Design application style •Design user interface 01/13/19 13Planning Process - 2•Define authoring process •Finalize content •Design major screens •Estimate project completion time and staffing requirements01/13/19 14Media for Publishing Application•Floppy diskette •Hard disk •CD-ROM •WORM CD •DVD •Digital tape (not common any more) •Computer network •WWW01/13/19 15Problems Facing MM Designers•Composition–How to combine media•Synchronization–Coordination of elements in presentation•Interaction–How to add interactivity to digital media•Database Integration–How top access large collections of digital media01/13/19 16Composition•Spatial composition (tiling and windowing)•Temporal composition (layout along time line)•Semantic composition (links among related elements)•Procedural composition (high level commands combining objects – e.g. clickable maps)•Component-based composition (activating and coordinating multiple sources – e.g. using MIDI sources to create an orchestra) 01/13/19 17Metaphors•Document metaphors (treelike doc organization)•Movie metaphor (works well for temporal composition)•Web metaphor (www pages and hypermedia)•Script metaphor (works well for procedural composition)•Circuit metaphor (component-based construction using plug-in sources)01/13/19 18Synchronization Problems•Clock errors (2 devices, 2 clocks)•Communication or processing delays•Response delays (latency – start/stop)•Tolerance variances between components•Changes in playback speed or direction•Changes in component load•Network reconfiguration•Stability (network restore lost synchronization)•Virtual clock used in place or real clock 01/13/19 19Synchronization Techniques•Interleaving streams (time sharing used in .avi files and Quicktime video)•Embedded time codes in stream (Quicktime uses them to decide when to drop frames)•Synchronization components (queue used to buffer info till synch is obtained)•Local scaling (input elements ignored till output bottleneck goes away)•Steam managers (e.g. Real Media Player)•Intelligent schedulers (dynamic load balancing)01/13/19 20What is need to integrate MM into generic cut and paste environments?•Multi-modal interaction–media stream treated like text streams•Input streams rather than input events–Current GUI stuff is event driven and this makes events into discrete and independent entities–Need to allow parallel activations of media events•Dialog independence–Makes application behavior independent of interaction device (everything – even speech – behaves like a mouse)•Interaction embedding–Application a generic player and the nature of the data determines the behavior01/13/19 21Database Integration - 1•Would be nice to store MM in a single database (lots of vendor issues to resolve)•Data quality–DB philosophy was to protect client from physical storage format–MM apps often need to