Universal DesignSlides based on Dix, HCI (3rd edition)Universal Design PrinciplesMulti-Sensory SystemsUsable SensesMulti-modal vs. Multi-mediaSpeech Recognition UsesSpeechSpeech Recognition Problems - 1Speech Recognition Problems - 2Phonetic Typewriter - 1Phonetic Typewriter - 2Phonetic Typewriter - 3Speech Recognition DemoSpeech SynthesisSpeech Synthesis UsesSpeech Generation DemoNon-Speech SoundsNon-Speech Sounds UsefulnessAuditory IconsSonicFinder for the MacintoshEarcons - 1Earcons - 2TouchHandwriting Recognition – 1Handwriting Recognition - 2Gesture-based Computing - 1Gesture-based Computing - 2Users with Special Needs - 1Users with Special Needs - 201/14/19 1Universal DesignCIS 577Bruce R. MaximUM-Dearborn01/14/19 2Slides based onDix, HCI (3rd edition)01/14/19 3Universal Design Principles •Equitable use•Flexibility in use•Simple and intuitive to use•Perceptible information•Tolerance for error•Low physical effort•Adequate size and space for approach and use01/14/19 4Multi-Sensory Systems•More than one sensory channel in interaction–sounds, music, text, hypertext, animation, video, gestures, vision•Used in a range of applications:–good for users with special needs, and virtual reality• Applications include –speech–non-speech sounds–handwriting01/14/19 5Usable Senses•Sight, sound, touch, taste and smell are used by people every day•Each of the senses is important on its own•Together, the senses provide a fuller interaction with the natural world•Computers rarely offer such a rich interaction•User interfaces might include use of sight, sound, or touch•User interfaces do not include the use taste or smell right now01/14/19 6Multi-modal vs. Multi-media•Multi-modal systems–use more than one sense (or mode ) of interaction–text processor may speak the words as well as echoing them to the screen (visual and aural)•Multi-media systems–use a number of different media to communicate information–computer-based teaching system: may use video, animation, text and still images (all visual)–could may also use sounds or speech two more media01/14/19 7Speech Recognition Uses•Single user or limited vocabulary systems e.g. computer dictation•Multi-user limited vocabulary systems can work satisfactorilye.g. some voice activated telephone systems•General user, wide vocabulary systems don’t work well•Great potential, for use by disabled user or users whose hands or eyes are occupied01/14/19 8Speech•Human beings have a great and natural mastery of speech•Makes it difficult to appreciate the complexities•It’s an easy medium for communication though01/14/19 9Speech Recognition Problems - 1•Different people speak differently:–accent, intonation, stress, idiom, volume, etc.•The syntax of semantically similar sentences may vary.•Background noises can interfere.•People often say “ummm.....” and “errr.....”01/14/19 10Speech Recognition Problems - 2•Words not enough - semantics needed as well•Requires intelligence to understand a sentence•Context of the utterance often has to be known•Also information about the subject and speaker•For example, What does this mean? “Errr.... I, um, don’t like this”01/14/19 11Phonetic Typewriter - 1•Developed for Finnish–a phonetic language, written as it is said•Trained on one speaker, will generalise to others.•A neural network is trained to cluster together similar sounds.•These sounds are then labelled with the corresponding character.01/14/19 12Phonetic Typewriter - 2•When recognizing speech, the sounds uttered are allocated to the closest corresponding output, and the character for that output is printed.•Requires large dictionary of minor variations to correct general mechanism•Noticeably poorer performance on speakers it has not been trained on01/14/19 13Phonetic Typewriter - 3a a aaaaoooo ooll umvhrah æp dsiyjgøtkvm hjhiuuvvvv. ...mar rrhhh æærmp p pppdkkpt t ttøøene elgnjjj jjii issyyhr khrhnn01/14/19 14Speech Recognition Demo01/14/19 15Speech Synthesis•Speech is natural and familiar way of receiving information•Speech synthesis problems are similar to those of speech recognition•Additional problems–intrusive - needs headphones–creates noise in the workplace–transient - harder to review and browse01/14/19 16Speech Synthesis Uses•Successful in certain constrained applications–User is particularly motivated to overcome problems–User has few alternatives•Examples:–screen readers read text displays to the visually impaired users–warning signals used spoken information sometimes presented to pilots whose visual and tactical skills are already fully occupied01/14/19 17Speech Generation Demo01/14/19 18Non-Speech Sounds•Boings, bangs, squeaks, clicks etc.•Commonly used for warnings and alarms•Evidence to show they are useful–fewer typing mistakes with key clicks–video games harder without sound•Language/culture independent, unlike speech01/14/19 19Non-Speech Sounds Usefulness•Dual mode displays–information presented along two different sensory channels–redundant presentation of information –resolution of ambiguity in one mode through information in another•Sound good for–transient information–background status information–Sound can be used as a redundant mode for GUI for almost any user action (file selection, window active, disk insert, search error, copy complete, etc.)01/14/19 20Auditory Icons•Use natural sounds to represent different types of object or action•Natural sounds have associated semantics that can be mapped onto similar interaction meanings (breaking glass sound - throwing things away)•Problem: not all things have associated meanings•Additional information can also be presented:–muffled sounds if object is obscured or action is in the background–use of stereo sounds allows positional information to be added user experience01/14/19 21SonicFinder for the Macintosh•Items and actions on the desktop have associated sounds–Folders have a papery noise–Moving files – dragging sound–Copying – sound of a liquid being poured into a receptacle rising pitch indicates the progress of the copy•Big files have louder sound than smaller ones01/14/19 22Earcons - 1•Synthetic sounds used to convey information•Structured combinations of notes (motives ) represent actions and objects•Motives combined to provide