Ubiquitous Computing and Augmented RealitiesThese slides are taken from the Dix textbookWhat is this about?Ubiquitous ComputingScales of DevicesDevice Scale Examples - 1Device Scale Examples - 2Device Scale Examples - 3Defining the Interaction ExperienceDifferent InputsThe output experienceMerging Physical and Digital WorldsApplication ThemesNew Opportunities for TheoryEvaluation ChallengesVirtual Reality TechnologyImmersionOn the DesktopVRML … VR on the webCommand and ControlAugmented Reality (AR)Applications of ARApplications of VRScientific and Technical DataStructured InformationVisualizing HierarchyWide hierarchies … use 3D?Networks in 2DTime and InteractivityBetween Two Worlds01/13/19 1Ubiquitous Computing andAugmented RealitiesCIS 577Bruce R. MaximUM-Dearborn01/13/19 2These slides are takenfrom the Dix textbook01/13/19 3What is this about?•Ubiquitous computing–filling the real world with computers•Virtual and augmented reality–making the real world in a computer!01/13/19 4Ubiquitous Computing•Any computing technology that permits human interaction away from a single workstation•Implications for–Technology defining the interactive experience–Applications or uses–Underlying theories of interaction01/13/19 5Scales of Devices•Weiser proposed–Inch–Foot–Yard•Implications for device size as well as relationship to people01/13/19 6Device Scale Examples - 1•Inch–PDAs–PARCTAB–Voice Recorders–smart phone•Individuals own many of them and they can all communicate with each other and environment.01/13/19 7Device Scale Examples - 2•Foot–notebooks–tablets–digital paper•Individual owns several but not assumed to be always with them.01/13/19 8Device Scale Examples - 3•Yard–electronic whiteboards–plasma displays–smart bulletin boards•Buildings or institutions own them and lots of people share them.01/13/19 9Defining the Interaction Experience•Implicit input–Sensor-based input–Extends traditional explicit input (e.g., keyboard and mouse)–Towards “awareness”–Use of recognition technologies–Introduces ambiguity because recognizers are not perfect01/13/19 10Different InputsCapacitive sensing on a tableSensors on a PDA01/13/19 11The output experience•More than eye-grabbing raster displays–Ambient: use features of the physical environment to signal information–Peripheral: designed to be in the background•Examples: –The Dangling String–The Water Lamp (shown)01/13/19 12Merging Physical and Digital Worlds•How can we remove the barrier?–Actions on physical objects have meaning electronically, and vice versa–Output from electronic world superimposed on physical world01/13/19 13Application Themes•Context-aware computing–Sensed phenomena facilitate easier interaction•Automated capture and access–Live experiences stored for future access•Toward continuous interaction–Everyday activities have no clear begin-end conditions01/13/19 14New Opportunities for Theory•Knowledge in the world–Ubiqicomp places more emphasis on the physical world•Activity theory–Goals and actions fluidly adjust to physical state of world•Situated action and distributed cognition–Emphasizes improvisational/opportunistic behavior versus planned actions•Ethnography–Deep descriptive understanding of activities in context01/13/19 15Evaluation Challenges•How can we adapt other HCI techiques to apply to ubicomp settings?–Ubicomp activities not so task-centric–Technologies are so new, it is often hard to get long-term authentic summative evaluation–Metric of success could be very different (playfulness, non-distraction versus efficiency)01/13/19 16Virtual Reality Technology•Headsets allow user to “see” the virtual world in 3D using “twin TV sets for eyes”•Gesture recognition achieved with DataGlove (lycra glove with optical sensors that measure hand and finger positions)•Eyegaze allows users to indicate direction with eyes alone•Whole body position sensed, walking etc.01/13/19 17Immersion•VR–computer simulation of the real world•mainly visual, but sound, haptic, gesture too–experience life-like situations•too dangerous, too expensive–see unseen things:•too small, too large, hidden, invisible–e.g. manipulating molecules•The Experience–aim is immersion, engagement, interaction01/13/19 18On the Desktop•Headset VR–expensive, uncomfortbale•Desktop VR–use ordinary monitor and PC•cheap and convenient•In games …•On the web–VRML – virtual reality markup language01/13/19 19VRML … VR on the web#VRML V1.0 asciiSeparator { Separator { # for sphere Material { emmissiveColor 0 0 1 # blue } Sphere { radius 1 } } Transform { translation 4 2 0 } Separator { # for cone Texture2 { filename "big_alan.jpg" } Cone { radius 1 # N.B. width=2*radius height 3} } }01/13/19 20Command and Control•Scenes projected on walls•Realistic environment•Hydraulic rams!•Real controls•Other people•Applications:–flight simulators–ships–military01/13/19 21Augmented Reality (AR)•Images projected over the real world–aircraft head-up display–semi-transparent goggles–projecting onto a desktop•Types of information–unrelated – e.g. reading email with wearable–related – e.g. virtual objects interacting with world•Issues–registration – aligning virtual and real–eye gaze direction01/13/19 22Applications of AR•Maintenance–overlay instructions–display schematics•Examples–photocopier engineers•registration critical arrows point to parts–aircraft wiring looms•registration perhaps too hard, use schematic01/13/19 23Applications of VR•Simulation–games, military, training•VR holidays–rainforest, safari, surf, ski and moon walk at home•Medical–surgery•scans and x-rays used to build model and allows ‘practice’ operations (force feedback)–phobia treatment•virtual lifts, spiders, etc.01/13/19 24Scientific and Technical Data•Number of virtual dimensions that are ‘real’•Three dimensional space–visualise invisible values (e.g. virtual wind tunnel)•Two dimensional space–can project data value up from plane (e.g. geographic data)•No ‘real’ dimensions–2D/3D histograms, scatter plots, pie charts, etc.01/13/19 25Structured Information•Scientific data – just numbers•Information systems … lots of kinds of