Android ProgrammingAndroid: Engineer's Dream!Innovative User InterfacesEyes-Free InteractionDevices That SenseEyes-Free Interaction With The Touch ScreenUsing A Touch ScreenOn-Screen TouchpadsPhone KeypadDialer FeedbackText Input Via Touch ScreenContact ManagerEntering Text InputKeypad LayoutManaging ContactsMarvin: Eyes-Free ShellEyes-Free ShellTalking CompassWeather ForecastDevice StatusConclusionSummaryWatch Computing Take Off!Eyes-Free User InteractionT. V. RamanGoogle Researchhttp://emacspeak.sf.net/ramanFebruary 9, 2009OverviewAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 2 / 25AndroidDialerContactsShellConclusionAndroid ProgrammingAndroid Dialer Contacts Shell ConclusionAndroid: Engineer’s Dream!Android Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 4 / 25■ Smart phones — computer in your pocket.■ Open platform — liberates innovation.■ New I/O — minimize user interaction.Bend technology to your will!Innovative User InterfacesAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 5 / 25Create innovative end-user solutions that:■ Fit into the user’s mode of working,■ Avoid one-size fits all solutions,■ Design context-aware solutions,■ Tailor solutions to user’s needs.Eyes-Free InteractionAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 6 / 25Not just for blind users!■ Eyes-busy environments —driving.■ One-handed use.■ Minimize explicit user interaction.Rethink all assumptions about the user.Devices That SenseAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 7 / 25Mobile phones make UI design exciting!■ Can capture user intent — keyboard, . . ..■ Can see — camera.■ Can hear and speak — speech I/O.■ Can sense and stimulate touch — touch screen and vibration.■ Can sense motion — accelerometer, compass.■ Can sense position — GPS, cell towers.■....Our Android has many eyes and ears!Eyes-Free Interaction With TheTouch ScreenAndroid Dialer Contacts Shell ConclusionUsing A Touch ScreenAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 9 / 25These are atomic tasks:■ Identifying the item to touch.■ Touching item to activate it.■ Eyes-free use requires help with the former.■ Activation should produce appropriate feedback.On-Screen TouchpadsAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 10 / 25Eyes-Free use of an on-screen touchpad:■ Need to see the buttons to know where to touch.■ Because: buttons use absolute positioning.■ Conclusion: (wrong) — you need to see to use a touch screen!■ Solution: Relax constraint of absolute positioning.What if we used relative positioning instead?Phone KeypadAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 11 / 25Eyes-free phone dialer:■ Keypad centered where you touch — relative positioning.■ Users already know layout of a phone keypad.■ Can stroke from the center for a desired digit.Enables eyes-free one-handed dialing!Dialer FeedbackAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 12 / 25Enables one-handed, eyes-free use:■ Buttons vibrate when you move over them.■ Augmented by auditory feedback.■ Pressing buttons produces spoken and tactile feedback.■ Shake to erase input.Synchronized auditory and haptic feedback is key.Text Input Via Touch ScreenAndroid Dialer Contacts Shell ConclusionContact ManagerAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 14 / 25No one dials phone numbers any more!■ Browsing contacts using touch screen.■ Finding contacts.■ Managing contacts.Entering Text InputAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 15 / 25Extending stroke dialer for textual input:■ Arrange letters in 4 concentric circles.■ Strokes access distinct circles.■ Choose desired letter by tracing chosen circle.Any letter is at most 3 steps away!Keypad LayoutAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 16 / 25Gesture: Pick circle, and trace to desired character.A I Q⊙ YU M ELetter GestureA Stroke up to the leftB Stroke A, trace rightE Stroke down to the rightManaging ContactsAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 17 / 25Contacts live in the cloud.■ Contacts managed over the network.■ Updated automatically on the device.■ No more editing contacts with two fingers!Marvin: Eyes-Free ShellAndroid Dialer Contacts Shell ConclusionEyes-Free ShellAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 19 / 25Marvin: Shell for launching eyes-free interaction.■ Home screen provides one-touch access to oft-used tasks.■ Uses stroke dialer for launching tasks.■ Code is Open Source.http://eyes-free.googlecode.comTalking CompassAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 20 / 25■ Compass is one touch away on the home screen.■ Speaks current bearing.■ Provides succinct or verbose feedback.■ Augmented by auditory tones and vibration for cardinal directions.Weather ForecastAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 21 / 25■ Android phone knows its location.■ Current weather is therefore just one touch away.■ Retrieves RSS weather forecast.■ Speaks weather headline.Device StatusAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 22 / 25■ Device status available as single touch gestures.■ Airplane mode.■ Date and time.■ Battery status.ConclusionAndroid Dialer Contacts Shell ConclusionSummaryAndroid Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 24 / 25■ Programmable smart phones are an engineer’s dream!■ I/O peripherals open up new UI avenues.■ Provide desired information with minimal user interaction.Profound impact on how we work and play!Watch Computing Take Off!Android Dialer Contacts Shell ConclusionEyes-Free Interaction Stanford February 2009 – 25 /