PowerPoint PresentationMotivationSurvey of Informal Ink AppsExample - Elec. Cocktail NapkinExample - FlatlandCommon FeaturesCommon FeaturesRelated WorkSlide 9ProblemsGoals of SATINOverviewDENIMSketchySPICESlide 15Slide 16Some MetricsSlide 18Architecture OverviewSlide 20RecognizersInterpretersInterpreters and RecognizersMulti-InterpretersScenegraphStroke DispatchingSlide 27Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33Slide 34Slide 35Slide 36Example - Cut ShallowExample - Cut DeepReusable Stroke LibrariesSlide 40WidgetsConclusionsDownload SATINExtra SlidesSlide 45Slide 46Slide 47Slide 48Nov 06 2000 1Jason Hong and James LandayUniversity of California BerkeleyGroup for User Interface ResearchNov 06 2000 2Motivation•Sketching is useful in many settingsNov 06 2000 3•Creative / Design•SILK•Music Notepad•Knight•Elec. Cocktail Napkin•Teddy•PatchWorkSurvey of Informal Ink Apps•Capture•ZenPad•NotePals•Dynomite•FiloChat•Others•Pegasus•XLibris•PerSketch•Kramer's Patches•Whiteboard•Tivoli•FlatlandNov 06 2000 4Example - Elec. Cocktail NapkinGross and Do (UIST96)Nov 06 2000 5Example - FlatlandIgarashi, Edwards, LaMarca, and Mynatt (AVI2000)Nov 06 2000 6Common Features•Pen input as ink or gesture•Stroke is pen input from dragging•Ink is a stroke that stays on-screen•Gesture is a stroke that activates a command•Recognize ink as objects•Transformation / clean-up of ink•Immediate / deferred processing of inkerase meNov 06 2000 7Common Features •Contain objects in addition to ink•Selecting and moving objects•Grouping of objects•Layering of objectsNov 06 2000 8Related Work•Commercial Software Support–Newton OS–PenPoint–Microsoft Windows for Pen Computing–Windows CE–Palm OS•Problems–Form-based and handwriting interfaces–Not easily extensible for building informal ink appsNov 06 2000 9Related Work•Research Software Support–ArtKit (Henry et. al. 1990)–Garnet Pen Int. (Landay et. al. 1993)–Patches (Kramer, 1994)–OOPS (Mankoff et. al. 2000)•Problems–Need more reusable libraries for handling and processing strokes–Need more extensibility so lots of kinds of informal ink apps can be builtNov 06 2000 10Problems•Software infrastructure support for informal ink apps rudimentary•Pen-based interaction techniques difficult to implement•Similar to GUI development in 1980s–Lots of applications + interaction techniques–Need cohesive frameworks + toolkitsNov 06 2000 11Goals of SATIN•Toolkit support for informal ink apps–Provide reusable mechanisms for handling + processing strokes–Separate mechanism from policy–Be extensible for new kinds of apps•Reusable widgets for pens•Distribute this toolkit for useNov 06 2000 12Overview Motivation Survey of Informal Ink Applications Two Applications Built with SATIN SATIN Architecture Overview ConclusionsNov 06 2000 13DENIMNov 06 2000 14SketchySPICENov 06 2000 15SketchySPICENov 06 2000 16SketchySPICENov 06 2000 17Some MetricsSATIN DENIM Sketchy SPICE#source files 180 76 7#methods 2192 642 63#classes 220 131 32#lines of code20000 9000 1000time 1 person18 months3 people 3 months1 person 3 daysNov 06 2000 18Overview Motivation Survey of Informal Ink Applications Two Applications Built with SATIN SATIN Architecture Overview ConclusionsNov 06 2000 19Architecture OverviewJava Core ClassesJava Virtual MachineJava2DSATINSwingApplicationSATIN relies on Java2D and SwingApplications use SATIN, can use Java2D and SwingNov 06 2000 20Architecture OverviewSATIN can be broken into twelve conceptsRecognizersInterpreters ScenegraphStrokeLibrariesWidgetsViewsRenderingTransitionsClipboardNotificationCommandEventsNov 06 2000 21Recognizers•Problem - many recognition techniques•Recognizers let new recognition technologies be plugged into SATIN•Given a stroke, return n-best list•No actions taken•SATIN comes with Rubine's Recognizer (implemented by Chris Long)Recognizer0.860.650.63CutCopyUndoNov 06 2000 22Interpreters•Problem – many ways of handling strokes•Interpreters let new ways of handling strokes be plugged into SATIN•e.g. straighten a line•e.g. issue a command Ink Interpreter Gesture Interpreter erase meNov 06 2000 23Interpreters and Recognizers•Decoupled recognition (Recognizers) from actions (Interpreters)•Interpreters can use recognizers•Both are modules that can be plugged into apps built with SATINInterpreter RecognizerCut0.860.650.63CutCopyUndoNov 06 2000 24Multi-Interpreters•Problem – Hard to combine and extend existing interpreters•Multi-Interpreters are a collection of interpreters + dispatch policy–e.g. dispatch to chain of interpreters–e.g. disable some interpreters on contextDefaultMulti InterpreterIntrp A Intrp Z…Semantic ZoomMulti InterpreterIntrp BIntrp A Intrp Z…Intrp BNov 06 2000 25Phrase(Patch)ScenegraphSheetDenim Label (Patch)Denim Sketch (Patch)Timed Stroke…SheetDenim Label (Patch)Denim Sketch (Patch)Phrase(Patch)Timed StrokeNov 06 2000 26Stroke Dispatching•Needed an approach that enabled:–Individual objects handle strokes on own–Priority to gestures over ink•Default policy is top-down dispatch1. Process stroke with its gesture interpreter2. Re-dispatch stroke to one of its children3. Process stroke with its ink interpreter4. Handle stroke in object itselfNov 06 2000 27Stroke DispatchingPhrase(Patch)SheetDenim Label (Patch)Denim Sketch (Patch)Timed StrokeNov 06 2000 28Stroke DispatchingPhrase(Patch)SheetDenim Label (Patch)Denim Sketch (Patch)Timed Stroke1. Process stroke with gesture interpreterSheetDefaultMulti InterpreterHold Select IntrpStandard Gesture Intrp…Nov 06 2000 29Stroke DispatchingPhrase(Patch)SheetDenim Label (Patch)Denim Sketch (Patch)Timed StrokeNov 06 2000 30Stroke DispatchingPhrase(Patch)SheetDenim Label (Patch)Denim Sketch (Patch)Timed StrokeSheetDefaultMulti InterpreterHold Select IntrpStandard Gesture Intrp…1. Process stroke with gesture interpreterNov 06 2000 31Stroke Dispatching2. Re-dispatch stroke to childrenPhrase(Patch)SheetDenim Label (Patch)Denim Sketch (Patch)Timed StrokeDenim Sketch (Patch)Nov 06 2000 32Stroke Dispatching1. Process stroke with gesture interpreterPhrase(Patch)SheetDenim Label (Patch)Denim Sketch (Patch)Timed StrokeDenim Sketch (Patch)Nov 06 2000 33Stroke Dispatching2. Re-dispatch stroke to childrenPhrase(Patch)SheetDenim Label (Patch)Denim Sketch (Patch)Timed StrokeDenim Sketch