An Interactive 3D Toolkit for Constructing 3D WidgetsRobert C. Zeleznik, Kenneth P. Herndon, Daniel C. Robbins,Nate Huang, Tom Meyer, Noah Parker and John F. HughesBrown UniversityDepartment of Computer ScienceProvidence, RI 02912(401) 863-7693;fbcz,kph,dcr,nth,twm,nfp,[email protected] CategoriesI.3.6 [Computer Graphics]: Methodology and Techniques; Inter-action TechniquesD.1.7 [Programming Languages]: ProgrammingTechniques; Visual Programming D.2.2 [Software Engineering]:Tools and Techniques;User Interfaces1 IntroductionToday’s user interfaces for most 3D graphics applications still de-pend heavily on 2D GUIs and keyboard input. There have beenseveral recent attempts both to extend these user interfaces into 3Dand to describe intermediary 3D widgets1that control applicationobjects [3; 4; 5; 7; 13; 15]. Even though this style of interaction isa straightforward extension of interaction through intermediary 2Dwidgets such as dials or sliders, we know of no efforts to developinteractive 3D toolkits akin to UIMX or Garnet [11].The Brown GraphicsGroup has had considerableexperience us-ing its Unified Graphics Architecture (UGA) system [16] to script3D widgets suchas deformation racks [14], interactive shadows[9],parameterized models, and other constrained 3D geometries. Us-ing this experience, we have developed an interactive toolkit tofacilitate the visual programming of the geometry and behavior ofsuch interactive models. The toolkit provides both a core set of 3Dwidget primitives for constructing interactive behaviors based onconstrained affine transformations, and an interactive 3D interfacefor combining these primitives into more complex widgets.This video paper describes the fundamental concepts of thetoolkit and its core set of primitives. In particular, we describe(i) the conceptual structure of the primitives, (ii) the criteria usedto select a particular primitive widget set that would be expressiveenough to let us construct a wide range of interactive 3D objects,and (iii) the constraint relationships among the primitives.2 Overview of our 3D ToolkitThe traditional approach to designing user interface toolkits is tocreate a library of software objects and customize them throughinstantiation and specialization within standard programming lan-guages[12; 15]. Although this approachis extremely powerful, ex-ploringthe full potentialrequires thatprogrammers be able tovisual-ize complex relationships among software objects (e.g., constraint1That is, encapsulations of geometry and behavior.networks, data- and control-flow graphs). A second paradigm,based on graphically manipulating function networks [1; 8; 10], ismore accessible to the non-programmer, but still suffers becauseinherently geometric relationships must be specified by wiring 2Dboxes together.Our toolkit uses direct manipulation of 3D widgets to model theconstruction of widgets and application objects whose geometriccomponentsare affinely constrained. This paradigm is more naturalthan scripting or dataflow programming because the process ofconstructing such objects is inherently geometric, and also enablesnon-programmers and designers to construct these objects visually.The scope of these constructions includes, for example, all of thewidgets we have built in the last few years and standard joints suchas slider, pin, and ball joints.We introduce the notion of primitive 3D widgets that can becombined with other primitive 3D widgets, using a process calledlinking, to establish one or more constraint relations between them.In some cases, the resulting composite objects are still consideredwidgets; in others, they are thought of as the behavioral scaffoldingto which the geometry of application objects can be attached. Thefact that the interface and application objects exist in the sameunderlying system, UGA, allows us to blur the distinction betweenthem. We feel that such blurring is natural for 3D applications ingeneral, and especially for virtual reality applications.Linkingis related to snapping[3], but differs in requiring explicitinteractive selection of source and destination objects, followedby explicit user confirmation. This protocol reduces clutter byeliminating alignment objects. In the interest of simplifying theuser interface, all linking operations are unparameterized, althoughin future work, parameterized linking for more advanced users andmore complicated widgets will be explored.3 Conceptual Structure of Widget PrimitivesA primitive widget combines the geometries and behaviors of itsportsand other moresimple primitives. A port isan encapsulation ofoneor more constraintvalues and a geometric representation. It canbe loosely considered a data type with the additional requirementthat its visual appearancesuggestthe meaning of the data. Ports arerelated to one another within a single widget via a network of bi-directional constraints. In addition, specific interaction techniquesare associated with each port. Each interaction technique tells howto modify a port while maintaining constraints on other ports. Forexample, if a user manipulates a point that is constrained to be ona line, the constraint could be resolved by moving the line withthe point, by restricting the user’s interaction sothat the point neverleavesthe line, or by a combination of thetwo. Wemust chooseoneof these as we implement the toolkit. These interaction techniquescan be thought of as hints to a constraint solver when the constraintnetwork is underdetermined so it can provide real-time, preciseinteractions.In addition to having an internal constraint network, a primitivewidget can be related to another primitive widget by linking a portof the former to a port of the latter. This establishes a constraint(bi- or uni-directional) between the two ports. Ports are alreadyconstrained by the internal constraint network of a primitive, andthe new constraintsmust be consistent with the existing constraints.Therefore, associated with each port is a function that determineshow to attach new constraints to that port and how to modify itsinteraction techniques so as to facilitate constraint maintenance.4 Description of the Toolkit PrimitivesHaving selected this framework to build our toolkit, we designed ageneral set of primitives to allow the interactive construction of notonlythe various 3Dwidgets previouslyscripted,butalsoapplicationobjects such as parameterized geometric models. These primitivesare intendedto be general enoughto allow