Adding Force Feedback to Graphics Systems:Issues and SolutionsWilliam R. Mark1 Scott C. Randolph2 Mark Finch3 James M. Van Verth4 Russell M. Taylor II5Department of Computer Science*University of North Carolina at Chapel HillABSTRACTIntegrating force feedback with a complete real-time virtualenvironment system presents problems which are moredifficult than those encountered in building simpler force-feedback systems. In particular, lengthy computations forgraphics or simulation require a decoupling of the haptic servoloop from the main application loop if high-quality forces areto be produced. We present some approaches to these problemsand describe our force-feedback software library whichimplements these techniques and provides other benefitsincluding haptic-textured surfaces, device independence,distributed operation and easy enhancement.CR Descriptors: H.1.2 [Models and Principles]:User/Machine Systems; C.3 [Special-Purpose and Application-Based Systems]: Real-time systems; I.3.7 [ComputerGraphics]: Three-Dimensional Graphics and Realism – VirtualReality; I.6.8 [Simulation and Modeling]: Types of Simulation– Distributed.Additional Keywords: haptic, force feedback, frictionmodel, intermediate surface representation, scientificvisualization, interactive graphics, virtual environment,virtual world.1. INTRODUCTIONAs designers of interactive computer systems work to increasethe information flow between the computer and the user,sensory modalities other than vision become increasinglyimportant. One such modality is force feedback. The sensingof forces is closely coupled to both the visual system and one’ssense of three-dimensional space; the eyes and hands work inconcert to explore and manipulate objects.Force feedback usefully enhances the capabilities ofvirtual environment systems; [17] showed that force feedbackincreases productivity in solving rigid-body placementproblems and [8] demonstrated an atomic-surface modificationsystem which would not have been feasible with graphicsalone.Virtual environment force displays use models andalgorithms described in the robotics and teleoperationliterature for low-level control—see for example [9][22][23].When combining a computer graphics engine, a simulation,and a force-feedback device into one system, there are severalareas of concern in addition to that of low-level control. Theforce-feedback component of such a system should:• Maintain a high update rate in the force servo loop.• Present high quality forces without detectable artifacts.• Transparently support different force-feedback devices.• Interface easily and cleanly with the rest of the system.We discuss some approaches to these problems and present theArmlib force-feedback library [13] as one solution.2. PROBLEMS AND SOLUTIONSIt has been clearly shown that it is necessary to run thesimulation and graphics loops of virtual environment (VE)systems asynchronously in order to maintain reasonabledisplay update rates (around 20 Hz) in the presence of longsimulation computations. [11][20]Such a decoupling is even more critical for force display,where update rates of several hundred Hz are required to producehigh-quality forces. The necessary rate depends somewhat onthe characteristics of the force-feedback device and controlalgorithm, but, for example, [1] required an update rate of 500Hz for their system. If the update rate falls below the requiredminimum, the user begins to notice high-frequencydiscontinuities and hard surfaces become either soft orunstable.We can decouple the simulation and haptic loops on asingle machine by using either multiple processors or veryfrequent context switches. However, it is often more practicalto dedicate one real-time machine to the haptic servo loop, anduse other machine(s) for the rest of the virtual environmenttasks (simulation, high-performance graphics, etc.). Thisstrategy allows each machine to be well matched to its task. Italso allows for flexible system configuration, which isparticularly useful in a research environment.The general case of such a split system connects the force-feedback device directly to a force server. This server tracks theprobe of the force-feedback device (held in the user’s hand) andexecutes the force-feedback servo loop. The applicationconnects to this force server through some communicationchannel, retrieving position information from the server and * CB #3175, Sitterson Hall; Chapel Hill, NC 27599. Tel. +1.919.962.1700Authors’ current organizations and contact information:1 UNC-CH; [email protected]; www.cs.unc.edu/~markw2 Spectrum Holobyte; [email protected]; www.holobyte.com3 Numerical Design, Ltd.; [email protected]; www.ndl.com/ndl4 Virtus Corp.; [email protected]; www.cs.unc.edu/~vanverth5 UNC-CH; [email protected]; www.cs.unc.edu/~taylorrsending descriptions of forces or force fields to it. Wecurrently use a TCP/IP Ethernet communications channelbecause we must connect to existing graphics and researchequipment; a low-latency, high-bandwidth channel such asshared memory would be superior.Kim et al. [12] did the first work in this area, showing thatteleoperation systems benefit from a decoupling of low-levelforce servo loops from higher-level control. Adachi et al. [1]were the first to apply the technique to virtual environmentforce-feedback systems. Rather than simply supplying asingle force vector to the force-feedback controller, theysupply an intermediate representation (their term, which weadopt) for a force model. This representation is updatedinfrequently by the application code, but is evaluated at a highupdate rate by the force-feedback controller.Gomez et al. [10] demonstrate a system which takesalmost the opposite approach. Their main simulation runs onthe force-feedback machine and sends state updates to agraphics machine.2.1 Intermediate representationsThe kind of intermediate representation that is most usefuldepends on the application. A molecular modeling systemmight use spheres of contact. An immersive design systemcould send a representation of nearby surfaces. A simulationmeant to teach understanding of physics force fields [3] mightsend equations to the server that describe the field.Mitsuishi et al. [16] demonstrate a remote milling systemwhich uses an intermediate representation of average toolforce.We describe two general intermediate representations,plane and probe and point-to-point