Chorus & Maestro: DistributedSynchronizationMatthew Jones ([email protected])Jacob Wiseman ([email protected])Faculty Advisors: Fernando Pereira and Benjamin Pi erceApril 11, 20051 AbstractThe need for distributed collaboration on documents has ta ken off recently.The Internet provides a medium for exchanging copies of documents betweenparties. However, with this growth, ensuring document integrity has becomea difficult problem. Documents may exist on various machines which aregeographically separated. It is often not practical to rely on locking mech-anisms for this type of document. Instead, we’d like to optimistically allowmultiple copies of a documents to exist, and then reconcile or synchronizeany changes.Harmony is a project at the University of Pennsylvania led by Dr. Ben-jamin Pierce. It gives users the power to reconcile structured documentswhich have been edited in non-conflicting ways (certain changes are un-reconcilable). At the current time, Harmony exists as a command line tool,and is not capable of doing remote synchronization.The goal of our project is to build systems f or practical deployment ofHarmony which makes it easy t o use in a distributed setting.12 Related Work2.1 CVS and SubversionCVS and subversion are version control systems. Both systems mediatechanges to collections of documents. These document can be replicated acrossmultiple systems giving many users a local copy. The systems have a cen-tral server which is respo nsible for mainta ining the official version of eachdocument, as well as a history. Client can freely edit local copies of theirdocuments, and then at some point the changes are sent ( or committed)to the server. The server is responsible for incorporating those changes (orrejecting them) and then passing those changes on to other clients.Since the entire goal of harmony is to reconcile changes made to differ-ent copies of a document, these systems provide an excellent model for howa distributed Ha r mony system could work. CVS and Subversion succeededbecause they provided both a well-designed system as well as a solid im-plementation of their standards. We hope to emulate this by designing oursystems in such a way that it should be simple for someone else to supplytheir own implementations of our protocols.These systems fail when different users make changes to the same file.In some cases, CVS and Subversion are a ble to merge separate changes toa file, but sometimes they cannot. By building our system on Harmony,we’ll be able to significantly reduce the number of instances which result inconflicts.[7][6]2.2 SubEthaEditSubEthaEdit is anot her example of a successful tool for distributed collab-oration on documents. SubEthaEdit’s biggest draw is the completely sim-ple setup which allows users to collaborate on a document nearly effort-lessly. SubEthaEdit is basically a text editor, which automatically findsother SubEthaEdit users on a network and allows them to simultaneouslyedit a document. It is a mature application which is very popular on theMacintosh platform.SubEthaEdit has a head start and has more features than we do at thispoint in time. However, SubEthaEdit is lacking in two key areas: (1) It’s MacOS X-only (2) It is an application, not an API. Using SubEthaEdit requiresusers to switch to the SubEthaEdit application, as it cannot be integrated2into their favorite editor.[12]2.3 Technologies and ToolsSome of the tools we used to build va r io us parts of the systems.2.4 WebDAVWebDAV stands for ”Web-based Distributed Authoring and Versioning”. Itis a set of extensions to the HTTP protocol which a llows users to collabo-ratively edit and manage files on remote web servers. (from the webDAVwebsite)[5]2.5 CadaverCadaver is a command-line WebDAV client for Unix. It supports file upload,download, on-screen display, name-space operations (move/copy), collectioncreation and deletion, and locking operations. (description from the Cadaverwebsite)[9]2.6 davServerPython davServer is a collection of classes which should make it easy towrite your own DAV server in Python. (Description from the davServerwebsite)[10]2.7 JmDNS/BonjourJmDNS is a Java implementation of the mDNS responder specified in Apple’sBonjour protocol[8][11]3 The ProblemsLooking at Subversion/CVS and SubEthaEdit, we see a distributed Harmonysystem as being able to being improvement s in two quite separate areas. Atfirst, we thought a single system could solve both problems. However, aswork has progressed, we have seen these as essentially two different problemsrequiring two different solutions:33.1 Problem 1: Small, local collaborationConsider collaborative note-taking in a meeting or lecture. Each audiencemember wants to contribute his or her own thoughts t o a growing outlineof the lecture built collaboratively. For participants, simplicity is crucial. Itshould be easy to create or connect to a document containing the outline aswell as to publish and receive changes to this document. At the current time,such collaboration is only poassibe using the SubEthaEdit platform.In the same class of scenarios is pair programming. Two developers maywish to change a piece of code simultaneously, while also viewing the other’schanges. Most developers would never give up Emacs in favor of an appli-cation like SubEthaEdit, however, using a a distributed Harmony API wecould integrate the needed functionality into Emacs.3.2 Problem 2: Large, distribute collaborationConsider now a large number o f people spread across the world sharing acommon document such as a contacts or calendar file. In this scenario per-formance a nd scalability are more crucial we may have hundreds or thousandsof users collaborating on much larger files over a larger geographic area. Easeof use (while not ignored) is not the driving concern, as there is more likelyto be an IT department to do t he initial setup.4 Technical ApproachIn order to meet t he requirements for these these divergent scenarios, we havedesigned and built two different architectures for supporting distribution ofharmony.4.1 Two systems4.1.1 Peer-to-Peer ArchitectureFor the small, local collaboration we employ a P2P network. This networkrelies on Apple’s Bonjour (formerly known as Rendezvous) to do automaticdiscovery of other peers on a local area network. Once on-line, each client cancreate and publish documents. Other clients can open and edit a document,4publishing their own changes. Each peer’s copy of Harmony