Secure Untrusted Data Repository SUNDR Jinyuan Li Maxwell Krohn David Mazie res and Dennis Shasha NYU Department of Computer Science Abstract SUNDR is a network file system designed to store data securely on untrusted servers SUNDR lets clients detect any attempts at unauthorized file modification by malicious server operators or users SUNDR s protocol achieves a property called fork consistency which guarantees that clients can detect any integrity or consistency failures as long as they see each other s file modifications An implementation is described that performs comparably with NFS sometimes better and sometimes worse while offering significantly stronger security 1 Introduction SUNDR is a network file system that addresses a longstanding tension between data integrity and accessibility Protecting data is often viewed as the problem of building a better fence around storage servers limiting the number of people with access disabling unnecessary software that might be remotely exploitable and staying current with security patches This approach has two drawbacks First experience shows that people frequently do not build high enough fences or sometimes entrust fences to administrators who are not completely trustworthy Second and more important high fences are inconvenient they restrict the ways in which people can access update and manage data This tension is particularly evident for free software source code repositories Free software projects often involve geographically dispersed developers committing source changes from all around the Internet making it impractical to fend off attackers with firewalls Hosting code repositories also requires a palette of tools such as CVS 4 and SSH 35 many of which have had remotely exploitable bugs Worse yet many projects rely on third party hosting services that centralize responsibility for large numbers of otherwise independent code repositories sourceforge net for example hosts CVS repositories now at MIT CS AI Lab for over 20 000 different software packages Many of these packages are bundled with various operating system distributions often without a meaningful audit By compromising sourceforge an attacker can therefore introduce subtle vulnerabilities in software that may eventually run on thousands or even millions of machines Such concerns are no mere academic exercise For example the Debian GNU Linux development cluster was compromised in 2003 2 An unauthorized attacker used a sniffed password and a kernel vulnerability to gain superuser access to Debian s primary CVS and Web servers After detecting the break in administrators were forced to freeze development for several days as they employed manual and ad hoc sanity checks to assess the extent of the damage Similar attacks have also succeeded against Apache 1 Gnome 32 and other popular projects Rather than hope for invulnerable servers we have developed SUNDR a network file system that reduces the need to trust storage servers in the first place SUNDR cryptographically protects all file system contents so that clients can detect any unauthorized attempts to change files In contrast to previous Byzantine fault tolerant file systems 6 27 that distribute trust but assume a threshold fraction of honest servers SUNDR vests the authority to write files entirely in users public keys Even a malicious user who gains complete administrative control of a SUNDR server cannot convince clients to accept altered contents of files he lacks permission to write Because of its security properties SUNDR also creates new options for managing data By using SUNDR organizations can outsource storage management without fear of server operators tampering with data SUNDR also enables new options for data backup and recovery after a disaster a SUNDR server can recover file system data from untrusted clients file caches Since clients always cryptographically verify the file system s state they are indifferent to whether data was recovered from untrusted clients or resided on the untrusted server all along This paper details the SUNDR file system s design and implementation We first describe SUNDR s security protocol and then present a prototype implementation that gives performance generally comparable to the popular Client NFS file system under both an example software development workload and microbenchmarks Our results show that applications like CVS can benefit from SUNDR s strong security guarantees while paying a digestible performance penalty Server application consistency server syscall 2 Setting SUNDR provides a file system interface to remote storage like NFS 29 and other network file systems To secure a source code repository for instance members of a project can mount a remote SUNDR file system on directory sundr and use sundr cvsroot as a CVS repository All checkouts and commits then take place through SUNDR ensuring users will detect any attempts by the hosting site to tamper with repository contents Figure 1 shows SUNDR s basic architecture When applications access the file system the client software internally translates their system calls into a series of fetch and modify operations where fetch means retrieving a file s contents or validating a cached local copy and modify means making new file system state visible to other users Fetch and modify in turn are implemented in terms of SUNDR protocol RPCs to the server Section 3 explains the protocol while Section 5 describes the server design To set up a SUNDR server one runs the server software on a networked machine with dedicated SUNDR disks or partitions The server can then host one or more file systems To create a file system one generates a public private superuser signature key pair and gives the public key to the server while keeping the private key secret The private key provides exclusive write access to the root directory of the file system It also directly or indirectly allows access to any file below the root However the privileges are confined to that one file system Thus when a SUNDR server hosts multiple file systems with different superusers no single person has write access to all files Each user of a SUNDR file system also has a signature key When establishing an account users exchange public keys with the superuser The superuser manages accounts with two superuser owned file in the root directory of the file system sundr users lists users public keys and numeric IDs while sundr group designates groups and their