THE BITTORRENT P2P FILE SHARING SYSTEM MEASUREMENTS AND ANALYSIS J A Pouwelse P Garbacki D H J Epema H J Sips Department of Computer Science Delft University of Technology the Netherlands j a pouwelse ewi tudelft nl ABSTRACT Of the many P2P file sharing prototypes in existence BitTorrent is one of the few that has managed to attract millions of users BitTorrent relies on other global components for file search employs a moderator system to ensure the integrity of file data and uses a bartering technique for downloading in order to prevent users from freeriding In this paper we present a measurement study of BitTorrent in which we focus on four issues viz availability integrity flashcrowd handling and download performance The purpose of this paper is to aid in the understanding of a real P2P system that apparently has the right mechanisms to attract a large user community to provide measurement data that may be useful in modeling P2P systems and to identify design issues in such systems 1 INTRODUCTION Even though many P2P file sharing systems have been proposed and implemented only very few have stood the test of intensive daily use by a very large user community The BitTorrent file sharing system is one of these systems Measurements on Internet backbones indicate that BitTorrent has evolved into one of the most popular networks 8 In fact BitTorrent traffic made up 53 of all P2P traffic in June 2004 12 As BitTorrent is only a file download protocol it relies on other global components such as web sites for finding files The most popular web site for this purpose is suprnova org There are different aspects that are important for the acceptance of a P2P system by a large user community First such a system should have a high availability Secondly users should almost always receive a good version of the content they request no fake files 10 Thirdly the system should be able to deal with flashcrowds Finally users should obtain a relatively high download speed In this paper we present a detailed measurement study of the combination of BitTorrent and Suprnova This measurements study addresses all four aforementioned aspects Our measurement data consist of detailed traces gathered over a period of 8 months Jun 03 to Mar 04 of more than two thousand global components In addition for one of the most popular files we followed all 90 155 downloading peers from the injection of the file until its disappearance several months In a period of two weeks we measured the bandwidth of 54 845 peers downloading over a hundred newly injected files This makes our measurement effort one of the largest ever conducted The contributions of this paper are the following first we add to the understanding of the operation of a P2P filesharing system that apparently by its user friendliness the quality of the content it delivers and its performance has the right mechanisms to attract millions of users Second the results of this paper can aid in the mathematical modeling of P2P systems For instance in the fluid model in 13 it is assumed that the arrival process and the abort and departure processes of downloaders are Poisson something that is in obvious contradiction with our measurements One of our main conclusions is that within P2P systems a tension exists between availability which is improved when there are no global components and data integrity which benefits from centralization 2 THE BITTORRENT FILE SHARING SYSTEM BitTorrent 5 in itself is only a file downloading protocol In BitTorrent files are split up into chunks on the order of a thousand per file and the downloaders of a file barter for chunks of it by uploading and downloading them in a tit for tat like manner to prevent parasitic behavior Each peer is responsible for maximizing its own download rate by contacting suitable peers and peers with high upload rates will with high probability also be able to download with high speeds When a peer has finished downloading a file it may become a seed by staying online for a while and sharing the file for free i e without bartering Site name Suprnova org Youceff com Piratebay org Lokitorrent com Available files 46 766 47 137 39 294 30 957 File transfers 2 267 463 1 145 889 749 133 816 435 Table 1 Popular BitTorrent web sites Oct 2004 To find a file in BitTorrent users access web sites which act as global directories of available files In Table 1 we show for the most popular of these web sites the number of different files and the number of active file transfers at a certain time In this paper we assume Suprnova as the directory web site The Suprnova web site uses a mirroring system to balance user requests across its mirror sites The web pages on Suprnova show for each available file the name and size the current numbers of downloaders and seeds and the name of the person who uploaded the file To start the download of a file a user clicks on a link pointing to torrent meta data file These meta data files are not stored on Suprnova or its mirrors but are distributed among a number of torrent file servers In turn each torrent file points to a tracker which keeps a global registry of all the downloaders and seeds of the corresponding file The tracker responds to a user s request with a list of some of the peers having part of the requested file with whom the user can establish direct connections to barter for chunks of the file One tracker can supervise the simultaneous downloads of multiple files New content is injected into BitTorrent by uploading a torrent file to the Suprnova web site and creating a seed with the first copy of the file In order to reduce the pollution level new content is first manually inspected by moderators who weed out fake content content with low perceptual quality and content with incorrect naming A normal user who injects content is called a moderated submitter To lower the burden on the moderators a user who frequently injects correct content is promoted to the rank of unmoderated submitter and is allowed to directly add content Unmoderated submitters can request a promotion to moderator status to existing moderators Together BitTorrent and Suprnova form a unique infrastructure that uses mirroring of the web servers with its directory structure meta data distribution for load balancing a bartering technique for fair resource sharing and a P2P moderation system to filter fake files 3 EXPERIMENTAL SETUP In this section we will discuss some details of our measurement software and the