Slide 1Distributed System ImplementationsSlide 3Modeling of P2P SystemsObserving P2PData LogKazaa ObjectsUsers are PatientUsers Slow DownUsers Slow DownClient ActivityWorkloadObject DynamicsZipfNon-ZipfNon-Zipf modelReally Non-Zipf?Locality AwarenessDiscussion PointsSlide 20Cluster ComputingAmazon AWSAn Example Application- GrepTheWebAn Example ApplicationAWS InterfaceAWS SecurityS3 Evaluation-Test ScenariosAverage Daily Read ThroughputCDF of Read and Write Throughput March 22-April8ResultsQuery VarianceResultsSlide 33Other ExperimentsExperience with EC2 and SQSConclusionDiscussion PointsCharacteristic Studiesof Distributed SystemsM a ryam R a hm a n i h e r i s & D a n ie l U hl i g1Distributed System ImplementationsCharacteristics of systems in real worldHow users and systems behaveP2P Track clients and traffic on KazaaExplore and model usage Cloud Computing Experiment with Amazon Web ServicesMeasure performance metrics2Krishna P. Gummadi, Richard J. Dunn, Stefan Saroiu, Steven D. Gribble, Henry M. Levy, And John ZahorjanMeasurement, Modeling, and Analysis of Peer-to-Peer File-Sharing Work LoadPeer To Peer CharacteristicsPresented by Daniel Uhlig3Modeling of P2P SystemsObserve clients and users behavior in real worldUsed large P2P system: KazaaClosed P2P protocalFasttrack network (still exists?)Peaked at 2.3 million users in 2003 (more than Napster’s peak)Now subscription DRM music store4Observing P2PRecorded all Kazaa traffic at U. of Washington60,000 faculty, staff, and students203 day trace (late spring semester past the end of fall semester)Protected privacy by making data anonymousAnalyze data, develop modelcompared to web: Is there a better comparison?5Data Log•Recorded all Kazaa traffic–Incoming and outgoing data transfer and searches–HTTP traffic with username in header–KazaaLite showed up 60 days into trace•Username hardcoded so used IP to differentiated. •20 terabytes of incoming data•1.6 Million Requests •25,000 users•8.85 TB of unique objects•Paper used requests from university peers to external peers. •Will this bias results?6Kazaa ObjectsFetch at most once 94% of Kazaa files vs 57% of web objects99% of Kazaa files fetch at most twiceClients download objects just once What is an object?Authors assume immutable object that is uniqueSame song Still fetch at most once?- Different filename?- Different bitrate?- Different Length?- Encoded by different user?7Users are Patient8•Web users want instant access, P2P users will wait •P2P users are patient •Small Objects –30% 1 hr –10% nearly a day•Large Objects–50% 1 day–20% wait 1 week•Is this accurate since client automatically restarts requests?Users Slow DownTook 30 day traces of ‘new’ usersBytes requested decrease with agePossible reasons?Loss of interestNew P2P appNew ID9Users Slow DownUsers leave the system foreverUsers request less data as they ageCore clients have constant activity levelBut, less data request 10Client Activity•How to measure activity: –Logged in vs downloading?•Average session length 2.4 minutes–Sessions could be split with a short term break in transfer•Activity over lifetime = 5.5%•Average transfer 17 minutes, but average session 2.4 minutes•Many transactions fail, looking for new host peer11WorkloadLarge (>100 MB) vs Small (<10 MB) filesRequest Volume vs Transfer volumeAudio clips vs Video clips12Object Dynamics13Clients fetch objects at-most-oncePopular objects quickly cycleNew objects are most popularMost requests are for old objectsSmall Objects Large ObjectsTop 10 Top 100Top 10Top 100Overlap between first and last 30 days0 of 10 5 of 1001 of 1044 of 100# of popular objects that a less 30 days old6 of 10 73 of 952 of 9 47 of 56How does the idea of distinct objects affect this?Zipf Distribution where most popular objects are most fetched. Classical result for web pages14Non-Zipf•Authors propose that Kazaa traffic is NOT modeled by Zipf distribution.•P2P differences from web –‘fetch-at-most once’–Immutable object (cnn.com changes regularly, a multimedia file does not)•Simulated model of behaviors and observed 15Non-Zipf model•Zipf seen as model in many places–Video on demand, video rental, movie tickets•Non-Zipf might better explain some of these models•Common characteristics–Birth of new objects–Fetch at most once–Immutable•Characteristics sometimes seen–Expensive to get object–Object remains•Does their non-zipf model explain everything?16Really Non-Zipf?•Multiple copies of the same object?–Does fetch at most once still hold•Requests for established files handled by internal users? (‘cached’)•Are objects immutable?–Changing names–Is new album/song a new object or an update from artist? •Non-Zipf in other multimedia–YouTube, Video Rental, DVD purchase, movie tickets?17Locality Awareness•Conserve University P2P bandwidth•86% of objects requested already at U of W–Cache data (legal issues)–Redirector so request stay internal when possible•Few key nodes can save significant bandwidth18Discussion Points•What is an unique item–Does this affect the distribution of popular objects?•Are objects immutable•Apply ideas to other multimedia: –YouTube video popularity–Still fetch at most once–Non-Zipf for DVD rental or purchase•How to define a unique object•Should P2P handle large and small objects differently•Caching or other forced locality vs P2P built in locality 19P r e s e n t e d b y : Maryam RahmaniherisU n i v e r s i t y o f I l l i n o i s a t U r b a n a - C h a m p a i g nC S 5 2 5 - S p r i n g 2 0 0 9 An Evaluation of Amazon's Grid Computing Services: EC2, S3, SQSSimon L. Garfinkel20Cluster ComputingBuilding your own clusterCostlySpaceCooling systemStaff…UnderutilizationOverutilization Cloud computing Computing as a utilityTime-multiplexing of resourcesNo need fro planning aheadElasticityAmazon’s AWSYou only need a working credit card21Amazon AWSEC2 (Elastic Compute Cloud)Linux virtual machines for 10 cents per CPU hourS3 (Simple Storage Service):Data storage for 15 cents per gigabyte per monthSQS (Simple Queue Service):Messaging service for 10 cents per thousands messages22An Example Application- GrepTheWeb23An Example
View Full Document
Unlocking...