CSCI 530, Spring 2010 Copyright © William C. Cheng T1CS530Scalable Wide-areaUpload[Bistro00]Bill Chenghttp://merlot.usc.edu/cs530-s10 CSCI 530, Spring 2010 Copyright © William C. Cheng T2a Platform for Building ScalableWide-Area Upload Applicationsa Platform for Building ScalableWide-Area Upload ApplicationsBistroBistro CSCI 530, Spring 2010 Copyright © William C. Cheng T3Scalable Data Transfer ApplicationsEnd-system / Application-level# of Receivers# of SendersOneOneManyMany CSCI 530, Spring 2010 Copyright © William C. Cheng T4Scalable Data Transfer ApplicationsEnd-system / Application-level# of Receivers# of SendersOneOneManyManyftptraditional apps...# of Receivers# of SendersOneOneManyManyweb downloadssoftware distributionvideo-on-demandserver push...ftptraditional apps... CSCI 530, Spring 2010 Copyright © William C. Cheng T5Scalable Data Transfer ApplicationsEnd-system / Application-levelchat roomsvideo conferencingmultiplayer games...# of Receivers# of SendersOneOneManyManyweb downloadssoftware distributionvideo-on-demandserver push...ftptraditional apps... CSCI 530, Spring 2010 Copyright © William C. Cheng T6Scalable Data Transfer ApplicationsEnd-system / Application-levelchat roomsvideo conferencingmultiplayer games...Bistro!!# of Receivers# of SendersOneOneManyManyweb downloadssoftware distributionvideo-on-demandserver push...ftptraditional apps... CSCI 530, Spring 2010 Copyright © William C. Cheng T7Scalable Data Transfer ApplicationsEnd-system / Application-level CSCI 530, Spring 2010 Copyright © William C. Cheng T8Who Is Working on Uploads?To the best of our knowledge, there is no existingwork on making many-to-one communication atthe application layer scalable and efficient CSCI 530, Spring 2010 Copyright © William C. Cheng T9What Are Upload Applications?Internet-basedComputingDistanceEducationDigitalDemocracyE-commerceInternet-basedStorageDigitalGovernmentDataWarehousingIRS income tax submissionHard deadlinespaper submissionreal-life eventsNo hard deadlinesInternet-based storageData warehousing CSCI 530, Spring 2010 Copyright © William C. Cheng T10Why is Upload Different?many-to-one data transferdata consumed later (will exploit this)contention for service rather than datareplication of services and resources for a singleevent is expensive, inflexible, & not scalableread vs. writetraditional solution such as replication of data(caching), replacement of data, etc. won’t helpfault tolerance, security CSCI 530, Spring 2010 Copyright © William C. Cheng T11Traditional Approaches(at the application layer)Increase capacitySpread the load ... over time, space, or bothExamplesdata replicationdata replacementservice replicationserver pushftp mirroring, web cachingmulti-resolution images, videoDNS lookup, NTPnews download, software distributionChange the workload CSCI 530, Spring 2010 Copyright © William C. Cheng T12Traditional Approaches (Cont...)Example: AkamaiRelieve web download hotspotsthrough data replication (caching)Use their own network of servers,with strategic placement of serversaround the worldClients include: Microsoft, Paramount, Wired, CBS Sports, Nike, BBC America, Apple, ...Why are there hotspots?real-life events> 2700 servers> 45 countries> 150 networksavailability ofnew dataCSCI 530, Spring 2010 Copyright © William C. Cheng T13Our GoalsA single infrastructure (termed Bistro) for alldata collection needsgood performance (for both service providers and users)scalable (shares resources among all service providers) secure (one service provider does not have to trustanother) CSCI 530, Spring 2010 Copyright © William C. Cheng T14Current State of Affairs for UploadingServer...Independent data transfers over the Internet, i.e., TCP/IPTCP/IP shares bandwidth fairlyindividual clients experience poor performancewhen number of clients is large (if transfer timeis long enough to see other connections)TCP/IP is here to stayClientsNot scalable! CSCI 530, Spring 2010 Copyright © William C. Cheng T15Key Observations(applications with deadlines)Existence of hot spots in uploads is largely due toapproaching deadlinesExacerbated by long transfer timesProblem: too much data ... too little time ...Client 1DestinationServerInternetbottleneckClient 2Client 3Client NData 1Data 2Data 3Data N CSCI 530, Spring 2010 Copyright © William C. Cheng T16Key Observations (Cont...)(applications with deadlines)What is actually needed is an assurance that specificdata was submitted before a specific timeThen the transfer of that data needs to be done in a timelymanner, but does not have to occur by the deadlinei.e., we need a commitment of what and when asubmission took placeunlink downloads, the data may not be consumedat the server right awayif a piece of data arrives after the deadline, we justneed to guarantee that it’s exactly the same pieceof data that was committed before the deadline CSCI 530, Spring 2010 Copyright © William C. Cheng T17Solution with BistroBefore deadline:TimeDeadlineTraffic at/near Destination Server:Client 1Client NDestinationServerInternetbottleneckHash(fingerprint)Hashes(fingerprints)Datawith contemporary cryptographytechnology, hash size isconstant (10s of bytes), nomatter how big a document is CSCI 530, Spring 2010 Copyright © William C. Cheng T18A Solution to Upload with Deadlines(b) upload with the Bistro Systemafter Bistro softwareis installed on the Server(a) upload without BistroServerClients...DataFlowDestination bistro(i.e., Server)Clients.........Bistro SystemDataFlowbistrosA bistro can be installed on an IRS server or a tax partner’s serverA bistro is like an e-Post Office, built to handle certified e-submissionsNote:Picutre above is for a single event, e.g., 2005 personal income tax submissionMultiple events may be going on concurrently or overlapping, eachwith a different destination serverCSCI 530, Spring 2010 Copyright © William C. Cheng T19A Solution to Upload with DeadlinesDestination bistro issues a timestamped and certified e-ticketA client generates a fingerprint for the document (tax return)Step 1: Real-time fingerprinting & timestamp(b) upload with the Bistro Systemafter Bistro softwareis installed on the Server(a) upload without BistroServerClients...DataFlowDestination bistro(i.e., Server)Clients.........Bistro SystemDataFlowbistros CSCI 530, Spring 2010 Copyright © William C. Cheng T20A Solution to Upload with DeadlinesStep 2: Low-latency upload to any intermediary
View Full Document
Unlocking...