DOC PREVIEW
U of I CS 525 - Advanced Topics in Distributed Systems

This preview shows page 1-2-3-23-24-25-26-46-47-48 out of 48 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 48 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

P2P AppsSlide 2Storage management & caching in PASTBackgroundOperationsOperations: InsertOperations: LookupOperations: ReclaimStorage ManagementReplica DiversionReplica Diversion (2)Replica Diversion (3)Replica Diversion (4)File DiversionMaintaining ReplicasMaintaining Replicas (2)OptimizationExperimental SetupExperimental ResultsExperimental Results (2)Experimental Results (3)Experimental Results (4)DiscussionOverCite A Distributed, Cooperative CiteSeerMotivationContributionArchitectureLife of a QueryGlobal Data StructureGlobal Data Structure (2)Local Data StructuresLocal Data Structures (2)Web CrawlerImplementationPerformancePerformance (2)Performance (3)Slide 38Colyseus A Distributed Architecture for Online Multiplier GamesBackground & MotivationSlide 41Colyseus ArchitectureColyseus Architecture (2)Colyseus Architecture: Object LocationRange Queriable DHT: OverviewSlide 46Experimental Results: Communication CostSlide 48P2P AppsCS525Advanced Topics in Distributed SystemsSpring 07Presented By: Imranul Hoque, Sonia Jahid2P2P Apps•Applications that use P2P techniques for better performanceOR•Applications that are built on P2P protocols•We’ll consider both–PAST: Built on top of Pastry–OverCite: Uses DHT instead of DB for storage–Colyseus: Uses range-queriable DHT for pub-subStorage management & caching in PASTAntony Rowstron, Peter Druschel4Background•P2P Storage Utility•Built on top of Pastry•Aim–Strong Persistence•k number of replicas–High Availability•Caching–Scalability•High utilization–Security•Quota, Key, Encrypted Routing Table5Operations•Insert–Parameters: name, owner-credentials, k, file–Returns: 160 bit identifier (fileId)•Lookup–Parameter: fileId–Returns: file from one of the (near) k nodes•Reclaim–Parameters: fileId, owner-credentials6Operations: InsertInsert1. fileId = SHA-1(name, public-keyowner, salt)2. storage = storage – k * filesize3. fileCert = fileId, SHA-1(file), k, salt, date, owner4. [file, fileCert, {fileCert}private-key] routed via Pastry5. Before insertion, verify the certificate 6. Forward insert request to other (k-1) nodes 7. Once done each of the k replicas issues store receipt8. Client gets the ack and does verification7Operations: LookupLookup1. Client node issues request message2. Pastry routes request message to a node containing file3. Node replies with [file, fileCert, {fileCert}private-key]4. Client verifies the file8Operations: ReclaimReclaim1. Client issues reclaim certificate2. Reclaim certificate routed via Pastry3. Replicas verify the reclaim certificate4. Each storing nodes issues a reclaim receipt5. A client receiving a reclaim receipt:a. Verifies the receiptb. storage = storage + filesizeReclaim vs. Delete9Storage Management•Goal–High global storage utilization–Graceful degradation at maximal utilization•Responsibility–Balancing free storage space–File copies are maintained by the k nodes with nodeIds closest to fileId–Conflicting Responsibility!•Solution:–Replica Diversion & File Diversion10Replica Diversion•Purpose: Balance remaining free storage among the nodes in a leaf set•Policy:if (fileSize/freeSpacepri > tpri){Choose a node div from the leaf set:- fileSize/freeSpacediv < tdiv - Not among the k closest nodes of fileId- Do not hold a replica of the same file- freeSpacediv is maximum among all such nodesIf no such node then:- Nodes already stored the replica discard them- Send a NACK to client causing File Diversion}11Replica Diversion (2)A1234Leaf Set of AFile12Replica Diversion (3)A1234Leaf Set of AFileA stores a pointer to 413Replica Diversion (4)A1234Leaf Set of AFileA stores a pointer to 4CA inserts a pointer in CC = (k+1)th closest node14File Diversion•Purpose: Balance remaining free storage space among different portions of nodeId space•Method:- Generate fileId with different salt- Retry insert operation- Repeat process for three times- If third attempt fails, report insert failure15Maintaining Replicas•Nodes exchange keep-alive messages to keep track of failed nodes•Unresponsive nodes replaced by new entries•New joining nodes also cause change in leaf set•If a joining node becomes one of the kth closest of a node N–The joining node keeps a pointer to the file table of N–Affected files are gradually migrated in background•A node discovering that a diverted replica is not a part of its leaf set–Gradually migrates files to a node within the node’s leaf set16Maintaining Replicas (2)•Node failure may cause storage shortage–Remaining nodes in leaf set can’t store files–Contact 2 most distant members of the leaf set to locate space–Fail otherwise•Open Pastry and Proprietary Pastry differ in replica management scheme17Optimization•File Encoding–Idea 1: Don’t store k replicas, add m checksum blocks (Pros and Cons?)–Idea 2: Store fragments of file at separate nodes (Pros and Cons?)•File Caching–Idea: Use unused portion of advertised disk space to cache files–Cache copies can be discarded or evicted at any time18Experimental Setup•2250 PAST Nodes•Number of replicas, k = 5•For Pastry, b = 4•Two different workloads:–Proxy logs from NLANR–File name and size from several file systems at Microsoft19Experimental Results 1. tdiv = 0.052. Lower tpri = higher success rate3. Lower tpri = lower utilization4. Why?tpri = 0.1tdiv = 0.0520Experimental Results (2)File diversions are negligible as long as storage utilization below 83%21Experimental Results (3)Even at 80% utilization less than 10% are diverted replicas22Experimental Results (4)1. At low storage utilization Hit Rate is high2. # of hop increases as utilization increases23Discussion•CFS: Built on top of Chord–CFS stores blocks rather than whole files–CFS relies on caching for small files only•Ivy: Read/Write P2P file system–Based on a set of logs and DHash–Provides an NFS like file system view to the user–Uses version vectors for synchronization•OceanStore: Uses un-trusted servers–Data is encrypted–Uses ACL for write access, key for read access–Data migrated based on access patternsOverCiteA Distributed, Cooperative CiteSeerJeremy Stribling, Jinyang Li, Isaac G. Councill, M. Frans Kaashoek, Robert Morris25Motivation•CiteSeer allows users to search and browse a large archive of research papers•Centralized design: Crawler, Database, Indexer•OverCite aggregates donated resources at multiple cites•Challenge:–Load balancing


View Full Document

U of I CS 525 - Advanced Topics in Distributed Systems

Documents in this Course
Epidemics

Epidemics

12 pages

LECTURE

LECTURE

7 pages

LECTURE

LECTURE

39 pages

LECTURE

LECTURE

41 pages

P2P Apps

P2P Apps

49 pages

Lecture

Lecture

48 pages

Epidemics

Epidemics

69 pages

GRIFFIN

GRIFFIN

25 pages

Load more
Download Advanced Topics in Distributed Systems
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Advanced Topics in Distributed Systems and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Advanced Topics in Distributed Systems 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?