1Indranil GuptaLecture 10Peer-to-peer systems – Part IIComputer Science 425Distributed SystemsComputer Science 425Distributed SystemsDHT=Distributed Hash Table• A hash table allows you to insert, lookup and delete objects with keys• A distributed hash table allows you to do the same in a distributed setting (objects=files)• Performance Concerns:– Load balancing– Fault-tolerance– Efficiency of lookups and inserts– Locality• Napster, Gnutella, FastTrack are all DHTs• So is Chord, a structured peer to peer system that we study nextComparative PerformanceO(N)O(N)O(N)GnutellaO(1)O(1)O(1)@client,O(N)@serverNapster#Messagesfor a lookupLookupLatencyMemoryComparative PerformanceO(log(N))O(log(N))O(log(N))ChordO(N)O(N)O(N)GnutellaO(1)O(1)O(1)(O(N)@server)Napster#Messagesfor a lookupLookupLatencyMemoryChord• Developers: I. Stoica, D. Karger, F. Kaashoek, H. Balakrishnan, R. Morris, Berkeley and MIT• Intelligent choice of neighbors to reduce latency and message cost of routing (lookups/inserts)2Chord• Uses Consistent Hashing on peer’s address– SHA-1 = Simple Hash Algorithm-1, a standard hashing function)– Difficult to reverse– SHA-1(ip_address,port) 160 bit string – Output truncated to m (< 160) bits– Called peer id (integer between 0 and )– Not unique but id conflicts very very unlikely– “Consistent”? Any node A can calculate the peer id of any other node B, given B’s ip address and port number– Can then map peers to one of logical points on a circlem212−mRing of peersN80N112N96N160Say m=7N32N45And 6 peers/nodesPeer pointers (1): successorsN800Say m=7N32N45N112N96N16(similarly predecessors)Peer pointers (2): finger tablesN8080 + 2080 + 2180 + 2280 + 2380 + 2480 + 2580 + 260Say m=7N32N45ith entry at peer with id n is first peer with id >= )2(mod2min +N112N96N16i ft[i]0 961 962 963 964 965 1126 16Finger Table at N80What about the files?• Filenames also mapped using same consistent hash function– SHA-1(filename) 160 bits, truncated to m bits=file id or key• File cnn.com/index.htmlthat maps to file id /key 42 is stored at first peer with id >= 42• Note that we are considering a different file-sharing application here : cooperative web caching– “Peer”=client browser, “files”=html objects– Peers can now fetch html objects from other peers that have them, rather than from server– The same discussion applies to any other file sharing application, including that of mp3 filesMapping FilesN800Say m=7N32N45N112N96N16File cnn.com/index.htmlwith key K42 stored here3SearchN800Say m=7N32N45File cnn.com/index.htmlwith key K42 stored hereWho has cnn.com/index.html?(hashes to K42)N112N96N16SearchN800Say m=7N32N45File cnn.com/index.htmlwith key K42 stored hereAt node n, send query for key k to largest successor/finger entry < k (all mod m)if none exist, send query to successor(n)N112N96N16Who has cnn.com/index.html?(hashes to K42)SearchN800Say m=7N32N45File cnn.com/index.htmlwith key K42 stored hereAll “arrows” are RPCsN112N96N16Who has cnn.com/index.html?(hashes to K42)At node n, send query for key k to largest successor/finger entry < k (all mod m)if none exist, send query to successor(n)AnalysisSearch takes O(log(N)) timeProof – (intuition): at each step, distance between query and peer-with-file reduces by a factor of at least 2(why?)Takes at most m steps: is at most a constant multiplicative factor above N, lookup is O(log(N))– (intuition): after log(N) forwardings, distance to key is at most (why?)Number of node identifiers in a range of is O(log(N)) with high probability (why? SHA-1!)So using successors in that range will be okNm/2Nm/2m2HereNext hopKeyAnalysis (contd.)• O(log(N)) search time holds for file insertions too (in general for routing to any key)– “Routing” can thus be used as a building block for other applications than file sharing [can you name one?]• O(log(N)) time true only if finger and successor entries correct• When might these entries be wrong?Analysis (contd.)• O(log(N)) search time holds for file insertions too (in general for routing to any key)– “Routing” can thus be used as a building block for other applications than file sharing [can you name one?]• O(log(N)) time true only if finger and successor entries correct• When might these entries be wrong?– When you have failures4Search under peer failuresN800Say m=7N32N45File cnn.com/index.htmlwith key K42 stored hereXXXLookup fails (N16 does not know N45)N112N96N16Who has cnn.com/index.html?(hashes to K42)Search under peer failuresN800Say m=7N32N45File cnn.com/index.htmlwith key K42 stored hereXOne solution: maintain r multiple successor entriesin case of failure, use successor entriesN112N96N16Who has cnn.com/index.html?(hashes to K42)Search under peer failures• Choosing r=2log(N) suffices to maintain correctness with high probability– Say 50% of nodes fail– Pr(for given node, at least one successor alive)=– Pr(above is true for all alive nodes)=2log211)21(1NN−=−1)11(212/2≈=−−NNeNSearch under peer failures (2)N800Say m=7N32N45File cnn.com/index.htmlwith key K42 stored hereXXLookup fails (N45 is dead)N112N96N16Who has cnn.com/index.html?(hashes to K42)Search under peer failures (2)N800Say m=7N32N45File cnn.com/index.htmlwith key K42 stored hereXOne solution: replicate file/key at r successors and predecessorsN112N96N16K42 replicatedK42 replicatedWho has cnn.com/index.html?(hashes to K42)Need to deal with dynamic changes Peers fail• New peers join• Peers leaveAll the time Need to update successors and fingers, and ensure keys reside in the right places5New peers joiningN800Say m=7N32N45N112N96N16N40Introducer directs N40 to N32N32 updates successor to N40N40 initializes successor to N45, and obtains fingers from itN40 periodically talks to neighbors to update own finger tableStabilization protocolNew peers joining (2)N800Say m=7N32N45N112N96N16N40N40 may need to copy some files/keys from N45(files with fileid between 32 and 40)K34,K38New peers joining (3)• A new peer affects O(log(N)) other finger entries in the system• Number of messages per peer join= O(log(N)*log(N)) • Similar set of operations for dealing with peers leavingStabilization Protocol• Concurrent peer joins, leaves, failures might cause loopiness of pointers, and failure of lookups– Chord peers periodically run a stabilization algorithm that checks and updates
View Full Document
Unlocking...