CS162 Operating Systems and Systems Programming Lecture 23 Distributed SystemsDistributed Systems are Everywhere!What makes building distributed systems interesting?Distributed ApplicationsUsing Messages: Send/Receive behaviorGeneral’s ParadoxDistributed Decision MakingDistributed TransactionsTwo-Phase Commit DetailsTwo-Phase Commit ExampleTwo-Phase Commit GotchasRemote Procedure CallRPC DetailsRPC Information FlowRPC BindingRPC TransparencyProblems with RPCAdministriviaMicrokernel operating systemsDistributed File SystemsVirtual File System (VFS)Simple Distributed File SystemClient caching to reduce network loadNetwork File System (NFS)NFS ContinuedNFS Cache ConsistencyNFS Pros and ConsAndrew File SystemAndrew File System (con’t)World Wide WebWWW CachingWWW Proxy CachesConclusionMessaging for Producer-Consumer StyleMessaging for Request/Response communicationByzantine General’s ProblemByzantine General’s Problem (con’t)Dealing with FailuresSchematic View of NFS ArchitectureSequential Ordering ConstraintsCS162Operating Systems andSystems ProgrammingLecture 23Distributed SystemsApril 15, 2010Benjamin Hindmanhttp://inst.eecs.berkeley.edu/~cs162Lec 23.204/15/10 Hindman CS162 ©UCB Spring 2010Distributed Systems are Everywhere!•We need (want?) to share physical devices (e.g., printers) and information (e.g., files)•Many applications are distributed in nature (e.g., ATM machines, airline reservations)•Many large problems can be solved by decomposing into lots of smaller problems that can be run in parallel (e.g., MapReduce, SETI@home)Lec 23.304/15/10 Hindman CS162 ©UCB Spring 2010What makes building distributed systems interesting?•Programming models•Transparency•Fault-tolerance•Performance•Scalability•Consistency•SecurityLec 23.404/15/10 Hindman CS162 ©UCB Spring 2010Distributed Applications•How do you actually program a distributed application?–Use networking building blocks to provide a basic send/receive abstraction (message passing)»Semantics: sender picks a specific receiver and receiver gets all or none of the message»Queue incoming messages on receive sideNetworkSendReceiveLec 23.504/15/10 Hindman CS162 ©UCB Spring 2010Using Messages: Send/Receive behavior•When should send return?–Asynchronous: return immediately–Synchronous: return after …»Receiver gets message? (i.e., ack received)»When message is safely buffered on destination?»Right away, if message is buffered on source node?•Main question here:–When can the sender be sure that receiver actually received the message?Lec 23.604/15/10 Hindman CS162 ©UCB Spring 2010•General’s paradox: –Constraints of problem: »Two generals, on separate mountains»Can only communicate via messengers»Messengers can be captured–Problem: need to coordinate attack»If they attack at different times, they all die»If they attack at same time, they win–Named after Custer, who died at Little Big Horn because he arrived a couple of days too early•Can messages over an unreliable network be used to guarantee two entities do something simultaneously?–Remarkably, “no”, even if all messages get throughYeah, but what if youDon’t get this ack?General’s Paradox11 am ok?So, 11 it is?Yes, 11 worksLec 23.704/15/10 Hindman CS162 ©UCB Spring 2010Distributed Decision Making•Why is distributed decision making desirable?–Fault Tolerance! A group of machines can come to a decision even if one or more of them fail during the processLec 23.804/15/10 Hindman CS162 ©UCB Spring 2010Distributed Transactions•Since we can’t solve the General’s Paradox, let’s solve a related problem, distributed transaction: N machines agree to do something, or not do it, atomically•Why should we care? Banks do this every day (every minute, every second, …)•Two-Phase Commit Protocol–Phase 1, coordinator sends out a request to commit»each participant responds with yes or no–Phase 2»If everyone says yes, coordinator sends out a commit»If someone says no, coordinator sends out an abortLec 23.904/15/10 Hindman CS162 ©UCB Spring 2010Two-Phase Commit Details•Each participant uses a local, persistent, corrupt-free log to keep track of whether a commit has happened»If a machine crashes, when it wakes up it first checks its log to recover state of world at time of crash–Log can be used to complete this process such that all machines either commit or don’t commit–Timeouts can be used to retry if coordinator doesn’t hear from all participantsLec 23.1004/15/10 Hindman CS162 ©UCB Spring 2010Two-Phase Commit Example•Simple Example: AWellsFargo, BChase–Phase 1:»A writes “Begin transaction” to logAB: OK to transfer funds to me?»Not enough funds:BA: transaction aborted; A writes “Abort” to log»Enough funds:B: Write new account balance & promise to commit to logBA: OK, I can commit–Phase 2: A can decide for both whether they will commit»A: write new account balance to log»Write “Commit” to log»Send message to B that commit occurred; wait for ack»Write “Got Commit” to log•What if B crashes at beginning? –Wakes up, does nothing; A will timeout, abort and retry•What if A crashes at beginning of phase 2?–Wakes up, sees that there is a transaction in progress; sends “Abort” to B•What if B crashes at beginning of phase 2?–B comes back up, looks at log; when A sends it “Commit” message, it will say, “oh, ok, commit”Lec 23.1104/15/10 Hindman CS162 ©UCB Spring 2010Two-Phase Commit Gotchas•Undesirable feature of Two-Phase Commit: blocking–One machine can be stalled until another site recovers:»Site B writes “prepared to commit” record to its log, sends a “yes” vote to the coordinator (site A) and crashes»Site A crashes»Site B wakes up, check its log, and realizes that it has voted “yes” on the update. It sends a message to site A asking what happened. At this point, B cannot decide to abort, because update may have committed»B is blocked until A comes back–A blocked site holds resources (locks on updated items, pages pinned in memory, etc) until learns fate of update•Alternatives such as “Three Phase Commit” don’t have this blocking problem•What happens if one or more of the participants is malicious?Lec 23.1204/15/10 Hindman CS162 ©UCB Spring 2010Remote Procedure Call•Raw messaging is a bit too low-level for programming•Another option: Remote Procedure Call (RPC)–Looks like a local procedure
View Full Document
Unlocking...