Berkeley Ninja ArchitectureACID vs BASE1.Strong Consistency2. Availability not considered3. Conservative--> Traditional databases1. Weak consistency2. Availability is a primary design element3. Aggressive--> large-scale distributed systemsCAP Theorem: Of the three different qualities(network partitions, consistency, availability), at most two of the three qualities can be maintained for any given system.Boundary between entities1. Remote Procedure Calls --> The way it is used currently is not sustainable for larger systems2. Trusting the other side --> need to check arguments before executing RPC3. Multiplexing between many different clients --> How this is done effects boundary definitionKey Messages1. Parallel programming tends to avoids the notion of availability, online evolution, checkpoint/restart (although currently this is changing)2. For Robustness in distributed systems, we must think probabilistically about system design qualities3. Message-Passing seems to be most effective solution, as boundaries must be clearly defined. 4. Need to have more support for partial failure, graceful degradation, and parallel I/ODiscussion1. Do you believe that techniques applied in distributed database community also can apply to large-scale distributed systems? Or does a completely new approach need to be taken?2. This work was presented in 2000. Do the principles of robustness apply for today's distributed systems? 3. Do you agree with the notion that without clear boundaries, large-scale distributed systems will remain unmaintainable?Cumulus: A FileSystem Backup to the CloudCumulus Design Choice1. Minimal Interface(4 commands)2. Highly portable 3. Efficient (through simulation)4. Practicality (Amazon S3 prototype)A Cloud Computing Design DecisionSoftware as a Service(thick cloud)1. Highly specific implies Better Performance2. Reduced FlexibilityUtility Computing(thin cloud)1. Abstract2. Portable3. Less EfficientWhat is the right choice? And is there a right choice?Comparison of Cumulus to Other Systems●Simplest backup system that most will be familiar with: tar, gzip●Others: rsync, rdiff-backup, Box Backup, Jungle Disk, Duplicity, Brackup--> In contrast to all other systems, Cumulus supports multiple snapshots, simple servers, incremental backups , sub-file disk storage, and encryption.Simple User CommandsGet : given a pathname, retrieve the contents of a file from the serverPut: Store the complete file on the server, given its pathnameList: Get the names of files stored on server Delete: Remove the given file from the server, reclaiming it's spaceWith these four commands, one can support incremental backups on a wide variety of systems.Snapshot Storage Format1. The above illustrates how snapshots are structured on a storage server, using Cumulus. 2. Two different snapshots are taken(on two different days), and each snapshot contains two separate files (labeled file1 and file2) 3. The file1 changes between the two days, while file2 is the same between the two snapshots. 4. The snapshot descriptor contains the date, root, and its corresponding segments.Cumulus Research QuestionsWhat is the penalty of using a thin cloud service with a very simple storage interface compared to a more sophisticated service?What are the monetary costs for using remote backup for two typical usage scenarios? How should remote backup strategies adapt to minimize monetary costs as the ratio of network and storage prices varies?How does our prototype implementation compare with other backup systems? What are the additional benefits (e.g., compression, sub-file incrementals) and overheads (e.g., metadata) of an implementation not captured in simulation? What is the performance of using an online service like Amazon S3 for backup?Experimental Setup for Simulation●Two traces are considered as representative workloads for simulation: file-server and user ●For both workloads, traces contain a daily record of meta-data of all files●Thin service model is compared to optimal backup, where only the needed storage/transfer is done, and no more. ● There are justifiable reasons that Cumulus does not try to store each file in one segment because of the other design goals it aims for(encryption, compression, etc.)●Statistics are established for both workloads, as shown below.Establishing Cleaning Threshold1. As the cost of storage increases, cleaning more aggressively gives advantage2. Ideal threshold stabilizes at .5 to .6, when storage is 10 times as expensive as networkCumulus Experimental SimulationBroader Impact“Can one build a competitive product economy around a cloud of abstract commodity resources, or do underlying technical reasons ultimately favor an integrated service-oriented architecture?”→ On one hand, if Cumulus is to be accepted as a general solution for file system backup, many more application must be tested and simulated.→ On the other hand, the need for standardization in the cloud is very important, and a solution like Cumulus should be adopted as quickly as possible.Discussion Questions for Cumulus1. Application-specific solutions vs. general light-weight, portable solutions?2. Who are the users of Cumulus? Would such a backup tool be easy to pick up for a novice?3. Is the interface provided adequate? Should there be more functionality? 4. Is the issue of security with backing up data adequately addressed?1Smoke and Mirrors: Reflecting Files at a Geographically Remote Location Without Loss of PerformanceUSENIX 092Why mirror data?Faster AccessBetter AvailabilityData protection against loss (Disaster Tolerance)3Synchronous Mirroring (Remote Sync)ApplicationMirroring AgentLocal StorageRemote Storage162354Mirroring Agent•Reliable•Slow (Application effectively pauses between step 1 and 6)4Semi-synchronous Mirroring 346ApplicationMirroring AgentLocal StorageRemote Storage12•Faster•Less ReliableMirroring Agent55Asynchronous Mirroring (Local Sync)346ApplicationMirroring AgentLocal StorageRemote Storage12•Faster•Least ReliableMirroring Agent56Mirroring Options:Mirroring SolutionsAsynchronousMirroringSemi-SynchronousMirroringSynchronousMirroringDecreasing Reliability, Decreasing Mirroring Latency7Failure ModelCan occur at any levelSimultaneous or in sequence (rolling disaster)Network elements can drop packets8Data Loss ModelData LossData LossData LossPrimary and MirrorData LossData LossNo LossPrimary and
View Full Document
Unlocking...