Examples Using a Distributed File SystemVelilind’s Laws of ExperimentationA Real-World ExampleWhat is Ficus?PropagationReconciliationGarbage CollectionFicus PerformanceOpen PerformanceRead/Write PerformanceDeletionSlide 12Cross-Machine InterferenceMeasurements of FicusFicus Measurement MethodologyBenchmarks UsedLocal Replica MeasurementsInterference MeasurementsExample of Raw Ficus ResultsAnalysis of Ficus MeasurementsWhat Can be Analyzed from Ficus Experiments?The “Standard” AnalysisEarning Some Self-RespectLearning Something About the SystemSample Analysis of Ficus ExperimentsSome Raw “cp” Benchmark DataIs Ficus Slower Than UFS?Is Ficus Faster Than Remote NFS?What is the Cost of a New Replica?Allocation of Variation for “cp” BenchmarkConfidence Intervals for “cp” RegressionScatter Plot of “cp” RegressionError Scatter of “cp” RegressionError Scatter by Experiment NumberQuantile-Quantile Plot of ErrorsF-test for Ficus “cp” RegressionSummary of Ficus “cp” AnalysisRegression Digression: A Bad ExampleInappropriate Use of RegressionThe Tale of the ResidualsWhite SlideExamples Using a Distributed File SystemAndy WangCIS 5930-03Computer SystemsPerformance Analysis2Velilind’s Lawsof Experimentation•If reproducibility may be a problem, conduct the test only once•If a straight-line fit is required, obtain only two data points3A Real-World Example•Overview of the Ficus file system•Summary of measured data•Analysis of measured data4What is Ficus?•Distributed, replicated file system•Individual computers store replicas of shared files–Fast local access–Shared data•Designed for robustness in face of network disconnections–Anyone can write any file, any time5Propagation•Any update generates a “best-effort” propagation message–Generated on every write system call–Broadcast to all known replicas–Notifies of change, not contents•Receiving site can ignore or can request latest version of file from generating site–Only when no conflict6Reconciliation•Correctness guarantees provided by reconciliation process•Runs periodically•Operates between pair of replicas–Transfers data in one direction only•Complex distributed algorithm–Proven to terminate correctly–Data is guaranteed to eventually get everywhere7Garbage Collection•Tricky to get deletion right•Example: Joe deletes foo while Mary renames it to bar•Need to globally agree that all names are gone•Requires complex two-phase distributed algorithm8Ficus Performance•File access (open) performance•Read/write performance•Aspects of deletion•Reconciliation•Cross-machine interference9Open Performance•Opening file requires–Finding file–Checking for conflicts–Local or remote (NFS-like) open•Finding file requires–Local or remote root access–Tracing path, changing machines as needed•Other steps are basically one RPC each10Read/Write Performance•Reading is same as local or NFS operation•Write is like local or NFS, plus:–Propagation (small outgoing packet)–Attribute update (beyond i-node update)11Deletion•Initially removing a file is reasonably cheap–Mark deleted–Remove from visible namespace–May actually be cheaper than UFS unlink•True cost is garbage collection–How long is space consumed?–CPU cost?–Still have to do unlink equivalent someday12Reconciliation•Runs periodically•Mechanism to suppress under high load•Must check every file–If updated, exchange info with remote–May also transfer data–Special handling, but similar, for new/deleted files•Primary cost is checking what’s updated13Cross-Machine Interference•If you store a replica, you pay costs:–Receiving propagation requests–Running reconciliation as client and server–Servicing remote access requests14Measurements of Ficus•Measurement methodology•Raw results15Ficus Measurement Methodology•Two classes of measurement–Local replica–Interference with remote replicas•Set up test volume•Populate with files•Run several “standard” benchmarks•Destroy volume after test16Benchmarks Used•Eight benchmarks: cp, find, findgrep, grep, ls, mab, rcp, rm•Most did single operation implied by name–cp copied locally within volume–rcp copied from remote machine–findgrep essentially did recursive grep–mab, Modified Andrew Benchmark, did more complex compile-edit-debug simulation17Local Replica Measurements•Set up UFS, remotely-accessed NFS, or Ficus volume–Ficus volume varies from 1 to 8 replicas•Run benchmarks on machine that stores local copy (except for NFS tests)•Ignore effect on machines holding other replicas18Interference Measurements•Set up UFS volume on “interfered” machine•On 1 to 3 other machines, set up 2-replica Ficus volume–Unique volume for each machine–Second replica stored on “interfered” machine•Run all 8 benchmarks simultaneously on all machines–Compare UFS time to uninterfered version19Example ofRaw Ficus Results/h/users/ficus/BENCH/FICUS/RESULTS/950531.211023/benchtimes:ficus mab 2 162.9 real 83.2 user 40.9 sys•Test was run on May 31, 1995, at 21:10:23•Ficus test with 2 replicas, MAB benchmark•162.9 seconds for run; 83.2 user time, 40.9 charged to system20Analysis of Ficus Measurements•What can be analyzed?•Sample of analysis21What Can be Analyzed from Ficus Experiments?•Easy, “standard” stuff•A bit of self-respect•Actually learning something22The “Standard” Analysis•Everybody publishes means, usually in nice tables or graphs•Standard deviations are becoming fairly common•Sometimes they even tell you how many runs they did–Allows you to generate confidence intervals23Earning Some Self-Respect•You should always provide the reader or listener with at least:–A mean of a specified number of runs–A confidence interval at 90% or higher–An analysis of whether the results are meaningful•Standard deviations are nice but not as important as confidence intervals24Learning SomethingAbout the System•Use confidence intervals to compare various parameters and results•Consider whether regression is meaningful•Can you do multivariate regression?•What about ANOVA?25Sample Analysisof Ficus Experiments•We will consider only the “cp” benchmark•Local-replica tests only•Questions to ask:–Is Ficus significantly slower than UFS?–Is Ficus faster than NFS remote access?–What is the cost of adding a remote replica?26Some Raw“cp” Benchmark