Consistency and ReplicationWhy replicate?Object ReplicationReplication and ScalingData-Centric Consistency ModelsStrict ConsistencySequential ConsistencyLinearizabilityLinearizability/Sequential Consistency ExampleCausal consistencyOther modelsOther ModelsSummary of Data-centric Consistency ModelsTACT: Tunable ConsistencyMid-term Exam CommentsCS677: Distributed OSComputer ScienceLecture 14, page 1Consistency and Replication•Introduction •Consistency models–Data-centric consistency models–Client-centric consistency modelsCS677: Distributed OSComputer ScienceLecture 14, page 2Why replicate?•Data replication: common technique in distributed systems•Reliability–If one replica is unavailable or crashes, use another–Protect against corrupted data•Performance–Scale with size of the distributed system (replicated web servers)–Scale in geographically distributed systems (web proxies)•Key issue: need to maintain consistency of replicated data–If one copy is modified, others become inconsistentCS677: Distributed OSComputer ScienceLecture 14, page 3Object Replication•Approach 1: application is responsible for replication–Application needs to handle consistency issues•Approach 2: system (middleware) handles replication–Consistency issues are handled by the middleware–Simplifies application development but makes object-specific solutions harderCS677: Distributed OSComputer ScienceLecture 14, page 4Replication and Scaling•Replication and caching used for system scalability•Multiple copies:–Improves performance by reducing access latency –But higher network overheads of maintaining consistency–Example: object is replicated N times•Read frequency R, write frequency W •If R<<W, high consistency overhead and wasted messages•Consistency maintenance is itself an issue–What semantics to provide?–Tight consistency requires globally synchronized clocks!•Solution: loosen consistency requirements–Variety of consistency semantics possibleCS677: Distributed OSComputer ScienceLecture 14, page 5Data-Centric Consistency Models•Consistency model (aka consistency semantics or constraints)–Contract between processes and the data store•If processes obey certain rules, data store will work correctly–All models attempt to return the results of the last write for a read operation•Differ in how “last” write is determined/definedCS677: Distributed OSComputer ScienceLecture 14, page 6Strict Consistency•Any read always returns the result of the most recent write–Implicitly assumes the presence of a global clock–A write is immediately visible to all processes•Difficult to achieve in real systems as network delays can be variableCS677: Distributed OSComputer ScienceLecture 14, page 7Sequential Consistency•Sequential consistency: weaker than strict consistency–Assumes all operations are executed in some sequential order and each process issues operations in program order•Any valid interleaving is allowed•All agree on the same interleaving•Each process preserves its program order•Nothing is said about “most recent write”CS677: Distributed OSComputer ScienceLecture 14, page 8Linearizability•Assumes sequential consistency and–If TS(x) < TS(y) then OP(x) should precede OP(y) in the sequence–Stronger than sequential consistency–Difference between linearizability and serializbility?•Granularity: reads/writes versus transactions•Example:Process P1 Process P2 Process P3x = 1;print ( y, z);y = 1;print (x, z);z = 1;print (x, y);CS677: Distributed OSComputer ScienceLecture 14, page 9Linearizability/Sequential Consistency Example•Four valid execution sequences for the processes of the previous slide. The vertical axis is time.x = 1;print ((y, z);y = 1;print (x, z);z = 1;print (x, y);Prints: 001011Signature: 001011 (a)x = 1;y = 1;print (x,z);print(y, z);z = 1;print (x, y);Prints: 101011Signature: 101011 (b)y = 1;z = 1;print (x, y);print (x, z);x = 1;print (y, z);Prints: 010111Signature: 110101 (c)y = 1;x = 1;z = 1;print (x, z);print (y, z);print (x, y);Prints: 111111Signature: 111111 (d)CS677: Distributed OSComputer ScienceLecture 14, page 10Causal consistency•Causally related writes must be seen by all processes in the same order. –Concurrent writes may be seen in different orders on different machinesNot permitted PermittedCS677: Distributed OSComputer ScienceLecture 14, page 11Other models•FIFO consistency: writes from a process are seen by others in the same order. Writes from different processes may be seen in different order (even if causally related)–Relaxes causal consistency–Simple implementation: tag each write by (Proc ID, seq #)•Even FIFO consistency may be too strong!–Requires all writes from a process be seen in order•Assume use of critical sections for updates–Send final result of critical section everywhere–Do not worry about propagating intermediate results•Assume presence of synchronization primitives to define semanticsCS677: Distributed OSComputer ScienceLecture 14, page 12Other Models •Weak consistency–Accesses to synchronization variables associated with a data store are sequentially consistent–No operation on a synchronization variable is allowed to be performed until all previous writes have been completed everywhere–No read or write operation on data items are allowed to be performed until all previous operations to synchronization variables have been performed.•Entry and release consistency–Assume shared data are made consistent at entry or exit points of critical sectionsCS677: Distributed OSComputer ScienceLecture 14, page 13Summary of Data-centric Consistency ModelsConsistency DescriptionStrict Absolute time ordering of all shared accesses matters.LinearizabilityAll processes must see all shared accesses in the same order. Accesses are furthermore ordered according to a (nonunique) global timestampSequential All processes see all shared accesses in the same order. Accesses are not ordered in timeCausal All processes see causally-related shared accesses in the same order.FIFOAll processes see writes from each other in the order they were used. Writes from different processes may not always be seen in that order(a)Consistency DescriptionWeak Shared data can be counted on to be consistent only after a synchronization is doneRelease Shared data are made consistent when a critical region is exitedEntry Shared data pertaining to a