COPS

Background

  • System setup
    • Multiple data centers, separated by long distance links
    • Each data center has many nodes, storage state is fully replicated at each data center
  • Desired performance:
    • Writes finish w/o waiting for remote sites (async. replication)
    • Reads contact local site only

What’s causal consistency?

  • Systems that obey the following set of partial orders
      1. if op1 and op2 are in the single thread of execution and op1 is issued before op2, then op1 –> op2. (On client1, op1: creates pid1, op2: adds pid1 to album aid. All node s see the effect of op2 after op1)
      1. If op2 reads the result written by op1, then op1–> op2 (On client1, op1: adds pid1 to album aid On client 2, op2: reads pid1 in album)
      1. if op1–>op2, op2–>op3, then op1–>op3 (On client 1, op1: adds pid1 to album aid On client 2, op2: reads pid1 in album, op3: deletes pid1. All nodes see op1—>op2—>op3, i.e. pid1 is deleted)

COPS’ approach

  • partition key-space among nodes
  • explicitly keep track explicit dependencies (partial orders) for each write
    • Site A performs a write, replicates it together with the dependencies to another site B
    • Site B waits until the write’s dependencies are satisfied in B before committing the write.

Client library

  • put(key,value,context); //put’s dependencies are set by context, new dependency includes the new put version.
  • value = get(key,context); //add dependencies of get to context
  • example
    Client 1:  put(x,1) ---> put(y,2)           
             ctx1:x1       store (x1) with y2, ctx=x1,y2
    Client 2:                get(y)=2 --> put(x,4)                         
                           ctx2:x1,y2   store (y2) with x4, ctx2:x4,y2
    Client 3:                             get(x)=4 --> put(z,5)                                      
                                        ctx3:x4,y2  store (x4,y2), ctx3:x4,y2,z5
    
  • Site A replicate y2 with dependency (x1) to site B.
  • Site B performs a dependency check locally to wait for x1 to commit before committing y2.

Anomalies under causal consistency

  • – write-write conflict
  • – do not capture causality caused by external communication. I posted a picture, call my friend to check it out.

Reasoning causal consistency

  • example (consider the case with and without c3)
    x,y=0 initially
    c1: put(x)=1        get(y)=?
    c2:      put(y)=1
    c3:                         get(y)=1  get(x)=0
      
    

Time to check to time to use

  • example 1
    Alice:               private ACL; upload photo
    Bob:   read public ACL,                         read photo 
    
  • example 2
    Alice: private ACL; upload photo
    Bob:                            read photo, read ACL 
    
  • example 2 ``` Alice: private ACL; upload photo; delete photo; public ACL Bob: read photo, read ACL