Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Eventual Consistency and Deterministic Dataflow Programming

Eventual Consistency and Deterministic Dataflow Programming

LADIS '14

Christopher Meiklejohn

October 23, 2014
Tweet

More Decks by Christopher Meiklejohn

Other Decks in Programming

Transcript

  1. Eventual Consistency
    and Deterministic Dataflow Programming
    Christopher Meiklejohn
    Basho Technologies, Inc. / SyncFree Consortium
    LADIS Workshop ’14
    Cambridge, United Kingdom
    October 23, 2014
    Meiklejohn (Basho) DerflowL
    LADIS ’14 1 / 17

    View full-size slide

  2. Overview
    1
    Motivation
    2
    Our Approach
    3
    Conclusions and Future Work
    4
    References
    Meiklejohn (Basho) DerflowL
    LADIS ’14 2 / 17

    View full-size slide

  3. Motivation
    Riak [7] is a Dynamo-inspired [4] key-value store
    Querying by key main mechanism for data storage and retrieval
    Three mechanisms presently for more expressive data access:
    MapReduce-like system [3, 6], secondary indexing, integration with
    Apache Solr
    Each additional mechanism contains drawbacks
    Mechanisms are not fault-tolerant
    Structure is rigid: need to know schema when storing data
    Not possible to perform composition of queries
    Meiklejohn (Basho) DerflowL
    LADIS ’14 3 / 17

    View full-size slide

  4. The Goal
    Building large scale distributed applications with strong properties
    Users want to be able to compute with their data in an efficient and
    composable manner while guaranteeing strong properties
    Provide a framework for building eventually consistent materialized
    views that have strong convergence properties
    Data types which have strong convergence properties
    Deterministic language to compose data types and preserve these
    strong convergence properties
    Meiklejohn (Basho) DerflowL
    LADIS ’14 4 / 17

    View full-size slide

  5. Our Approach
    Leverage conflict-free replicated data types (CRDTs)
    Builds on top of Derflow [1], a deterministic dataflow language
    Extends Derflow with CRDTs, similar to LVars [8] and Bloom
    L
    [2]
    Applies techniques from C-CRDTs [10] for incremental computation
    Uses version vectors with exceptions for tracking program state [9]
    Uses Dynamo-style quorums for aggregating computation results
    Meiklejohn (Basho) DerflowL
    LADIS ’14 5 / 17

    View full-size slide

  6. CRDTs
    Conflict-free Replicated Data Types [11]
    Strong Eventual Consistency (SEC)
    Data structures which contain resolution logic
    Based on mathematical properties
    Figure : Set where element can be added and removed once. 1
    2
    1
    Can be generalized to arbitrary adds and removes.
    2
    Iwan Briquemont, Optimising Client-side Geo-replication with Partially Replicated
    Data Structures, Master’s thesis, Université catholique de Louvain, Aug. 2014.
    Meiklejohn (Basho) DerflowL
    LADIS ’14 6 / 17

    View full-size slide

  7. Derflow / Derflow
    L
    Distributed deterministic programming language
    Similar to Mozart/Oz, Ozma [5]
    Relies on a distributed variable store
    Extend the model to state-based CRDTs and C-CRDTs
    Inputs and outputs of programs are CRDTs
    Meiklejohn (Basho) DerflowL
    LADIS ’14 7 / 17

    View full-size slide

  8. Computational CRDTs
    Defines CRDTs which observe partial state
    Allows for chaining, but requires explicit mapping function
    Defines a merge function to combine partial results
    Maps well to Dynamo data storage model
    Replicas within a replica set should have equal state,
    Disjoint between replica sets
    Meiklejohn (Basho) DerflowL
    LADIS ’14 8 / 17

    View full-size slide

  9. Version Vectors with Exceptions
    Each program tracks the versions of inputs used in computations
    Data is tracked in a version vector with exceptions
    Treat computations as values in Dynamo-style system
    Can be used for anti-entropy, read repair, etc.
    Meiklejohn (Basho) DerflowL
    LADIS ’14 9 / 17

    View full-size slide

  10. Dynamo Quorums
    Identify covering set of
    nodes for key space
    Can include multiple copies
    from the same replica set
    Merge (LUB) across replica
    sets
    Merge (Delta) across copies
    of the same replica
    Figure : Ring with 32 partitions and
    3 nodes
    Meiklejohn (Basho) DerflowL
    LADIS ’14 10 / 17

    View full-size slide

  11. Dynamo Quorums
    Identify covering set of
    nodes for key space
    Can include multiple copies
    from the same replica set
    Merge (LUB) across replica
    sets
    Merge (Delta) across copies
    of the same replica
    Figure : Ring with 32 partitions and
    3 nodes with covering set identified
    in red
    Meiklejohn (Basho) DerflowL
    LADIS ’14 11 / 17

    View full-size slide

  12. Dynamo Quorums
    Identify covering set of
    nodes for key space
    Can include multiple copies
    from the same replica set
    Merge (LUB) across replica
    sets
    Merge (Delta) across copies
    of the same replica
    Figure : Ring with 32 partitions and
    3 nodes with fault-tolerant covering
    set identified in red and blue
    Meiklejohn (Basho) DerflowL
    LADIS ’14 12 / 17

    View full-size slide

  13. Conclusions and Future Work
    Current status:
    DerflowL implemented on Riak Core, extended with CRDTs
    Implemented a test harness for DerflowL using open source work from
    Basho to test distribution and operation in partitions
    Future work:
    Composing CRDTs currently is very explicit and cumbersome
    Will be solved as part of the SyncFree research project
    Program analysis for detection of incorrect programs
    Meiklejohn (Basho) DerflowL
    LADIS ’14 13 / 17

    View full-size slide

  14. References I
    M. Bravo, Z. Li, P. Van Roy, and C. Meiklejohn.
    Derflow: Distributed deterministic dataflow programming for erlang.
    In Proceedings of the Thirteenth ACM SIGPLAN Workshop on Erlang,
    Erlang ’14, pages 51–60, New York, NY, USA, 2014. ACM.
    N. Conway, W. Marczak, P. Alvaro, J. M. Hellerstein, and D. Maier.
    Logic and lattices for distributed programming.
    Technical Report UCB/EECS-2012-167, EECS Department, University of
    California, Berkeley, Jun 2012.
    J. Dean and S. Ghemawat.
    Mapreduce: Simplified data processing on large clusters.
    Commun. ACM, 51(1):107–113, Jan. 2008.
    Meiklejohn (Basho) DerflowL
    LADIS ’14 14 / 17

    View full-size slide

  15. References II
    G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman,
    A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels.
    Dynamo: Amazon’s highly available key-value store.
    In Proceedings of Twenty-first ACM SIGOPS Symposium on Operating
    Systems Principles, SOSP ’07, pages 205–220, New York, NY, USA, 2007.
    ACM.
    S. Doeraene and P. Van Roy.
    A new concurrency model for scala based on a declarative dataflow core.
    In Proceedings of the 4th Workshop on Scala, SCALA ’13, pages 4:1–4:10,
    New York, NY, USA, 2013. ACM.
    B. Fink.
    Distributed computation on dynamo-style distributed storage: Riak pipe.
    In Proceedings of the Eleventh ACM SIGPLAN Workshop on Erlang
    Workshop, Erlang ’12, pages 43–50, New York, NY, USA, 2012. ACM.
    Meiklejohn (Basho) DerflowL
    LADIS ’14 15 / 17

    View full-size slide

  16. References III
    R. Klophaus.
    Riak core: Building distributed applications without shared state.
    In ACM SIGPLAN Commercial Users of Functional Programming, CUFP ’10,
    pages 14:1–14:1, New York, NY, USA, 2010. ACM.
    L. Kuper and R. R. Newton.
    Lvars: Lattice-based data structures for deterministic parallelism.
    In Proceedings of the 2Nd ACM SIGPLAN Workshop on Functional
    High-performance Computing, FHPC ’13, pages 71–84, New York, NY, USA,
    2013. ACM.
    D. Malkhi and D. Terry.
    Concise version vectors in winfs.
    In Proceedings of the 19th International Conference on Distributed
    Computing, DISC’05, pages 339–353, Berlin, Heidelberg, 2005.
    Springer-Verlag.
    Meiklejohn (Basho) DerflowL
    LADIS ’14 16 / 17

    View full-size slide

  17. References IV
    D. Navalho, S. Duarte, N. Preguiça, and M. Shapiro.
    Incremental stream processing using computational conflict-free replicated
    data types.
    In Proceedings of the 3rd International Workshop on Cloud Data and
    Platforms, CloudDP ’13, pages 31–36, New York, NY, USA, 2013. ACM.
    M. Shapiro, N. Preguiça, C. Baquero, and M. Zawirski.
    A comprehensive study of Convergent and Commutative Replicated Data
    Types.
    Rapport de recherche RR-7506, INRIA, Jan. 2011.
    Meiklejohn (Basho) DerflowL
    LADIS ’14 17 / 17

    View full-size slide