$30 off During Our Annual Pro Sale. View Details »

A Certain Tendency of the Database Community

A Certain Tendency of the Database Community

Salon des Refusés, Programming 2017
Brussels, Belgium

Christopher Meiklejohn

April 04, 2017
Tweet

More Decks by Christopher Meiklejohn

Other Decks in Research

Transcript

  1. A Certain Tendency of
    the Database Community
    Christopher S. Meiklejohn
    Université catholique de Louvain, Belgium
    Instituto Superior Técnico, Portugal
    1
    LIGHT ONE

    View Slide

  2. Certain Tendency
    • Certain tendency

    Replicated databases are treated as a “single” system; the databases are
    the “source of truth”
    2

    View Slide

  3. Certain Tendency
    • Certain tendency

    Replicated databases are treated as a “single” system; the databases are
    the “source of truth”
    • Data ownership by clients

    Data is “owned” by the clients that create the data and data exists as
    soon as it is created
    2

    View Slide

  4. Certain Tendency
    • Certain tendency

    Replicated databases are treated as a “single” system; the databases are
    the “source of truth”
    • Data ownership by clients

    Data is “owned” by the clients that create the data and data exists as
    soon as it is created
    • Database is an optimization, bottleneck

    Databases serve as a “convenience” that make it easier to write
    applications: think: shared memory registers: however, reduced
    availability
    2

    View Slide

  5. Certain Tendency
    • Certain tendency

    Replicated databases are treated as a “single” system; the databases are
    the “source of truth”
    • Data ownership by clients

    Data is “owned” by the clients that create the data and data exists as
    soon as it is created
    • Database is an optimization, bottleneck

    Databases serve as a “convenience” that make it easier to write
    applications: think: shared memory registers: however, reduced
    availability
    • The edge is the source of truth!

    We need models and abstractions that allow us to write correct applications
    that operate with distributed data where it is being generated: the edge
    2

    View Slide

  6. Background
    Consistency
    3

    View Slide

  7. Consistency Models
    • Contract

    Between the application developer and the system
    that application will be deployed on
    4

    View Slide

  8. Consistency Models
    • Contract

    Between the application developer and the system
    that application will be deployed on
    • Guaranteed outcomes following certain rules

    Event interleaving, possible partial-orders, update
    visibility, when and where, etc.
    4

    View Slide

  9. Consistency Models
    • Contract

    Between the application developer and the system
    that application will be deployed on
    • Guaranteed outcomes following certain rules

    Event interleaving, possible partial-orders, update
    visibility, when and where, etc.
    • Required for building applications

    Otherwise, we may pick a system to deploy our
    application on where our application returns
    incorrect results
    4

    View Slide

  10. Strong vs. Weak
    • Strong

    Linearizability is the strongest; respects
    the “real-time” order of events
    5

    View Slide

  11. Strong vs. Weak
    • Strong

    Linearizability is the strongest; respects
    the “real-time” order of events
    • Weak

    Eventual consistency; informally
    specified, no bound on when an update
    may be visible
    5

    View Slide

  12. Eventual Consistency
    6
    “...the storage system guarantees that if no new updates
    are made to the [replicated, shared] object, eventually all
    accesses [to any replica] will return the
    last updated value.” - W. Vogels

    View Slide

  13. Eventual Consistency
    6
    “...the storage system guarantees that if no new updates
    are made to the [replicated, shared] object, eventually all
    accesses [to any replica] will return the
    last updated value.” - W. Vogels
    Rather weak model, but used by many large-scale
    distributed systems today…

    View Slide

  14. 7
    Why pick a weaker model
    when stronger models exist?

    View Slide

  15. CAP Theorem
    • Consistency is at odds with availability

    If systems wish to remain functioning under
    network partitions, systems must sacrifice one
    or the other
    8

    View Slide

  16. CAP Theorem
    • Consistency is at odds with availability

    If systems wish to remain functioning under
    network partitions, systems must sacrifice one
    or the other
    • Consistency

    Guarantees on event order and event visibility
    8

    View Slide

  17. CAP Theorem
    • Consistency is at odds with availability

    If systems wish to remain functioning under
    network partitions, systems must sacrifice one
    or the other
    • Consistency

    Guarantees on event order and event visibility
    • Availability

    Ability for a system to keep servicing requests
    under network partitions and/or failures
    8

    View Slide

  18. CAP Example
    • Two replicas of an reservation system…

    Replicated for fault-tolerance to ensure system availability
    9

    View Slide

  19. CAP Example
    • Two replicas of an reservation system…

    Replicated for fault-tolerance to ensure system availability
    • Two concurrent requests…

    Tom and Chris attempt to reserve the last available seat on a plane
    9

    View Slide

  20. CAP Example
    • Two replicas of an reservation system…

    Replicated for fault-tolerance to ensure system availability
    • Two concurrent requests…

    Tom and Chris attempt to reserve the last available seat on a plane
    • Two possible paths…

    If one replica of the system can not reach the other replica, we have two
    choices:
    9

    View Slide

  21. CAP Example
    • Two replicas of an reservation system…

    Replicated for fault-tolerance to ensure system availability
    • Two concurrent requests…

    Tom and Chris attempt to reserve the last available seat on a plane
    • Two possible paths…

    If one replica of the system can not reach the other replica, we have two
    choices:
    • [Favoring Consistency] Prevent booking

    Return an error to the user and prevent both bookings
    9

    View Slide

  22. CAP Example
    • Two replicas of an reservation system…

    Replicated for fault-tolerance to ensure system availability
    • Two concurrent requests…

    Tom and Chris attempt to reserve the last available seat on a plane
    • Two possible paths…

    If one replica of the system can not reach the other replica, we have two
    choices:
    • [Favoring Consistency] Prevent booking

    Return an error to the user and prevent both bookings
    • [Favoring Availability] Allow concurrent requests

    However, now the seat is double booked and we must have a “conflict
    resolution” function for returning the system to a consistent state
    9

    View Slide

  23. Real World Analogies
    Eventual Consistency
    10

    View Slide

  24. Real World Analogies
    Eventual Consistency
    10
    Does the physical world
    favor availability over
    consistency?

    View Slide

  25. Recorded Knowledge
    • Approximation

    Approximation of globally known
    knowledge that is periodically recorded
    11

    View Slide

  26. Recorded Knowledge
    • Approximation

    Approximation of globally known
    knowledge that is periodically recorded
    • “Potentially outdated”

    Act of recording this information produces
    an artifact that is already outdated unless
    the system has quiesced
    11

    View Slide

  27. Message Passing
    • Exchange messages

    Members of the same system exchange
    messages asynchronously
    12

    View Slide

  28. Message Passing
    • Exchange messages

    Members of the same system exchange
    messages asynchronously
    • Dropped or delayed

    Messages can either be dropped or delayed
    12

    View Slide

  29. Message Passing
    • Exchange messages

    Members of the same system exchange
    messages asynchronously
    • Dropped or delayed

    Messages can either be dropped or delayed
    • Examples

    Letters via the postal service;

    Text messages;

    Telephone calls
    12

    View Slide

  30. Primary Site
    • Ownership of information

    Each member in the system owns the primary copy of their information
    13

    View Slide

  31. Primary Site
    • Ownership of information

    Each member in the system owns the primary copy of their information
    • Coordinates updates

    Members coordinate updates to information they are the primary site for
    13

    View Slide

  32. Primary Site
    • Ownership of information

    Each member in the system owns the primary copy of their information
    • Coordinates updates

    Members coordinate updates to information they are the primary site for
    • Information can be cached

    Information from other sites can be cached by other members in the system
    13

    View Slide

  33. Primary Site
    • Ownership of information

    Each member in the system owns the primary copy of their information
    • Coordinates updates

    Members coordinate updates to information they are the primary site for
    • Information can be cached

    Information from other sites can be cached by other members in the system
    • Local or incomplete replica

    Use memory
    13

    View Slide

  34. Primary Site
    • Ownership of information

    Each member in the system owns the primary copy of their information
    • Coordinates updates

    Members coordinate updates to information they are the primary site for
    • Information can be cached

    Information from other sites can be cached by other members in the system
    • Local or incomplete replica

    Use memory
    • Stale replica

    Outdated printed map
    13

    View Slide

  35. Primary Site
    • Ownership of information

    Each member in the system owns the primary copy of their information
    • Coordinates updates

    Members coordinate updates to information they are the primary site for
    • Information can be cached

    Information from other sites can be cached by other members in the system
    • Local or incomplete replica

    Use memory
    • Stale replica

    Outdated printed map
    • Primary site

    Google Maps or the USGS, etc.
    13

    View Slide

  36. What really are
    Databases?
    14

    View Slide

  37. Database: an Optimization
    • Graph of primary copy locations

    Represents all members in the system with the data they create
    and are responsible for
    15

    View Slide

  38. Database: an Optimization
    • Graph of primary copy locations

    Represents all members in the system with the data they create
    and are responsible for
    • Contract edges for subgraph

    Reduce several vertices in the graph to a single vertex: database
    for those entities
    15

    View Slide

  39. Database: an Optimization
    • Graph of primary copy locations

    Represents all members in the system with the data they create
    and are responsible for
    • Contract edges for subgraph

    Reduce several vertices in the graph to a single vertex: database
    for those entities
    • Geo-replicated, EC database

    Contracted edges per country, placing a replica in each country
    that served as the primary copy
    15

    View Slide

  40. Database: an Optimization
    • Graph of primary copy locations

    Represents all members in the system with the data they create
    and are responsible for
    • Contract edges for subgraph

    Reduce several vertices in the graph to a single vertex: database
    for those entities
    • Geo-replicated, EC database

    Contracted edges per country, placing a replica in each country
    that served as the primary copy
    • Wikipedia (for a given topic)

    Information about a given topic is stored here, written and
    coordinated by multiple authors
    15

    View Slide

  41. Why Optimize?
    • Expensive

    Retrieval from the primary site is expensive,
    if the primary site is geographically distant
    [latency] or unavailable [availability]
    16

    View Slide

  42. Why Optimize?
    • Expensive

    Retrieval from the primary site is expensive,
    if the primary site is geographically distant
    [latency] or unavailable [availability]
    • Replication introduces challenges

    Replication can make maintaining
    consistency much more challenges if
    caching/replication is pervasive
    16

    View Slide

  43. IoT and Mobile Applications
    • Centralized won’t scale

    Storing all data at a central location for
    processing won’t scale due to power and
    DC requirements
    17

    View Slide

  44. IoT and Mobile Applications
    • Centralized won’t scale

    Storing all data at a central location for
    processing won’t scale due to power and
    DC requirements
    • Today’s systems assume centralization

    Both programming models and
    applications used today assume
    centralization of data (ie. Spark, etc.)
    17

    View Slide

  45. 18
    We are moving towards
    large-scale edge computation!

    View Slide

  46. Edge Computation
    The Living Database
    19

    View Slide

  47. Database as a Constraint
    Satisfaction Problem
    • Where do we route requests?

    When we need to retrieve a certain piece of data, how do we know
    where to route the request to?
    20

    View Slide

  48. Database as a Constraint
    Satisfaction Problem
    • Where do we route requests?

    When we need to retrieve a certain piece of data, how do we know
    where to route the request to?
    • How we do specify acceptable staleness?

    Do we need to route to the primary site or can we use a cache? Does
    that cache provide a value within an acceptable value of staleness?
    20

    View Slide

  49. Database as a Constraint
    Satisfaction Problem
    • Where do we route requests?

    When we need to retrieve a certain piece of data, how do we know
    where to route the request to?
    • How we do specify acceptable staleness?

    Do we need to route to the primary site or can we use a cache? Does
    that cache provide a value within an acceptable value of staleness?
    • How do we bound latency?

    How do we select an appropriate cache? How do we choose between
    a cache and a primary site given we have to match a latency bound?
    20

    View Slide

  50. Database as a Constraint
    Satisfaction Problem
    • Where do we route requests?

    When we need to retrieve a certain piece of data, how do we know
    where to route the request to?
    • How we do specify acceptable staleness?

    Do we need to route to the primary site or can we use a cache? Does
    that cache provide a value within an acceptable value of staleness?
    • How do we bound latency?

    How do we select an appropriate cache? How do we choose between
    a cache and a primary site given we have to match a latency bound?
    • How do we reason about staleness?

    Across multiple requests for the same object, how do we know which
    version is newer or older?
    20

    View Slide

  51. Solution #1
    Mergeable Data Structures
    • Abstract data types for AP/EC systems

    Encapsulate AP replication concerns and exist in time and space
    21

    View Slide

  52. Solution #1
    Mergeable Data Structures
    • Abstract data types for AP/EC systems

    Encapsulate AP replication concerns and exist in time and space
    • Merge to most “recent” result

    Conflict resolution and provenance information
    21

    View Slide

  53. Solution #1
    Mergeable Data Structures
    • Abstract data types for AP/EC systems

    Encapsulate AP replication concerns and exist in time and space
    • Merge to most “recent” result

    Conflict resolution and provenance information
    • One example: CRDTs

    Conflict-free Replicated Data Types (Shapiro et al. 2011)
    21

    View Slide

  54. Solution #1
    Mergeable Data Structures
    • Abstract data types for AP/EC systems

    Encapsulate AP replication concerns and exist in time and space
    • Merge to most “recent” result

    Conflict resolution and provenance information
    • One example: CRDTs

    Conflict-free Replicated Data Types (Shapiro et al. 2011)
    • Causality

    Capture causality for object mutations and can identify concurrent
    operations
    21

    View Slide

  55. Solution #1
    Mergeable Data Structures
    • Abstract data types for AP/EC systems

    Encapsulate AP replication concerns and exist in time and space
    • Merge to most “recent” result

    Conflict resolution and provenance information
    • One example: CRDTs

    Conflict-free Replicated Data Types (Shapiro et al. 2011)
    • Causality

    Capture causality for object mutations and can identify concurrent
    operations
    • Concurrency

    Resolve concurrent operations using a bias (think: concurrent
    add(e) || remove(e) on the same set for same element)
    21

    View Slide

  56. Solution #2
    Programming Model
    • Based on mergeable data structures

    Mergeable data structures form the core data abstraction for a programming
    model
    22

    View Slide

  57. Solution #2
    Programming Model
    • Based on mergeable data structures

    Mergeable data structures form the core data abstraction for a programming
    model
    • Programming through composition of mergeable data structures

    Ensure the mergeability property holds through program transformations, data
    composition
    22

    View Slide

  58. Solution #2
    Programming Model
    • Based on mergeable data structures

    Mergeable data structures form the core data abstraction for a programming
    model
    • Programming through composition of mergeable data structures

    Ensure the mergeability property holds through program transformations, data
    composition
    • One example: Lasp

    Lattice Processing (Meiklejohn, Van Roy 2015)
    22

    View Slide

  59. Solution #2
    Programming Model
    • Based on mergeable data structures

    Mergeable data structures form the core data abstraction for a programming
    model
    • Programming through composition of mergeable data structures

    Ensure the mergeability property holds through program transformations, data
    composition
    • One example: Lasp

    Lattice Processing (Meiklejohn, Van Roy 2015)
    • Correct-by-construction

    Correct-by-construction distributed programs for infrastructure that
    provides weak guarantees
    22

    View Slide

  60. Solution #2
    Programming Model
    • Based on mergeable data structures

    Mergeable data structures form the core data abstraction for a programming
    model
    • Programming through composition of mergeable data structures

    Ensure the mergeability property holds through program transformations, data
    composition
    • One example: Lasp

    Lattice Processing (Meiklejohn, Van Roy 2015)
    • Correct-by-construction

    Correct-by-construction distributed programs for infrastructure that
    provides weak guarantees
    • Result provenance

    Extends CRDT causality/concurrency tracking through to results of
    applications providing mergeable outcomes
    22

    View Slide

  61. Solution #3
    Remove Role Dichotomy
    • Eliminate client-server dichotomy

    Servers shouldn’t be responsible for canonical data and data sharing,
    but rather serve as a location where particular code will run with clients
    data
    23

    View Slide

  62. Solution #3
    Remove Role Dichotomy
    • Eliminate client-server dichotomy

    Servers shouldn’t be responsible for canonical data and data sharing,
    but rather serve as a location where particular code will run with clients
    data
    • Clients communicate other clients

    Exchange state for latency reduction, serve as the primary site for their
    information
    23

    View Slide

  63. Solution #3
    Remove Role Dichotomy
    • Eliminate client-server dichotomy

    Servers shouldn’t be responsible for canonical data and data sharing,
    but rather serve as a location where particular code will run with clients
    data
    • Clients communicate other clients

    Exchange state for latency reduction, serve as the primary site for their
    information
    • Servers as business entities

    Necessary for latency reduction of large data sets, durability, location of
    “exactly-once” side-effects: ie. charge credit card
    23

    View Slide

  64. Solution #3
    Remove Role Dichotomy
    • Eliminate client-server dichotomy

    Servers shouldn’t be responsible for canonical data and data sharing,
    but rather serve as a location where particular code will run with clients
    data
    • Clients communicate other clients

    Exchange state for latency reduction, serve as the primary site for their
    information
    • Servers as business entities

    Necessary for latency reduction of large data sets, durability, location of
    “exactly-once” side-effects: ie. charge credit card
    • One example: Skype

    Completely peer-to-peer for operation, but a central server is used for
    authentication and storage of users “address book.”
    23

    View Slide

  65. Addendum
    Causality
    24

    View Slide

  66. What about Causality?
    • Misnomer “Eventually Consistent World”

    We know that causality drives interactions in
    the physical world: relativity, light cones, etc.
    25

    View Slide

  67. What about Causality?
    • Misnomer “Eventually Consistent World”

    We know that causality drives interactions in
    the physical world: relativity, light cones, etc.
    • Causality in distributed systems

    Happens-before relationship (Lamport
    1978) describes capturing causal
    relationships between entities in a
    distributed system
    25

    View Slide

  68. Causality Tradeoffs
    • Benefits

    Simplifies the development of systems

    Reason about cause/effect; eliminates storage
    and maintenance of redundant information
    26

    View Slide

  69. Causality Tradeoffs
    • Benefits

    Simplifies the development of systems

    Reason about cause/effect; eliminates storage
    and maintenance of redundant information
    • Negatives

    Expensive in storage and maintenance of
    causal message delivery channels; methods
    for reduction in state introduce false
    dependencies
    26

    View Slide

  70. Where’s the disconnect?
    • What about causal consistency?

    Does causal consistency provide a better formalism for
    describing consistency in the world given we know causal
    relationships hold?
    27

    View Slide

  71. Where’s the disconnect?
    • What about causal consistency?

    Does causal consistency provide a better formalism for
    describing consistency in the world given we know causal
    relationships hold?
    • Data has decay

    Some information may no longer be important after a given
    period of time, and forgotten
    27

    View Slide

  72. Where’s the disconnect?
    • What about causal consistency?

    Does causal consistency provide a better formalism for
    describing consistency in the world given we know causal
    relationships hold?
    • Data has decay

    Some information may no longer be important after a given
    period of time, and forgotten
    • Causality formalism needs explicit decay

    The formalism for describing causal consistency requires
    data be explicitly “decayed” through messages (or
    tombstones — keeping data forever to satisfy the causal
    relationship)
    27

    View Slide

  73. Causality Example
    By Analogy
    • Driver’s license example

    I learned a bunch of rules about driving a car and passed a
    driver’s test obtaining a license, which I renewed several years
    later
    28

    View Slide

  74. Causality Example
    By Analogy
    • Driver’s license example

    I learned a bunch of rules about driving a car and passed a
    driver’s test obtaining a license, which I renewed several years
    later
    • What does causality imply?

    Causality captures that I would not have had the license if I
    had not learned the rules and passed the test
    28

    View Slide

  75. Causality Example
    By Analogy
    • Driver’s license example

    I learned a bunch of rules about driving a car and passed a
    driver’s test obtaining a license, which I renewed several years
    later
    • What does causality imply?

    Causality captures that I would not have had the license if I
    had not learned the rules and passed the test
    • What does causal consistency imply?

    Does causal consistency imply that I can still recover the
    information that I originally used to pass the test? Does it imply
    that if I can recall my driver’s license number that I should be
    able to recall the rules that are implied by that license number?
    28

    View Slide

  76. Conclusion
    • Some adoption of ideas: “Uber goes Unconventional”

    Places canonical ride state on the device to bootstrap
    datacenters under failure: device is source of truth
    29

    View Slide

  77. Conclusion
    • Some adoption of ideas: “Uber goes Unconventional”

    Places canonical ride state on the device to bootstrap
    datacenters under failure: device is source of truth
    • Datacenter-focused designs are limiting

    Impractical from a storage, bandwidth, and power perspective
    29

    View Slide

  78. Conclusion
    • Some adoption of ideas: “Uber goes Unconventional”

    Places canonical ride state on the device to bootstrap
    datacenters under failure: device is source of truth
    • Datacenter-focused designs are limiting

    Impractical from a storage, bandwidth, and power perspective
    • Emerging countries have limited access

    Sneakernet, USB, Bluetooth still the pervasive model of
    communication
    29

    View Slide

  79. Conclusion
    • Some adoption of ideas: “Uber goes Unconventional”

    Places canonical ride state on the device to bootstrap
    datacenters under failure: device is source of truth
    • Datacenter-focused designs are limiting

    Impractical from a storage, bandwidth, and power perspective
    • Emerging countries have limited access

    Sneakernet, USB, Bluetooth still the pervasive model of
    communication
    • Peer-to-peer designs can provide higher-scale

    Grow to planetary scale, new programming models needed to
    embrace these network designs, new abstractions needed
    29

    View Slide

  80. 30
    Thanks!
    Christopher Meiklejohn

    @cmeik
    http://www.lasp-lang.org

    View Slide