Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Life Beyond The Illusion Of Present

Life Beyond The Illusion Of Present

The idea of the present is an illusion. Everything we see, hear and feel is just an echo from the past. But this illusion has influenced us and the way we view the world in so many ways; from Newton’s physics with a linearly progressing timeline accruing absolute knowledge along the way to the von Neumann machine with its total ordering of instructions updating mutable state with full control of the “present”. But unfortunately this is not how the world works. There is no present, all we have is facts derived from the merging of multiple pasts. The truth is closer to Einstein’s physics where everything is relative to one’s perspective.

As developers we need to wake up and break free from the perceived reality of living in a single globally consistent present. The advent of multicore and cloud computing architectures meant that most applications today are distributed systems—multiple cores separated by the memory bus or multiple nodes separated by the network—which puts a harsh end to this illusion. Facts travel at the speed of light (at best), which makes the distinction between past and perceived present even more apparent in a distributed system where latency is higher and where facts (messages) can get lost. 

E0b5787d1a1935a2800e0bbffc81c196?s=128

Jonas Bonér

June 10, 2015
Tweet

Transcript

  1. Life BEYOND the ILLUSION of PRESENT Jonas Bonér CTO Typesafe

    @jboner
  2. "Time is what prevents everything from happening at once." -

    John Archibald Wheeler
  3. Newton's PHYSICS

  4. THIS SIMPLIFIED MODEL IS VERY APPEALING TO US

  5. von Neumann ARCHITECTURE

  6. BACK THEN LIFE WAS GOOD

  7. THEN, ALONG CAME CONCURRENCY MADE LIFE MISERABLE

  8. Jim Gray's TRANSACTIONS SAVE THE DAY

  9. None
  10. WELL, ALONG CAME DISTRIBUTION MADE LIFE MISERABLE, again...

  11. But don't be surprised UNFORTUNATELY, THIS IS NOT HOW THE

    WORLD WORKS
  12. "The future is a function of the past." - A.

    P. Robertson
  13. "The (local) present is a merge function of multiple concurrent

    pasts." — Me
  14. val newLocalPresent = observedPasts. foldLeft(oldLocalPresent) { _ merge _ }

  15. WE NEED TO EXPLICITLY MODEL THE LOCAL PRESENT AS FACTS

    DERIVED FROM THE MERGING OF MULTIPLE CONCURRENT PASTS
  16. INFORMATION IS ALWAYS FROM THE PAST

  17. THE TRUTH IS CLOSER TO EINSTEIN'S PHYSICS

  18. INFORMATION HAS LATENCY

  19. THE COST OF MAINTAINING THIS ILLUSION IS INCREASED CONTENTION &

    COHERENCY
  20. AS LATENCY GETS HIGHER, THE ILLUSION CRACKS EVEN MORE

  21. DISTRIBUTED SYSTEMS EVERYWHERE

  22. THE NETWORK IS RELIABLE...NAT

  23. "If a tree falls in a forest and no one

    is around to hear it, does it make a sound?" — Charles Riborg Mann
  24. Information CAN (and will) GET LOST

  25. HOW DO WE DEAL WITH INFORMATION LOSS IN REAL LIFE?

  26. WE USE A SIMPLE PROTOCOL OF Confirm, Wait & Repeat

  27. We fill in THE BLANKS

  28. ...AND IF WE ARE WRONG, WE TAKE COMPENSATING ACTION APOLOGY-ORIENTED

    PROGRAMMING - PAT HELLAND (IN MEMORIES, GUESSES, AND APOLOGIES)
  29. The bottom line: WE CAN'T FORCE THE WORLD INTO A

    SINGLE GLOBALLY CONSISTENT PRESENT
  30. Should we just GIVE UP?

  31. I BELIEVE THAT THERE IS A PATH FORWARD

  32. WE NEED TO TREAT TIME AS A FIRST CLASS CONSTRUCT

  33. WHAT IS TIME, really?

  34. TIME IS THE SUCCESSION OF CAUSALLY RELATED EVENTS

  35. How can we MANAGE TIME?

  36. Think in FACTS

  37. What is a FACT?

  38. "A Fact is something that truly exists or happens: something

    that has actual existence, a true piece of information." — Merriam Webster
  39. IMMUTABILITY is a requirement

  40. So, do variables HAVE A PURPOSE IN LIFE?

  41. "The assignment statement is the von Neumann bottleneck of programming

    languages and keeps us thinking in word-at-a-time terms in much the same way the computer's bottleneck does." — John Backus (Turing Award lecture 1977)
  42. MUTABLE STATE NEEDS TO BE CONTAINED

  43. Ok, but how should we MANAGE FACTS?

  44. Functional PROGRAMMING

  45. Logic PROGRAMMING

  46. Dataflow PROGRAMMING

  47. NEVER DELETE FACTS

  48. "When bookkeeping was done with clay tablets or paper and

    ink, accountants developed some clear rules about good accounting practices. One never alters the books; if an error is made, it is annotated and a new compensating entry is made in the books. The books are thus a complete history of the transactions of the business. Update-in-place strikes many systems designers as a cardinal sin: it violates traditional accounting practices that have been observed for hundreds of years." — Jim Gray (1981)
  49. CRUD

  50. "Database is a cache of a subset of the log."

    — Pat Helland (2007)
  51. Store facts in an EVENT LOG

  52. The log allows TIME TRAVEL

  53. Can we REWRITE THE PAST?

  54. Allows us to shift our focus from DATA AT REST,

    to DATA IN MOTION
  55. Stream Processing

  56. CONSTRUCTING A SUFFICIENTLY CONSISTENT LOCAL PRESENT MEANS EMPLOYING CONSISTENCY MECHANISMS

  57. Consistency WHAT? WHY? WHEN?

  58. WE NEED TO DECOMPOSE THE SYSTEM USING CONSISTENCY BOUNDARIES

  59. INSIDE DATA: OUR CURRENT PRESENT OUTSIDE DATA: BLAST FROM THE

    PAST BETWEEN SERVICES: HOPE FOR THE FUTURE — PAT HELLAND (DATA ON THE INSIDE VS DATA ON THE OUTSIDE)
  60. MicroSERVICE

  61. AGGREGATE Root

  62. WITHIN THE CONSISTENCY BOUNDARY

  63. EVENT Sourcing

  64. BETWEEN THE Consistency Boundaries IT'S A ZOO

  65. Decoupling in TIME / SPACE

  66. STRONG CONSISTENCY The wrong default

  67. Here, we are living in the LOOMING SHADOW OF IMPOSSIBILITY

    THEOREMS
  68. FLP CONSENSUS IS IMPOSSIBLE

  69. PROTOCOLS CLIMB THE LADDER OF KNOWLEDGE Cɸ: Common Knowledge (infinite

    number of i) Eiɸ: (Everyone knows * i) ɸ E3ɸ: (Everyone knows * 3) ɸ E2ɸ: Everyone knows Everyone knows ɸ E1ɸ: Everyone knows ɸ Sɸ: Someone knows ɸ
  70. COMMON KNOWLEDGE IS NOT ATTAINABLE VIA PROTOCOL - JOSEPH HALPERN

  71. CAP CONSISTENCY IS IMPOSSIBLE

  72. Dissecting CAP

  73. "The first principle of successful scalability is to batter the

    consistency mechanisms down to a minimum." – James Hamilton
  74. EVENTUAL CONSISTENCY What does it really mean?

  75. Tracking TIME is tracking CAUSALITY

  76. RELYING ON TIMESTAMPS IS A BAD IDEA

  77. Instead, rely on LOGICAL TIME

  78. Lamport CLOCKS GLOBAL CAUSAL ORDERING BETWEEN

  79. Vector CLOCKS PARTIAL CAUSAL ORDERING BETWEEN EVENTS

  80. Causal CONSISTENCY

  81. What CONSISTENCY DO YOU REALLY NEED AND when?

  82. ACID 2.0 ASSOCIATIVE COMMUTATIVE IDEMPOTENT DISTRIBUTED

  83. CONFLICT-FREE REPLICATED DATA TYPES

  84. DISORDERLY PROGRAMMING CALM THEOREM

  85. WE ARE JUST GETTING STARTED

  86. WE HAVE A LONG ROAD AHEAD OF US...

  87. None
  88. Thanks FOR LISTENING

  89. Life BEYOND the ILLUSION of PRESENT Jonas Bonér CTO Typesafe

    @jboner