Life Beyond The Illusion Of Present

Slide 1

Slide 1 text

Life BEYOND the ILLUSION of PRESENT Jonas Bonér CTO Typesafe @jboner

Slide 2

Slide 2 text

"Time is what prevents everything from happening at once." - John Archibald Wheeler

Slide 3

Slide 3 text

Newton's PHYSICS

Slide 4

Slide 4 text

THIS SIMPLIFIED MODEL IS VERY APPEALING TO US

Slide 5

Slide 5 text

von Neumann ARCHITECTURE

Slide 6

Slide 6 text

BACK THEN LIFE WAS GOOD

Slide 7

Slide 7 text

THEN, ALONG CAME CONCURRENCY MADE LIFE MISERABLE

Slide 8

Slide 8 text

Jim Gray's TRANSACTIONS SAVE THE DAY

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

WELL, ALONG CAME DISTRIBUTION MADE LIFE MISERABLE, again...

Slide 11

Slide 11 text

But don't be surprised UNFORTUNATELY, THIS IS NOT HOW THE WORLD WORKS

Slide 12

Slide 12 text

"The future is a function of the past." - A. P. Robertson

Slide 13

Slide 13 text

"The (local) present is a merge function of multiple concurrent pasts." — Me

Slide 14

Slide 14 text

val newLocalPresent = observedPasts. foldLeft(oldLocalPresent) { _ merge _ }

Slide 15

Slide 15 text

WE NEED TO EXPLICITLY MODEL THE LOCAL PRESENT AS FACTS DERIVED FROM THE MERGING OF MULTIPLE CONCURRENT PASTS

Slide 16

Slide 16 text

INFORMATION IS ALWAYS FROM THE PAST

Slide 17

Slide 17 text

THE TRUTH IS CLOSER TO EINSTEIN'S PHYSICS

Slide 18

Slide 18 text

INFORMATION HAS LATENCY

Slide 19

Slide 19 text

THE COST OF MAINTAINING THIS ILLUSION IS INCREASED CONTENTION & COHERENCY

Slide 20

Slide 20 text

AS LATENCY GETS HIGHER, THE ILLUSION CRACKS EVEN MORE

Slide 21

Slide 21 text

DISTRIBUTED SYSTEMS EVERYWHERE

Slide 22

Slide 22 text

THE NETWORK IS RELIABLE...NAT

Slide 23

Slide 23 text

"If a tree falls in a forest and no one is around to hear it, does it make a sound?" — Charles Riborg Mann

Slide 24

Slide 24 text

Information CAN (and will) GET LOST

Slide 25

Slide 25 text

HOW DO WE DEAL WITH INFORMATION LOSS IN REAL LIFE?

Slide 26

Slide 26 text

WE USE A SIMPLE PROTOCOL OF Conﬁrm, Wait & Repeat

Slide 27

Slide 27 text

We ﬁll in THE BLANKS

Slide 28

Slide 28 text

...AND IF WE ARE WRONG, WE TAKE COMPENSATING ACTION APOLOGY-ORIENTED PROGRAMMING - PAT HELLAND (IN MEMORIES, GUESSES, AND APOLOGIES)

Slide 29

Slide 29 text

The bottom line: WE CAN'T FORCE THE WORLD INTO A SINGLE GLOBALLY CONSISTENT PRESENT

Slide 30

Slide 30 text

Should we just GIVE UP?

Slide 31

Slide 31 text

I BELIEVE THAT THERE IS A PATH FORWARD

Slide 32

Slide 32 text

WE NEED TO TREAT TIME AS A FIRST CLASS CONSTRUCT

Slide 33

Slide 33 text

WHAT IS TIME, really?

Slide 34

Slide 34 text

TIME IS THE SUCCESSION OF CAUSALLY RELATED EVENTS

Slide 35

Slide 35 text

How can we MANAGE TIME?

Slide 36

Slide 36 text

Think in FACTS

Slide 37

Slide 37 text

What is a FACT?

Slide 38

Slide 38 text

"A Fact is something that truly exists or happens: something that has actual existence, a true piece of information." — Merriam Webster

Slide 39

Slide 39 text

IMMUTABILITY is a requirement

Slide 40

Slide 40 text

So, do variables HAVE A PURPOSE IN LIFE?

Slide 41

Slide 41 text

"The assignment statement is the von Neumann bottleneck of programming languages and keeps us thinking in word-at-a-time terms in much the same way the computer's bottleneck does." — John Backus (Turing Award lecture 1977)

Slide 42

Slide 42 text

MUTABLE STATE NEEDS TO BE CONTAINED

Slide 43

Slide 43 text

Ok, but how should we MANAGE FACTS?

Slide 44

Slide 44 text

Functional PROGRAMMING

Slide 45

Slide 45 text

Logic PROGRAMMING

Slide 46

Slide 46 text

Dataﬂow PROGRAMMING

Slide 47

Slide 47 text

NEVER DELETE FACTS

Slide 48

Slide 48 text

"When bookkeeping was done with clay tablets or paper and ink, accountants developed some clear rules about good accounting practices. One never alters the books; if an error is made, it is annotated and a new compensating entry is made in the books. The books are thus a complete history of the transactions of the business. Update-in-place strikes many systems designers as a cardinal sin: it violates traditional accounting practices that have been observed for hundreds of years." — Jim Gray (1981)

Slide 49

Slide 49 text

CRUD

Slide 50

Slide 50 text

"Database is a cache of a subset of the log." — Pat Helland (2007)

Slide 51

Slide 51 text

Store facts in an EVENT LOG

Slide 52

Slide 52 text

The log allows TIME TRAVEL

Slide 53

Slide 53 text

Can we REWRITE THE PAST?

Slide 54

Slide 54 text

Allows us to shift our focus from DATA AT REST, to DATA IN MOTION

Slide 55

Slide 55 text

Stream Processing

Slide 56

Slide 56 text

CONSTRUCTING A SUFFICIENTLY CONSISTENT LOCAL PRESENT MEANS EMPLOYING CONSISTENCY MECHANISMS

Slide 57

Slide 57 text

Consistency WHAT? WHY? WHEN?

Slide 58

Slide 58 text

WE NEED TO DECOMPOSE THE SYSTEM USING CONSISTENCY BOUNDARIES

Slide 59

Slide 59 text

INSIDE DATA: OUR CURRENT PRESENT OUTSIDE DATA: BLAST FROM THE PAST BETWEEN SERVICES: HOPE FOR THE FUTURE — PAT HELLAND (DATA ON THE INSIDE VS DATA ON THE OUTSIDE)

Slide 60

Slide 60 text

MicroSERVICE

Slide 61

Slide 61 text

AGGREGATE Root

Slide 62

Slide 62 text

WITHIN THE CONSISTENCY BOUNDARY

Slide 63

Slide 63 text

EVENT Sourcing

Slide 64

Slide 64 text

BETWEEN THE Consistency Boundaries IT'S A ZOO

Slide 65

Slide 65 text

Decoupling in TIME / SPACE

Slide 66

Slide 66 text

STRONG CONSISTENCY The wrong default

Slide 67

Slide 67 text

Here, we are living in the LOOMING SHADOW OF IMPOSSIBILITY THEOREMS

Slide 68

Slide 68 text

FLP CONSENSUS IS IMPOSSIBLE

Slide 69

Slide 69 text

PROTOCOLS CLIMB THE LADDER OF KNOWLEDGE Cɸ: Common Knowledge (infinite number of i) Eiɸ: (Everyone knows * i) ɸ E3ɸ: (Everyone knows * 3) ɸ E2ɸ: Everyone knows Everyone knows ɸ E1ɸ: Everyone knows ɸ Sɸ: Someone knows ɸ

Slide 70

Slide 70 text

COMMON KNOWLEDGE IS NOT ATTAINABLE VIA PROTOCOL - JOSEPH HALPERN

Slide 71

Slide 71 text

CAP CONSISTENCY IS IMPOSSIBLE

Slide 72

Slide 72 text

Dissecting CAP

Slide 73

Slide 73 text

"The first principle of successful scalability is to batter the consistency mechanisms down to a minimum." – James Hamilton