Slide 1

Slide 1 text

Accidentally Distributed

Slide 2

Slide 2 text

Dramatis Personae Serge Danzanvilliers (beard) @alolcool Yann Schwartz (no beard) @abolibibelot

Slide 3

Slide 3 text

2009 We got some data

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

2010 We got mucho data

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable. Leslie Lamport

Slide 8

Slide 8 text

2012 Wait what’s this data anyway?

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

2013 Invented Elsewhere

Slide 13

Slide 13 text

Kafka! ● Clustered ● Failover ● Replication ● Non texty ● Pull model ● Streaming

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

(Interlude) The Case of the Missing Minutes

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

Reality has a known theoretical bias

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

Buffers, buffers everywhere ● Caches, buffers, batches, queues : everything comes in chunks ● Unbounded queues : never-ending tears ● Operation triage: lose some data or lose your system

Slide 22

Slide 22 text

TL;DR ● Ignorance is not bliss ● System entropy always increases ● Worse is better until it’s not