Topics in Distributed Systems

The fallacies of distribute computing, Lamport Clocks, CRDTs

Transcript

1. Topics in Distributed Systems Arnon Rotem-Gal-Oz
2. What’s a “distributed system”? You know you have a distributed system when the crash of a computer you’ve never heard of stops you from getting any work done. —LESLIE LAMPORT
3. Your mission, should you choose to accept it: • Read data from one “place” • Write it to another “place”
4. mov eax, [ebx] mov [ecx],eax (try (let [[partitioner msg] (channel/pull chan)] (kp/send-message @producer (kp/message topic (.getBytes ^String partitioner) (.getBytes ^String msg))) (counter-fn)) (catch Exception ex …
5. System Event Actual Latency Scaled Latency One CPU cycle 0.4 ns 1 s Level 1 cache access 0.9 ns 2 s Level 2 cache access 2.8 ns 7 s Level 3 cache access 28 ns 1 min Main memory access (DDR DIMM) ~100 ns 4 min Intel® Optane™ DC persistent memory access ~350 ns 15 min Intel® Optane™ DC SSD I/O <10 μs 7 hrs NVMe SSD I/O ~25 μs 17 hrs SSD I/O 50–150 μs 1.5–4 days Rotational disk I/O 1–10 ms 1–9 months Internet call: San Francisco to New York City 65 ms 5 years Internet call: San Francisco to Hong Kong 141 ms 11 years Systems Performance: Enterprise and the Cloud, Brendan
6. System Event Actual Latency Scaled Latency One CPU cycle 0.4 ns 1 s Level 1 cache access 0.9 ns 2 s Level 2 cache access 2.8 ns 7 s Level 3 cache access 28 ns 1 min Main memory access (DDR DIMM) ~100 ns 4 min Intel® Optane™ DC persistent memory access ~350 ns 15 min Intel® Optane™ DC SSD I/O <10 μs 7 hrs NVMe SSD I/O ~25 μs 17 hrs SSD I/O 50–150 μs 1.5–4 days Rotational disk I/O 1–10 ms 1–9 months Internet call: San Francisco to New York City 65 ms 5 years Internet call: San Francisco to Hong Kong 141 ms 11 years Systems Performance: Enterprise and the Cloud, Brendan
7. System Event Actual Latency Scaled Latency One CPU cycle 0.4 ns 1 s Level 1 cache access 0.9 ns 2 s Level 2 cache access 2.8 ns 7 s Level 3 cache access 28 ns 1 min Main memory access (DDR DIMM) ~100 ns 4 min Intel® Optane™ DC persistent memory access ~350 ns 15 min Intel® Optane™ DC SSD I/O <10 μs 7 hrs NVMe SSD I/O ~25 μs 17 hrs SSD I/O 50–150 μs 1.5–4 days Rotational disk I/O 1–10 ms 1–9 months Internet call: San Francisco to New York City 65 ms 5 years Internet call: San Francisco to Hong Kong 141 ms 11 years Systems Performance: Enterprise and the Cloud, Brendan mov eax, [ebx] mov [ecx],eax (try (let [[partitioner msg] (cha (kp/send-message @pr message topic (.getBytes ^Str (.getBytes ^String msg)))
8. Request network
9. The network is reliable skb rides the rocket…
10. Latency is zero
11. System Event Actual Latency Scaled Latency One CPU cycle 0.4 ns 1 s Level 1 cache access 0.9 ns 2 s Level 2 cache access 2.8 ns 7 s Level 3 cache access 28 ns 1 min Main memory access (DDR DIMM) ~100 ns 4 min Intel® Optane™ DC persistent memory access ~350 ns 15 min Intel® Optane™ DC SSD I/O <10 μs 7 hrs NVMe SSD I/O ~25 μs 17 hrs SSD I/O 50–150 μs 1.5–4 days Rotational disk I/O 1–10 ms 1–9 months Internet call: San Francisco to New York City 65 ms 5 years Internet call: San Francisco to Hong Kong 141 ms 11 years Systems Performance: Enterprise and the Cloud, Brendan
12. Bandwidth is infinite
13. The network is secure
14. Topology doesn’t change
15. There is one administrator
16. Transport cost is zero
17. Network is homogeneous
18. Instances are free
19. Instances have 
 identities
20. Latency is zero
21. Latency is constant
23. What’s ”Happened Before”?
24. Concurrent Q A B X Y Firewall blocks all traffic: P can’t communicate to Q P
25. P Q A B P sends M Q receives M X Causal reation
26. Q LogicalClockQ A P sends M 0 1 2 3 0 1 2 Q receives M B X P LogicalClockP
27. Q computes: LogicalClockQ = max(0, 3) + 1 P LogicalClockP Q LogicalClockQ A P sends M 0 1 2 3 0 1 4 5 Q receives M B LogicalClockM = 3 X Y
28. Counter
29. Counter take 2
30. Decrements?
31. Sets ?
32. • Don’t take distributed actions lightly • Be careful when using abstractions that hide distributed calls • Big data means low- probability problems are daily occurances
33. Read more • Fallacies of distributed computing • Vector clocks • CRDTs - https://www.serverless.com/blog/crdt-explained- supercharge-serverless-at-edge • https://bartoszsypytkowski.com/the-state-of-a-state-based-crdts/ • Google Spanner https://static.googleusercontent.com/media/ research.google.com/en//archive/spanner-osdi2012.pdf • https://research.google/pubs/pub45855/