Bloom: A Language For Disorderly Distributed Programming

3e09fee7b359be847ed5fa48f524a3d3?s=47 Christopher Meiklejohn
October 13, 2013
Programming
22
9.1k

Bloom: A Language For Disorderly Distributed Programming

Wicked Good Ruby '13.

3e09fee7b359be847ed5fa48f524a3d3?s=128

Christopher Meiklejohn

October 13, 2013
Tweet

More Decks by Christopher Meiklejohn

See All by Christopher Meiklejohn
3e09fee7b359be847ed5fa48f524a3d3?s=48 cmeiklejohn
0
270
3e09fee7b359be847ed5fa48f524a3d3?s=48 cmeiklejohn
0
230
3e09fee7b359be847ed5fa48f524a3d3?s=48 cmeiklejohn
3
230
3e09fee7b359be847ed5fa48f524a3d3?s=48 cmeiklejohn
0
330
3e09fee7b359be847ed5fa48f524a3d3?s=48 cmeiklejohn
4
2.1k
3e09fee7b359be847ed5fa48f524a3d3?s=48 cmeiklejohn
3
210
3e09fee7b359be847ed5fa48f524a3d3?s=48 cmeiklejohn
1
150
3e09fee7b359be847ed5fa48f524a3d3?s=48 cmeiklejohn
3
880
3e09fee7b359be847ed5fa48f524a3d3?s=48 cmeiklejohn
0
380

Other Decks in Programming

See All in Programming
88493e31e8ac7540094d35f7cda46c8a?s=48 lnit
8
3.2k
Fb1b9f3d7332a7a7e262b70013b5f7dd?s=48 mtsmfm
0
120
9cbac3a8b2ea3f8f9c79d01aeb050ad3?s=48 shikugawa
1
670
38a509643276213e62730e92ef220405?s=48 masatakamiki
0
200
29bfc9e2bc82b775a30af70be3196386?s=48 hannaprinz
0
110
Fa54d97af93719f9aa82873de3acb3b3?s=48 decoc
1
460
Fc358d63dcdcbad7a5c29198785dec2c?s=48 naototty
1
270
A6a5e01b331d18dbd9c5de1f2cc2879f?s=48 erukiti
2
340
B7caf463b7dbc6c305ba290ec3c16521?s=48 eroispaziali
0
300
0ca0c0ec6efe3a7f5220332a910e6da0?s=48 shinyoshiaki
1
520
C302e84057a922dce0ecbe80207e3fcc?s=48 mu_zaru
0
290
3e049ed33195d00a8b745b16c63dce6e?s=48 leew
0
130

Featured

See All Featured
24fc194843a71f10949be18d5a692682?s=48 gr2m
82
11k
A60068bce2e73de3a37ca9d2dbe36092?s=48 bryan
28
3.1k
95d9f37115fbb0d13135d0f77132f856?s=48 morganepeng
86
11k
56c4053438af8e8b90d6f53cbb7573be?s=48 jakevdp
766
190k
Eb8975af8e49e19e3dd6b6b84a542e26?s=48 qrush
282
18k
245cee81a9c424266e5e401d844ea881?s=48 lara
168
9.1k
719435d98d452de7ac367c828266cf01?s=48 erikaheidi
5
3.5k
016edace78ffe826f2348d1e0c504afd?s=48 maggiecrowley
2
180
20bfe76b3d6105641f879fe45cfc9272?s=48 bkeepers
50
3.9k
96270e4c3e5e9806cf7245475c00b275?s=48 addyosmani
494
110k
96270e4c3e5e9806cf7245475c00b275?s=48 addyosmani
308
21k
6bb82b13cda86d3b821fa44100335ddf?s=48 thoeni
2
300

Transcript

  1. Bloom: A Language For Disorderly Distributed Programming Christopher Meiklejohn Wicked

    Good Ruby ’13 Sunday, October 13, 13

  2. cmeiklejohn / @cmeik My Name is My Name Sunday, October

    13, 13

  3. What do you do? Sunday, October 13, 13

  4. Sunday, October 13, 13

  5. Sunday, October 13, 13

  6. Sunday, October 13, 13

  7. What did you do? Sunday, October 13, 13

  8. Sunday, October 13, 13

  9. Sunday, October 13, 13

  10. UC Berkeley Sunday, October 13, 13

  11. BOOM Sunday, October 13, 13

  12. The Outline Sunday, October 13, 13

  13. Theory, language. The Outline Sunday, October 13, 13

  14. von Neumann Sunday, October 13, 13

  15. Instructions, mutable state, executed in order. von Neumann Sunday, October

    13, 13

  16. Imposes a total order. von Neumann Sunday, October 13, 13

  17. Disorderly actors Sunday, October 13, 13

  18. Allow disorder and races; illusion of total order. Disorderly actors

    Sunday, October 13, 13

  19. The idea behind transactions; serializability. Disorderly actors Sunday, October 13,

    13

  20. Tricky to scale. Disorderly actors Sunday, October 13, 13

  21. Scaling Sunday, October 13, 13

  22. Multiple machines; sharing computation. Scaling Sunday, October 13, 13

  23. CAP Theorem Sunday, October 13, 13

  24. Consistency, availability, partition tolerance. CAP Theorem Sunday, October 13, 13

  25. Negative result: linearizability impossible. CAP Theorem Sunday, October 13, 13

  26. Whoa, Theory! Sunday, October 13, 13

  27. Distributed computing Sunday, October 13, 13

  28. The “new normal”. Distributed computing Sunday, October 13, 13

  29. Order is di!cult to impose. Distributed computing Sunday, October 13,

    13

  30. Synchronization; coordination. Distributed computing Sunday, October 13, 13

  31. FLP result; delays, partitions, partial writes. Distributed computing Sunday, October

    13, 13

  32. Inverse methods; commutative operations. Distributed computing Sunday, October 13, 13

  33. ACID 2.0 Sunday, October 13, 13

  34. Associativity, commutativity, idempotence. ACID 2.0 Sunday, October 13, 13

  35. (X · Y) · Z = X · (Y ·

    Z) ACID 2.0: Associativity Sunday, October 13, 13

  36. X · Y = Y · X ACID 2.0: Commutativity

    Sunday, October 13, 13

  37. X · X = X ACID 2.0: Idempotence Sunday, October

    13, 13

  38. Associativity, commutativity, idempotence. ACID 2.0 Sunday, October 13, 13

  39. Familiar? Sunday, October 13, 13

  40. Bounded Join Semilattices Sunday, October 13, 13

  41. Partially ordered set; least upper bound; ACI. Bounded Join Semilattices

    Sunday, October 13, 13

  42. Objects grow over time; merge computes LUB. Bounded Join Semilattices

    Sunday, October 13, 13

  43. Monotonic and con"uent; convergent. Bounded Join Semilattices Sunday, October 13,

    13

  44. Map into another via monotone functions. Bounded Join Semilattices Sunday,

    October 13, 13

  45. Examples. Bounded Join Semilattices Sunday, October 13, 13

  46. b a c a, b a, c a, b, c

    Set; merge function: union. b, c Sunday, October 13, 13

  47. 3 5 7 5 7 7 Increaing natual; merge function:

    max. Sunday, October 13, 13

  48. F F T F T T Booleans; merge function: or.

    Sunday, October 13, 13

  49. x <= y montone f(x) <= f(y) Sunday, October 13,

    13

  50. Sunday, October 13, 13

  51. CvRDT examples. Sunday, October 13, 13

  52. [ [{1, a}], [] ] [ [{1, a}], [] ]

    Sunday, October 13, 13

  53. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}], [{1, a}] ] [ [{1, a}], [{1, a}] ] Sunday, October 13, 13

  54. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}], [{1, a}] ] [ [{1, a}], [{1, a}] ] [ [{1, a}, {2, a}], [{1, a}] ] Sunday, October 13, 13

  55. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}], [{1, a}] ] [ [{1, a}], [{1, a}] ] [ [{1, a}, {2, a}], [{1, a}] ] [ [{1, a}, {2, a}], [{1, a}] ] Sunday, October 13, 13

  56. [ [{1, a}], [] ] [ [{1, a}], [] ]

    Sunday, October 13, 13

  57. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}, {2, b}], [] ] Sunday, October 13, 13

  58. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}, {2, b}], [] ] [ [{1, a}], [{1, a}] ] Sunday, October 13, 13

  59. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}, {2, b}], [] ] [ [{1, a}], [{1, a}] ] [ [{1, a}, {2, b}], [{1, a}] ] Sunday, October 13, 13

  60. Just data; what about computation? Bounded Join Semilattices Sunday, October

    13, 13

  61. Composability desired. Bounded Join Semilattices Sunday, October 13, 13

  62. CALM Theorem Sunday, October 13, 13

  63. Consistency as Logical Monotonicity CALM Theorem Sunday, October 13, 13

  64. Consistent given async; all monotone functions. CALM Theorem Sunday, October

    13, 13

  65. Identify non-monotone functions. CALM Theorem Sunday, October 13, 13

  66. Disorderly Programming Sunday, October 13, 13

  67. Order-independent, explicit order-dependent. Disorderly Programming Sunday, October 13, 13

  68. Prototype language: Bloom Disorderly Programming Sunday, October 13, 13

  69. Bloom Sunday, October 13, 13

  70. Based in a logic called Dedalus. Bloom Sunday, October 13,

    13

  71. Disorderly language. Bloom Sunday, October 13, 13

  72. Lattices and mappings between lattices. Bloom Sunday, October 13, 13

  73. Highlights non-monotonicity. Bloom Sunday, October 13, 13

  74. Operational model. Bloom Sunday, October 13, 13

  75. Local clock; local execution; local state. Bloom Sunday, October 13,

    13

  76. Timestepped execution. Bloom Sunday, October 13, 13

  77. Local updates; system events; network. Sunday, October 13, 13

  78. Bloom rules; merge functions. Sunday, October 13, 13

  79. Network; next. Sunday, October 13, 13

  80. Chat server. Bloom Sunday, October 13, 13

  81. module ChatProtocol state do channel :connect, [:@addr, :client] => [:nick]

    channel :mcast end end examples/chat/chat_protocol.rb Sunday, October 13, 13

  82. class ChatServer include Bud include ChatProtocol state { table :nodelist

    } bloom do nodelist <= connect.map { |c| [c.client, c.nick] } mcast <~ (mcast * nodelist).pairs { |m,n| [n.key, m.val] } end end examples/chat/chat_server.rb Sunday, October 13, 13

  83. Lattices; monotonic functions. Bloom Sunday, October 13, 13

  84. class ChatServer include Bud include ChatProtocol state do table :nodelist

    channel :disconnect end bloom do nodelist <= connect.map { |c| [c.client, c.nick] } mcast <~ (mcast * nodelist).pairs { |m,n| [n.key, m.val] } nodelist <- disconnect.map { |c| [c.client, c.nick] } end end examples/chat/chat_server.rb Sunday, October 13, 13

  85. Non-monotonic; not eventually consistent. Bloom Sunday, October 13, 13

  86. bloom do mcast <~ stdio do |s| [@server, [ip_port, @nick,

    Time.new.strftime("%I:%M.%S"), s.line]] end stdio <~ mcast { |m| [pretty_print(m.val)] } end examples/chat/chat.rb Sunday, October 13, 13

  87. KV Store. Bloom Sunday, October 13, 13

  88. module BasicKVS include KVSProtocol state do table :kvstate, [:key] =>

    [:value] end bloom :mutate do kvstate <+ kvput {|s| [s.key, s.value]} kvstate <- (kvstate * kvput).lefts(:key => :key) end bloom :get do temp :getj <= (kvget * kvstate).pairs(:key => :key) kvget_response <= getj do |g, t| [g.reqid, t.key, t.value] end end bloom :delete do kvstate <- (kvstate * kvdel).lefts(:key => :key) end end kvs/kvs.rb Sunday, October 13, 13

  89. module ReplicatedKVS include KVSProtocol include MulticastProtocol include MembershipProtocol import BasicKVS

    => :kvs bloom :local_indir do kvs.kvdel <= kvdel kvs.kvget <= kvget kvget_response <= kvs.kvget_response end bloom :puts do # if I am the master, multicast store requests mcast_send <= kvput do |k| unless member.include? [k.client] [k.reqid, ["put", [@addy, k.key, k.reqid, k.value]]] end end kvs.kvput <= mcast_done do |m| if m.payload[0] == "put" m.payload[1] end end # if I am a replica, store the payload of the multicast kvs.kvput <= pipe_out do |d| if d.payload.fetch(1) != @addy and d.payload[0] == "put" d.payload[1] end end end bloom :dels do mcast_send <= kvdel do |k| unless member.include? [k.client] [k.reqid, ["del", [@addy, k.key, k.reqid]]] end end kvs.kvdel <= mcast_done do |m| if m.payload[0] == "del" m.payload[1] end end kvs.kvdel <= pipe_out do |d| if d.payload.fetch(1) != @addy and d.payload[0] == "del" d.payload[1] end end end end kvs/kvs.rb Sunday, October 13, 13

  90. CALM Analysis Sunday, October 13, 13

  91. Label asynchronous, non-monotonic. CALM Analysis Sunday, October 13, 13

  92. Identify where coordination is needed. CALM Analysis Sunday, October 13,

    13

  93. The Conclusion Sunday, October 13, 13

  94. Prototype language. The Conclusion Sunday, October 13, 13

  95. https://github.com/bloom-lang/bud https://github.com/bloom-lang/bud-sandbox Thanks! Questions? Sunday, October 13, 13