Concurrent and Resilient Connections to Outside the BEAM

Transcript

1. outside CONCURRENT RESILIENT and CONNECTIONS to the BEAM
2. ERLANG VM mnesia message passing ets
3. ERLANG VM key-value store relational db message queue
4. ERLANG VM key-value store relational db message queue
5. the outside world is SCARY
6. ANDREA LEOPARDI @WHATYOUHIDE
7. Gothenburg, Sweden
8. FOOTBALL ADDICTS
9. GENSERVER TCP
10. SILLY the way
11. defmodule Redis do def command(cmd) do end end
12. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...) :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end
13. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...) :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end
14. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...) :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end
15. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...) :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end
16. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...) :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end
17. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...) :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end
18. opening NEW connections is EXPENSIVE
19. GENSERVER keeping the socket in a
20. TWO WAYS blocking non-blocking
21. TWO WAYS blocking
22. CLIENT GENSERVER socket socket send() recv()
23. defmodule Redis do def command(conn, cmd) do sock = checkout(conn) # send and recv checkin(conn, sock) end end
24. defmodule Redis do def command(conn, cmd) do sock = checkout(conn) # send and recv checkin(conn, sock) end end
25. {:error, :checked_out} :queue.in() VS
26. POOLING :|
27. TWO WAYS non-blocking
28. active: true {:noreply, _} + GenServer.reply/2 +
29. client Redis client client GenServer
30. tcp is FULL DUPLEX
31. client Redis client client Sender Receiver
32. TWO WAYS blocking non-blocking copies less data + needs pooling - doesn't use full duplex - enc/dec on client + - copies more data + doesn't need pooling + uses full duplex - enc/dec in server
33. RESIL IENCY
34. Redis GenServer
35. Redis GenServer
36. {:tcp_closed, reason} backoff + reconnect
37. hex.pm/packages/connection gen_
38. Connection GenServer
39. init/1 connect/2 disconnect/2 handle_call/3 handle_cast/2 handle_info/2 terminate/2 code_change/3 {:disconnect, reason, state} {:backoff, timeout, state}
40. sync/async CONNECT
41. SYNC start_link {:ok, pid} init() connect
42. ASYNC start_link {:ok, pid} init() connect connect()
43. You now allow initializations with fewer guarantees: they went from "the connection is available" to "the connection manager is available". Fred Hebert
44. CONNECTION TALKING CONCURRENCY RESILIENCY
45. @whatyouhide