Concurrent and Resilient Connections to Outside the BEAM

Transcript

1. outside CONCURRENT RESILIENT and CONNECTIONS to the BEAM

2. ERLANG VM mnesia message passing ets

3. ERLANG VM key-value store relational db message queue

4. ERLANG VM key-value store relational db message queue

5. the outside world is SCARY

6. ANDREA LEOPARDI @WHATYOUHIDE

7. Gothenburg, Sweden

8. FOOTBALL ADDICTS

9. GENSERVER TCP

10. SILLY the way

11. defmodule Redis do def command(cmd) do end end

12. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...)

    :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end

13. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...)

    :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end

14. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...)

    :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end

15. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...)

    :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end

16. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...)

    :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end

17. defmodule Redis do def command(cmd) do {:ok, sock} = :gen_tcp.connect(...)

    :gen_tcp.send(sock, encode(cmd)) {:ok, data} = :gen_tcp.recv(sock, 0) :gen_tcp.close(sock) decode(data) end end

18. opening NEW connections is EXPENSIVE

19. GENSERVER keeping the socket in a

20. TWO WAYS blocking non-blocking

21. TWO WAYS blocking

22. CLIENT GENSERVER socket socket send() recv()

23. defmodule Redis do def command(conn, cmd) do sock = checkout(conn)

    # send and recv checkin(conn, sock) end end

24. defmodule Redis do def command(conn, cmd) do sock = checkout(conn)

    # send and recv checkin(conn, sock) end end

25. {:error, :checked_out} :queue.in() VS

26. POOLING :|

27. TWO WAYS non-blocking

28. active: true {:noreply, _} + GenServer.reply/2 +

29. client Redis client client GenServer

30. tcp is FULL DUPLEX

31. client Redis client client Sender Receiver

32. TWO WAYS blocking non-blocking copies less data + needs pooling

    - doesn't use full duplex - enc/dec on client + - copies more data + doesn't need pooling + uses full duplex - enc/dec in server

33. RESIL IENCY

34. Redis GenServer

35. Redis GenServer

36. {:tcp_closed, reason} backoff + reconnect

37. hex.pm/packages/connection gen_

38. Connection GenServer

39. init/1 connect/2 disconnect/2 handle_call/3 handle_cast/2 handle_info/2 terminate/2 code_change/3 {:disconnect, reason,

    state} {:backoff, timeout, state}

40. sync/async CONNECT

41. SYNC start_link {:ok, pid} init() connect

42. ASYNC start_link {:ok, pid} init() connect connect()

43. You now allow initializations with fewer guarantees: they went from

    "the connection is available" to "the connection manager is available". Fred Hebert

44. CONNECTION TALKING CONCURRENCY RESILIENCY

45. @whatyouhide