Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Docker and OTP: Friends or Foes?

Daniel Azuma
September 07, 2018

Docker and OTP: Friends or Foes?

Presentation by Daniel Azuma at ElixirConf 2018. Discusses ideas for making OTP applications work together with container-based deployments. More information at http://daniel-azuma.com/elixirconf2018

Daniel Azuma

September 07, 2018
Tweet

More Decks by Daniel Azuma

Other Decks in Programming

Transcript

  1. Docker and OTP:
    Friends or Foes?
    Daniel Azuma

    View Slide

  2. Docker and OTP:
    Friends or Foes?
    Daniel Azuma

    View Slide

  3. View Slide

  4. Web client
    Javascript

    View Slide

  5. Web client
    Javascript
    HTTP/
    websocket

    View Slide

  6. Web client
    Tanx Phoenix app
    Javascript
    HTTP/
    websocket

    View Slide

  7. Web client
    Tanx Phoenix app
    Game
    Server
    Javascript
    HTTP/
    websocket

    View Slide

  8. Web client
    Tanx Phoenix app
    Game
    Server
    Game
    Updater
    Javascript
    HTTP/
    websocket

    View Slide

  9. Web client
    Tanx Phoenix app
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket

    View Slide

  10. Web client
    Tanx Phoenix app
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    Javascript

    View Slide

  11. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    Javascript

    View Slide

  12. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    Load Balancer
    Javascript

    View Slide

  13. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Load Balancer
    Javascript

    View Slide

  14. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Load Balancer
    Javascript

    View Slide

  15. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Load Balancer
    Javascript

    View Slide

  16. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Load Balancer
    Javascript

    View Slide

  17. Deployment

    View Slide

  18. View Slide

  19. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Docker Container
    Javascript

    View Slide

  20. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Docker Container
    Updated Container
    Javascript

    View Slide

  21. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Docker Container
    Updated Container
    Javascript

    View Slide

  22. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Docker Container
    Updated Container
    Javascript

    View Slide

  23. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Docker Container
    Updated Container

    View Slide

  24. Containers
    vs
    OTP

    View Slide

  25. View Slide

  26. View Slide

  27. View Slide

  28. I want to
    terminate
    containers!
    I want

    long-running
    GenServers!

    View Slide

  29. I want to host

    in the cloud!
    I want total
    control of
    resources!

    View Slide

  30. ?

    View Slide

  31. View Slide

  32. View Slide

  33. ☺ Familiar

    View Slide

  34. ☺ Familiar
    Isolating

    View Slide

  35. View Slide

  36. Phoenix == Rails??

    View Slide

  37. View Slide

  38. View Slide

  39. https://www.flickr.com/photos/bcgrote/2884221396

    View Slide

  40. View Slide

  41. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  42. Problem 1:
    Maintaining an Erlang Cluster

    View Slide

  43. https://github.com/bitwalker/libcluster

    View Slide

  44. View Slide

  45. Problem 2:
    Handling node shutdown

    View Slide

  46. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  47. Channels

    View Slide

  48. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  49. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  50. Load Balancer
    Web client
    Tanx Phoenix app
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    Node 2 Node 1

    View Slide

  51. Load Balancer
    Web client
    Tanx Phoenix app
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    Node 2 Node 1

    View Slide

  52. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  53. GenServers

    View Slide

  54. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  55. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  56. “Distributed” Supervisor

    View Slide

  57. https://github.com/derekkraan/horde

    View Slide

  58. Node 2 Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  59. Node 2 Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  60. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  61. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  62. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Game
    Server
    Game
    Updater
    Game Supervisor
    Tanx Phoenix app

    View Slide

  63. Problem 3:
    Preserving process state

    View Slide

  64. “Distributed” Agent

    View Slide

  65. “Distributed” Agent
    == CRDT

    View Slide

  66. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  67. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Tanx Phoenix app

    View Slide

  68. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Tanx Phoenix app

    View Slide

  69. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Tanx Phoenix app
    SIGTERM

    View Slide

  70. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Tanx Phoenix app
    SIGTERM

    View Slide

  71. Node 1
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  72. Node 1
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Horde
    Supervisor
    Game
    Server
    Game
    Updater
    Game Supervisor
    Tanx Phoenix app

    View Slide

  73. Node 1
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Horde
    Supervisor
    Game
    Server
    Game
    Updater
    Game Supervisor
    Tanx Phoenix app

    View Slide

  74. Problem 4:
    Preserving communication

    View Slide

  75. “Distributed” Registry

    View Slide

  76. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Channel
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT

    View Slide

  77. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Channel
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Horde
    Registry
    Handoff
    CRDT
    Horde
    Registry

    View Slide

  78. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Channel
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Horde
    Registry
    Handoff
    CRDT
    Horde
    Registry
    register

    View Slide

  79. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Channel
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Horde
    Registry
    Handoff
    CRDT
    Horde
    Registry
    m
    essages

    View Slide

  80. Node 1
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Horde
    Supervisor
    Game
    Server
    Game
    Updater
    Game Supervisor
    Tanx Phoenix app
    Channel
    Horde
    Registry
    Horde
    Registry
    reregister

    View Slide

  81. Connect Erlang cluster —
    Move/restart processes —
    Hand off process state —
    Maintain communication —
    LibCluster
    Horde Supervisor
    CRDT
    Horde Registry

    View Slide

  82. Does it work?

    View Slide

  83. audience

    participation

    demo!
    http://tanx.elixir-seattle.net

    View Slide

  84. Load Balancer
    v1
    Javascript
    HTTP/
    websocket
    v1
    http://tanx.elixir-seattle.net

    View Slide

  85. Load Balancer
    v1
    Javascript
    HTTP/
    websocket
    v1
    http://tanx.elixir-seattle.net

    View Slide

  86. Load Balancer
    v1
    Javascript
    HTTP/
    websocket
    v1
    v2
    v2
    http://tanx.elixir-seattle.net

    View Slide

  87. v2
    Load Balancer
    v1
    Javascript
    HTTP/
    websocket
    v1
    v2
    http://tanx.elixir-seattle.net

    View Slide

  88. View Slide

  89. Processes ❤ Containers

    View Slide

  90. https://github.com/bitwalker/swarm
    https://lasp-lang.org

    View Slide

  91. View Slide

  92. View Slide

  93. View Slide

  94. Friends?

    View Slide

  95. Thank you!
    http://daniel-azuma.com/elixirconf2018

    View Slide

  96. Thank you!
    http://daniel-azuma.com/elixirconf2018
    http://tanx.elixir-seattle.net

    View Slide

  97. View Slide