Docker and OTP: Friends or Foes?

E06aa8f63d2a1753a2b352bc1cabbde2?s=47 Daniel Azuma
September 07, 2018

Docker and OTP: Friends or Foes?

Presentation by Daniel Azuma at ElixirConf 2018. Discusses ideas for making OTP applications work together with container-based deployments. More information at http://daniel-azuma.com/elixirconf2018

E06aa8f63d2a1753a2b352bc1cabbde2?s=128

Daniel Azuma

September 07, 2018
Tweet

Transcript

  1. Docker and OTP: Friends or Foes? Daniel Azuma

  2. Docker and OTP: Friends or Foes? Daniel Azuma

  3. None
  4. Web client Javascript

  5. Web client Javascript HTTP/ websocket

  6. Web client Tanx Phoenix app Javascript HTTP/ websocket

  7. Web client Tanx Phoenix app Game Server Javascript HTTP/ websocket

  8. Web client Tanx Phoenix app Game Server Game Updater Javascript

    HTTP/ websocket
  9. Web client Tanx Phoenix app Game Server Game Updater Game

    Supervisor Javascript HTTP/ websocket
  10. Web client Tanx Phoenix app Game Server Game Updater Game

    Supervisor HTTP/ websocket Javascript
  11. Web client Tanx Phoenix app Channel Game Server Game Updater

    Game Supervisor HTTP/ websocket Javascript
  12. Web client Tanx Phoenix app Channel Game Server Game Updater

    Game Supervisor HTTP/ websocket Load Balancer Javascript
  13. Web client Tanx Phoenix app Channel Game Server Game Updater

    Game Supervisor HTTP/ websocket messages Load Balancer Javascript
  14. Web client Tanx Phoenix app Channel Game Server Game Updater

    Game Supervisor HTTP/ websocket messages Load Balancer Javascript
  15. Web client Tanx Phoenix app Channel Game Server Game Updater

    Game Supervisor HTTP/ websocket messages Load Balancer Javascript
  16. Web client Tanx Phoenix app Channel Game Server Game Updater

    Game Supervisor HTTP/ websocket messages Load Balancer Javascript
  17. Deployment

  18. None
  19. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor HTTP/ websocket messages Docker Container Javascript
  20. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor HTTP/ websocket messages Docker Container Updated Container Javascript
  21. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor HTTP/ websocket messages Docker Container Updated Container Javascript
  22. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor HTTP/ websocket messages Docker Container Updated Container Javascript
  23. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor Javascript HTTP/ websocket messages Docker Container Updated Container
  24. Containers vs OTP

  25. None
  26. None
  27. None
  28. I want to terminate containers! I want long-running GenServers!

  29. I want to host in the cloud! I want total

    control of resources!
  30. ?

  31. None
  32. None
  33. ☺ Familiar

  34. ☺ Familiar Isolating

  35. None
  36. Phoenix == Rails??

  37. None
  38. None
  39. https://www.flickr.com/photos/bcgrote/2884221396

  40. None
  41. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor Javascript HTTP/ websocket messages Node 2 Node 1
  42. Problem 1: Maintaining an Erlang Cluster

  43. https://github.com/bitwalker/libcluster

  44. None
  45. Problem 2: Handling node shutdown

  46. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor Javascript HTTP/ websocket messages Node 2 Node 1
  47. Channels

  48. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor Javascript HTTP/ websocket messages Node 2 Node 1
  49. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor Javascript HTTP/ websocket messages Node 2 Node 1 ❗
  50. Load Balancer Web client Tanx Phoenix app Game Server Game

    Updater Game Supervisor Javascript HTTP/ websocket Node 2 Node 1
  51. Load Balancer Web client Tanx Phoenix app Game Server Game

    Updater Game Supervisor Javascript HTTP/ websocket Node 2 Node 1
  52. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor Javascript HTTP/ websocket messages Node 2 Node 1
  53. GenServers

  54. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor Javascript HTTP/ websocket messages Node 2 Node 1
  55. Load Balancer Web client Tanx Phoenix app Channel Game Server

    Game Updater Game Supervisor Javascript HTTP/ websocket messages Node 2 Node 1
  56. “Distributed” Supervisor

  57. https://github.com/derekkraan/horde

  58. Node 2 Node 1 Horde Supervisor Horde Supervisor Tanx Phoenix

    app
  59. Node 2 Node 1 Horde Supervisor Horde Supervisor Tanx Phoenix

    app
  60. Node 2 Game Server Game Updater Game Supervisor Node 1

    Horde Supervisor Horde Supervisor Tanx Phoenix app
  61. Node 2 Game Server Game Updater Game Supervisor Node 1

    Horde Supervisor Horde Supervisor Tanx Phoenix app
  62. Node 2 Game Server Game Updater Game Supervisor Node 1

    Horde Supervisor Horde Supervisor Game Server Game Updater Game Supervisor Tanx Phoenix app
  63. Problem 3: Preserving process state

  64. “Distributed” Agent

  65. “Distributed” Agent == CRDT

  66. Node 2 Game Server Game Updater Game Supervisor Node 1

    Horde Supervisor Horde Supervisor Tanx Phoenix app
  67. Node 2 Game Server Game Updater Game Supervisor Node 1

    Horde Supervisor Horde Supervisor Handoff CRDT Handoff CRDT Tanx Phoenix app
  68. Node 2 Game Server Game Updater Game Supervisor Node 1

    Horde Supervisor Horde Supervisor Handoff CRDT Handoff CRDT Tanx Phoenix app
  69. Node 2 Game Server Game Updater Game Supervisor Node 1

    Horde Supervisor Horde Supervisor Handoff CRDT Handoff CRDT Tanx Phoenix app SIGTERM
  70. Node 2 Game Server Game Updater Game Supervisor Node 1

    Horde Supervisor Horde Supervisor Handoff CRDT Handoff CRDT Tanx Phoenix app SIGTERM
  71. Node 1 Horde Supervisor Handoff CRDT Handoff CRDT Node 2

    Game Server Game Updater Game Supervisor Horde Supervisor Tanx Phoenix app
  72. Node 1 Horde Supervisor Handoff CRDT Handoff CRDT Node 2

    Game Server Game Updater Game Supervisor Horde Supervisor Game Server Game Updater Game Supervisor Tanx Phoenix app
  73. Node 1 Horde Supervisor Handoff CRDT Handoff CRDT Node 2

    Game Server Game Updater Game Supervisor Horde Supervisor Game Server Game Updater Game Supervisor Tanx Phoenix app
  74. Problem 4: Preserving communication

  75. “Distributed” Registry

  76. Node 2 Game Server Game Updater Game Supervisor Channel Node

    1 Horde Supervisor Horde Supervisor Handoff CRDT Handoff CRDT
  77. Node 2 Game Server Game Updater Game Supervisor Channel Node

    1 Horde Supervisor Horde Supervisor Handoff CRDT Horde Registry Handoff CRDT Horde Registry
  78. Node 2 Game Server Game Updater Game Supervisor Channel Node

    1 Horde Supervisor Horde Supervisor Handoff CRDT Horde Registry Handoff CRDT Horde Registry register
  79. Node 2 Game Server Game Updater Game Supervisor Channel Node

    1 Horde Supervisor Horde Supervisor Handoff CRDT Horde Registry Handoff CRDT Horde Registry m essages
  80. Node 1 Horde Supervisor Handoff CRDT Handoff CRDT Node 2

    Game Server Game Updater Game Supervisor Horde Supervisor Game Server Game Updater Game Supervisor Tanx Phoenix app Channel Horde Registry Horde Registry reregister
  81. Connect Erlang cluster — Move/restart processes — Hand off process

    state — Maintain communication — LibCluster Horde Supervisor CRDT Horde Registry
  82. Does it work?

  83. audience participation demo! http://tanx.elixir-seattle.net

  84. Load Balancer v1 Javascript HTTP/ websocket v1 http://tanx.elixir-seattle.net

  85. Load Balancer v1 Javascript HTTP/ websocket v1 http://tanx.elixir-seattle.net

  86. Load Balancer v1 Javascript HTTP/ websocket v1 v2 v2 http://tanx.elixir-seattle.net

  87. v2 Load Balancer v1 Javascript HTTP/ websocket v1 v2 http://tanx.elixir-seattle.net

  88. None
  89. Processes ❤ Containers

  90. https://github.com/bitwalker/swarm https://lasp-lang.org

  91. None
  92. None
  93. None
  94. Friends?

  95. Thank you! http://daniel-azuma.com/elixirconf2018

  96. Thank you! http://daniel-azuma.com/elixirconf2018 http://tanx.elixir-seattle.net

  97. None