Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Docker and OTP: Friends or Foes?

Daniel Azuma
September 07, 2018
Programming
5
640

Docker and OTP: Friends or Foes?

Presentation by Daniel Azuma at ElixirConf 2018. Discusses ideas for making OTP applications work together with container-based deployments. More information at http://daniel-azuma.com/elixirconf2018

Daniel Azuma

September 07, 2018
Tweet

More Decks by Daniel Azuma

See All by Daniel Azuma
Ruby Ate My DSL!
dazuma
0
290
Yes! You should provide a client library for your API!
dazuma
1
1.6k
Containerizing Rails: Techniques, Pitfalls, & Best Practices
dazuma
2
1.2k
Dispelling the Dark Magic: Inside a Ruby Debugger
dazuma
0
190
What's my app *really* doing in production?
dazuma
0
140
Humanity On Rails
dazuma
2
540
Getting Down To Earth: Geospatial Analysis With Rails
dazuma
10
2.2k

Other Decks in Programming

See All in Programming
NSSpain 2023: An overview of different approaches to share code across platforms
terhechte
0
120
はじめてのPHPコマンドラインオプション〜PHP開発を今よりちょっと楽しくする〜
kotomin_m
2
100
Rediscover the joy of coding with Creative Coding
clown0082
0
110
iOSアプリ開発における効果的なUnitTestの戦略とその実装
mot_techtalk
1
200
第4回 AWSとGitHub勉強会 - GitHub Actionsを使ったCD環境の構築 -
ogiwarat
0
150
Domain-Driven Design (Tutorial)
hschwentner
13
18k
【PHPカンファレンス沖縄 2023】素朴で考慮漏れのある PHP コードをテストコードとともに補強していく（ライブコーディング補足資料） / #phpcon_okinawa 2023 livecoding supplementary material
okashoi
2
140
第6回 AWSとGitHub勉強会 - AWS ECS Fargate環境の構築 -
ogiwarat
0
160
スキル差があるペア_モブプロで効果的な_ドライバーナビゲータ以外のロールの分け方.pdf
yatasunshine
1
220
Help Collisions, Isolate the Worlds
tagomoris
0
100
外部APIとズブズブな開発どうしてますか？
kin29
1
350
認知負荷で考えるフロントエンドの組織体制
shibe23
0
870

Featured

See All Featured
How GitHub Uses GitHub to Build GitHub
holman
467
280k
The Straight Up "How To Draw Better" Workshop
denniskardys
226
130k
Optimising Largest Contentful Paint
csswizardry
6
1.8k
A Tale of Four Properties
chriscoyier
150
21k
Design by the Numbers
sachag
272
18k
Creatively Recalculating Your Daily Design Routine
revolveconf
208
11k
GraphQLとの向き合い方2022年版
quramy
25
11k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
112
17k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
15
1.6k
Build your cross-platform service in a week with App Engine
jlugia
223
17k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
26
5.5k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
6
7.1k

Transcript

  1. Docker and OTP:
    Friends or Foes?
    Daniel Azuma

    View Slide

  2. Docker and OTP:
    Friends or Foes?
    Daniel Azuma

    View Slide

  3. View Slide

  4. Web client
    Javascript

    View Slide

  5. Web client
    Javascript
    HTTP/
    websocket

    View Slide

  6. Web client
    Tanx Phoenix app
    Javascript
    HTTP/
    websocket

    View Slide

  7. Web client
    Tanx Phoenix app
    Game
    Server
    Javascript
    HTTP/
    websocket

    View Slide

  8. Web client
    Tanx Phoenix app
    Game
    Server
    Game
    Updater
    Javascript
    HTTP/
    websocket

    View Slide

  9. Web client
    Tanx Phoenix app
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket

    View Slide

  10. Web client
    Tanx Phoenix app
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    Javascript

    View Slide

  11. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    Javascript

    View Slide

  12. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    Load Balancer
    Javascript

    View Slide

  13. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Load Balancer
    Javascript

    View Slide

  14. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Load Balancer
    Javascript

    View Slide

  15. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Load Balancer
    Javascript

    View Slide

  16. Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Load Balancer
    Javascript

    View Slide

  17. Deployment

    View Slide

  18. View Slide

  19. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Docker Container
    Javascript

    View Slide

  20. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Docker Container
    Updated Container
    Javascript

    View Slide

  21. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Docker Container
    Updated Container
    Javascript

    View Slide

  22. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    HTTP/
    websocket
    messages
    Docker Container
    Updated Container
    Javascript

    View Slide

  23. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Docker Container
    Updated Container

    View Slide

  24. Containers
    vs
    OTP

    View Slide

  25. View Slide

  26. View Slide

  27. View Slide

  28. I want to
    terminate
    containers!
    I want

    long-running
    GenServers!

    View Slide

  29. I want to host

    in the cloud!
    I want total
    control of
    resources!

    View Slide

  30. ?

    View Slide

  31. View Slide

  32. View Slide

  33. ☺ Familiar

    View Slide

  34. ☺ Familiar
    Isolating

    View Slide

  35. View Slide

  36. Phoenix == Rails??

    View Slide

  37. View Slide

  38. View Slide

  39. https://www.flickr.com/photos/bcgrote/2884221396

    View Slide

  40. View Slide

  41. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  42. Problem 1:
    Maintaining an Erlang Cluster

    View Slide

  43. https://github.com/bitwalker/libcluster

    View Slide

  44. View Slide

  45. Problem 2:
    Handling node shutdown

    View Slide

  46. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  47. Channels

    View Slide

  48. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  49. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1
    ❗

    View Slide

  50. Load Balancer
    Web client
    Tanx Phoenix app
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    Node 2 Node 1

    View Slide

  51. Load Balancer
    Web client
    Tanx Phoenix app
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    Node 2 Node 1

    View Slide

  52. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  53. GenServers

    View Slide

  54. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  55. Load Balancer
    Web client
    Tanx Phoenix app
    Channel
    Game
    Server
    Game
    Updater
    Game Supervisor
    Javascript
    HTTP/
    websocket
    messages
    Node 2 Node 1

    View Slide

  56. “Distributed” Supervisor

    View Slide

  57. https://github.com/derekkraan/horde

    View Slide

  58. Node 2 Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  59. Node 2 Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  60. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  61. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  62. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Game
    Server
    Game
    Updater
    Game Supervisor
    Tanx Phoenix app

    View Slide

  63. Problem 3:
    Preserving process state

    View Slide

  64. “Distributed” Agent

    View Slide

  65. “Distributed” Agent
    == CRDT

    View Slide

  66. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  67. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Tanx Phoenix app

    View Slide

  68. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Tanx Phoenix app

    View Slide

  69. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Tanx Phoenix app
    SIGTERM

    View Slide

  70. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Tanx Phoenix app
    SIGTERM

    View Slide

  71. Node 1
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Horde
    Supervisor
    Tanx Phoenix app

    View Slide

  72. Node 1
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Horde
    Supervisor
    Game
    Server
    Game
    Updater
    Game Supervisor
    Tanx Phoenix app

    View Slide

  73. Node 1
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Horde
    Supervisor
    Game
    Server
    Game
    Updater
    Game Supervisor
    Tanx Phoenix app

    View Slide

  74. Problem 4:
    Preserving communication

    View Slide

  75. “Distributed” Registry

    View Slide

  76. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Channel
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT

    View Slide

  77. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Channel
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Horde
    Registry
    Handoff
    CRDT
    Horde
    Registry

    View Slide

  78. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Channel
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Horde
    Registry
    Handoff
    CRDT
    Horde
    Registry
    register

    View Slide

  79. Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Channel
    Node 1
    Horde
    Supervisor
    Horde
    Supervisor
    Handoff
    CRDT
    Horde
    Registry
    Handoff
    CRDT
    Horde
    Registry
    m
    essages

    View Slide

  80. Node 1
    Horde
    Supervisor
    Handoff
    CRDT
    Handoff
    CRDT
    Node 2
    Game
    Server
    Game
    Updater
    Game Supervisor
    Horde
    Supervisor
    Game
    Server
    Game
    Updater
    Game Supervisor
    Tanx Phoenix app
    Channel
    Horde
    Registry
    Horde
    Registry
    reregister

    View Slide

  81. Connect Erlang cluster —
    Move/restart processes —
    Hand off process state —
    Maintain communication —
    LibCluster
    Horde Supervisor
    CRDT
    Horde Registry

    View Slide

  82. Does it work?

    View Slide

  83. audience

    participation

    demo!
    http://tanx.elixir-seattle.net

    View Slide

  84. Load Balancer
    v1
    Javascript
    HTTP/
    websocket
    v1
    http://tanx.elixir-seattle.net

    View Slide

  85. Load Balancer
    v1
    Javascript
    HTTP/
    websocket
    v1
    http://tanx.elixir-seattle.net

    View Slide

  86. Load Balancer
    v1
    Javascript
    HTTP/
    websocket
    v1
    v2
    v2
    http://tanx.elixir-seattle.net

    View Slide

  87. v2
    Load Balancer
    v1
    Javascript
    HTTP/
    websocket
    v1
    v2
    http://tanx.elixir-seattle.net

    View Slide

  88. View Slide

  89. Processes ❤ Containers

    View Slide

  90. https://github.com/bitwalker/swarm
    https://lasp-lang.org

    View Slide

  91. View Slide

  92. View Slide

  93. View Slide

  94. Friends?

    View Slide

  95. Thank you!
    http://daniel-azuma.com/elixirconf2018

    View Slide

  96. Thank you!
    http://daniel-azuma.com/elixirconf2018
    http://tanx.elixir-seattle.net

    View Slide

  97. View Slide