Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Pathfinder - Building a Container Platform in Ruby Ecosystem

Pathfinder - Building a Container Platform in Ruby Ecosystem

This session will discuss about an attempt to build container platform in ruby/mruby ecosystem, the current situation and lesson-learned that we can discern to improve it further.

Further on, for those whom are unfamiliar, this session will also touch a bit about container platform/orchestrator and the generic architecture behind it. So that as a developer, we understand the abstraction that it provides.

*This deck was presented in RubyKaigi 2019*

Giovanni Sakti

April 18, 2019
Tweet

More Decks by Giovanni Sakti

Other Decks in Programming

Transcript

  1. Building a Container Platform
    in ruby ecosystem
    Rubykaigi 2019 @giosakti

    View Slide

  2. Hello

    View Slide

  3. 名前わギオ

    View Slide

  4. 名前わギオ
    My name is Gio

    View Slide

  5. Was the Kanji correct?
    Sorry I used google translate

    View Slide

  6. I come from Jakarta

    View Slide

  7. View Slide

  8. I work at GO-JEK

    View Slide

  9. I work at GO-JEK
    Anybody knows GO-JEK or ゴ-ジェック?

    View Slide

  10. View Slide

  11. View Slide

  12. Super
    App.

    View Slide

  13. And we ❤ Ruby

    View Slide

  14. Ruby is one of 5 most
    used languages
    in the company

    View Slide

  15. I also organized ruby
    meetups in Jakarta

    View Slide

  16. View Slide

  17. ruby.id/conf

    View Slide

  18. Moving on…

    View Slide

  19. Building a Container Platform
    in ruby ecosystem

    View Slide

  20. This session will discuss about an attempt to build
    container platform in ruby/mruby ecosystem, the
    current situation and lesson-learned that we can
    discern to improve it further.
    Further on, for those whom are unfamiliar, this session
    will also touch a bit about container
    platform/orchestrator and the generic architecture
    behind it. So that as a developer, we understand the
    abstraction that it provides.

    View Slide

  21. Background

    View Slide

  22. This started early last
    year...

    View Slide

  23. Converging to
    centralized platform

    View Slide

  24. Principles & Considerations
    Hybrid
    Infrastructures
    Explore
    Containers
    Seamless
    Transition
    Open Source

    View Slide

  25. One of projects that I
    involved was Logging

    View Slide

  26. github.com/BaritoLog

    View Slide

  27. We decided to use
    LXC to host Barito
    components

    View Slide

  28. Why LXC?
    1. Drop-in VM replacement
    2. We have cookbooks for all the components

    View Slide

  29. But, we need a system
    to manage those LXCs

    View Slide

  30. View Slide

  31. github.com/pathfinder-cm

    View Slide

  32. A container platform.

    View Slide

  33. Written in Ruby and
    Golang

    View Slide

  34. But, what exactly is a
    container platform?

    View Slide

  35. But, what exactly is a
    container platform?
    (sometimes it is also called orchestrator or manager)

    View Slide

  36. I found this online

    View Slide

  37. http://splasho.com/upgoer5/

    View Slide

  38. View Slide

  39. Neat.

    View Slide

  40. a software that allows us to
    manage many computers as if
    they were a single big computer
    and assign jobs into it

    View Slide

  41. What it means is that
    we…

    View Slide

  42. Abstract away CPU & RAM

    View Slide

  43. Abstract away Storage

    View Slide

  44. Abstract away Network

    View Slide

  45. The Architecture

    View Slide

  46. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Scheduler
    The Brain
    Container #1
    Container #2
    Container #3
    Pathfinder
    Agent
    The Sidecar
    Container
    Runtime
    Pathfinder CLI

    View Slide

  47. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Scheduler
    The Brain
    Container #1
    Container #2
    Container #3
    Pathfinder
    Agent
    The Sidecar
    Container
    Runtime
    Pathfinder CLI

    View Slide

  48. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Scheduler
    The Brain
    Container #1
    Container #2
    Container #3
    Pathfinder
    Agent
    The Sidecar
    Container
    Runtime
    Pathfinder CLI

    View Slide

  49. Data Storage
    Pathfinder Mono
    State Server & UI
    Pathfinder
    Scheduler
    Container #1
    Container #2
    Container #3
    Pathfinder
    Agent
    Container
    Runtime
    (LXD, Docker,
    Containerd,
    etc)
    Pathfinder CLI
    The Memory
    The Conscious
    The Brain
    The Sidecar

    View Slide

  50. Pathfinder in Action

    View Slide

  51. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Agent
    The Sidecar
    Container
    Runtime
    Self-registration

    View Slide

  52. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder CLI

    View Slide

  53. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder CLI

    View Slide

  54. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder CLI

    View Slide

  55. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Scheduler
    The Brain

    View Slide

  56. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Scheduler
    The Brain

    View Slide

  57. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Agent
    The Sidecar
    Container
    Runtime

    View Slide

  58. View Slide

  59. Pathfinder in Production
    1. 50+ worker nodes
    2. ~1000 containers
    3. Already live for 6 months+
    4. Serving our logging infrastructures (for now)

    View Slide

  60. Pathfinder in Production
    Roll-out process is still ongoing.
    Expect at least 3-4x the current traffic.

    View Slide

  61. Pathfinder in Ruby

    View Slide

  62. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Scheduler
    The Brain
    Container #1
    Container #2
    Container #3
    Pathfinder
    Agent
    The Sidecar
    Container
    Runtime
    Pathfinder CLI

    View Slide

  63. Mruby is particularly interesting
    1. Easily create executable
    2. Low footprint
    3. Uniform language

    View Slide

  64. Possible challenges
    Ecosystem?

    View Slide

  65. Let's Find Out

    View Slide

  66. Writing Pathfinder Agent
    in mruby

    View Slide

  67. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Scheduler
    The Brain
    Container #1
    Container #2
    Container #3
    Pathfinder
    Agent
    The Sidecar
    Container
    Runtime
    Pathfinder CLI

    View Slide

  68. What the agent does
    Interact with the state server
    • Do self-registration
    • Make queries

    View Slide

  69. What the agent does
    Interact with local container daemon
    • Make queries
    • Execute changes

    View Slide

  70. What the agent does
    Collect and send metrics

    View Slide

  71. Let’s discuss the need of
    each use-case

    View Slide

  72. Use case:
    Interact with the state server
    • Do self-registration
    • Make queries
    Need:
    a way to communicate with REST API server via
    HTTP/HTTPS.

    View Slide

  73. Used:
    github.com/matsumotory/mruby-
    simplehttp

    View Slide

  74. Use case:
    Interact with local container daemon
    • Make queries
    • Execute changes
    Need:
    • a way to communicate with REST API server
    via UNIX socket
    • Communicate by following LXC daemon
    (LXD) API documentation

    View Slide

  75. Used:
    github.com/matsumotory/mruby-
    simplehttp

    View Slide

  76. Extracted:
    github.com/giosakti/mruby-lxd

    View Slide

  77. Use case:
    Collect and send metrics
    Need:
    It needs to gather system metrics (CPU & RAM
    utilization at minimum)

    View Slide

  78. Extracted:
    github.com/giosakti/mruby-
    sysinfo

    View Slide

  79. Use case:
    Build self-contained
    executable

    View Slide

  80. Used:
    github.com/hone/mruby-cli

    View Slide

  81. Writing Pathfinder CLI
    in mruby

    View Slide

  82. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Scheduler
    The Brain
    Container #1
    Container #2
    Container #3
    Pathfinder
    Agent
    The Sidecar
    Container
    Runtime
    Pathfinder CLI

    View Slide

  83. What the CLI does
    Allow user to interact with the APIs of state
    server

    View Slide

  84. The requirements are
    similar to agent

    View Slide

  85. Extracted:
    pathfinder-cm/pathfinder-mruby-client

    View Slide

  86. Extracted:
    giosakti/mruby-cmdr

    View Slide

  87. 2 other components are
    already in ruby

    View Slide

  88. Pathfinder Mono
    The state server

    View Slide

  89. Pathfinder Scheduler

    View Slide

  90. Now

    View Slide

  91. Data Storage
    The Memory
    Pathfinder Mono
    State Server & UI
    The Conscious
    Pathfinder
    Scheduler
    The Brain
    Container #1
    Container #2
    Container #3
    Pathfinder
    Agent
    The Sidecar
    Container
    Runtime
    Pathfinder CLI

    View Slide

  92. List of Projects - Reusable libraries
    • giosakti/mruby-lxd
    Libraries for interacting with LXC Daemon (LXD)
    • pathfinder-cm/mruby-pathfinder-client
    Libraries for interacting with Pathfinder state server
    • giosakti/mruby-cmdr
    Mini frameworks for structuring CLI apps

    View Slide

  93. List of Projects – Rewritten Components
    • pathfinder-cm/pathfinder-agent-mruby
    Agent that will be installed in the worker nodes
    • pathfinder-cm/pfi-mruby
    CLI for interacting with Pathfinder state server

    View Slide

  94. List of Projects – Existing Components
    • pathfinder-cm/pathfinder-mono
    State server & scheduler on one repo

    View Slide

  95. Wrap-up

    View Slide

  96. No Gemfile ☺

    View Slide

  97. mruby has almost all the
    libraries we need to do
    such projects

    View Slide

  98. Building the executable for
    mruby application is fairly
    straightforward

    View Slide

  99. The agent mruby executable
    is smaller than the go one
    Need to investigate further

    View Slide

  100. We have the assertion
    library for test just fine, but
    no mocking library yet

    View Slide

  101. Concurrency on mruby
    Thread is not there, but fiber is there I think

    View Slide

  102. Need to explore catching
    signals for graceful
    shutdown

    View Slide

  103. Documentation of mruby is
    a bit lacking
    One area of improvement that we can all help

    View Slide

  104. Thank You!
    Come greet me for questions (and stickers!!)
    Or tweet me @giosakti

    View Slide