Pathfinder - Building a Container Platform in Ruby Ecosystem

Pathfinder - Building a Container Platform in Ruby Ecosystem

This session will discuss about an attempt to build container platform in ruby/mruby ecosystem, the current situation and lesson-learned that we can discern to improve it further.

Further on, for those whom are unfamiliar, this session will also touch a bit about container platform/orchestrator and the generic architecture behind it. So that as a developer, we understand the abstraction that it provides.

*This deck was presented in RubyKaigi 2019*

0fe18dfd87b3e48c0a45280e07cf96c6?s=128

Giovanni Sakti

April 18, 2019
Tweet

Transcript

  1. Building a Container Platform in ruby ecosystem Rubykaigi 2019 @giosakti

  2. Hello

  3. 名前わギオ

  4. 名前わギオ My name is Gio

  5. Was the Kanji correct? Sorry I used google translate

  6. I come from Jakarta

  7. None
  8. I work at GO-JEK

  9. I work at GO-JEK Anybody knows GO-JEK or ゴ-ジェック?

  10. None
  11. None
  12. Super App.

  13. And we ❤ Ruby

  14. Ruby is one of 5 most used languages in the

    company
  15. I also organized ruby meetups in Jakarta

  16. None
  17. ruby.id/conf

  18. Moving on…

  19. Building a Container Platform in ruby ecosystem

  20. This session will discuss about an attempt to build container

    platform in ruby/mruby ecosystem, the current situation and lesson-learned that we can discern to improve it further. Further on, for those whom are unfamiliar, this session will also touch a bit about container platform/orchestrator and the generic architecture behind it. So that as a developer, we understand the abstraction that it provides.
  21. Background

  22. This started early last year...

  23. Converging to centralized platform

  24. Principles & Considerations Hybrid Infrastructures Explore Containers Seamless Transition Open

    Source
  25. One of projects that I involved was Logging

  26. github.com/BaritoLog

  27. We decided to use LXC to host Barito components

  28. Why LXC? 1. Drop-in VM replacement 2. We have cookbooks

    for all the components
  29. But, we need a system to manage those LXCs

  30. None
  31. github.com/pathfinder-cm

  32. A container platform.

  33. Written in Ruby and Golang

  34. But, what exactly is a container platform?

  35. But, what exactly is a container platform? (sometimes it is

    also called orchestrator or manager)
  36. I found this online

  37. http://splasho.com/upgoer5/

  38. None
  39. Neat.

  40. a software that allows us to manage many computers as

    if they were a single big computer and assign jobs into it
  41. What it means is that we…

  42. Abstract away CPU & RAM

  43. Abstract away Storage

  44. Abstract away Network

  45. The Architecture

  46. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
  47. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
  48. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
  49. Data Storage Pathfinder Mono State Server & UI Pathfinder Scheduler

    Container #1 Container #2 Container #3 Pathfinder Agent Container Runtime (LXD, Docker, Containerd, etc) Pathfinder CLI The Memory The Conscious The Brain The Sidecar
  50. Pathfinder in Action

  51. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Agent The Sidecar Container Runtime Self-registration
  52. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder CLI
  53. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder CLI
  54. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder CLI
  55. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain
  56. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain
  57. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Agent The Sidecar Container Runtime
  58. None
  59. Pathfinder in Production 1. 50+ worker nodes 2. ~1000 containers

    3. Already live for 6 months+ 4. Serving our logging infrastructures (for now)
  60. Pathfinder in Production Roll-out process is still ongoing. Expect at

    least 3-4x the current traffic.
  61. Pathfinder in Ruby

  62. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
  63. Mruby is particularly interesting 1. Easily create executable 2. Low

    footprint 3. Uniform language
  64. Possible challenges Ecosystem?

  65. Let's Find Out

  66. Writing Pathfinder Agent in mruby

  67. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
  68. What the agent does Interact with the state server •

    Do self-registration • Make queries
  69. What the agent does Interact with local container daemon •

    Make queries • Execute changes
  70. What the agent does Collect and send metrics

  71. Let’s discuss the need of each use-case

  72. Use case: Interact with the state server • Do self-registration

    • Make queries Need: a way to communicate with REST API server via HTTP/HTTPS.
  73. Used: github.com/matsumotory/mruby- simplehttp

  74. Use case: Interact with local container daemon • Make queries

    • Execute changes Need: • a way to communicate with REST API server via UNIX socket • Communicate by following LXC daemon (LXD) API documentation
  75. Used: github.com/matsumotory/mruby- simplehttp

  76. Extracted: github.com/giosakti/mruby-lxd

  77. Use case: Collect and send metrics Need: It needs to

    gather system metrics (CPU & RAM utilization at minimum)
  78. Extracted: github.com/giosakti/mruby- sysinfo

  79. Use case: Build self-contained executable

  80. Used: github.com/hone/mruby-cli

  81. Writing Pathfinder CLI in mruby

  82. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
  83. What the CLI does Allow user to interact with the

    APIs of state server
  84. The requirements are similar to agent

  85. Extracted: pathfinder-cm/pathfinder-mruby-client

  86. Extracted: giosakti/mruby-cmdr

  87. 2 other components are already in ruby

  88. Pathfinder Mono The state server

  89. Pathfinder Scheduler

  90. Now

  91. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
  92. List of Projects - Reusable libraries • giosakti/mruby-lxd Libraries for

    interacting with LXC Daemon (LXD) • pathfinder-cm/mruby-pathfinder-client Libraries for interacting with Pathfinder state server • giosakti/mruby-cmdr Mini frameworks for structuring CLI apps
  93. List of Projects – Rewritten Components • pathfinder-cm/pathfinder-agent-mruby Agent that

    will be installed in the worker nodes • pathfinder-cm/pfi-mruby CLI for interacting with Pathfinder state server
  94. List of Projects – Existing Components • pathfinder-cm/pathfinder-mono State server

    & scheduler on one repo
  95. Wrap-up

  96. No Gemfile ☺

  97. mruby has almost all the libraries we need to do

    such projects
  98. Building the executable for mruby application is fairly straightforward

  99. The agent mruby executable is smaller than the go one

    Need to investigate further
  100. We have the assertion library for test just fine, but

    no mocking library yet
  101. Concurrency on mruby Thread is not there, but fiber is

    there I think
  102. Need to explore catching signals for graceful shutdown

  103. Documentation of mruby is a bit lacking One area of

    improvement that we can all help
  104. Thank You! Come greet me for questions (and stickers!!) Or

    tweet me @giosakti