Pathfinder - Building a Container Platform in Ruby Ecosystem

This session will discuss about an attempt to build container platform in ruby/mruby ecosystem, the current situation and lesson-learned that we can discern to improve it further.

Further on, for those whom are unfamiliar, this session will also touch a bit about container platform/orchestrator and the generic architecture behind it. So that as a developer, we understand the abstraction that it provides.

*This deck was presented in RubyKaigi 2019*

Transcript

1. Building a Container Platform in ruby ecosystem Rubykaigi 2019 @giosakti
2. Hello
3. 名前わギオ
4. 名前わギオ My name is Gio
5. Was the Kanji correct? Sorry I used google translate
6. I come from Jakarta
8. I work at GO-JEK
9. I work at GO-JEK Anybody knows GO-JEK or ゴ-ジェック?
12. Super App.
13. And we ❤ Ruby
14. Ruby is one of 5 most used languages in the company
15. I also organized ruby meetups in Jakarta
17. ruby.id/conf
18. Moving on…
19. Building a Container Platform in ruby ecosystem
20. This session will discuss about an attempt to build container platform in ruby/mruby ecosystem, the current situation and lesson-learned that we can discern to improve it further. Further on, for those whom are unfamiliar, this session will also touch a bit about container platform/orchestrator and the generic architecture behind it. So that as a developer, we understand the abstraction that it provides.
21. Background
22. This started early last year...
23. Converging to centralized platform
24. Principles & Considerations Hybrid Infrastructures Explore Containers Seamless Transition Open Source
25. One of projects that I involved was Logging
26. github.com/BaritoLog
27. We decided to use LXC to host Barito components
28. Why LXC? 1. Drop-in VM replacement 2. We have cookbooks for all the components
29. But, we need a system to manage those LXCs
31. github.com/pathfinder-cm
32. A container platform.
33. Written in Ruby and Golang
34. But, what exactly is a container platform?
35. But, what exactly is a container platform? (sometimes it is also called orchestrator or manager)
36. I found this online
37. http://splasho.com/upgoer5/
39. Neat.
40. a software that allows us to manage many computers as if they were a single big computer and assign jobs into it
41. What it means is that we…
42. Abstract away CPU & RAM
43. Abstract away Storage
44. Abstract away Network
45. The Architecture
46. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
47. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
48. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
49. Data Storage Pathfinder Mono State Server & UI Pathfinder Scheduler Container #1 Container #2 Container #3 Pathfinder Agent Container Runtime (LXD, Docker, Containerd, etc) Pathfinder CLI The Memory The Conscious The Brain The Sidecar
50. Pathfinder in Action
51. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Agent The Sidecar Container Runtime Self-registration
52. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder CLI
53. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder CLI
54. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder CLI
55. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain
56. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain
57. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Agent The Sidecar Container Runtime
59. Pathfinder in Production 1. 50+ worker nodes 2. ~1000 containers 3. Already live for 6 months+ 4. Serving our logging infrastructures (for now)
60. Pathfinder in Production Roll-out process is still ongoing. Expect at least 3-4x the current traffic.
61. Pathfinder in Ruby
62. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
63. Mruby is particularly interesting 1. Easily create executable 2. Low footprint 3. Uniform language
64. Possible challenges Ecosystem?
65. Let's Find Out
66. Writing Pathfinder Agent in mruby
67. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
68. What the agent does Interact with the state server • Do self-registration • Make queries
69. What the agent does Interact with local container daemon • Make queries • Execute changes
70. What the agent does Collect and send metrics
71. Let’s discuss the need of each use-case
72. Use case: Interact with the state server • Do self-registration • Make queries Need: a way to communicate with REST API server via HTTP/HTTPS.
73. Used: github.com/matsumotory/mruby- simplehttp
74. Use case: Interact with local container daemon • Make queries • Execute changes Need: • a way to communicate with REST API server via UNIX socket • Communicate by following LXC daemon (LXD) API documentation
75. Used: github.com/matsumotory/mruby- simplehttp
76. Extracted: github.com/giosakti/mruby-lxd
77. Use case: Collect and send metrics Need: It needs to gather system metrics (CPU & RAM utilization at minimum)
78. Extracted: github.com/giosakti/mruby- sysinfo
79. Use case: Build self-contained executable
80. Used: github.com/hone/mruby-cli
81. Writing Pathfinder CLI in mruby
82. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
83. What the CLI does Allow user to interact with the APIs of state server
84. The requirements are similar to agent
85. Extracted: pathfinder-cm/pathfinder-mruby-client
86. Extracted: giosakti/mruby-cmdr
87. 2 other components are already in ruby
88. Pathfinder Mono The state server
89. Pathfinder Scheduler
90. Now
91. Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI
92. List of Projects - Reusable libraries • giosakti/mruby-lxd Libraries for interacting with LXC Daemon (LXD) • pathfinder-cm/mruby-pathfinder-client Libraries for interacting with Pathfinder state server • giosakti/mruby-cmdr Mini frameworks for structuring CLI apps
93. List of Projects – Rewritten Components • pathfinder-cm/pathfinder-agent-mruby Agent that will be installed in the worker nodes • pathfinder-cm/pfi-mruby CLI for interacting with Pathfinder state server
94. List of Projects – Existing Components • pathfinder-cm/pathfinder-mono State server & scheduler on one repo
95. Wrap-up
96. No Gemfile ☺
97. mruby has almost all the libraries we need to do such projects
98. Building the executable for mruby application is fairly straightforward
99. The agent mruby executable is smaller than the go one Need to investigate further
100. We have the assertion library for test just fine, but no mocking library yet
101. Concurrency on mruby Thread is not there, but fiber is there I think
102. Need to explore catching signals for graceful shutdown
103. Documentation of mruby is a bit lacking One area of improvement that we can all help
104. Thank You! Come greet me for questions (and stickers!!) Or tweet me @giosakti