Pathfinder - Building a Container Platform in Ruby Ecosystem

Slide 1

Slide 1 text

Building a Container Platform in ruby ecosystem Rubykaigi 2019 @giosakti

Slide 2

Slide 2 text

Hello

Slide 3

Slide 3 text

名前わギオ

Slide 4

Slide 4 text

名前わギオ My name is Gio

Slide 5

Slide 5 text

Was the Kanji correct? Sorry I used google translate

Slide 6

Slide 6 text

I come from Jakarta

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

I work at GO-JEK

Slide 9

Slide 9 text

I work at GO-JEK Anybody knows GO-JEK or ゴ-ジェック?

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Super App.

Slide 13

Slide 13 text

And we ❤ Ruby

Slide 14

Slide 14 text

Ruby is one of 5 most used languages in the company

Slide 15

Slide 15 text

I also organized ruby meetups in Jakarta

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

ruby.id/conf

Slide 18

Slide 18 text

Moving on…

Slide 19

Slide 19 text

Building a Container Platform in ruby ecosystem

Slide 20

Slide 20 text

This session will discuss about an attempt to build container platform in ruby/mruby ecosystem, the current situation and lesson-learned that we can discern to improve it further. Further on, for those whom are unfamiliar, this session will also touch a bit about container platform/orchestrator and the generic architecture behind it. So that as a developer, we understand the abstraction that it provides.

Slide 21

Slide 21 text

Background

Slide 22

Slide 22 text

This started early last year...

Slide 23

Slide 23 text

Converging to centralized platform

Slide 24

Slide 24 text

Principles & Considerations Hybrid Infrastructures Explore Containers Seamless Transition Open Source

Slide 25

Slide 25 text

One of projects that I involved was Logging

Slide 26

Slide 26 text

github.com/BaritoLog

Slide 27

Slide 27 text

We decided to use LXC to host Barito components

Slide 28

Slide 28 text

Why LXC? 1. Drop-in VM replacement 2. We have cookbooks for all the components

Slide 29

Slide 29 text

But, we need a system to manage those LXCs

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

github.com/pathfinder-cm

Slide 32

Slide 32 text

A container platform.

Slide 33

Slide 33 text

Written in Ruby and Golang

Slide 34

Slide 34 text

But, what exactly is a container platform?

Slide 35

Slide 35 text

But, what exactly is a container platform? (sometimes it is also called orchestrator or manager)

Slide 36

Slide 36 text

I found this online

Slide 37

Slide 37 text

http://splasho.com/upgoer5/

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

Neat.

Slide 40

Slide 40 text

a software that allows us to manage many computers as if they were a single big computer and assign jobs into it

Slide 41

Slide 41 text

What it means is that we…

Slide 42

Slide 42 text

Abstract away CPU & RAM

Slide 43

Slide 43 text

Abstract away Storage

Slide 44

Slide 44 text

Abstract away Network

Slide 45

Slide 45 text

The Architecture

Slide 46

Slide 46 text

Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI

Slide 47

Slide 47 text

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Data Storage Pathfinder Mono State Server & UI Pathfinder Scheduler Container #1 Container #2 Container #3 Pathfinder Agent Container Runtime (LXD, Docker, Containerd, etc) Pathfinder CLI The Memory The Conscious The Brain The Sidecar

Slide 50

Slide 50 text

Pathfinder in Action

Slide 51

Slide 51 text

Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Agent The Sidecar Container Runtime Self-registration

Slide 52

Slide 52 text

Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder CLI

Slide 53

Slide 53 text

Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder CLI

Slide 54

Slide 54 text

Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder CLI

Slide 55

Slide 55 text

Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain

Slide 56

Slide 56 text

Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Scheduler The Brain

Slide 57

Slide 57 text

Data Storage The Memory Pathfinder Mono State Server & UI The Conscious Pathfinder Agent The Sidecar Container Runtime

Slide 58

Slide 58 text

No content

Slide 59

Slide 59 text

Pathfinder in Production 1. 50+ worker nodes 2. ~1000 containers 3. Already live for 6 months+ 4. Serving our logging infrastructures (for now)

Slide 60

Slide 60 text

Pathfinder in Production Roll-out process is still ongoing. Expect at least 3-4x the current traffic.

Slide 61

Slide 61 text

Pathfinder in Ruby

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Mruby is particularly interesting 1. Easily create executable 2. Low footprint 3. Uniform language

Slide 64

Slide 64 text

Possible challenges Ecosystem?

Slide 65

Slide 65 text

Let's Find Out

Slide 66

Slide 66 text

Writing Pathfinder Agent in mruby

Slide 67

Slide 67 text

Slide 68

Slide 68 text

What the agent does Interact with the state server • Do self-registration • Make queries

Slide 69

Slide 69 text

What the agent does Interact with local container daemon • Make queries • Execute changes

Slide 70

Slide 70 text

What the agent does Collect and send metrics

Slide 71

Slide 71 text

Let’s discuss the need of each use-case

Slide 72

Slide 72 text

Use case: Interact with the state server • Do self-registration • Make queries Need: a way to communicate with REST API server via HTTP/HTTPS.

Slide 73

Slide 73 text

Used: github.com/matsumotory/mruby- simplehttp

Slide 74

Slide 74 text

Use case: Interact with local container daemon • Make queries • Execute changes Need: • a way to communicate with REST API server via UNIX socket • Communicate by following LXC daemon (LXD) API documentation

Slide 75

Slide 75 text

Used: github.com/matsumotory/mruby- simplehttp

Slide 76

Slide 76 text

Extracted: github.com/giosakti/mruby-lxd

Slide 77

Slide 77 text

Use case: Collect and send metrics Need: It needs to gather system metrics (CPU & RAM utilization at minimum)

Slide 78

Slide 78 text

Extracted: github.com/giosakti/mruby- sysinfo

Slide 79

Slide 79 text

Use case: Build self-contained executable

Slide 80

Slide 80 text

Used: github.com/hone/mruby-cli

Slide 81

Slide 81 text

Writing Pathfinder CLI in mruby

Slide 82

Slide 82 text

Slide 83

Slide 83 text

What the CLI does Allow user to interact with the APIs of state server

Slide 84

Slide 84 text

The requirements are similar to agent

Slide 85

Slide 85 text

Extracted: pathfinder-cm/pathfinder-mruby-client

Slide 86

Slide 86 text

Extracted: giosakti/mruby-cmdr

Slide 87

Slide 87 text

2 other components are already in ruby

Slide 88

Slide 88 text

Pathfinder Mono The state server

Slide 89

Slide 89 text

Pathfinder Scheduler

Slide 90

Slide 90 text

Now

Slide 91

Slide 91 text

Slide 92

Slide 92 text

List of Projects - Reusable libraries • giosakti/mruby-lxd Libraries for interacting with LXC Daemon (LXD) • pathfinder-cm/mruby-pathfinder-client Libraries for interacting with Pathfinder state server • giosakti/mruby-cmdr Mini frameworks for structuring CLI apps

Slide 93

Slide 93 text

List of Projects – Rewritten Components • pathfinder-cm/pathfinder-agent-mruby Agent that will be installed in the worker nodes • pathfinder-cm/pfi-mruby CLI for interacting with Pathfinder state server

Slide 94

Slide 94 text

List of Projects – Existing Components • pathfinder-cm/pathfinder-mono State server & scheduler on one repo

Slide 95

Slide 95 text

Wrap-up

Slide 96

Slide 96 text

No Gemfile ☺

Slide 97

Slide 97 text

mruby has almost all the libraries we need to do such projects

Slide 98

Slide 98 text

Building the executable for mruby application is fairly straightforward

Slide 99

Slide 99 text

The agent mruby executable is smaller than the go one Need to investigate further

Slide 100

Slide 100 text

We have the assertion library for test just fine, but no mocking library yet

Slide 101

Slide 101 text

Concurrency on mruby Thread is not there, but fiber is there I think

Slide 102

Slide 102 text

Need to explore catching signals for graceful shutdown

Slide 103

Slide 103 text

Documentation of mruby is a bit lacking One area of improvement that we can all help

Slide 104

Slide 104 text

Thank You! Come greet me for questions (and stickers!!) Or tweet me @giosakti