How Ruby Survives in the Cloud Native World

Slide 1

Slide 1 text

...in the World of the Cloud-Native Uchio Kondo / GMO Pepabo, Inc. RubyKaigi 2018 How Ruby Survives

Slide 2

Slide 2 text

Software Engineer @ GMO Pepabo Uchio Kondo @udzura R&D/Developper Productivity Team / From Fukuoka https://blog.udzura.jp/

Slide 3

Slide 3 text

Books • Perfect Ruby/Perfect Ruby on Rails

Slide 4

Slide 4 text

OSS Products • yao - Yet Another OpenStack client https://github.com/yaocloud/yao • Haconiwa - mruby on containers http://haconiwa.mruby.org/

Slide 5

Slide 5 text

Community • Fukuoka.rb Meetup @ Fukuoka • Chief Organizer of Fukuoka Regional RubyKaigi 02

Slide 6

Slide 6 text

Splatoon 2 Main Weapon • (Forge) Splattershot Pro • Back to A- in all rules before RubyKaigi ;)

Slide 7

Slide 7 text

What is Cloud Native? §1

Slide 8

Slide 8 text

Refer to CNCF’s definition • CNCF = Cloud Native Computing Foundation • There is the description of cloud-native on CNCF’s website

Slide 9

Slide 9 text

1. Containerized. 2. Dynamically orchestrated. 3. Microservices oriented. From FAQ - https://www.cncf.io/about/faq/

Slide 10

Slide 10 text

Emerging now but... • As many of you know, containers, orchestrations and microservices are the next key technologies for both web developers and operators. • But... • Why now? Why these technologies suddenly gathered attention at the same time? • How can we optimize or leverage Ruby across these technologies?? We are Rubyists, so we would like to continue utilize Ruby in cloud.

Slide 11

Slide 11 text

To answer these questions • I need to talk about my own experience, I mean case study. • Now I am going to talk about what I have been doing for these 3 years. • It’s “To solve issues of linux containers and orchestrations.”

Slide 12

Slide 12 text

How I decided to create  my own container runtime §2

Slide 13

Slide 13 text

I was a operator of a SaaS • My company had been running a SaaS: • http://sqale.jp • The service had been using containers (even this service was released on 2012 August!) • Utilizing very early version of LXC (0.7.5...)

Slide 14

Slide 14 text

Got some troubles... • Managing container config was hard • Create config file with ERB via chef... • Cannot change resource on the fly • Changing CPU/memory needs restart • Too old to upgrade • Changelog was too big after LXC 1.0

Slide 15

Slide 15 text

So I decided • To create my own container runtime • Aiming to make containers operation easy • I learned about container internal, which seems super interesting and cool for me • ...With mruby!!! • mruby fits these kind of systems programming

Slide 16

Slide 16 text

The result: Haconiwa • I ended up releasing this and talking in RubyKaigi 2016 about my container: • http://rubykaigi.org/2016/presentations/udzura.html

Slide 17

Slide 17 text

Ruby DSL for Haconiwa Namespaces to enable, mount point and resources (e.g. CPU, memory, IP...) are customizable via DSL! Hooks are available (Signal handlers, lifecycle, interval...)

Slide 18

Slide 18 text

How I released web hosting platform using my own containers with custom stack §3

Slide 19

Slide 19 text

Haconiwa was released!! • Available now on github • I can create my own containers with Ruby DSL and play with it • This was satisfactory to me, but...

Slide 20

Slide 20 text

A “ngx_mruby man” said: • ry: “This product is so cool!!” • ud: “Thanks!” • ry: “But I wonder what is the strong or distinctive point of this container, comparing with other containers like Docker or LXC?”

Slide 21

Slide 21 text

The strong/ distinctive point

Slide 22

Slide 22 text

Thought deeply • Finally I found Haconiwa’s distinctive points are that: • Haconiwa has 2 features: • Composability: Haconiwa can combine many of Linux container functionalities. Namespace, cgroup, capabilityes, seccomp... • Extendibility: Haconiwa has “hooks”, and extends its lifecycle and features with programming using Ruby

Slide 23

Slide 23 text

For “new hosting” • Originally, Haconiwa’s idea was from a hosting service, so Haconiwa’s these features are friendly for a hosting. • Configuration control • Dynamic resource management • Then, my colleague proposed new web hosting service using Haconiwa! • After all, the project was started!!!

Slide 24

Slide 24 text

Required features • Server efficiency/highly integrated architecture: • This directly affects pricing! • Continuous upgrade: • Containers should be kept being “managed”. Library update, security, ... • And ... Availability

Slide 25

Slide 25 text

A new technology is long-awaited

Slide 26

Slide 26 text

“FastContainer” architecture

Slide 27

Slide 27 text

FastContainer • A container lifecycle management architecture: • 1. Container will be up when required, e.g. on the first request • 2. Container will be running until the “Lifetime” comes • 3. After the lifetime, container will be dropped, then restart once the next request comes. • cf. “Phoenix Server Pattern” in IaC

Slide 28

Slide 28 text

Stateless container • Containers states are persisted only on 2 places. • 1. Contents Management DB (CMDB) • Containers’ desired spec and state are on it • Containers are converged to CMDB’s state • 2. Shared storage (e.g. NFS or Managed DB) • User’s contents are on it • Container process itself is stateless

Slide 29

Slide 29 text

8FC 1SPYZ 8FC 3FRVFTU %JTQBUDIFS 'BTU$POUBJOFS 3VOUJNF $.%# 1. Get container’s information from CMDB. Then, check container is alive by this info. If not, invoke new container at this time. ❌

Slide 30

Slide 30 text

8FC 1SPYZ 8FC 3FRVFTU %JTQBUDIFS $.%# 2. If container is up, just forward the request to its content 'BTU$POUBJOFS 3VOUJNF

Slide 31

Slide 31 text

8FC 1SPYZ 8FC 3FRVFTU %JTQBUDIFS $.%# 'BTU$POUBJOFS 3VOUJNF ,JMMFE 3. Kill container after “lifetime”, e.g. 20 minutes. Return to 1.

Slide 32

Slide 32 text

Merits for hosting • Resource is allocated just when required • If there are less accesses, less resource is used • Processes are continuously refreshed • Because lifetime is limited. This prohibit containers from getting too fat • Containers have “host server transparency” • Containers are forced to be immutable by refresh. • So they can be invoked in any host servers

Slide 33

Slide 33 text

Implementation Overview

Slide 34

Slide 34 text

Used technologies • ngx_mruby • mruby-cli for system tools  (e.g. LVS update by database) • Haconiwa • Core API to control clusters (written in go) • Scheduler for provisioning containers  (written in go... compatible with sidekiq redis) • Dashboard SPA using Nuxt.js  (Sorry for “not by rails” :bow: )

Slide 35

Slide 35 text

Sample of FastContainer 1. Gets container’s spec via CMDB 2. Check specific IP and port are listening. If listening, just return these IP:port to nginx, and request will be forwarded 3. If not listening, compose the command to invoke and run it, then wait until the container is up. (Note: this code skips sanitization...)

Slide 36

Slide 36 text

Comparing with k8s stack runc Kubernetes Flannel Calico CRI-O Containerd Kernel syscall Clustered Node Pool Haconiwa ngx_mruby  based  service mesh HTTP API  (Jardin) Bridge + veth CMDB(Postgres) etcd Runtime Orchestration Networking

Slide 37

Slide 37 text

New orchestration stack • We had to re-implement this layer • We wanted very detailed control in resource allocation and container lifecycle management. This was difficult by existing software stack in that time. • Ruby tag (ngx_mruby and Haconiwa) was helpful to implement this stack • because it’s all Ruby!

Slide 38

Slide 38 text

With this architecture, a brand-new hosting was released!

Slide 39

Slide 39 text

How I’m creating a new orchestrator for containers using Ruby §4

Slide 40

Slide 40 text

Service is running, but... • There are a lot of tasks and issues • One of this is about “container efficiency” • We want to provide more user containers using less resource as possible • Imagine that: over 10,000 containers in a server.  Is this realizable?

Slide 41

Slide 41 text

“10k containers” issue • 10k containers in one host is challenging: • Bridge limit: a bridge can have just 1,024 interfaces!! • Namespace creation speed: • “ip netns add” is slow using older iproute2 • Fat slab cache makes unshare(2) operation slow, ... https://ten-snapon.com/archives/1913

Slide 42

Slide 42 text

I want a “sandbox” for containers

Slide 43

Slide 43 text

Smaller, simpler one • Required features: • Use FastContainer for lifecycle management • Require less roles to deploy (I want just 1 or 2 VMs to setup) • Customizable Hacofiles and rootfs

Slide 44

Slide 44 text

Now I remember... • My company was using a smaller, simpler VM manager than OpenStack for development

Slide 45

Slide 45 text

mizzy/maglica • Simplistic libvirt wrapper, using zeromq for RPC • (BTW, mizzy-san was an chief architect of Sqale...) https://www.slideshare.net/mizzy/maglica-techkayac

Slide 46

Slide 46 text

maglica for containers • Core technologies: zeromq (for simple RPC) and Serverengine • Serverengine: a concise framework for CRuby to build a valid UNIX daemon/Windows service. • It helps us to write a multi process worker server.

Slide 47

Slide 47 text

udzura/marfusha

Slide 48

Slide 48 text

marfusha’s architecture marfd controller marfd subscriber marfd provisioner dispatcher CMDB (Consul based) ⚙ ⚙ Front API Async via zeromq sync Sync rootfs/config files (Hacofiles!) Setup/Invoke via API FastContainer here

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Ruby for Cloud-Native §5

Slide 51

Slide 51 text

Cloud-Native seems a must • Containers: • For application and environment portability • Orchestrations: • All operations should be coded • Microservices: • As developers are organized in smaller teams  (e.g. Spotify’s “tribes”)

Slide 52

Slide 52 text

CNCF incubators: • Ruby product is only one: • Fluentd!! (Great Job)

Slide 53

Slide 53 text

From my experience • Ruby has its own strong points to implement cloud- native platforms: 1. Learning DSL is easier for non-devs 2. Combination of CRuby and mruby

Slide 54

Slide 54 text

1. DSL’s merit for learning • In cloud-native operations, everything is programmable. • Operators should learn programming languages. But this is not always realistic • Because they aren’t all programmers. • IMO DSL is one of good solutions: • Ruby’s power of “effectivity” is also good to non- programmers to learn and use!

Slide 55

Slide 55 text

c.f. Static format • e.g. YAML is good, but it’s prone to becoming a “high wall”. • ...And is not even “typed”. • Ruby 3’s soft-typing will be helpful for such type of configuration as code

Slide 56

Slide 56 text

2. CRuby and mruby • We can use both CRuby and mruby in (almost) one grammar. • We can use CRuby in: • Generic server/cli tools programming • And we can use mruby in: • Systems programming!! • If you are required to control middleware or even Linux, you can use mruby to access them easily

Slide 57

Slide 57 text

mruby for systems programming • Easier memory management • Just write object alloc/free functions, then we can pass object management to GC • Simpler C bridging API • Its rule is simpler than other languages based on C • Ecosystem • There are many mruby gems, which are reusable.

Slide 58

Slide 58 text

On the other hand • What I wanted in Ruby were: 1. Better RPC solution 2. Easier way to use many core

Slide 59

Slide 59 text

1. Better RPC solution • gRPC supports Ruby • But not officially for mruby! • C/C++ is supported, so creating a binging is a kind of solutions. • But this spoils one of gRPC’s merits: • auto-generation of all of RPC code.

Slide 60

Slide 60 text

2. Use many core • Using current Ruby, the only good way is workers using fork(2) • Going beyond GVL is hard • Multi-process is a classic UNIX way, and reasonable for now • In the future, Guild will be a good solution!!

Slide 61

Slide 61 text

The Future §6

Slide 62

Slide 62 text

Cloud Native’s future • Kubernetes will be a de-facto standard for developers or operators who want to dynamically control containers. • Most of applications will be deployed to server-less platforms, especially applications that will be required to be small and agile.

Slide 63

Slide 63 text

K8s is agnostic to containers • In the near future, Haconiwa would be run under k8s, if we make an adapter that speaks CRI!  (PR welcome) • Containers other than haconiwa might be created runc rkt railcar CRI-O Containerd CRI is a interfece between container and Kubernetes ... ... (haconiwa adapter?) haconiwa

Slide 64

Slide 64 text

Server-less is also agnostic • A server-less provider just provides “API”. The only thing developers have to learn is this, and then they concentrate their resources to develop applications • This means providers can choose any technologies in the back of API to solve developers' tasks and issues

Slide 65

Slide 65 text

Cloud-Native for Ruby! • Somebody can provide a new function as a service using Ruby and mruby, in which Ruby is fully available! • Because using Ruby is reasonable to solve developers’ issue that “We want to use Ruby in server-less!!” • I imagine one like AWS greengrass • but all in mruby!!

Slide 66

Slide 66 text

We can create yet another cloud-native platform using Ruby/mruby

Slide 67

Slide 67 text

To be continued...

Slide 68

Slide 68 text

Font used in the slide • Sinkin sans: https://www.fontsquirrel.com/fonts/ sinkin-sans • Under Apache License v2.00 • See https://www.fontsquirrel.com/fonts/sinkin-sans#eula