The fastest way to bootstrap Ruby on Rails

by KONDO Uchio

Slide 1

Slide 1 text

The fastest way to bootstrap Ruby on Rails Presentation by Uchio Kondo @

Slide 2

Slide 2 text

Engineer @ GMO Pepabo, Inc. Uchio Kondo / @udzura https://blog.udzura.jp/ * Local Organizer of * Chair of CloudNative Days Fukuoka 2019 * Author of Japanese Ruby Books * Founder of Haconiwa (mruby on container) * 10 y.o. Rubyist * Container Enthusiast * System Programming Novice

Slide 3

Slide 3 text

Living and working in Fukuoka 3 ☕ We’re here! GMO Pepabo Ofﬁce Yakuin, Fukuoka’s Best ☕ Town

Slide 4

Slide 4 text

Fukuoka.rb • @nagachika, @jimlock, @udzura(me) and many helpers/organizers from Fukuoka.rb !! We welcome Rubyists!! 4

Slide 5

Slide 5 text

A Creator of Haconiwa 5

Slide 6

Slide 6 text

A Japanese Script 6 • Or please check @udzura ’s newest tweet #rubykaigia .

Slide 7

Slide 7 text

Introduction to CRIU

Slide 8

Slide 8 text

CRIU (pronounced like screw’s crew) 8 https://criu.org/

Slide 9

Slide 9 text

I’ve sent a patch to CRIU • And I have read many parts of CRIU’s code (almost all in C language) 9

Slide 10

Slide 10 text

Have you heard • About “CRIU” ? 10

Slide 11

Slide 11 text

Have you heard • About “Process” ? 11

Slide 12

Slide 12 text

Processes (for Linux) • Processes are very basic concept on which OS executes tasks/programs. 12

Slide 13

Slide 13 text

CRIU • Using CRIU, We can: • Dump processes information info “image” ﬁles • Restore processes from dumped image ﬁles • CRIU stands for “Checkpoint and Restore In Userspace” • CRIU is a tool for processes, but it can be applied to containers C/R. 13

Slide 14

Slide 14 text

CRIU Internal: • CRIU uses /proc ﬁlesystems, cgroup attributes and stats, TCP checkpoint feature after Kernel 3.5, and many syscalls to get process info 14

Slide 15

Slide 15 text

Where CRIU is useful • 1) Migration • 2) Making apps fast in bootstrap 15

Slide 16

Slide 16 text

Migration example • cf. KVM’s live migration 16 Dump! Copy Restore! Reboot Host A Container Image Same Container Another Host

Slide 17

Slide 17 text

Faster bootstrap example • When you have a heavier app in bootstrap, like Jenkins, Redmine, or legacy monolithic Rails app, you will hesitate to reboot these. • e.g. Jenkins requires about 5 sec to boot  (using docker with downloaded image). 17

Slide 18

Slide 18 text

Ahead-of-time process dump • CRIU can skip initialization process. • e.g. read() scripts and code evaluation. • Using CRIU program will be loaded directory from “memory dump” 18

Slide 19

Slide 19 text

Ahead-of-time process dump 19 Boot from scratch Boot using predump Memory Eval Direct Load Initialized

Slide 20

Slide 20 text

Example of CRIU utilization in Ruby on Rails

Slide 21

Slide 21 text

Detailed CRIU examples • Apply CRIU to Ruby on Rails applications!! 21

Slide 22

Slide 22 text

Environments • We have a application platform using containers: 22 Application Containers Proxy Routing by vhost Requests Deployment by users

Slide 23

Slide 23 text

Automated host migration • When one host is crashed, the new container is invoked automatically in another host. But this requires RESTART of a container... 23 Crash! Proxy Requests Restart and Redirect Accesses

Slide 24

Slide 24 text

Can we reduce restart time? • CRIU looked like a very good choice for this system. 24

Slide 25

Slide 25 text

Preliminary experiment • Created instances on vagrant as below: • Checking response time, this includes restart time 25

Slide 26

Slide 26 text

How to measure boot time 26 ngx_mruby Haconiwa Containers On the next request, restart an application container and forward access to this container in one transaction Force to stop by operation Haconiwa

Slide 27

Slide 27 text

The response time is composed of 27 Normal Container Response First Time Response with Boot Middleware work Middleware work App response App response Container bootstrap process

Slide 28

Slide 28 text

Result 28 Boot from scratch CRIU

Slide 29

Slide 29 text

Experiment on staging • Deployed simple Welcome Rails application to staging • Ran apache bench and plot response times • Invoke relocation between hosts, then check reboot time. 29

Slide 30

Slide 30 text

Experiments on staging 30 Relocate while the bench is running Proxy Then boot and serve

Slide 31

Slide 31 text

Result #1 • Without any technologies to fasten bootstrap(no bootsnap/no CRIU) 31

Slide 32

Slide 32 text

Result #2 • Activated bootsnap, but no CRIU utilization 32

Slide 33

Slide 33 text

Result #3 • Images were copied to all hosts, and utilized CRIU on bootstrap 33

Slide 34

Slide 34 text

Result of all 34

Slide 35

Slide 35 text

FYI: Scale out using CRIU • We can restore and spawn as many containers as you like from one CRIU's container image • This means CRIU can reduce costs of containers scale-out 35 Proxy Host

Slide 36

Slide 36 text

A tool to deploy applications fast - Grenadine

Slide 37

Slide 37 text

OK, CRIU is so effective for “container” bootstrapping. 37

Slide 38

Slide 38 text

But I guess you are thinking: • “Our application is deployed to VMs, NOT containers” 38

Slide 39

Slide 39 text

Migration VM->container is hard • Many aspects of system characteristics are varying between VMs and containers • VMs has an init process • daemons such as sshd and rsyslogd • a full-stack ﬁlesystem to manage whole operation system • ... all of which containers are usually omitting 39

Slide 40

Slide 40 text

Is there a tool to introduce checkpoint/restorte for “normal”, historical VMs? 40

Slide 41

Slide 41 text

Yes, there is. • I named the tool “Grenadine”, which is written in mruby for 41

Slide 42

Slide 42 text

Let me show you a DEMO. 42

Slide 43

Slide 43 text

The internal of Grenadine Advanced Topic!!

Slide 44

Slide 44 text

Warning: • Following talk is about Linux container runtimes implementation • Requires some of Linux kernel's container feature • But I’ll show you (m)ruby codes, which I guess you can read :) 44

Slide 45

Slide 45 text

What grenadine daemon do indeed • grenadine daemon creates a minimalistic container internally. 45 Grenadine daemon Application Minimalistic container described below App App wait() We’re here!

Slide 46

Slide 46 text

How to write a container in Ruby pid = Process.fork do  isolate_namespace_using_unshare(flag) assign_to_cgroup(name) if enabled chroot_or_pivot_root(to: rootfs) # Doing other options exec(“/path/to/command”, “--opt1”, “--opt2”) end status = Process.waitpid2(pid) # will block and wait() puts status • NOTE: program should be single-threaded, so mruby is better 46

Slide 47

Slide 47 text

Grenadine unshare-s only PID Namespace and Mount Namespace • To avoid PID conﬂict: 47 12345 12347 12351 12345 12347 12351 12345 is Already taken! <- -> 1 6 4 1 6 4 No existing PID Root namespace Using isolated namespace Restore

Slide 48

Slide 48 text

48 clone(CLONE_NEWNS| CLONE_NEWPID) wait() from Parent Re-mount /proc in new root

Slide 49

Slide 49 text

“Working with Unix Processes” 49 ʮͳΔ΄ͲUnixϓϩηεʯ

Slide 50

Slide 50 text

Grenadine requires no additional root ﬁlesystem • Grenadine creates ad-hoc root ﬁlesystem using bind-mount 50 / /run/gren/aaaa-bbbb /tmp /run/gren/aaaa-bbbb/tmp /var/log /run/gren/aaaa-bbbb/var/log /... /run/gren/aaaa-bbbb/... Bind-mount pivot_root !!

Slide 51

Slide 51 text

51 Generates new root For bind-mount dest Bind-mount “/“ and any important directories Mount namespace is isolated, so it is invisible from host

Slide 52

Slide 52 text

To make checkpoint easily • The processes Grenadine creates look like a normal, VM-based application daemon • Internally some of attributes are unshared • To allow CRIU to create checkpoints easily • Grenadine is aimed that it can be introduced into historical VM-based application environments with least changes 52

Slide 53

Slide 53 text

Toward “Cloud Native”...

Slide 54

Slide 54 text

Tha age of Cloud Native • Now is the time everything becomes containerized, orchestrated and microservice-based... 54

Slide 55

Slide 55 text

Grenadine in the future • Grenadine can be used for historical VMs now • I want people to use Grenadine as a checkpoint/restore tool for Kubernetes or other container-based systems • It is cloud-native middleware!! (Like ones on CNCF incubators) 55

Slide 56

Slide 56 text

Please try Grenadine • Current Grenadine have great chemistry with smaller applications • But soon it can be applied with larger apps :) • I will release rpm or deb ﬁle - as soon as possible. :( 56

Slide 57

Slide 57 text

Enjoy using next generation kernel features! 57

Slide 58

Slide 58 text

Thanks!!

Slide 59

Slide 59 text

References • Papers: • দຊ྄հ, ۙ౻Ӊஐ࿕, CRIUΛར༻ͨ͠HTTPϦΫΤετ୯ҐͰίϯςφ Λ࠶഑ஔͰ͖Δ௿ίετͰߴ଎ͳεέδϡʔϦϯάख๏, ৘ใॲཧֶձ ݚڀใࠂΠϯλʔωοτͱӡ༻ٕज़ʢIOTʣ, No.2019-IOT-44, Vol.21, pp.1-8, 2018೥3݄. 59

Slide 60

Slide 60 text

References • Websites: • CRIU.org https://criu.org/Main_Page • Introduction to CRIU https://speakerdeck.com/udzura/introduction-to- criu • Grenadine on GitHub: https://github.com/udzura/grenadine • Japanese article: https://udzura.hatenablog.jp/entry/ 2019/03/07/202615 60