How Ruby Survives in the Cloud Native World

At RubyKaigi 2018
http://rubykaigi.org/2018/presentations/udzura.html#jun01

Transcript

1. ...in the World of the Cloud-Native Uchio Kondo / GMO Pepabo, Inc. RubyKaigi 2018 How Ruby Survives
2. Software Engineer @ GMO Pepabo Uchio Kondo @udzura R&D/Developper Productivity Team / From Fukuoka https://blog.udzura.jp/
3. Books • Perfect Ruby/Perfect Ruby on Rails
4. OSS Products • yao - Yet Another OpenStack client https://github.com/yaocloud/yao • Haconiwa - mruby on containers http://haconiwa.mruby.org/
5. Community • Fukuoka.rb Meetup @ Fukuoka • Chief Organizer of Fukuoka Regional RubyKaigi 02
6. Splatoon 2 Main Weapon • (Forge) Splattershot Pro • Back to A- in all rules before RubyKaigi ;)
7. What is Cloud Native? §1
8. Refer to CNCF’s definition • CNCF = Cloud Native Computing Foundation • There is the description of cloud-native on CNCF’s website
9. 1. Containerized. 2. Dynamically orchestrated. 3. Microservices oriented. From FAQ - https://www.cncf.io/about/faq/
10. Emerging now but... • As many of you know, containers, orchestrations and microservices are the next key technologies for both web developers and operators. • But... • Why now? Why these technologies suddenly gathered attention at the same time? • How can we optimize or leverage Ruby across these technologies?? We are Rubyists, so we would like to continue utilize Ruby in cloud.
11. To answer these questions • I need to talk about my own experience, I mean case study. • Now I am going to talk about what I have been doing for these 3 years. • It’s “To solve issues of linux containers and orchestrations.”
12. How I decided to create
 my own container runtime §2
13. I was a operator of a SaaS • My company had been running a SaaS: • http://sqale.jp • The service had been using containers (even this service was released on 2012 August!) • Utilizing very early version of LXC (0.7.5...)
14. Got some troubles... • Managing container config was hard • Create config file with ERB via chef... • Cannot change resource on the fly • Changing CPU/memory needs restart • Too old to upgrade • Changelog was too big after LXC 1.0
15. So I decided • To create my own container runtime • Aiming to make containers operation easy • I learned about container internal, which seems super interesting and cool for me • ...With mruby!!! • mruby fits these kind of systems programming
16. The result: Haconiwa • I ended up releasing this and talking in RubyKaigi 2016 about my container: • http://rubykaigi.org/2016/presentations/udzura.html
17. Ruby DSL for Haconiwa Namespaces to enable, mount point and resources (e.g. CPU, memory, IP...) are customizable via DSL! Hooks are available (Signal handlers, lifecycle, interval...)
18. How I released web hosting platform using my own containers with custom stack §3
19. Haconiwa was released!! • Available now on github • I can create my own containers with Ruby DSL and play with it • This was satisfactory to me, but...
20. A “ngx_mruby man” said: • ry: “This product is so cool!!” • ud: “Thanks!” • ry: “But I wonder what is the strong or distinctive point of this container, comparing with other containers like Docker or LXC?”
21. The strong/ distinctive point
22. Thought deeply • Finally I found Haconiwa’s distinctive points are that: • Haconiwa has 2 features: • Composability: Haconiwa can combine many of Linux container functionalities. Namespace, cgroup, capabilityes, seccomp... • Extendibility: Haconiwa has “hooks”, and extends its lifecycle and features with programming using Ruby
23. For “new hosting” • Originally, Haconiwa’s idea was from a hosting service, so Haconiwa’s these features are friendly for a hosting. • Configuration control • Dynamic resource management • Then, my colleague proposed new web hosting service using Haconiwa! • After all, the project was started!!!
24. Required features • Server efficiency/highly integrated architecture: • This directly affects pricing! • Continuous upgrade: • Containers should be kept being “managed”. Library update, security, ... • And ... Availability
25. A new technology is long-awaited
26. “FastContainer” architecture
27. FastContainer • A container lifecycle management architecture: • 1. Container will be up when required, e.g. on the first request • 2. Container will be running until the “Lifetime” comes • 3. After the lifetime, container will be dropped, then restart once the next request comes. • cf. “Phoenix Server Pattern” in IaC
28. Stateless container • Containers states are persisted only on 2 places. • 1. Contents Management DB (CMDB) • Containers’ desired spec and state are on it • Containers are converged to CMDB’s state • 2. Shared storage (e.g. NFS or Managed DB) • User’s contents are on it • Container process itself is stateless
29. 8FC
    1SPYZ 8FC
    3FRVFTU
    %JTQBUDIFS 'BTU$POUBJOFS 3VOUJNF $.%# 1. Get container’s information from CMDB. Then, check container is alive by this info. If not, invoke new container at this time. ❌
30. 8FC
    1SPYZ 8FC
    3FRVFTU
    %JTQBUDIFS $.%# 2. If container is up, just forward the request to its content 'BTU$POUBJOFS 3VOUJNF
31. 8FC
    1SPYZ 8FC
    3FRVFTU
    %JTQBUDIFS $.%# 'BTU$POUBJOFS 3VOUJNF ,JMMFE 3. Kill container after “lifetime”, e.g. 20 minutes. Return to 1.
32. Merits for hosting • Resource is allocated just when required • If there are less accesses, less resource is used • Processes are continuously refreshed • Because lifetime is limited. This prohibit containers from getting too fat • Containers have “host server transparency” • Containers are forced to be immutable by refresh. • So they can be invoked in any host servers
33. Implementation Overview
34. Used technologies • ngx_mruby • mruby-cli for system tools
 (e.g. LVS update by database) • Haconiwa • Core API to control clusters (written in go) • Scheduler for provisioning containers
 (written in go... compatible with sidekiq redis) • Dashboard SPA using Nuxt.js
 (Sorry for “not by rails” :bow: )
35. Sample of FastContainer 1. Gets container’s spec via CMDB 2. Check specific IP and port are listening. If listening, just return these IP:port to nginx, and request will be forwarded 3. If not listening, compose the command to invoke and run it, then wait until the container is up. (Note: this code skips sanitization...)
36. Comparing with k8s stack runc Kubernetes Flannel Calico CRI-O Containerd Kernel syscall Clustered Node Pool Haconiwa ngx_mruby
 based
 service mesh HTTP API
 (Jardin) Bridge + veth CMDB(Postgres) etcd Runtime Orchestration Networking
37. New orchestration stack • We had to re-implement this layer • We wanted very detailed control in resource allocation and container lifecycle management. This was difficult by existing software stack in that time. • Ruby tag (ngx_mruby and Haconiwa) was helpful to implement this stack • because it’s all Ruby!
38. With this architecture, a brand-new hosting was released!
39. How I’m creating a new orchestrator for containers using Ruby §4
40. Service is running, but... • There are a lot of tasks and issues • One of this is about “container efficiency” • We want to provide more user containers using less resource as possible • Imagine that: over 10,000 containers in a server.
 Is this realizable?
41. “10k containers” issue • 10k containers in one host is challenging: • Bridge limit: a bridge can have just 1,024 interfaces!! • Namespace creation speed: • “ip netns add” is slow using older iproute2 • Fat slab cache makes unshare(2) operation slow, ... https://ten-snapon.com/archives/1913
42. I want a “sandbox” for containers
43. Smaller, simpler one • Required features: • Use FastContainer for lifecycle management • Require less roles to deploy (I want just 1 or 2 VMs to setup) • Customizable Hacofiles and rootfs
44. Now I remember... • My company was using a smaller, simpler VM manager than OpenStack for development
45. mizzy/maglica • Simplistic libvirt wrapper, using zeromq for RPC • (BTW, mizzy-san was an chief architect of Sqale...) https://www.slideshare.net/mizzy/maglica-techkayac
46. maglica for containers • Core technologies: zeromq (for simple RPC) and Serverengine • Serverengine: a concise framework for CRuby to build a valid UNIX daemon/Windows service. • It helps us to write a multi process worker server.
47. udzura/marfusha
48. marfusha’s architecture marfd controller marfd subscriber marfd provisioner dispatcher CMDB (Consul based) ⚙ ⚙ Front API Async via zeromq sync Sync rootfs/config files (Hacofiles!) Setup/Invoke via API FastContainer here
49. marfusha’s architecture marfd controller marfd subscriber marfd provisioner dispatcher CMDB (Consul based) ⚙ ⚙ Front API Async via zeromq sync Sync rootfs/config files (Hacofiles!) Setup/Invoke via API FastContainer here Developing!!! https://github.com/udzura/ marfusha
50. Ruby for Cloud-Native §5
51. Cloud-Native seems a must • Containers: • For application and environment portability • Orchestrations: • All operations should be coded • Microservices: • As developers are organized in smaller teams
 (e.g. Spotify’s “tribes”)
52. CNCF incubators: • Ruby product is only one: • Fluentd!! (Great Job)
53. From my experience • Ruby has its own strong points to implement cloud- native platforms: 1. Learning DSL is easier for non-devs 2. Combination of CRuby and mruby
54. 1. DSL’s merit for learning • In cloud-native operations, everything is programmable. • Operators should learn programming languages. But this is not always realistic • Because they aren’t all programmers. • IMO DSL is one of good solutions: • Ruby’s power of “effectivity” is also good to non- programmers to learn and use!
55. c.f. Static format • e.g. YAML is good, but it’s prone to becoming a “high wall”. • ...And is not even “typed”. • Ruby 3’s soft-typing will be helpful for such type of configuration as code
56. 2. CRuby and mruby • We can use both CRuby and mruby in (almost) one grammar. • We can use CRuby in: • Generic server/cli tools programming • And we can use mruby in: • Systems programming!! • If you are required to control middleware or even Linux, you can use mruby to access them easily
57. mruby for systems programming • Easier memory management • Just write object alloc/free functions, then we can pass object management to GC • Simpler C bridging API • Its rule is simpler than other languages based on C • Ecosystem • There are many mruby gems, which are reusable.
58. On the other hand • What I wanted in Ruby were: 1. Better RPC solution 2. Easier way to use many core
59. 1. Better RPC solution • gRPC supports Ruby • But not officially for mruby! • C/C++ is supported, so creating a binging is a kind of solutions. • But this spoils one of gRPC’s merits: • auto-generation of all of RPC code.
60. 2. Use many core • Using current Ruby, the only good way is workers using fork(2) • Going beyond GVL is hard • Multi-process is a classic UNIX way, and reasonable for now • In the future, Guild will be a good solution!!
61. The Future §6
62. Cloud Native’s future • Kubernetes will be a de-facto standard for developers or operators who want to dynamically control containers. • Most of applications will be deployed to server-less platforms, especially applications that will be required to be small and agile.
63. K8s is agnostic to containers • In the near future, Haconiwa would be run under k8s, if we make an adapter that speaks CRI!
 (PR welcome) • Containers other than haconiwa might be created runc rkt railcar CRI-O Containerd CRI is a interfece between container and Kubernetes ... ... (haconiwa adapter?) haconiwa
64. Server-less is also agnostic • A server-less provider just provides “API”. The only thing developers have to learn is this, and then they concentrate their resources to develop applications • This means providers can choose any technologies in the back of API to solve developers' tasks and issues
65. Cloud-Native for Ruby! • Somebody can provide a new function as a service using Ruby and mruby, in which Ruby is fully available! • Because using Ruby is reasonable to solve developers’ issue that “We want to use Ruby in server-less!!” • I imagine one like AWS greengrass • but all in mruby!!
66. We can create yet another cloud-native platform using Ruby/mruby
67. To be continued...
68. Font used in the slide • Sinkin sans: https://www.fontsquirrel.com/fonts/ sinkin-sans • Under Apache License v2.00 • See https://www.fontsquirrel.com/fonts/sinkin-sans#eula