orchestrations and microservices are the next key technologies for both web developers and operators. • But... • Why now? Why these technologies suddenly gathered attention at the same time? • How can we optimize or leverage Ruby across these technologies?? We are Rubyists, so we would like to continue utilize Ruby in cloud.
my own experience, I mean case study. • Now I am going to talk about what I have been doing for these 3 years. • It’s “To solve issues of linux containers and orchestrations.”
had been running a SaaS: • http://sqale.jp • The service had been using containers (even this service was released on 2012 August!) • Utilizing very early version of LXC (0.7.5...)
Create config file with ERB via chef... • Cannot change resource on the fly • Changing CPU/memory needs restart • Too old to upgrade • Changelog was too big after LXC 1.0
• Aiming to make containers operation easy • I learned about container internal, which seems super interesting and cool for me • ...With mruby!!! • mruby fits these kind of systems programming
cool!!” • ud: “Thanks!” • ry: “But I wonder what is the strong or distinctive point of this container, comparing with other containers like Docker or LXC?”
that: • Haconiwa has 2 features: • Composability: Haconiwa can combine many of Linux container functionalities. Namespace, cgroup, capabilityes, seccomp... • Extendibility: Haconiwa has “hooks”, and extends its lifecycle and features with programming using Ruby
hosting service, so Haconiwa’s these features are friendly for a hosting. • Configuration control • Dynamic resource management • Then, my colleague proposed new web hosting service using Haconiwa! • After all, the project was started!!!
will be up when required, e.g. on the first request • 2. Container will be running until the “Lifetime” comes • 3. After the lifetime, container will be dropped, then restart once the next request comes. • cf. “Phoenix Server Pattern” in IaC
places. • 1. Contents Management DB (CMDB) • Containers’ desired spec and state are on it • Containers are converged to CMDB’s state • 2. Shared storage (e.g. NFS or Managed DB) • User’s contents are on it • Container process itself is stateless
• If there are less accesses, less resource is used • Processes are continuously refreshed • Because lifetime is limited. This prohibit containers from getting too fat • Containers have “host server transparency” • Containers are forced to be immutable by refresh. • So they can be invoked in any host servers
LVS update by database) • Haconiwa • Core API to control clusters (written in go) • Scheduler for provisioning containers (written in go... compatible with sidekiq redis) • Dashboard SPA using Nuxt.js (Sorry for “not by rails” :bow: )
Check specific IP and port are listening. If listening, just return these IP:port to nginx, and request will be forwarded 3. If not listening, compose the command to invoke and run it, then wait until the container is up. (Note: this code skips sanitization...)
• We wanted very detailed control in resource allocation and container lifecycle management. This was difficult by existing software stack in that time. • Ruby tag (ngx_mruby and Haconiwa) was helpful to implement this stack • because it’s all Ruby!
tasks and issues • One of this is about “container efficiency” • We want to provide more user containers using less resource as possible • Imagine that: over 10,000 containers in a server. Is this realizable?
challenging: • Bridge limit: a bridge can have just 1,024 interfaces!! • Namespace creation speed: • “ip netns add” is slow using older iproute2 • Fat slab cache makes unshare(2) operation slow, ... https://ten-snapon.com/archives/1913
and Serverengine • Serverengine: a concise framework for CRuby to build a valid UNIX daemon/Windows service. • It helps us to write a multi process worker server.
(Consul based) ⚙ ⚙ Front API Async via zeromq sync Sync rootfs/config files (Hacofiles!) Setup/Invoke via API FastContainer here Developing!!! https://github.com/udzura/ marfusha
environment portability • Orchestrations: • All operations should be coded • Microservices: • As developers are organized in smaller teams (e.g. Spotify’s “tribes”)
is programmable. • Operators should learn programming languages. But this is not always realistic • Because they aren’t all programmers. • IMO DSL is one of good solutions: • Ruby’s power of “effectivity” is also good to non- programmers to learn and use!
and mruby in (almost) one grammar. • We can use CRuby in: • Generic server/cli tools programming • And we can use mruby in: • Systems programming!! • If you are required to control middleware or even Linux, you can use mruby to access them easily
write object alloc/free functions, then we can pass object management to GC • Simpler C bridging API • Its rule is simpler than other languages based on C • Ecosystem • There are many mruby gems, which are reusable.
not officially for mruby! • C/C++ is supported, so creating a binging is a kind of solutions. • But this spoils one of gRPC’s merits: • auto-generation of all of RPC code.
good way is workers using fork(2) • Going beyond GVL is hard • Multi-process is a classic UNIX way, and reasonable for now • In the future, Guild will be a good solution!!
for developers or operators who want to dynamically control containers. • Most of applications will be deployed to server-less platforms, especially applications that will be required to be small and agile.
Haconiwa would be run under k8s, if we make an adapter that speaks CRI! (PR welcome) • Containers other than haconiwa might be created runc rkt railcar CRI-O Containerd CRI is a interfece between container and Kubernetes ... ... (haconiwa adapter?) haconiwa
“API”. The only thing developers have to learn is this, and then they concentrate their resources to develop applications • This means providers can choose any technologies in the back of API to solve developers' tasks and issues
as a service using Ruby and mruby, in which Ruby is fully available! • Because using Ruby is reasonable to solve developers’ issue that “We want to use Ruby in server-less!!” • I imagine one like AWS greengrass • but all in mruby!!
udzura
napplecomputer
yukima0707
cybozuinsideout
line_developers
kazamori
kargo113
tillrohrmann
oracle4engineer
mii3king
toranoana
akatsuki174
yuki_saito
aarron
tmm1
reverentgeek
brettharned
maltzj
sachag
bkeepers
tanoku
jakevdp
yeseniaperezcruz
keithpitt
davidbonilla
