How Ruby Survives in the Cloud Native World

At RubyKaigi 2018
http://rubykaigi.org/2018/presentations/udzura.html#jun01

Transcript

1. ...in the World of the Cloud-Native
   Uchio Kondo / GMO Pepabo, Inc.
   
   RubyKaigi 2018
   How Ruby Survives
   

   View Slide

2. Software Engineer @ GMO Pepabo
   Uchio Kondo @udzura
   R&D/Developper Productivity Team / From Fukuoka
   https://blog.udzura.jp/
   

   View Slide

3. Books
   • Perfect Ruby/Perfect Ruby on Rails
   

   View Slide

4. OSS Products
   • yao - Yet Another OpenStack client
   
   https://github.com/yaocloud/yao
   
   • Haconiwa - mruby on containers
   
   http://haconiwa.mruby.org/
   

   View Slide

5. Community
   • Fukuoka.rb Meetup @ Fukuoka
   
   • Chief Organizer of Fukuoka Regional RubyKaigi 02
   

   View Slide

6. Splatoon 2 Main Weapon
   • (Forge) Splattershot Pro
   
   • Back to A- in all rules before RubyKaigi ;)
   

   View Slide

7. What is Cloud Native?
   §1
   

   View Slide

8. Refer to CNCF’s definition
   • CNCF = Cloud Native Computing Foundation
   
   • There is the description of cloud-native on CNCF’s
   website
   

   View Slide

9. 1. Containerized.
   
   2. Dynamically orchestrated.
   
   3. Microservices oriented.
   From FAQ - https://www.cncf.io/about/faq/
   

   View Slide

10. Emerging now but...
    • As many of you know, containers, orchestrations
    and microservices are the next key technologies for
    both web developers and operators.
    
    • But...
    
    • Why now? Why these technologies suddenly
    gathered attention at the same time?
    
    • How can we optimize or leverage Ruby across
    these technologies?? We are Rubyists, so we
    would like to continue utilize Ruby in cloud.
    

    View Slide

11. To answer these questions
    • I need to talk about my own experience, I mean
    case study.
    
    • Now I am going to talk about what I have been
    doing for these 3 years.
    
    • It’s “To solve issues of linux containers and
    orchestrations.”
    

    View Slide

12. How I decided
    
    to create

    my own container
    runtime
    §2
    

    View Slide

13. I was a operator of a SaaS
    • My company had been running a SaaS:
    
    • http://sqale.jp
    
    • The service had been using containers (even this
    service was released on 2012 August!)
    
    • Utilizing very early version of LXC (0.7.5...)
    

    View Slide

14. Got some troubles...
    • Managing container config was hard
    
    • Create config file with ERB via chef...
    
    • Cannot change resource on the fly
    
    • Changing CPU/memory needs restart
    
    • Too old to upgrade
    
    • Changelog was too big after LXC 1.0
    

    View Slide

15. So I decided
    • To create my own container runtime
    
    • Aiming to make containers operation easy
    
    • I learned about container internal, which seems
    super interesting and cool for me
    
    • ...With mruby!!!
    
    • mruby fits these kind of systems programming
    

    View Slide

16. The result: Haconiwa
    • I ended up releasing this and talking in RubyKaigi
    2016 about my container:
    
    • http://rubykaigi.org/2016/presentations/udzura.html
    

    View Slide

17. Ruby DSL for Haconiwa
    Namespaces to enable, mount point
    and resources (e.g. CPU, memory, IP...)
    are customizable via DSL!
    Hooks are available
    
    (Signal handlers,
    lifecycle, interval...)
    

    View Slide

18. How I released
    
    web hosting platform
    
    using
    
    my own containers
    
    with custom stack
    §3
    

    View Slide

19. Haconiwa was released!!
    • Available now on github
    
    • I can create my own containers with Ruby DSL and
    play with it
    
    • This was satisfactory to me, but...
    

    View Slide

20. A “ngx_mruby man” said:
    • ry: “This product is so cool!!”
    
    • ud: “Thanks!”
    
    • ry: “But I wonder what is the strong or distinctive
    point of this container, comparing with other
    containers like Docker or LXC?”
    

    View Slide

21. The strong/
    
    distinctive point
    

    View Slide

22. Thought deeply
    • Finally I found Haconiwa’s distinctive points are
    that:
    
    • Haconiwa has 2 features:
    
    • Composability: Haconiwa can combine many of
    Linux container functionalities. Namespace,
    cgroup, capabilityes, seccomp...
    
    • Extendibility: Haconiwa has “hooks”, and extends
    its lifecycle and features with programming using
    Ruby
    

    View Slide

23. For “new hosting”
    • Originally, Haconiwa’s idea was from a hosting
    service, so Haconiwa’s these features are friendly
    for a hosting.
    
    • Configuration control
    
    • Dynamic resource management
    
    • Then, my colleague proposed new web hosting
    service using Haconiwa!
    
    • After all, the project was started!!!
    

    View Slide

24. Required features
    • Server efficiency/highly integrated architecture:
    
    • This directly affects pricing!
    
    • Continuous upgrade:
    
    • Containers should be kept being “managed”.
    Library update, security, ...
    
    • And ... Availability
    

    View Slide

25. A new technology
    
    is long-awaited
    

    View Slide

26. “FastContainer”
    architecture
    

    View Slide

27. FastContainer
    • A container lifecycle management architecture:
    
    • 1. Container will be up when required, e.g. on the
    first request
    
    • 2. Container will be running until the “Lifetime”
    comes
    
    • 3. After the lifetime, container will be dropped,
    then restart once the next request comes.
    
    • cf. “Phoenix Server Pattern” in IaC
    

    View Slide

28. Stateless container
    • Containers states are persisted only on 2 places.
    
    • 1. Contents Management DB (CMDB)
    
    • Containers’ desired spec and state are on it
    
    • Containers are converged to CMDB’s state
    
    • 2. Shared storage (e.g. NFS or Managed DB)
    
    • User’s contents are on it
    
    • Container process itself is stateless
    

    View Slide

29. 8FC
    1SPYZ
    8FC
    3FRVFTU
    %JTQBUDIFS
    'BTU$POUBJOFS
    3VOUJNF
    $.%#
    1. Get container’s information from CMDB.
    
    Then, check container is alive by this info.
    
    If not, invoke new container at this time.
    ❌
    

    View Slide

30. 8FC
    1SPYZ
    8FC
    3FRVFTU
    %JTQBUDIFS
    $.%#
    2. If container is
    up, just forward
    the request to
    its content
    'BTU$POUBJOFS
    3VOUJNF
    

    View Slide

31. 8FC
    1SPYZ
    8FC
    3FRVFTU
    %JTQBUDIFS
    $.%#
    'BTU$POUBJOFS
    3VOUJNF
    ,JMMFE
    3. Kill container
    after “lifetime”,
    
    e.g. 20 minutes.
    
    Return to 1.
    

    View Slide

32. Merits for hosting
    • Resource is allocated just when required
    
    • If there are less accesses, less resource is used
    
    • Processes are continuously refreshed
    
    • Because lifetime is limited. This prohibit containers
    from getting too fat
    
    • Containers have “host server transparency”
    
    • Containers are forced to be immutable by refresh.
    
    • So they can be invoked in any host servers
    

    View Slide

33. Implementation
    
    Overview
    

    View Slide

34. Used technologies
    • ngx_mruby
    
    • mruby-cli for system tools

    (e.g. LVS update by database)
    
    • Haconiwa
    
    • Core API to control clusters (written in go)
    
    • Scheduler for provisioning containers

    (written in go... compatible with sidekiq redis)
    
    • Dashboard SPA using Nuxt.js

    (Sorry for “not by rails” :bow: )
    

    View Slide

35. Sample of FastContainer
    1. Gets container’s spec via CMDB
    2. Check specific IP and port are listening.
    
    If listening, just return these IP:port to
    nginx, and request will be forwarded
    3. If not listening, compose the
    command to invoke and run it, then
    wait until the container is up.
    
    (Note: this code skips sanitization...)
    

    View Slide

36. Comparing with k8s stack
    runc
    Kubernetes
    Flannel
    Calico
    CRI-O
    Containerd
    Kernel syscall
    Clustered Node Pool
    Haconiwa
    ngx_mruby

    based

    service mesh
    HTTP API

    (Jardin)
    Bridge +
    
    veth
    CMDB(Postgres)
    etcd
    Runtime Orchestration Networking
    

    View Slide

37. New orchestration stack
    • We had to re-implement this layer
    
    • We wanted very detailed control in resource
    allocation and container lifecycle management. This
    was difficult by existing software stack in that time.
    
    • Ruby tag (ngx_mruby and Haconiwa) was helpful to
    implement this stack
    
    • because it’s all Ruby!
    

    View Slide

38. With this architecture,
    
    a brand-new hosting was
    
    released!
    

    View Slide

39. How I’m creating
    
    a new orchestrator
    
    for containers
    
    using Ruby
    §4
    

    View Slide

40. Service is running, but...
    • There are a lot of tasks and issues
    
    • One of this is about “container efficiency”
    
    • We want to provide more user containers using
    less resource as possible
    
    • Imagine that: over 10,000 containers in a server.

    Is this realizable?
    

    View Slide

41. “10k containers” issue
    • 10k containers in one host is challenging:
    
    • Bridge limit: a bridge can have just 1,024
    interfaces!!
    
    • Namespace creation speed:
    
    • “ip netns add” is slow using older iproute2
    
    • Fat slab cache makes unshare(2) operation
    slow, ...
    https://ten-snapon.com/archives/1913
    

    View Slide

42. I want a “sandbox”
    
    for containers
    

    View Slide

43. Smaller, simpler one
    • Required features:
    
    • Use FastContainer for lifecycle management
    
    • Require less roles to deploy (I want just 1 or 2 VMs
    to setup)
    
    • Customizable Hacofiles and rootfs
    

    View Slide

44. Now I remember...
    • My company was using a smaller, simpler VM
    manager than OpenStack for development
    

    View Slide

45. mizzy/maglica
    • Simplistic libvirt wrapper, using zeromq for RPC
    
    • (BTW, mizzy-san was an chief architect of Sqale...)
    https://www.slideshare.net/mizzy/maglica-techkayac
    

    View Slide

46. maglica for containers
    • Core technologies: zeromq (for simple RPC) and
    Serverengine
    
    • Serverengine: a concise framework for CRuby to
    build a valid UNIX daemon/Windows service.
    
    • It helps us to write a multi process worker server.
    

    View Slide

47. udzura/marfusha
    

    View Slide

48. marfusha’s architecture
    marfd controller marfd subscriber
    marfd provisioner
    dispatcher
    CMDB
    
    (Consul based)
    ⚙ ⚙
    Front
    API
    Async via
    zeromq
    sync
    Sync rootfs/config
    files (Hacofiles!)
    Setup/Invoke
    via API
    FastContainer
    here
    

    View Slide

49. marfusha’s architecture
    marfd controller marfd subscriber
    marfd provisioner
    dispatcher
    CMDB
    
    (Consul based)
    ⚙ ⚙
    Front
    API
    Async via
    zeromq
    sync
    Sync rootfs/config
    files (Hacofiles!)
    Setup/Invoke
    via API
    FastContainer
    here
    Developing!!!
    
    https://github.com/udzura/
    marfusha
    

    View Slide

50. Ruby for Cloud-Native
    §5
    

    View Slide

51. Cloud-Native seems a must
    • Containers:
    
    • For application and environment portability
    
    • Orchestrations:
    
    • All operations should be coded
    
    • Microservices:
    
    • As developers are organized in smaller teams

    (e.g. Spotify’s “tribes”)
    

    View Slide

52. CNCF incubators:
    • Ruby product is only one:
    
    • Fluentd!! (Great Job)
    

    View Slide

53. From my experience
    • Ruby has its own strong points to implement cloud-
    native platforms:
    
    1. Learning DSL is easier for non-devs
    
    2. Combination of CRuby and mruby
    

    View Slide

54. 1. DSL’s merit for learning
    • In cloud-native operations, everything is
    programmable.
    
    • Operators should learn programming languages.
    But this is not always realistic
    
    • Because they aren’t all programmers.
    
    • IMO DSL is one of good solutions:
    
    • Ruby’s power of “effectivity” is also good to non-
    programmers to learn and use!
    

    View Slide

55. c.f. Static format
    • e.g. YAML is good, but it’s prone to becoming a
    “high wall”.
    
    • ...And is not even “typed”.
    
    • Ruby 3’s soft-typing will be helpful for such type of
    configuration as code
    

    View Slide

56. 2. CRuby and mruby
    • We can use both CRuby and mruby in (almost) one
    grammar.
    
    • We can use CRuby in:
    
    • Generic server/cli tools programming
    
    • And we can use mruby in:
    
    • Systems programming!!
    
    • If you are required to control middleware or even
    Linux, you can use mruby to access them easily
    

    View Slide

57. mruby for systems programming
    • Easier memory management
    
    • Just write object alloc/free functions, then we can
    pass object management to GC
    
    • Simpler C bridging API
    
    • Its rule is simpler than other languages based on C
    
    • Ecosystem
    
    • There are many mruby gems, which are reusable.
    

    View Slide

58. On the other hand
    • What I wanted in Ruby were:
    
    1. Better RPC solution
    
    2. Easier way to use many core
    

    View Slide

59. 1. Better RPC solution
    • gRPC supports Ruby
    
    • But not officially for mruby!
    
    • C/C++ is supported, so creating a binging is a kind
    of solutions.
    
    • But this spoils one of gRPC’s merits:
    
    • auto-generation of all of RPC code.
    

    View Slide

60. 2. Use many core
    • Using current Ruby, the only good way is workers
    using fork(2)
    
    • Going beyond GVL is hard
    
    • Multi-process is a classic UNIX way, and reasonable
    for now
    
    • In the future, Guild will be a good solution!!
    

    View Slide

61. The Future
    §6
    

    View Slide

62. Cloud Native’s future
    • Kubernetes will be a de-facto standard for
    developers or operators who want to dynamically
    control containers.
    
    • Most of applications will be deployed to server-less
    platforms, especially applications that will be
    required to be small and agile.
    

    View Slide

63. K8s is agnostic to containers
    • In the near future, Haconiwa would be run under
    k8s, if we make an adapter that speaks CRI!

    (PR welcome)
    
    • Containers other than haconiwa might be created
    runc
    rkt
    railcar
    CRI-O
    Containerd
    CRI is a interfece
    
    between container
    and Kubernetes
    ... ...
    (haconiwa adapter?)
    haconiwa
    

    View Slide

64. Server-less is also agnostic
    • A server-less provider just provides “API”. The only
    thing developers have to learn is this, and then
    they concentrate their resources to develop
    applications
    
    • This means providers can choose any technologies
    in the back of API to solve developers' tasks and
    issues
    

    View Slide

65. Cloud-Native for Ruby!
    • Somebody can provide a new function as a service
    using Ruby and mruby, in which Ruby is fully
    available!
    
    • Because using Ruby is reasonable to solve
    developers’ issue that “We want to use Ruby in
    server-less!!”
    
    • I imagine one like AWS greengrass
    
    • but all in mruby!!
    

    View Slide

66. We can create
    
    yet another
    
    cloud-native platform
    
    using Ruby/mruby
    

    View Slide

67. To be continued...
    

    View Slide

68. Font used in the slide
    • Sinkin sans: https://www.fontsquirrel.com/fonts/
    sinkin-sans
    
    • Under Apache License v2.00
    
    • See https://www.fontsquirrel.com/fonts/sinkin-sans#eula
    

    View Slide