Pathfinder - Building a Container Platform in Ruby Ecosystem

Transcript

1. Building a Container Platform in ruby ecosystem Rubykaigi 2019 @giosakti

2. Hello

3. 名前わギオ

4. 名前わギオ My name is Gio

5. Was the Kanji correct? Sorry I used google translate

6. I come from Jakarta

7. None

8. I work at GO-JEK

9. I work at GO-JEK Anybody knows GO-JEK or ゴ-ジェック?

10. None
11. None

12. Super App.

13. And we ❤ Ruby

14. Ruby is one of 5 most used languages in the

    company

15. I also organized ruby meetups in Jakarta

16. None

17. ruby.id/conf

18. Moving on…

19. Building a Container Platform in ruby ecosystem

20. This session will discuss about an attempt to build container

    platform in ruby/mruby ecosystem, the current situation and lesson-learned that we can discern to improve it further. Further on, for those whom are unfamiliar, this session will also touch a bit about container platform/orchestrator and the generic architecture behind it. So that as a developer, we understand the abstraction that it provides.

21. Background

22. This started early last year...

23. Converging to centralized platform

24. Principles & Considerations Hybrid Infrastructures Explore Containers Seamless Transition Open

    Source

25. One of projects that I involved was Logging

26. github.com/BaritoLog

27. We decided to use LXC to host Barito components

28. Why LXC? 1. Drop-in VM replacement 2. We have cookbooks

    for all the components

29. But, we need a system to manage those LXCs

30. None

31. github.com/pathfinder-cm

32. A container platform.

33. Written in Ruby and Golang

34. But, what exactly is a container platform?

35. But, what exactly is a container platform? (sometimes it is

    also called orchestrator or manager)

36. I found this online

37. http://splasho.com/upgoer5/

38. None

39. Neat.

40. a software that allows us to manage many computers as

    if they were a single big computer and assign jobs into it

41. What it means is that we…

42. Abstract away CPU & RAM

43. Abstract away Storage

44. Abstract away Network

45. The Architecture

46. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI

47. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI

48. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI

49. Data Storage Pathfinder Mono State Server & UI Pathfinder Scheduler

    Container #1 Container #2 Container #3 Pathfinder Agent Container Runtime (LXD, Docker, Containerd, etc) Pathfinder CLI The Memory The Conscious The Brain The Sidecar

50. Pathfinder in Action

51. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Agent The Sidecar Container Runtime Self-registration

52. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder CLI

53. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder CLI

54. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder CLI

55. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain

56. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain

57. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Agent The Sidecar Container Runtime
58. None

59. Pathfinder in Production 1. 50+ worker nodes 2. ~1000 containers

    3. Already live for 6 months+ 4. Serving our logging infrastructures (for now)

60. Pathfinder in Production Roll-out process is still ongoing. Expect at

    least 3-4x the current traffic.

61. Pathfinder in Ruby

62. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI

63. Mruby is particularly interesting 1. Easily create executable 2. Low

    footprint 3. Uniform language

64. Possible challenges Ecosystem?

65. Let's Find Out

66. Writing Pathfinder Agent in mruby

67. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI

68. What the agent does Interact with the state server •

    Do self-registration • Make queries

69. What the agent does Interact with local container daemon •

    Make queries • Execute changes

70. What the agent does Collect and send metrics

71. Let’s discuss the need of each use-case

72. Use case: Interact with the state server • Do self-registration

    • Make queries Need: a way to communicate with REST API server via HTTP/HTTPS.

73. Used: github.com/matsumotory/mruby- simplehttp

74. Use case: Interact with local container daemon • Make queries

    • Execute changes Need: • a way to communicate with REST API server via UNIX socket • Communicate by following LXC daemon (LXD) API documentation

75. Used: github.com/matsumotory/mruby- simplehttp

76. Extracted: github.com/giosakti/mruby-lxd

77. Use case: Collect and send metrics Need: It needs to

    gather system metrics (CPU & RAM utilization at minimum)

78. Extracted: github.com/giosakti/mruby- sysinfo

79. Use case: Build self-contained executable

80. Used: github.com/hone/mruby-cli

81. Writing Pathfinder CLI in mruby

82. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI

83. What the CLI does Allow user to interact with the

    APIs of state server

84. The requirements are similar to agent

85. Extracted: pathfinder-cm/pathfinder-mruby-client

86. Extracted: giosakti/mruby-cmdr

87. 2 other components are already in ruby

88. Pathfinder Mono The state server

89. Pathfinder Scheduler

90. Now

91. Data Storage The Memory Pathfinder Mono State Server & UI

    The Conscious Pathfinder Scheduler The Brain Container #1 Container #2 Container #3 Pathfinder Agent The Sidecar Container Runtime Pathfinder CLI

92. List of Projects - Reusable libraries • giosakti/mruby-lxd Libraries for

    interacting with LXC Daemon (LXD) • pathfinder-cm/mruby-pathfinder-client Libraries for interacting with Pathfinder state server • giosakti/mruby-cmdr Mini frameworks for structuring CLI apps

93. List of Projects – Rewritten Components • pathfinder-cm/pathfinder-agent-mruby Agent that

    will be installed in the worker nodes • pathfinder-cm/pfi-mruby CLI for interacting with Pathfinder state server

94. List of Projects – Existing Components • pathfinder-cm/pathfinder-mono State server

    & scheduler on one repo

95. Wrap-up

96. No Gemfile ☺

97. mruby has almost all the libraries we need to do

    such projects

98. Building the executable for mruby application is fairly straightforward

99. The agent mruby executable is smaller than the go one

    Need to investigate further

100. We have the assertion library for test just fine, but

     no mocking library yet

101. Concurrency on mruby Thread is not there, but fiber is

     there I think

102. Need to explore catching signals for graceful shutdown

103. Documentation of mruby is a bit lacking One area of

     improvement that we can all help

104. Thank You! Come greet me for questions (and stickers!!) Or

     tweet me @giosakti