mruby: a packaging story filled with freedom - eurucamp 2015

Ruby is a great language for building CLIs. Libraries such as Thor, GLI, or even OptionParser make creating command line applications even easier. Many tools we use everyday are built upon these libraries such as `chef`, the heroku toolbelt, @sferik's twitter client `t`, and of course the `rails` command line tool.

Building a CLI in Ruby is so easy, however distributing your tool to end users is often the most challenging part. We often run into problems because the Ruby runtime is a dependency and it's not always available or compatible with our applications. Many of the solutions that exist are problematic, such as packaging the full runtime as part of the application. We are starting to see many people switching to Go for it's portability like Vagrant, CloudFoundry, and `hub`, the Github command-line tool.

But there's still hope! In 2012, Matz released a lightweight embeddable ruby called mruby which is designed for being embedded in other compiled languages. mruby: a packaging story filled with freedom. In this talk we'll look how we can write CLI tools using mruby to produce a self-contained binary that can be shipped to end users.

Transcript

1. mruby A packaging story filled with freedom
2. Today is not Friday...
3. But I believe in hugs everyday
4. #rubykaraoke
5. Terence Lee @hone02
6. Austin, TX
7. 23.10.2015
9. Packaging in Ruby
10. story of the heroku toolbelt
11. gem install heroku
12. Requires Ruby to be installed
13. Can't guarantee same version of ruby locally
15. Packaging was a nightmare
16. hk
18. require in ruby is slow
20. Traveling Ruby • no need to rewrite your app • packages your app as a directory • runtime packaging is taken care of • can leverage rubygems ecosystem • native gems limited to precompiled exts
21. No great packaging story
23. sferik
24. I want to build things in Ruby
25. Making Packaging Possible
26. mruby-cli http://github.com/hone/mruby-cli
27. writing in ruby
28. ├── [4.0K] mrblib │ ├── [ 697] cli.rb │ ├── [ 424] help.rb │ ├── [ 53] mruby-cli.rb │ ├── [ 403] option.rb │ ├── [1023] options.rb │ ├── [7.8K] setup.rb │ ├── [ 238] util.rb │ └── [ 206] version.rb
29. performance is a feature
30. MRI # hello_world.rb puts 'Hello World'
31. MRI # hello_world.rb puts 'Hello World' $ time ruby hello.rb Hello World real 0m0.041s user 0m0.028s sys 0m0.008s
32. mruby-cli # mrblib/hello.rb def __main__(argv) puts "Hello World" end
33. mruby-cli # mrblib/hello.rb def __main__(argv) puts "Hello World" end $ time mruby/build/host/bin/hello Hello World real 0m0.003s user 0m0.000s sys 0m0.000s
34. No require
35. a single binary for every major platform
36. $ docker-compose run compile Build summary: ================================================ Config Name: host Output Directory: build/host Binaries: mrbc Included Gems: mruby-print - standard print/puts/p mruby-sprintf - standard Kernel#sprintf method mruby-time - standard Time class mruby-io mruby-mtest hello - hello - Binaries: hello mruby-compiler - mruby compiler library mruby-bin-mrbc - mruby compiler executable ================================================
37. ================================================ Config Name: x86_64-apple-darwin14 Output Directory: build/x86_64-apple-darwin14 Included Gems: mruby-print - standard print/puts/p mruby-sprintf - standard Kernel#sprintf method mruby-time - standard Time class mruby-io mruby-mtest hello - hello - Binaries: hello ================================================
38. ================================================ Config Name: x86_64-w64-mingw32 Output Directory: build/x86_64-w64-mingw32 Included Gems: mruby-print - standard print/puts/p mruby-sprintf - standard Kernel#sprintf method mruby-time - standard Time class mruby-io mruby-mtest hello - hello - Binaries: hello ================================================
39. $ ls -nlh mruby-cli | awk '{print $5}' 421K
40. simple setup
41. Docker FROM hone/mruby-cli
42. Hello World Example $ mruby-cli --setup hello
43. Hello World Example $ mruby-cli --setup hello $ cd hello
44. Hello World Example $ mruby-cli --setup hello $ cd hello $ docker-compose run compile
45. Hello World Example $ mruby-cli --setup hello $ cd hello $ docker-compose run compile $ docker-compose run shell # mruby/build/host/bin/hello
46. Hello World Example $ mruby-cli --setup hello $ cd hello $ docker-compose run compile $ docker-compose run shell # mruby/build/host/bin/hello Hello World
47. Hello World Example $ mruby/bin/mruby-cli --setup hello create .gitignore create mrbgem.rake create build_config.rb create Rakefile create Dockerfile create docker-compose.yml create tools/ create tools/hello/ create tools/hello/hello.c create mrblib/ create mrblib/hello.rb create bintest/ create bintest/hello.rb create test/ create test/test_hello.rb
48. MRuby
49. mruby is the lightweight implementation of the Ruby language complying with part of the ISO standard. mruby can be linked and embedded within your application.
50. used in the wild https://github.com/ximenzi/openwrt-mruby
51. Contributors
52. Differences with MRI
53. no built in File Socket I/O
54. not threadsafe
55. no thread no fork
56. subset of Ruby 2.1
57. procs blocks DHH freedom patching meta programming literals
58. Executing mruby code
59. Interpret (*.rb) $ mruby/bin/mruby -e "puts 'Hello World'" Hello World $ mruby/bin/mruby hello_world.rb Hello World
60. Interactive (mirb) $ mruby/bin/mirb mirb - Embeddable Interactive Ruby Shell > puts 'Hello World' Hello World => nil
61. bytecode (*.mrb) $ mruby/bin/mrbc hello_world.rb
62. bytecode (*.mrb) $ mruby/bin/mrbc hello_world.rb $ ls hello_world.* hello_world.rb hello_world.mrb
63. bytecode (*.mrb) $ mruby/bin/mrbc hello_world.rb $ ls hello_world.* hello_world.rb hello_world.mrb $ mruby/bin/mruby -b hello_world.mrb Hello World
64. Binary C Code (*.c) $ mruby/bin/mrbc -o hello_world.c - Bhello_world hello_world.rb $ cat hello_world.c /* dumped in little endian order. use `mrbc -E` option for big endian CPU. */ #include <stdint.h> const uint8_t #if defined __GNUC__ __attribute__((aligned(4))) #elif defined _MSC_VER __declspec(align(4)) #endif hello_world[] = { 0x45,0x54,0x49,0x52,0x30,0x30,0x30,0x33,0xcb,0xa4,0x00, 0x00,0x00,0x65,0x4d,0x41,...}; This is an FYI No need to read this C code
65. int main(void) { mrb_state *mrb = mrb_open(); mrbc_context *c; mrb_value v; c = mrbc_context_new(mrb); mrb_load_irep_cxt(mrb, hello_world, c); mrbc_context_free(mrb, c); return 0; }
66. int main(void) { mrb_state *mrb = mrb_open(); mrbc_context *c; mrb_value v; c = mrbc_context_new(mrb); mrb_load_irep_cxt(mrb, hello_world, c); mrbc_context_free(mrb, c); return 0; }
67. tools/hello/hello.c #include <stdlib.h> #include <stdio.h> /* Include the mruby header */ #include <mruby.h> #include <mruby/array.h> int main(int argc, char *argv[]) { mrb_state *mrb = mrb_open(); mrb_value ARGV = mrb_ary_new_capa(mrb, argc); int i; int return_value; for (i = 0; i < argc; i++) { mrb_ary_push(mrb, ARGV, mrb_str_new_cstr(mrb, argv[i])); } mrb_define_global_const(mrb, "ARGV", ARGV); // call __main__(ARGV) mrb_funcall(mrb, mrb_top_self(mrb), "__main__", 1, ARGV); return_value = EXIT_SUCCESS; if (mrb->exc) { mrb_print_error(mrb); return_value = EXIT_FAILURE; } mrb_close(mrb); return return_value; } This is an FYI No need to read this C code
68. # mrblib/hello.rb def __main__(argv) puts "Hello World" end
69. mrblib ├── [4.0K] mrblib │ ├── [ 697] cli.rb │ ├── [ 424] help.rb │ ├── [ 53] mruby-cli.rb │ ├── [ 403] option.rb │ ├── [1023] options.rb │ ├── [7.8K] setup.rb │ ├── [ 238] util.rb │ └── [ 206] version.rb
70. Rakefile $ rake -T rake all # build all targets... rake clean # clean all built.. rake compile # compile all the binaries rake release # generate release tarballs rake test # run all tests rake test:bintest # run integration tests rake test:mtest # run mruby & unit tests
71. build_config.rb MRuby::Build.new do |conf| toolchain :gcc conf.enable_bintest gem_config(conf) end
72. build_config.rb MRuby::CrossBuild.new('x86_64-apple-darwin14') do |conf| toolchain :clang [conf.cc, conf.linker].each do |cc| cc.command = 'x86_64-apple-darwin14-clang' end conf.cxx.command = 'x86_64-apple-darwin14-clang++' conf.archiver.command = 'x86_64-apple-darwin14-ar' conf.build_target = 'x86_64-pc-linux-gnu' conf.host_target = 'x86_64-apple-darwin14' conf.build_mrbtest_lib_only gem_config(conf) end
73. $ docker-compose run compile Build summary: ================================================ Config Name: host Output Directory: build/host Binaries: mrbc Included Gems: mruby-print - standard print/puts/p mruby-sprintf - standard Kernel#sprintf method mruby-time - standard Time class mruby-io mruby-mtest hello - hello - Binaries: hello mruby-compiler - mruby compiler library mruby-bin-mrbc - mruby compiler executable ================================================
74. mrbgem.rake MRuby::Gem::Specification.new('hello') do |spec| spec.license = 'MIT' spec.authors = ['Terence Lee'] spec.summary = 'hello world' spec.bins = ['hello'] spec.add_dependency 'mruby-print', :core => 'mruby- print' spec.add_dependency 'mruby-mtest', :mgem => 'mruby- mtest' spec.add_dependency 'mruby-yaml', :github => 'hone/mruby-yaml' end
75. mrbgem.rake MRuby::Gem::Specification.new('hello') do |spec| spec.license = 'MIT' spec.authors = ['Terence Lee'] spec.summary = 'hello world' spec.bins = ['hello'] spec.add_dependency 'mruby-print', :core => 'mruby- print' spec.add_dependency 'mruby-mtest', :mgem => 'mruby- mtest' spec.add_dependency 'mruby-yaml', :github => 'hone/mruby-yaml' end
76. core mrbgems ├── [4.0K] mruby-array-ext ├── [4.0K] mruby-bin-debugger ├── [4.0K] mruby-bin-mirb ├── [4.0K] mruby-bin-mrbc ├── [4.0K] mruby-bin-mruby ├── [4.0K] mruby-bin-mruby-config ├── [4.0K] mruby-bin-strip ├── [4.0K] mruby-compiler ├── [4.0K] mruby-enumerator
77. mrbgem.rake MRuby::Gem::Specification.new('hello') do |spec| spec.license = 'MIT' spec.authors = ['Terence Lee'] spec.summary = 'hello world' spec.bins = ['hello'] spec.add_dependency 'mruby-print', :core => 'mruby- print' spec.add_dependency 'mruby-mtest', :mgem => 'mruby- mtest' spec.add_dependency 'mruby-yaml', :github => 'hone/mruby-yaml' end
78. mgem list
79. mrbgem.rake MRuby::Gem::Specification.new('hello') do |spec| spec.license = 'MIT' spec.authors = ['Terence Lee'] spec.summary = 'hello world' spec.bins = ['hello'] spec.add_dependency 'mruby-print', :core => 'mruby- print' spec.add_dependency 'mruby-mtest', :mgem => 'mruby- mtest' spec.add_dependency 'mruby-yaml', :github => 'hone/mruby-yaml' end
80. build_config.rb MRuby::Build.new do |conf| toolchain :gcc conf.enable_bintest gem_config(conf) end
81. build_config.rb def gem_config(conf) # be sure to include this gem conf.gem File.expand_path(File. dirname(__FILE__)) conf.gem :github => 'hone/mruby-yaml' end
82. Testing
83. mtest module MRubyCLI class TestUtil < MTest::Unit::TestCase def test_camelize assert_equal "GoodNightMoon", Util.camelize ("good_night_moon") assert_equal "FooBarBaz", Util.camelize("foo- bar-baz") end end end MTest::Unit.new.run
84. mtest (cont.) $ docker-compose run mtest >>> Test host <<< mrbtest - Embeddable Ruby Test .....S... # Running tests: .. Finished tests in 0.000305s, 6557.3770 tests/s, 26229.5082 assertions/s. 2 tests, 8 assertions, 0 failures, 0 errors, 0 skips
85. bintest require 'open3' require 'tmpdir' BIN_PATH = File.join(File.dirname(__FILE__), ".. /mruby/bin/hello") assert('setup') do output, status = Open3.capture2("#{BIN_PATH}") assert_true status.success?, "Process did not exit cleanly" assert_include output, "Hello World" end
86. bintest (cont.) $ docker-compose run bintest . Total: 1 OK: 1 KO: 0 Crash: 0 Time: 0.01 seconds
87. Gotchas
88. Can't leverage Rubygems
89. mrbgems that have native extensions need to support cross compiling
90. Roadmap
91. Releases $ docker-compose run release Creating releases/v0.0.1/hello-0.0.1-x86_64-pc-linux-gnu.tgz releases/v0.0.1/hello-0.0.1-i686-pc-linux-gnu.tgz releases/v0.0.1/hello-0.0.1-x86_64-apple-darwin14.tgz releases/v0.0.1/hello-0.0.1-i386-apple-darwin14.tgz releases/v0.0.1/hello-0.0.1-x86_64-w64-mingw32.tgz releases/v0.0.1/hello-0.0.1-i686-w64-mingw32.tgz releases/v0.0.1/hello-0.0.1.tgz
92. deb
93. rpm
94. dmg
95. windows installer
96. Upgrading $ docker-compose upgrade conflict Rakefile [r]eplace, [d]iff, [s]kip? d 61c64 < task :test => ['test:bintest', 'test:mtest'] --- > task :test => ["test:mtest", "test:bintest"] 66,102d68 < end < < desc "generate a release tarball" < task :release do < require 'tmpdir' < require 'fileutils' < require_relative 'mrblib/version'
97. Per Architecture Compiling $ docker-compose run compile x86_64- apple-darwin14
98. Curated Set of mrbgems
99. Upstream Cross Compile Toolchains MRuby::CrossBuild.new('64-bit OS X') do |conf| toolchain :x86_64_apple_darwin14 conf.build_mrbtest_lib_only gem_config(conf) end
100. Documentation Support
101. mruby-cli & you
102. Young Community
103. Building a CLI App • download the mruby-cli binary • install Docker/docker-compose • mruby-cli --setup foo • docker-compose run compile • modify, recompile
104. Thank You @hone02 https://github.com/hone/mruby-cli