mruby: a packaging story filled with freedom - Rocky Mountain Ruby 2015

Ruby is a great language for building CLIs. Libraries such as Thor, GLI, or even OptionParser make creating command line applications even easier. Many tools we use everyday are built upon these libraries such as `chef`, the heroku toolbelt, @sferik’s twitter client `t`, and of course the `rails` command line tool.

Building a CLI in Ruby is so easy, however distributing your tool to end users is often the most challenging part. We often run into problems because the Ruby runtime is a dependency and it’s not always available or compatible with our applications. Many of the solutions that exist are problematic, such as packaging the full runtime as part of the application. We are starting to see many people switching to Go for it’s portability like Vagrant, CloudFoundry, and `hub`, the Github command-line tool.

But there’s still hope! In 2012, Matz released a lightweight embeddable ruby called mruby which is designed for being embedded in other compiled languages. mruby: a packaging story filled with freedom. In this talk we’ll look how we can write CLI tools using mruby to produce a self-contained binary that can be shipped to end users.

mruby: a packaging story filled with freedom - Madison+ Ruby 2015 by hone

Published August 21, 2015 in Programming

Ruby is a great language for building CLIs. Libraries such as Thor, GLI, or even OptionParser make creating command line applications even easier. Many tools we use everyday are built upon these libraries such as `chef`, the heroku toolbelt, @sferik’s twitter client `t`, and of course the `rails` command line tool.

Building a CLI in Ruby is so easy, however distributing your tool to end users is often the most challenging part. We often run into problems because the Ruby runtime is a dependency and it’s not always available or compatible with our applications. Many of the solutions that exist are problematic, such as packaging the full runtime as part of the application. We are starting to see many people switching to Go for it’s portability like Vagrant, CloudFoundry, and `hub`, the Github command-line tool.

But there’s still hope! In 2012, Matz released a lightweight embeddable ruby called mruby which is designed for being embedded in other compiled languages. mruby: a packaging story filled with freedom. In this talk we’ll look how we can write CLI tools using mruby to produce a self-contained binary that can be shipped to end users.

Transcript

1. mruby A packaging story filled with freedom
2. Terence Lee @hone02
3. Austin, TX
7. Tipsy Chicken grilled chicken grilled corn green chiles baby spinach maple bacon bourbon marmalade cheddar cheese chipotle ranch
8. 10.23.2015
9. #rubykaraoke
13. Packaging in Ruby
14. story of the heroku toolbelt
15. gem install heroku
16. Requires Ruby to be installed
17. Can't guarantee same version of ruby locally
19. Packaging was a nightmare
20. hk
22. require in ruby is slow
25. Traveling Ruby • no need to rewrite your app • packages your app as a directory • runtime packaging is taken care of • can leverage rubygems ecosystem • native gems limited to precompiled exts
26. sferik
27. No great packaging story
28. I want to build things in Ruby
29. Making Packaging Possible
30. mruby-cli http://github.com/hone/mruby-cli
31. writing in ruby
32. mrblib ├── [4.0K] mrblib │ ├── [4.0K] mruby-cli │ │ ├── [ 697] cli.rb │ │ ├── [ 424] help.rb │ │ ├── [ 403] option.rb │ │ ├── [1022] options.rb │ │ ├── [8.0K] setup.rb │ │ ├── [ 238] util.rb │ │ └── [ 206] version.rb │ └── [ 53] mruby-cli.rb
33. performance is a feature
34. MRI # hello_world.rb puts 'Hello World'
35. MRI # hello_world.rb puts 'Hello World' $ time ruby hello.rb Hello World real 0m0.041s user 0m0.028s sys 0m0.008s
36. mruby-cli # mrblib/hello.rb def __main__(argv) puts "Hello World" end
37. mruby-cli # mrblib/hello.rb def __main__(argv) puts "Hello World" end $ time mruby/build/host/bin/hello Hello World real 0m0.003s user 0m0.000s sys 0m0.000s
38. No require
39. a single binary for every major platform
40. $ docker-compose run compile Build summary: ================================================ Config Name: x86_64-pc-linux-gnu Output Directory: build/host Included Gems: mruby-print - standard print/puts/p mruby-sprintf - standard Kernel#sprintf method mruby-time - standard Time class mruby-io mruby-mtest hello - hello - Binaries: hello ================================================
41. ================================================ Config Name: x86_64-apple-darwin14 Output Directory: build/x86_64-apple-darwin14 Included Gems: mruby-print - standard print/puts/p mruby-sprintf - standard Kernel#sprintf method mruby-time - standard Time class mruby-io mruby-mtest hello - hello - Binaries: hello ================================================
42. ================================================ Config Name: x86_64-w64-mingw32 Output Directory: build/x86_64-w64-mingw32 Included Gems: mruby-print - standard print/puts/p mruby-sprintf - standard Kernel#sprintf method mruby-time - standard Time class mruby-io mruby-mtest hello - hello - Binaries: hello ================================================
43. $ ls -nlh mruby-cli | awk '{print $5}' 421K
44. simple setup
45. Docker FROM hone/mruby-cli
46. Hello World Example $ mruby-cli --setup hello
47. Hello World Example $ mruby-cli --setup hello $ cd hello
48. Hello World Example $ mruby-cli --setup hello $ cd hello $ docker-compose run compile
49. Hello World Example $ mruby-cli --setup hello $ cd hello $ docker-compose run compile $ docker-compose run shell # mruby/build/host/bin/hello
50. Hello World Example $ mruby-cli --setup hello $ cd hello $ docker-compose run compile $ docker-compose run shell # mruby/build/host/bin/hello Hello World
51. Hello World Example $ mruby/bin/mruby-cli --setup hello create .gitignore create mrbgem.rake create build_config.rb create Rakefile create Dockerfile create docker-compose.yml create tools/ create tools/hello/ create tools/hello/hello.c create mrblib/ create mrblib/hello.rb create mrblib/hello/ create mrblib/hello/version.rb create bintest/ create bintest/hello.rb create test/ create test/test_hello.rb
52. mruby
53. mruby is the lightweight implementation of the Ruby language complying with part of the ISO standard. mruby can be linked and embedded within your application.
54. used in the wild https://github.com/ximenzi/openwrt-mruby
55. Contributors
56. Differences with MRI
57. no built in File Socket I/O
58. not threadsafe
59. no thread no fork
60. subset of Ruby 2.1
61. procs blocks DHH freedom patching meta programming literals
62. Executing mruby code
63. Interpret (*.rb) $ mruby/bin/mruby -e "puts 'Hello World'" Hello World $ mruby/bin/mruby hello_world.rb Hello World
64. Interactive (mirb) $ mruby/bin/mirb mirb - Embeddable Interactive Ruby Shell > puts 'Hello World' Hello World => nil
65. bytecode (*.mrb) $ mruby/bin/mrbc hello_world.rb
66. bytecode (*.mrb) $ mruby/bin/mrbc hello_world.rb $ ls hello_world.* hello_world.rb hello_world.mrb
67. bytecode (*.mrb) $ mruby/bin/mrbc hello_world.rb $ ls hello_world.* hello_world.rb hello_world.mrb $ mruby/bin/mruby -b hello_world.mrb Hello World
68. Binary C Code (*.c) $ mruby/bin/mrbc -o hello_world.c - Bhello_world hello_world.rb $ cat hello_world.c /* dumped in little endian order. use `mrbc -E` option for big endian CPU. */ #include <stdint.h> const uint8_t #if defined __GNUC__ __attribute__((aligned(4))) #elif defined _MSC_VER __declspec(align(4)) #endif hello_world[] = { 0x45,0x54,0x49,0x52,0x30,0x30,0x30,0x33,0xcb,0xa4,0x00, 0x00,0x00,0x65,0x4d,0x41,...}; This is an FYI No need to read this C code
69. int main(void) { mrb_state *mrb = mrb_open(); mrbc_context *c; mrb_value v; c = mrbc_context_new(mrb); mrb_load_irep_cxt(mrb, hello_world, c); mrbc_context_free(mrb, c); return 0; }
70. int main(void) { mrb_state *mrb = mrb_open(); mrbc_context *c; mrb_value v; c = mrbc_context_new(mrb); mrb_load_irep_cxt(mrb, hello_world, c); mrbc_context_free(mrb, c); return 0; }
71. tools/hello/hello.c #include <stdlib.h> #include <stdio.h> /* Include the mruby header */ #include <mruby.h> #include <mruby/array.h> int main(int argc, char *argv[]) { mrb_state *mrb = mrb_open(); mrb_value ARGV = mrb_ary_new_capa(mrb, argc); int i; int return_value; for (i = 0; i < argc; i++) { mrb_ary_push(mrb, ARGV, mrb_str_new_cstr(mrb, argv[i])); } mrb_define_global_const(mrb, "ARGV", ARGV); // call __main__(ARGV) mrb_funcall(mrb, mrb_top_self(mrb), "__main__", 1, ARGV); return_value = EXIT_SUCCESS; if (mrb->exc) { mrb_print_error(mrb); return_value = EXIT_FAILURE; } mrb_close(mrb); return return_value; } This is an FYI No need to read this C code
72. # mrblib/hello.rb def __main__(argv) puts "Hello World" end
73. mrblib ├── [4.0K] mrblib │ ├── [4.0K] mruby-cli │ │ ├── [ 697] cli.rb │ │ ├── [ 424] help.rb │ │ ├── [ 403] option.rb │ │ ├── [1022] options.rb │ │ ├── [8.0K] setup.rb │ │ ├── [ 238] util.rb │ │ └── [ 206] version.rb │ └── [ 53] mruby-cli.rb
74. Rakefile $ rake -T rake all # build all targets... rake clean # clean all built.. rake compile # compile all the binaries rake release # generate release tarballs rake test # run all tests rake test:bintest # run integration tests rake test:mtest # run mruby & unit tests
75. build_config.rb MRuby::Build.new do |conf| toolchain :gcc conf.enable_bintest conf.enable_debug conf.enable_test gem_config(conf) end
76. build_config.rb MRuby::CrossBuild.new('x86_64-apple-darwin14') do |conf| toolchain :clang [conf.cc, conf.linker].each do |cc| cc.command = 'x86_64-apple-darwin14-clang' end conf.cxx.command = 'x86_64-apple-darwin14-clang++' conf.archiver.command = 'x86_64-apple-darwin14-ar' conf.build_target = 'x86_64-pc-linux-gnu' conf.host_target = 'x86_64-apple-darwin14' gem_config(conf) end
77. $ docker-compose run compile Build summary: ================================================ Config Name: host Output Directory: build/host Binaries: mrbc Included Gems: mruby-print - standard print/puts/p mruby-sprintf - standard Kernel#sprintf method mruby-time - standard Time class mruby-io mruby-mtest hello - hello - Binaries: hello mruby-compiler - mruby compiler library mruby-bin-mrbc - mruby compiler executable ================================================
78. mrbgem.rake MRuby::Gem::Specification.new('hello') do |spec| spec.license = 'MIT' spec.authors = ['Terence Lee'] spec.summary = 'hello world' spec.bins = ['hello'] spec.add_dependency 'mruby-print', :core => 'mruby- print' spec.add_dependency 'mruby-mtest', :mgem => 'mruby- mtest' spec.add_dependency 'mruby-yaml', :github => 'hone/mruby-yaml' end
79. mrbgem.rake MRuby::Gem::Specification.new('hello') do |spec| spec.license = 'MIT' spec.authors = ['Terence Lee'] spec.summary = 'hello world' spec.bins = ['hello'] spec.add_dependency 'mruby-print', :core => 'mruby- print' spec.add_dependency 'mruby-mtest', :mgem => 'mruby- mtest' spec.add_dependency 'mruby-yaml', :github => 'hone/mruby-yaml' end
80. core mrbgems ├── [4.0K] mruby-array-ext ├── [4.0K] mruby-bin-debugger ├── [4.0K] mruby-bin-mirb ├── [4.0K] mruby-bin-mrbc ├── [4.0K] mruby-bin-mruby ├── [4.0K] mruby-bin-mruby-config ├── [4.0K] mruby-bin-strip ├── [4.0K] mruby-compiler ├── [4.0K] mruby-enumerator
81. mrbgem.rake MRuby::Gem::Specification.new('hello') do |spec| spec.license = 'MIT' spec.authors = ['Terence Lee'] spec.summary = 'hello world' spec.bins = ['hello'] spec.add_dependency 'mruby-print', :core => 'mruby- print' spec.add_dependency 'mruby-mtest', :mgem => 'mruby- mtest' spec.add_dependency 'mruby-yaml', :github => 'hone/mruby-yaml' end
82. mgem list
83. mrbgem.rake MRuby::Gem::Specification.new('hello') do |spec| spec.license = 'MIT' spec.authors = ['Terence Lee'] spec.summary = 'hello world' spec.bins = ['hello'] spec.add_dependency 'mruby-print', :core => 'mruby- print' spec.add_dependency 'mruby-mtest', :mgem => 'mruby- mtest' spec.add_dependency 'mruby-yaml', :github => 'hone/mruby-yaml' end
84. build_config.rb MRuby::Build.new do |conf| toolchain :gcc conf.enable_bintest conf.enable_debug conf.enable_test gem_config(conf) end
85. build_config.rb def gem_config(conf) # be sure to include this gem conf.gem File.expand_path(File. dirname(__FILE__)) conf.gem :github => 'hone/mruby-yaml' end
86. Testing
87. mtest module MRubyCLI class TestUtil < MTest::Unit::TestCase def test_camelize assert_equal "GoodNightMoon", Util.camelize ("good_night_moon") assert_equal "FooBarBaz", Util.camelize("foo- bar-baz") end end end MTest::Unit.new.run
88. mtest (cont.) $ docker-compose run mtest >>> Test host <<< mrbtest - Embeddable Ruby Test .....S... # Running tests: .. Finished tests in 0.000305s, 6557.3770 tests/s, 26229.5082 assertions/s. 2 tests, 8 assertions, 0 failures, 0 errors, 0 skips
89. bintest require 'open3' require 'tmpdir' BIN_PATH = File.join(File.dirname(__FILE__), ".. /mruby/bin/hello") assert('setup') do output, status = Open3.capture2("#{BIN_PATH}") assert_true status.success?, "Process did not exit cleanly" assert_include output, "Hello World" end
90. bintest (cont.) $ docker-compose run bintest . Total: 1 OK: 1 KO: 0 Crash: 0 Time: 0.01 seconds
91. Gotchas
92. no stdlib
93. Can't leverage Rubygems
94. mrbgems that have native extensions need to support cross compiling
95. jruby-launcher
96. jruby/jruby -- bin/
97. jruby/jruby-launcher
98. jruby.cpp /* * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS HEADER. * * Copyright 1997-2008, 2010 Sun Microsystems, Inc. All rights reserved. * * The contents of this file are subject to the terms of either the GNU * General Public License Version 2
99. mjruby https://github.com/jkutner/mjruby
100. mjruby/test
101. mruby-cli & you
102. Young Community
103. mruby/mruby on github
105. Building a CLI App • download the mruby-cli binary • install Docker Toolbox • mruby-cli --setup calculator • add new functionality ◦ modify to build a calculator or just start with addition • docker-compose run compile • modify, recompile • tweet at me (@hone02)
106. Let's build things in Ruby
107. Thank You @hone02 https://github.com/hone/mruby-cli