Building maintainable command-line tools with MRuby

Slide 1

Slide 1 text

Slide 2

Slide 2 text

MRuby & mruby-cli Eric Hodel — @drbrain

Slide 3

Slide 3 text

About Me • Ruby commi?er • RubyGems commi?er • Rake commi?er • Author of many gems • On vacaEon

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Fastly + Ruby www.ruby-lang.org rubygems.org

Slide 7

Slide 7 text

@FastlyJapan

Slide 8

Slide 8 text

Part Ⅰ

Slide 9

Slide 9 text

Web API Wrapper

Slide 10

Slide 10 text

Why not Ruby? • Install Ruby • Installing gems • Expensive support

Slide 11

Slide 11 text

MRuby =~ Ruby 1. Prototype in CRuby 2. Port to MRuby 3. Maintain in MRuby

Slide 12

Slide 12 text

MRuby

Slide 13

Slide 13 text

MRuby • Lightweight ruby • Embeddable • Hardware • SoWware

Slide 14

Slide 14 text

mrbgems • CRuby stdlib parts • mruby libraries • C extensions

Slide 15

Slide 15 text

Other mruby Talks 2016: Welcome to haconiwa 2016: Recent Advances in HTTP 2016: System calls hijacking with (m)ruby 2015: Making robot with mruby 2015: Building CLI Apps for Everyone 2014: MRuby as Development Plaborm for Payments 2014: MRuby on LEGO Mindstorms EV3 ® 2014: Resource Control Architecture scripEng with mruby

Slide 16

Slide 16 text

Using MRuby

Slide 17

Slide 17 text

build_config.rb • Which mrbgems to use • Which compiler to use • Library & include path • Can have many builds

Slide 18

Slide 18 text

mrbgem.rake • Gem metadata • Gem dependencies • Gem binaries

Slide 19

Slide 19 text

rake

Slide 20

Slide 20 text

rake Results • bin/your-cli • bin/mruby • bin/mirb • bin/mrbtest

Slide 21

Slide 21 text

mruby-cli

Slide 22

Slide 22 text

Building CLI Apps for Everyone

Slide 23

Slide 23 text

Write Ruby release on Linux, OS X, Windows

Slide 24

Slide 24 text

Using mruby-cli 1. Write MRuby 2. Compile 3. GOTO 1 unless tests pass 4. Release executables

Slide 25

Slide 25 text

MRuby Challenges

Slide 26

Slide 26 text

MRuby != CRuby

Slide 27

Slide 27 text

Modular stdlib • Minimal core library • mruby-array-ext • Easy to forget for mrbgems

Slide 28

Slide 28 text

Language Issues • No pre-deﬁned globals • Some syntax bugs • No keyword arguments for def • Regexp replacements

Slide 29

Slide 29 text

ExcepEons • Inaccurate backtrace • More C exploraEon

Slide 30

Slide 30 text

PorEng Libraries • Easy overall • Beware restricted syntax • PorEng tests is hard

Slide 31

Slide 31 text

mrbgems • Like Ruby 1.6.x • Libraries have narrow focus • Expect to submit patches

Slide 32

Slide 32 text

Patches mruby-onig-regexp mruby-curl mruby-io mruby-h?psclient mruby-cli

Slide 33

Slide 33 text

Build InformaEon • Needed for cross-compile • Hard to read without rake

Slide 34

Slide 34 text

mruby-cli Challenges

Slide 35

Slide 35 text

Docker

Slide 36

Slide 36 text

Docker • Ready to cross-compile • Clumsy to use • Best opEon available

Slide 37

Slide 37 text

docker-compose run compile • Too long to type • run shell to debug • Unfamiliar environment

Slide 38

Slide 38 text

Build System

Slide 39

Slide 39 text

Task OrganizaEon • MRuby and mruby-cli mixed • Hard to add new tasks

Slide 40

Slide 40 text

Improvements • Build MRuby separately • Run docker from a task

Slide 41

Slide 41 text

Docker cd mruby; rake rake compile rake -f in-docker Outer Inner

Slide 42

Slide 42 text

"Outer" Tasks • Download MRuby • Global test setup • Release tasks

Slide 43

Slide 43 text

"Inner" Tasks • Cross-compile libraries • Compile MRuby • Run tests

Slide 44

Slide 44 text

AddiEonal Beneﬁts • Hooks for custom tasks • Faster rake test

Slide 45

Slide 45 text

C Libraries

Slide 46

Slide 46 text

StaEc Linking • Single ﬁle • Re-release on vulnerabiliEes • Larger

Slide 47

Slide 47 text

Dynamic Linking • Easy vulnerability management • Adds a dependency

Slide 48

Slide 48 text

Cross Compiling

Slide 49

Slide 49 text

Cross Compiling • Download release • Cross-compile • MRuby compiler conﬁguraEon

Slide 50

Slide 50 text

CrossLibrary CrossLibrary.new 'curl' do |cross| cross.release_name = 'curl-7.28.0' cross.url = "https://curl.haxx.se/…" # TODO: cross-compile OpenSSL too. cross.configure_flags << '--without-ssl' end

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Part Ⅱ

Slide 56

Slide 56 text

Customer Support Tool

Slide 57

Slide 57 text

Problem • API-only to start • Complicated setup • Support burden

Slide 58

Slide 58 text

SoluEon • Automate • Validate inputs • Reduce errors

Slide 59

Slide 59 text

Prototypes

Slide 60

Slide 60 text

Single File Script • Easy to write • Inﬂexible

Slide 61

Slide 61 text

MulEple Scripts • One purpose per script • More ﬂexible • Well documented

Slide 62

Slide 62 text

Extract classes • JSON-API • Argument parsing

Slide 63

Slide 63 text

Lessons

Slide 64

Slide 64 text

Document • No API docs • 50% comments • Users can suggest changes

Slide 65

Slide 65 text

Simple Scripts • Minimum dependencies • Minimum opEons • One script per task • Very few large scripts

Slide 66

Slide 66 text

Hard to Maintain • CRuby version • Clone and edit by support

Slide 67

Slide 67 text

Switch to mruby-cli

Slide 68

Slide 68 text

RubyGems Pa?ern • Subclass per subcommand • mruby-optparse • setup for opEons • execute to run

Slide 69

Slide 69 text

Example Command class ConditionList < Command def setup option_service_id required: true option_version required: true end

Slide 70

Slide 70 text

Example Command class ConditionList < Command def execute results = api.send_request … results.each do |result| puts "…" end end

Slide 71

Slide 71 text

mruby-optparse • Familiar • Type checking • Argument compleEon • Shell compleEon

Slide 72

Slide 72 text

mruby-curl • Most mature HTTP library • HTTP + HTTPS • Persistent connecEons • libcurl dependency

Slide 73

Slide 73 text

TesEng • mruby-mtest for asserEons • mruby-test as a runner • Test case setup is painful

Slide 74

Slide 74 text

TesEng

Slide 75

Slide 75 text

mruby-mtest • Like minitest • Test subclass • test_* methods • setup / teardown • Remember: MTest::Unit.new.run

Slide 76

Slide 76 text

mruby-test • Test runner • One MRuby VM per test ﬁle • Complete isolaEon

Slide 77

Slide 77 text

ComplicaEons • Global setup hard • docker-compose documentaEon • Slow startup

Slide 78

Slide 78 text

Workarounds • Setup before all tests • Pass data out-of-band • Compile only host for tesEng

Slide 79

Slide 79 text

rake bintest • IntegraEon tests • Assert output from input

Slide 80

Slide 80 text

Part Ⅲ

Slide 81

Slide 81 text

Future Work

Slide 82

Slide 82 text

mruby-cli • Improved build system • DocumentaEon • Easier upgrades

Slide 83

Slide 83 text

MRuby • Accurate backtraces • build_config.rb library • Keyword args in methods

Slide 84

Slide 84 text

CRuby • mrb_get_args() • AutomaEc type check • More expressive than rb_scan_args()