Slide 1

Slide 1 text

Native Extensions for Everyone

Slide 2

Slide 2 text

Terence Lee @hone02

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Austin, TX

Slide 6

Slide 6 text

Austin, TX

Slide 7

Slide 7 text

Native Extensions for Everyone

Slide 8

Slide 8 text

Ruby is Awesome ● Beautiful syntax ● Optimize for happiness ● Community ● Mature Ecosystem

Slide 9

Slide 9 text

Ruby is Slow ● Usually it doesn't matter ● Most workloads are I/O bound ● But occasionally it does...

Slide 10

Slide 10 text

"Best of both worlds" ● Native extensions ● JSON gem, nokogiri gem, etc…. ● stdlib too: Date, Pathname, etc... ● Very fast ● Transparent to the user

Slide 11

Slide 11 text

Case Study: Discourse ● String#blank? Is a hotspot for Discourse ● Sam Saffron's fast_blank ● 50 LoC in C ● 20x speedup of String#blank?

Slide 12

Slide 12 text

Why don't we do it more?

Slide 13

Slide 13 text

But C... ● Unsafe ● Risky - segfaults! ● Maintenance burden ● Contribution barrier ● Cross Platform Support

Slide 14

Slide 14 text

Case Study #2: New Relic RPM ● Monitoring agent built in Ruby ● Hard to minimize overhead blog.scoutapp.com/articles/2016/02/07/overhead-benchmarks-new-relic-vs-scout

Slide 15

Slide 15 text

Case Study #3: Skylight Agent ● Started with Ruby ● Too much overhead ● Native extension! ● But C… ● AppSignal has written parts of their agent in Rust too

Slide 16

Slide 16 text

Meet Rust

Slide 17

Slide 17 text

Meet Rust ● Like C: compiled, statically typed, very fast ● Unlike C: enjoyable to use, guarantees safety ● "If it compiles, it doesn't crash" ● Same guarantee as Ruby with memory safety ● Multiplatform Support

Slide 18

Slide 18 text

Cargo - Dependency Manager

Slide 19

Slide 19 text

Cargo.toml [package] name = "hello_world" version = "0.1.0" authors = ["Your Name "] [dependencies] time = "0.1.12" regex = "0.1.41"

Slide 20

Slide 20 text

cargo build $ cargo build Updating registry `https://github.com/rust-lang/crates.io-index` Downloading memchr v0.1.5 Downloading libc v0.1.10 Downloading regex-syntax v0.2.1 Downloading memchr v0.1.5 Downloading aho-corasick v0.3.0 Downloading regex v0.1.41 Compiling memchr v0.1.5 Compiling libc v0.1.10 Compiling regex-syntax v0.2.1 Compiling memchr v0.1.5 Compiling aho-corasick v0.3.0 Compiling regex v0.1.41 Compiling hello_world v0.1.0 (file:///path/to/project/hello_world)

Slide 21

Slide 21 text

Cargo.lock [root] name = "hello_world" version = "0.1.0" dependencies = [ "regex 0.1.80 (registry+https://github.com/rust-lang/crates.io-index)", "time 0.1.36 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "aho-corasick" version = "0.5.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", ]

Slide 22

Slide 22 text

Rust Multiplatform Support https://blog.rust-lang.org/2016/05/13/rustup.html rustup - a toolchain manager for Rust. "rustup works the same on Windows as it does on Unix" - rustup README Targets: $ rustup target list | wc -l 36 Including: ● x86_64-apple-darwin (installed) ● x86_64-pc-windows-gnu (installed) ● x86_64-unknown-linux-gnu (default) ● x86_64-unknown-linux-musl (installed)

Slide 23

Slide 23 text

Shipping Rust in Firefox https://hacks.mozilla.org/2016/07/shipping-rust-in-firefox/ "Starting with Firefox 48, Mozilla is shipping its first production Rust code, with more to come!" -Dave Herman, Director of Strategy at Mozilla Research

Slide 24

Slide 24 text

Zero-cost abstractions™ ● In Ruby: tension between abstractions and performance ● hash.keys.each vs hash.each_key ● array.shuffle.first vs array.sample ● In Rust: no such tradeoff ● Compiler is magic

Slide 25

Slide 25 text

Zero-cost abstractions™ pub fn main() { let array = [1,2,3]; let mut sum = 0; for i in 0..3 { sum += array[i] } println!("{:?}", sum); }

Slide 26

Slide 26 text

Zero-cost abstractions™ pub fn main() { let array = [1,2,3]; let sum = array.iter().fold(0, |sum, &val| sum + val); println!("{:?}", sum); }

Slide 27

Slide 27 text

fast_blank in Rust* pub extern "C" in fast_blank(buf: Buf) -> bool { buf.as_slice().chars().all(|c| c.is_whitespace()) } * without boilerplate code

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

fast_blank in Helix #[macro_use] extern crate helix; declare_types! { reopen class RubyString { def is_blank(self) -> bool { self.chars().all(|c| c.is_whitespace()) } } }

Slide 30

Slide 30 text

The Vision ● Keep writing the Ruby you love… ● ...without the fear of eventually hitting a wall ● Start with Ruby ● Move to Helix when appropriate

Slide 31

Slide 31 text

Build a Helix Rails App

Slide 32

Slide 32 text

1. Generate Rails App $ rails new helix-flipper --skip-active-record

Slide 33

Slide 33 text

2. Add helix-rails dependency Gemfile: gem 'helix-rails', '~> 0.5.0' $ bundle install

Slide 34

Slide 34 text

3. Rails Generator $ bin/rails generate helix:crate text_transform Running via Spring preloader in process 24825 create crates/text_transform/Cargo.toml create crates/text_transform/src/lib.rs create crates/text_transform/text_transform.gemspec create crates/text_transform/Gemfile create crates/text_transform/lib/tasks/helix_runtime.rake create crates/text_transform/lib/text_transform.rb create crates/text_transform/.gitignore create crates/text_transform/Rakefile append crates/text_transform/Rakefile run bundle from "./crates/text_transform" append Gemfile

Slide 35

Slide 35 text

Slide 36

Slide 36 text

crates/text_transform/src/lib.rs #[macro_use] extern crate helix; ruby! { class TextTransform { def hello() { println!("Hello from text_transform!"); } } }

Slide 37

Slide 37 text

$ bundle exec rake build $ bundle exec irb > require 'text_transform' => true > TextTransform.hello Hello from text_transform! => nil

Slide 38

Slide 38 text

4. Implement flip method def flip(text: String) -> String { text.chars().rev().map(|char| { match char { '!' => '¡', '"' => '„', '&' => '⅋', '\'' => '‚', '(' => ')', ')' => '(', ',' => '‘', '.' => '˙', ... // Flip back ... _ => char, } }).collect() end

Slide 39

Slide 39 text

Slide 40

Slide 40 text

Slide 41

Slide 41 text

crates/text_transform/spec/text_transform.rb require "text_transform" describe "TextTransform" do it "can flip text" do expect(TextTransform.flip("Hello Aaron (@tenderlove)!")).to eq("¡(ǝʌolɹǝpuǝʇ@) uoɹɐ∀ ollǝH") end it "can flip the text back" do expect(TextTransform.flip("¡(ǝʌolɹǝpuǝʇ@) uoɹɐ∀ ollǝH")).to eq("Hello Aaron (@tenderlove)!") end end

Slide 42

Slide 42 text

5. Flipping Tables! def flip(text: String) -> String { if text == "┬──┬ ノ( ゜-゜ノ)" { return "(╯°□°）╯︵ ┻━┻".to_string(); } else if text == "(╯°□°）╯︵ ┻━┻" { return "┬──┬ ノ( ゜-゜ノ)".to_string(); } // rest of method … end

Slide 43

Slide 43 text

6. Using in Rails $ bin/rails generate helix:crate text_transform Running via Spring preloader in process 24825 create crates/text_transform/Cargo.toml create crates/text_transform/src/lib.rs create crates/text_transform/text_transform.gemspec create crates/text_transform/Gemfile create crates/text_transform/lib/tasks/helix_runtime.rake create crates/text_transform/lib/text_transform.rb create crates/text_transform/.gitignore create crates/text_transform/Rakefile append crates/text_transform/Rakefile run bundle from "./crates/text_transform" append Gemfile

Slide 44

Slide 44 text

6. Using in Rails Gemfile: gem 'text_transform', path: 'crates/text_transform'

Slide 45

Slide 45 text

6. Using in Rails (Routes) config/routes.rb: resources :flips, path: "/", only: [:index, :create]

Slide 46

Slide 46 text

6. Using in Rails (Controller) app/controllers/flips_controller.rb: class FlipsController < ApplicationController def index @text = params[:text] || "Hello world!" end def create @text = TextTransform.flip(params[:text]) render :index end end

Slide 47

Slide 47 text

6. Using in Rails (View) app/views/flips/index.html.erb:

Flipper

<%= form_tag do %> <%= text_field_tag :text, @text %> <%= submit_tag "Flip!" %> <% end %>

Slide 48

Slide 48 text

7. Deploy to Heroku Setup rustc on production machine $ heroku create $ heroku buildpacks:add https://github.com/emk/heroku-buildpack-rust $ heroku buildpacks:add heroku/ruby $ git push heroku master

Slide 49

Slide 49 text

7. Deploy to Heroku Setup rustc on production machine $ heroku create $ heroku buildpacks:add https://github.com/emk/heroku-buildpack-rust $ heroku buildpacks:add heroku/ruby $ git push heroku master

Slide 50

Slide 50 text

Distribution (Precompile Binaries) ● Use CI to build binaries and publish to rubygems ○ Travis/CircleCI: Linux + OS X ○ Appveyor: Windows ● Don't need rust compiler on host machine ● Next version of helix CLI will generate CI config ○ helix bootstrap hello_helix ○ Add env var RUBYGEMS_AUTH_TOKEN to CI services to publish

Slide 51

Slide 51 text

7. Extract text_transform gem

Slide 52

Slide 52 text

7. Extract text_transform gem Gemfile: gem 'text_transform' $ bundle install

Slide 53

Slide 53 text

8. Deploy to Heroku $ heroku create $ git push heroku master -----> Ruby app detected -----> Compiling Ruby/Rails -----> Using Ruby version: ruby-2.3.4 -----> Installing dependencies using bundler 1.15.2 … Installing text_transform 0.1.0 (x86_64-linux)

Slide 54

Slide 54 text

Table Flip!

Slide 55

Slide 55 text

Table Flip!

Slide 56

Slide 56 text

Whoops… It worked!™

Slide 57

Slide 57 text

Good use cases ● CPU-bound ● Simple inputs ○ JSON ○ Filenames ○ data tables ● Avoid chatty APIs (boundary cost)

Slide 58

Slide 58 text

Measure # of words in Shakespeare Ruby: 3 seconds Rust example: 30 ms

Slide 59

Slide 59 text

Good use cases ruby! { class WordCount { def search(path: String, search: String) -> i32 { let mut file = File::open(path).expect("could not open file."); // … lines.into_par_iter(). .map(|line| wc_line(line, search)) .sum() } } }

Slide 60

Slide 60 text

Good use cases ● Use Rust libraries ● Leverage Rust web browser tech ● Mailer, Background job, Action Cable

Slide 61

Slide 61 text

ruby! { class InlineCSS { def inline(html: String, css: String) -> String { let doc = kuchiki::parse_html().one(html); let mut parser = parse_css::CSSParser::new(css); let rules = ...; for rule in rules { let matching_elements = ...; for matching_element in matching_elements { // insert style attribute } } // serialize } } } Good use cases

Slide 62

Slide 62 text

Roadmap ● Greenfield project ● Drop-in replacement ● Reopen class ● Ship to production ● Non-traditional use-cases ● Performance parity with C ● Misc features and QoL improvements