Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Helix: Native Extensions for Everyone - RubyConf Colombia 2017

B87c43d4be875c9b41cd436f5c364f75?s=47 hone
September 08, 2017

Helix: Native Extensions for Everyone - RubyConf Colombia 2017

You can write a native extension for Ruby using Rust through Helix. With Helix your code will be screaming fast, while ensuring thread and memory safety and preventing segfaults. Fast, reliable, and productive. Come learn how we can have our cake and eat it too!

Writing Faster Ruby: https://speakerdeck.com/sferik/writing-fast-ruby

B87c43d4be875c9b41cd436f5c364f75?s=128

hone

September 08, 2017
Tweet

More Decks by hone

Other Decks in Programming

Transcript

  1. Native Extensions for Everyone

  2. Terence Lee @hone02

  3. None
  4. None
  5. Austin, TX

  6. Austin, TX

  7. Native Extensions for Everyone

  8. Ruby is Awesome • Beautiful syntax • Optimize for happiness

    • Community • Mature Ecosystem
  9. Ruby is Slow • Usually it doesn't matter • Most

    workloads are I/O bound • But occasionally it does...
  10. "Best of both worlds" • Native extensions • JSON gem,

    nokogiri gem, etc…. • stdlib too: Date, Pathname, etc... • Very fast • Transparent to the user
  11. Case Study: Discourse • String#blank? Is a hotspot for Discourse

    • Sam Saffron's fast_blank • 50 LoC in C • 20x speedup of String#blank?
  12. Why don't we do it more?

  13. But C... • Unsafe • Risky - segfaults! • Maintenance

    burden • Contribution barrier • Cross Platform Support
  14. Case Study #2: New Relic RPM • Monitoring agent built

    in Ruby • Hard to minimize overhead blog.scoutapp.com/articles/2016/02/07/overhead-benchmarks-new-relic-vs-scout
  15. Case Study #3: Skylight Agent • Started with Ruby •

    Too much overhead • Native extension! • But C… • AppSignal has written parts of their agent in Rust too
  16. Meet Rust

  17. Meet Rust • Like C: compiled, statically typed, very fast

    • Unlike C: enjoyable to use, guarantees safety • "If it compiles, it doesn't crash" • Same guarantee as Ruby with memory safety • Multiplatform Support
  18. Cargo - Dependency Manager

  19. Cargo.toml [package] name = "hello_world" version = "0.1.0" authors =

    ["Your Name <you@example.com>"] [dependencies] time = "0.1.12" regex = "0.1.41"
  20. cargo build $ cargo build Updating registry `https://github.com/rust-lang/crates.io-index` Downloading memchr

    v0.1.5 Downloading libc v0.1.10 Downloading regex-syntax v0.2.1 Downloading memchr v0.1.5 Downloading aho-corasick v0.3.0 Downloading regex v0.1.41 Compiling memchr v0.1.5 Compiling libc v0.1.10 Compiling regex-syntax v0.2.1 Compiling memchr v0.1.5 Compiling aho-corasick v0.3.0 Compiling regex v0.1.41 Compiling hello_world v0.1.0 (file:///path/to/project/hello_world)
  21. Cargo.lock [root] name = "hello_world" version = "0.1.0" dependencies =

    [ "regex 0.1.80 (registry+https://github.com/rust-lang/crates.io-index)", "time 0.1.36 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "aho-corasick" version = "0.5.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", ]
  22. Rust Multiplatform Support https://blog.rust-lang.org/2016/05/13/rustup.html rustup - a toolchain manager for

    Rust. "rustup works the same on Windows as it does on Unix" - rustup README Targets: $ rustup target list | wc -l 36 Including: • x86_64-apple-darwin (installed) • x86_64-pc-windows-gnu (installed) • x86_64-unknown-linux-gnu (default) • x86_64-unknown-linux-musl (installed)
  23. Shipping Rust in Firefox https://hacks.mozilla.org/2016/07/shipping-rust-in-firefox/ "Starting with Firefox 48, Mozilla

    is shipping its first production Rust code, with more to come!" -Dave Herman, Director of Strategy at Mozilla Research
  24. Zero-cost abstractions™ • In Ruby: tension between abstractions and performance

    • hash.keys.each vs hash.each_key • array.shuffle.first vs array.sample • In Rust: no such tradeoff • Compiler is magic
  25. Zero-cost abstractions™ pub fn main() { let array = [1,2,3];

    let mut sum = 0; for i in 0..3 { sum += array[i] } println!("{:?}", sum); }
  26. Zero-cost abstractions™ pub fn main() { let array = [1,2,3];

    let sum = array.iter().fold(0, |sum, &val| sum + val); println!("{:?}", sum); }
  27. fast_blank in Rust* pub extern "C" in fast_blank(buf: Buf) ->

    bool { buf.as_slice().chars().all(|c| c.is_whitespace()) } * without boilerplate code
  28. None
  29. fast_blank in Helix #[macro_use] extern crate helix; declare_types! { reopen

    class RubyString { def is_blank(self) -> bool { self.chars().all(|c| c.is_whitespace()) } } }
  30. The Vision • Keep writing the Ruby you love… •

    ...without the fear of eventually hitting a wall • Start with Ruby • Move to Helix when appropriate
  31. Build a Helix Rails App

  32. 1. Generate Rails App $ rails new helix-flipper --skip-active-record

  33. 2. Add helix-rails dependency Gemfile: gem 'helix-rails', '~> 0.5.0' $

    bundle install
  34. 3. Rails Generator $ bin/rails generate helix:crate text_transform Running via

    Spring preloader in process 24825 create crates/text_transform/Cargo.toml create crates/text_transform/src/lib.rs create crates/text_transform/text_transform.gemspec create crates/text_transform/Gemfile create crates/text_transform/lib/tasks/helix_runtime.rake create crates/text_transform/lib/text_transform.rb create crates/text_transform/.gitignore create crates/text_transform/Rakefile append crates/text_transform/Rakefile run bundle from "./crates/text_transform" append Gemfile
  35. 3. Rails Generator $ bin/rails generate helix:crate text_transform Running via

    Spring preloader in process 24825 create crates/text_transform/Cargo.toml create crates/text_transform/src/lib.rs create crates/text_transform/text_transform.gemspec create crates/text_transform/Gemfile create crates/text_transform/lib/tasks/helix_runtime.rake create crates/text_transform/lib/text_transform.rb create crates/text_transform/.gitignore create crates/text_transform/Rakefile append crates/text_transform/Rakefile run bundle from "./crates/text_transform" append Gemfile
  36. crates/text_transform/src/lib.rs #[macro_use] extern crate helix; ruby! { class TextTransform {

    def hello() { println!("Hello from text_transform!"); } } }
  37. $ bundle exec rake build $ bundle exec irb >

    require 'text_transform' => true > TextTransform.hello Hello from text_transform! => nil
  38. 4. Implement flip method def flip(text: String) -> String {

    text.chars().rev().map(|char| { match char { '!' => '¡', '"' => '„', '&' => '⅋', '\'' => '‚', '(' => ')', ')' => '(', ',' => '‘', '.' => '˙', ... // Flip back ... _ => char, } }).collect() end
  39. 4. Implement flip method def flip(text: String) -> String {

    text.chars().rev().map(|char| { match char { '!' => '¡', '"' => '„', '&' => '⅋', '\'' => '‚', '(' => ')', ')' => '(', ',' => '‘', '.' => '˙', ... // Flip back ... _ => char, } }).collect() end
  40. 4. Implement flip method def flip(text: String) -> String {

    text.chars().rev().map(|char| { match char { '!' => '¡', '"' => '„', '&' => '⅋', '\'' => '‚', '(' => ')', ')' => '(', ',' => '‘', '.' => '˙', ... // Flip back ... _ => char, } }).collect() end
  41. crates/text_transform/spec/text_transform.rb require "text_transform" describe "TextTransform" do it "can flip text"

    do expect(TextTransform.flip("Hello Aaron (@tenderlove)!")).to eq("¡(ǝʌolɹǝpuǝʇ@) uoɹɐ∀ ollǝH") end it "can flip the text back" do expect(TextTransform.flip("¡(ǝʌolɹǝpuǝʇ@) uoɹɐ∀ ollǝH")).to eq("Hello Aaron (@tenderlove)!") end end
  42. 5. Flipping Tables! def flip(text: String) -> String { if

    text == "┬──┬ ノ( ゜-゜ノ)" { return "(╯°□°)╯︵ ┻━┻".to_string(); } else if text == "(╯°□°)╯︵ ┻━┻" { return "┬──┬ ノ( ゜-゜ノ)".to_string(); } // rest of method … end
  43. 6. Using in Rails $ bin/rails generate helix:crate text_transform Running

    via Spring preloader in process 24825 create crates/text_transform/Cargo.toml create crates/text_transform/src/lib.rs create crates/text_transform/text_transform.gemspec create crates/text_transform/Gemfile create crates/text_transform/lib/tasks/helix_runtime.rake create crates/text_transform/lib/text_transform.rb create crates/text_transform/.gitignore create crates/text_transform/Rakefile append crates/text_transform/Rakefile run bundle from "./crates/text_transform" append Gemfile
  44. 6. Using in Rails Gemfile: gem 'text_transform', path: 'crates/text_transform'

  45. 6. Using in Rails (Routes) config/routes.rb: resources :flips, path: "/",

    only: [:index, :create]
  46. 6. Using in Rails (Controller) app/controllers/flips_controller.rb: class FlipsController < ApplicationController

    def index @text = params[:text] || "Hello world!" end def create @text = TextTransform.flip(params[:text]) render :index end end
  47. 6. Using in Rails (View) app/views/flips/index.html.erb: <h1>Flipper</h1> <%= form_tag do

    %> <%= text_field_tag :text, @text %> <%= submit_tag "Flip!" %> <% end %>
  48. 7. Deploy to Heroku Setup rustc on production machine $

    heroku create $ heroku buildpacks:add https://github.com/emk/heroku-buildpack-rust $ heroku buildpacks:add heroku/ruby $ git push heroku master
  49. 7. Deploy to Heroku Setup rustc on production machine $

    heroku create $ heroku buildpacks:add https://github.com/emk/heroku-buildpack-rust $ heroku buildpacks:add heroku/ruby $ git push heroku master
  50. Distribution (Precompile Binaries) • Use CI to build binaries and

    publish to rubygems ◦ Travis/CircleCI: Linux + OS X ◦ Appveyor: Windows • Don't need rust compiler on host machine • Next version of helix CLI will generate CI config ◦ helix bootstrap hello_helix ◦ Add env var RUBYGEMS_AUTH_TOKEN to CI services to publish
  51. 7. Extract text_transform gem

  52. 7. Extract text_transform gem Gemfile: gem 'text_transform' $ bundle install

  53. 8. Deploy to Heroku $ heroku create $ git push

    heroku master -----> Ruby app detected -----> Compiling Ruby/Rails -----> Using Ruby version: ruby-2.3.4 -----> Installing dependencies using bundler 1.15.2 … Installing text_transform 0.1.0 (x86_64-linux)
  54. Table Flip!

  55. Table Flip!

  56. Whoops… It worked!™

  57. Good use cases • CPU-bound • Simple inputs ◦ JSON

    ◦ Filenames ◦ data tables • Avoid chatty APIs (boundary cost)
  58. Measure # of words in Shakespeare Ruby: 3 seconds Rust

    example: 30 ms
  59. Good use cases ruby! { class WordCount { def search(path:

    String, search: String) -> i32 { let mut file = File::open(path).expect("could not open file."); // … lines.into_par_iter(). .map(|line| wc_line(line, search)) .sum() } } }
  60. Good use cases • Use Rust libraries • Leverage Rust

    web browser tech • Mailer, Background job, Action Cable
  61. ruby! { class InlineCSS { def inline(html: String, css: String)

    -> String { let doc = kuchiki::parse_html().one(html); let mut parser = parse_css::CSSParser::new(css); let rules = ...; for rule in rules { let matching_elements = ...; for matching_element in matching_elements { // insert style attribute } } // serialize } } } Good use cases
  62. Roadmap • Greenfield project • Drop-in replacement • Reopen class

    • Ship to production • Non-traditional use-cases • Performance parity with C • Misc features and QoL improvements
  63. Let's keep writing the ruby we love without the fear

    we'll hit a wall.
  64. usehelix.com Thank You @hone02