Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Leveraging Rust with mruby: Loving our fellow Windows® users - FOSDEM 2017

B87c43d4be875c9b41cd436f5c364f75?s=47 hone
February 04, 2017

Leveraging Rust with mruby: Loving our fellow Windows® users - FOSDEM 2017

Even if mruby core works on Windows, any mruby app is may not work if the libraries it depends on don’t compile and run on Windows. Using Rust, we can work towards building an ecosystem that is Windows compatible. This talk walks through building mruby libraries that do that.

Unlike some other languages like JavaScript, Java, or Python, Ruby is not known for it’s ability to run well on Windows. The Windows environment is often treated as a second or third class citizen even though there are many potential users. Part of this is due to Ruby’s strong POSIX oriented standard library. This has mostly been ok since most Ruby applications are hosted and run on UNIX servers.

mruby is a lightweight Ruby that can be linked and embedded inside other applications. Being embeddable means that the runtime host could be Windows. mruby also has no Operating System specific standard library. Since the ecosystem is young, there’s potential for an ecosystem that is Windows friendly. But writing cross platform native extensions in C for mruby is still difficult.

Rust is a modern systems programming language that is fast and built to prevent segfaults and guarantee thread safety. It’s more than just a compiler, it’s a platform that has a great dependency manager and cares about cross compilation out of the box. This makes it a great candidate over C to write native extensions.

In this talk, we’ll go over some basic Rust and how we can integrate this to write mruby libraries that compile on Linux, OS X, and Windows. It’ll dig into not only writing the library code, but also the mruby C API as well as the mruby/Rust binding layer that makes this all possible.

Video: https://fosdem.org/2017/schedule/event/ruby_leverging_rust_to_build_a_windows_friendly_ecosystem_for_mruby/

B87c43d4be875c9b41cd436f5c364f75?s=128

hone

February 04, 2017
Tweet

More Decks by hone

Other Decks in Programming

Transcript

  1. Leveraging Rust with mruby Loving our fellow Windows® users +

    <3
  2. Terence Lee @hone02

  3. None
  4. None
  5. Austin, TX

  6. Austin, TX

  7. Leveraging Rust with mruby Loving our fellow Windows® users

  8. Agenda • Ruby on Windows • Intro to mruby &

    rust • mruby + rust • Future?
  9. Ruby on Windows

  10. Heroku CLI

  11. None
  12. None
  13. Python https://www.python.org/downloads/windows/

  14. Installing Ruby on Windows • chruby ◦ Requirements: bash >=

    3 or zsh • RVM ◦ https://rvm.io/support/faq#does-rvm-work-on-windows-will-it-in-the-future ◦ Does RVM work on windows? Will it in the future? ▪ NO. If you would like to manage multiple versions of ruby on windows please use pik which is an excellent tool by Gordon Thiesfeld. You can find it on GitHub. There are plans to include windows support in RVM 2.0 => https://www.bountysource.com/fundraisers/489-rvm-2-0. • ruby-build ◦ bash script
  15. RubyInstaller

  16. Extensions on Windows is Hard

  17. Extensions on Windows is Hard Nokogiri

  18. Ruby is missing a Windows Ecosystem

  19. mruby

  20. mruby is the lightweight implementation of the Ruby language complying

    with part of the ISO standard. mruby can be linked and embedded within your application.
  21. Differences with MRI

  22. no built in File Socket I/O

  23. not threadsafe

  24. no thread no fork

  25. subset of Ruby 2.1

  26. procs blocks DHH freedom patching meta programming literals

  27. mrubygems?

  28. mrbgem.rake MRuby::Gem::Specification.new('hello') do |spec| spec.license = 'MIT' spec.authors = ['Terence

    Lee'] spec.summary = 'hello world' spec.bins = ['hello'] spec.add_dependency 'mruby-print', :core => 'mruby-print' spec.add_dependency 'mruby-mtest', :mgem => 'mruby-mtest' spec.add_dependency 'mruby-yaml', :github => 'hone/mruby-yaml' end
  29. core mrbgems ├── [4.0K] mruby-array-ext ├── [4.0K] mruby-bin-debugger ├── [4.0K]

    mruby-bin-mirb ├── [4.0K] mruby-bin-mrbc ├── [4.0K] mruby-bin-mruby ├── [4.0K] mruby-bin-mruby-config ├── [4.0K] mruby-bin-strip ├── [4.0K] mruby-compiler ├── [4.0K] mruby-enumerator
  30. mgem list

  31. mrbgems & Windows?

  32. mruby-socket

  33. None
  34. Still making the same mistakes

  35. None
  36. Rust is a systems programming language that runs blazingly fast,

    prevents segfaults, and guarantees thread safety.
  37. Enabler

  38. issues even for seasoned maintainers C Extensions are Hard

  39. If you have successfully compiled a Rust program, it will

    not segfault at runtime.
  40. Ownership

  41. Zero Cost Abstractions

  42. Zero Cost Abstractions pub fn main() { let array =

    [1,2,3]; let mut sum = 0; for i in 0..3 { sum += array[i] } println!("{:?}", sum); }
  43. Zero Cost Abstractions pub fn main() { let array =

    [1,2,3]; let sum = array.iter().fold(0, |sum, &val| sum + val); println!("{:?}", sum); }
  44. Rust 1.0: Stability as a Deliverable (2014) https://blog.rust-lang.org/2014/10/30/Stability.html 6 week

    release cycle Code compiles on Rust stable 1.0 should compile with Rust stable 1.x
  45. Rust Cross-compilation https://blog.rust-lang.org/2016/05/13/rustup.html rustup - a toolchain manager for Rust.

    "rustup works the same on Windows as it does on Unix" - rustup README Targets: $ rustup target list | wc -l 36 Including: • x86_64-apple-darwin (installed) • x86_64-pc-windows-gnu (installed) • x86_64-unknown-linux-gnu (default) • x86_64-unknown-linux-musl (installed)
  46. Shipping Rust in Firefox https://hacks.mozilla.org/2016/07/shipping-rust-in-firefox/ "Starting with Firefox 48, Mozilla

    is shipping its first production Rust code, with more to come!" -Dave Herman, Director of Strategy at Mozilla Research
  47. Cargo

  48. Cargo.toml [package] name = "hello_world" version = "0.1.0" authors =

    ["Your Name <you@example.com>"] [dependencies] time = "0.1.12" regex = "0.1.41"
  49. cargo build $ cargo build Updating registry `https://github.com/rust-lang/crates.io-index` Downloading memchr

    v0.1.5 Downloading libc v0.1.10 Downloading regex-syntax v0.2.1 Downloading memchr v0.1.5 Downloading aho-corasick v0.3.0 Downloading regex v0.1.41 Compiling memchr v0.1.5 Compiling libc v0.1.10 Compiling regex-syntax v0.2.1 Compiling memchr v0.1.5 Compiling aho-corasick v0.3.0 Compiling regex v0.1.41 Compiling hello_world v0.1.0 (file:///path/to/project/hello_world)
  50. Cargo.lock [root] name = "hello_world" version = "0.1.0" dependencies =

    [ "regex 0.1.80 (registry+https://github.com/rust-lang/crates.io-index)", "time 0.1.36 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "aho-corasick" version = "0.5.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", ]
  51. mruby + rust

  52. Creating a mrbgem with C

  53. ├── [4.0K] bintest │ └── [ 437] mruby-hello-world.rb ├── [2.3K]

    build_config.rb ├── [ 298] mrbgem.rake ├── [4.0K] mrblib │ └── [ 42] mruby-hello-world.rb ├── [1.9K] Rakefile ├── [4.0K] src │ └── [ 363] hello_world_gem.c ├── [4.0K] test │ └── [ 119] test_mruby-hello-world.rb └── [4.0K] tools └── [4.0K] mruby-hello-world └── [ 755] mruby-hello-world.c
  54. ├── [4.0K] bintest │ └── [ 437] mruby-hello-world.rb ├── [2.3K]

    build_config.rb ├── [ 298] mrbgem.rake ├── [4.0K] mrblib │ └── [ 42] mruby-hello-world.rb ├── [1.9K] Rakefile ├── [4.0K] src │ └── [ 363] hello_world_gem.c ├── [4.0K] test │ └── [ 119] test_mruby-hello-world.rb └── [4.0K] tools └── [4.0K] mruby-hello-world └── [ 755] mruby-hello-world.c
  55. hello_world_gem.c #include "mruby.h" mrb_value c_hello(mrb_state * mrb, mrb_value self) {

    return mrb_str_new_cstr(mrb, "Hello"); } void mrb_mruby_hello_world_gem_init(mrb_state * mrb) { struct RClass *klass = mrb_define_module(mrb, "Hello"); mrb_define_class_method(mrb, klass, "hello", c_hello, MRB_ARGS_NONE()); } void mrb_mruby_hello_world_gem_final(mrb_state * mrb) { }
  56. hello_world_gem.c #include "mruby.h" mrb_value c_hello(mrb_state * mrb, mrb_value self) {

    return mrb_str_new_cstr(mrb, "Hello"); } void mrb_mruby_hello_world_gem_init(mrb_state * mrb) { struct RClass *klass = mrb_define_module(mrb, "Hello"); mrb_define_class_method(mrb, klass, "hello", c_hello, MRB_ARGS_NONE()); } void mrb_mruby_hello_world_gem_final(mrb_state * mrb) { }
  57. hello_world_gem.c #include "mruby.h" mrb_value c_hello(mrb_state * mrb, mrb_value self) {

    return mrb_str_new_cstr(mrb, "Hello"); } void mrb_mruby_hello_world_gem_init(mrb_state * mrb) { struct RClass *klass = mrb_define_module(mrb, "Hello"); mrb_define_class_method(mrb, klass, "hello", c_hello, MRB_ARGS_NONE()); } void mrb_mruby_hello_world_gem_final(mrb_state * mrb) { }
  58. hello_world_gem.c #include "mruby.h" mrb_value c_hello(mrb_state * mrb, mrb_value self) {

    return mrb_str_new_cstr(mrb, "Hello"); } void mrb_mruby_hello_world_gem_init(mrb_state * mrb) { struct RClass *klass = mrb_define_module(mrb, "Hello"); mrb_define_class_method(mrb, klass, "hello", c_hello, MRB_ARGS_NONE()); } void mrb_mruby_hello_world_gem_final(mrb_state * mrb) { }
  59. ├── [4.0K] bintest │ └── [ 437] mruby-hello-world.rb ├── [2.3K]

    build_config.rb ├── [ 298] mrbgem.rake ├── [4.0K] mrblib │ └── [ 42] mruby-hello-world.rb ├── [1.9K] Rakefile ├── [4.0K] src │ └── [ 363] hello_world_gem.c ├── [4.0K] test │ └── [ 119] test_mruby-hello-world.rb └── [4.0K] tools └── [4.0K] mruby-hello-world └── [ 755] mruby-hello-world.c
  60. mruby-hello-world.rb def __main__(argv) puts Hello.hello end

  61. Rust FFI https://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywhere.html > Rust makes it easy to communicate

    with C APIs without overhead, and to leverage its ownership system to provide much stronger safety guarantees for those APIs at the same time. #[no_mangle] pub extern "C" fn double_input(input: i32) -> i32 { input * 2 }
  62. ├── [4.0K] bintest │ └── [ 437] mruby-rust-hello-world.rb ├── [2.3K]

    build_config.rb ├── [2.4K] mrbgem.rake ├── [4.0K] mrblib │ ├── [4.0K] mruby-rust-hello-world │ └── [ 41] mruby-rust-hello-world.rb ├── [1.9K] Rakefile ├── [4.0K] rust │ ├── [ 460] Cargo.lock │ ├── [ 196] Cargo.toml │ └── [4.0K] src │ └── [ 710] lib.rs ├── [4.0K] test │ └── [ 119] test_mruby-rust.rb └── [4.0K] tools └── [4.0K] mruby-rust-hello-world └── [ 759] mruby-rust-hello-world.c
  63. lib.rs #[no_mangle] pub extern "C" fn mrb_rust_hello(mrb: *mut mrb_state, selfie:

    mrb_value) -> mrb_value { unsafe { mrb_str_new_cstr(mrb, cstr!("Hello")) } } #[no_mangle] pub extern "C" fn mrb_mruby_rust_hello_world_gem_init(mrb: *mut mrb_state) { unsafe { let klass = mrb_define_module(mrb, cstr!("Rust")); mrb_define_class_method(mrb, klass, cstr!("hello"), mrb_rust_hello as mrb_func_t, MRB_ARGS_NONE()); } } #[no_mangle] pub extern "C" fn mrb_mruby_rust_hello_world_gem_final(mrb: *mut mrb_state) { }
  64. Cargo.toml [package] name = "hello" version = "0.1.0" authors =

    ["Terence Lee <hone02@gmail.com>"] [dependencies] mferuby = { git = "https://github.com/hone/mferuby.git" } [lib] crate-type = ["staticlib"]
  65. mferuby Rust bindings for the mruby C API

  66. libmruby-sys crate ├── [4.0K] crates │ ├── [4.0K] libmruby-sys │

    │ ├── [ 150] Cargo.toml │ │ └── [4.0K] src │ │ └── [5.1K] lib.rs │ └── [4.0K] mferuby │ ├── [ 203] Cargo.toml │ └── [4.0K] src │ ├── [ 588] lib.rs │ └── [ 815] mrb.rs low level rust binding to mruby C API
  67. crates/libmruby-sys/src/lib.rs extern { #[link_name = "tmrb_nil_value"] pub fn nil() ->

    mrb_value; #[link_name = "tmrb_true_value"] pub fn mrb_true() -> mrb_value; #[link_name = "tmrb_false_value"] pub fn mrb_false() -> mrb_value; #[link_name = "tmrb_fixnum_value"] pub fn fixnum(i: c_int) -> mrb_value; #[link_name = "tmrb_float_value"] pub fn float(mrb: *mut mrb_state, f: c_double) -> mrb_value; pub fn mrb_open() -> *mut mrb_state; pub fn mrb_define_module(mrb: *mut mrb_state, name: *const c_char) -> *mut RClass; pub fn mrb_define_class(mrb: *mut mrb_state, name: *const c_char, super_class: *mut RClass) -> *mut RClass; pub fn mrb_define_class_under(mrb: *mut mrb_state, outer: *mut RClass, name: *const c_char, super_class: *mut RClass) -> *mut
  68. mferuby crate ├── [4.0K] crates │ ├── [4.0K] libmruby-sys │

    │ ├── [ 150] Cargo.toml │ │ └── [4.0K] src │ │ └── [5.1K] lib.rs │ └── [4.0K] mferuby │ ├── [ 203] Cargo.toml │ └── [4.0K] src │ ├── [ 588] lib.rs │ └── [ 815] mrb.rs high level rust API on top of libmruby-sys
  69. crates/mferuby/src/lib.rs pub fn mruby_str_to_rust_string(mruby_string: sys::mrb_value) -> Result<String, std::str::Utf8Error> { let

    size = unsafe { sys::RSTRING_LEN(mruby_string) }; let ptr = unsafe { sys::RSTRING_PTR(mruby_string) }; let slice = unsafe { std::slice::from_raw_parts(ptr as *const u8, size as usize) }; let s = try!(std::str::from_utf8(slice)); Ok(s.to_string()) }
  70. mferuby-runtime mrbgem ├── [2.3K] build_config.rb ├── [ 412] mrbgem.rake ├──

    [1.9K] Rakefile ├── [4.0K] src │ ├── [1.4K] mferuby_runtime.c │ ├── [ 138] mferuby_runtime_gem.c │ └── [ 37] mruby.rs └── [4.0K] test └── [ 131] test_mferuby-runtime.rb compiles against mruby and exposes non-public C API to rust
  71. src/mferuby_runtime.c #include <mruby/compile.h> #include <mruby/value.h> mrb_value tmrb_float_value(struct mrb_state *mrb, mrb_float

    f) { return mrb_float_value(mrb, f); } mrb_value tmrb_fixnum_value(mrb_int i) { return mrb_fixnum_value(i); } mrb_value tmrb_nil_value() { return mrb_nil_value(); } mrb_value tmrb_true_value() { return mrb_true_value(); }
  72. mruby-docopt

  73. cargo help

  74. CPP, still Hard

  75. class CommandLine USAGE = <<USAGE Heroku Docker Ruby Util. Usage:

    ruby-util detect-ruby <gemfile-path> ruby-util detect-gem <profiled-path> <gem> ruby-util install-ruby <gemfile-path> <install-path> <profiled-path> Options: -h --help Show this message USAGE def run(argv) options = Docopt.parse(USAGE, argv) if options["detect-ruby"] puts CLI::DetectRuby.new(File.read(options["<gemfile-path>"])).ruby_ver sion
  76. mruby-docopt pub extern "C" fn parse(mrb: *mut sys::mrb_state, this: sys::mrb_value)

    -> sys::mrb_value { let mrb_obj = mferuby::Mrb::new(mrb); let mut usage: sys::mrb_value = unsafe { mem::uninitialized() }; let mut argv: sys::mrb_value = unsafe { mem::uninitialized() }; unsafe { sys::mrb_get_args(mrb, cstr!("SA"), &mut usage, &mut argv); } let rust_usage = mferuby::mruby_str_to_rust_string(usage).unwrap(); let argc = unsafe { sys::RARRAY_LEN(argv) }; let mut vec_args: Vec<String> = vec![]; for i in 0..argc { let element = unsafe { sys::mrb_ary_ref(mrb, argv, i) }; vec_args.push(mferuby::mruby_str_to_rust_string(element).unwrap()) ;
  77. mruby-docopt pub extern "C" fn parse(mrb: *mut sys::mrb_state, this: sys::mrb_value)

    -> sys::mrb_value { let mrb_obj = mferuby::Mrb::new(mrb); let mut usage: sys::mrb_value = unsafe { mem::uninitialized() }; let mut argv: sys::mrb_value = unsafe { mem::uninitialized() }; unsafe { sys::mrb_get_args(mrb, cstr!("SA"), &mut usage, &mut argv); } let rust_usage = mferuby::mruby_str_to_rust_string(usage).unwrap(); let argc = unsafe { sys::RARRAY_LEN(argv) }; let mut vec_args: Vec<String> = vec![]; for i in 0..argc { let element = unsafe { sys::mrb_ary_ref(mrb, argv, i) }; vec_args.push(mferuby::mruby_str_to_rust_string(element).unwrap()) ;
  78. mruby-docopt pub extern "C" fn parse(mrb: *mut sys::mrb_state, this: sys::mrb_value)

    -> sys::mrb_value { let mrb_obj = mferuby::Mrb::new(mrb); let mut usage: sys::mrb_value = unsafe { mem::uninitialized() }; let mut argv: sys::mrb_value = unsafe { mem::uninitialized() }; unsafe { sys::mrb_get_args(mrb, cstr!("SA"), &mut usage, &mut argv); } let rust_usage = mferuby::mruby_str_to_rust_string(usage).unwrap(); let argc = unsafe { sys::RARRAY_LEN(argv) }; let mut vec_args: Vec<String> = vec![]; for i in 0..argc { let element = unsafe { sys::mrb_ary_ref(mrb, argv, i) }; vec_args.push(mferuby::mruby_str_to_rust_string(element).unwrap()) ;
  79. let result = docopt::Docopt::new(rust_usage) .and_then(|d| d.help(false).argv(vec_args.into_iter()).parse()); match result { Ok(args)

    => { let args = Box::new(args); let klass = unsafe { sys::mrb_class_get_under(mrb, sys::mrb_module_get(mrb, cstr!("Docopt")), cstr!("Options")) }; unsafe { sys::mrb_obj_value(mrb_obj.data_object_alloc::<docopt::ArgvMap>(kl ass, args, &docopt_option_type)) } }, Err(e) => unsafe { println!("ERROR: {:?}", e); sys::nil() }, } }
  80. let result = docopt::Docopt::new(rust_usage) .and_then(|d| d.help(false).argv(vec_args.into_iter()).parse()); match result { Ok(args)

    => { let args = Box::new(args); let klass = unsafe { sys::mrb_class_get_under(mrb, sys::mrb_module_get(mrb, cstr!("Docopt")), cstr!("Options")) }; unsafe { sys::mrb_obj_value(mrb_obj.data_object_alloc::<docopt::ArgvMap>(kl ass, args, &docopt_option_type)) } }, Err(e) => unsafe { println!("ERROR: {:?}", e); sys::nil() }, } }
  81. let result = docopt::Docopt::new(rust_usage) .and_then(|d| d.help(false).argv(vec_args.into_iter()).parse()); match result { Ok(args)

    => { let args = Box::new(args); let klass = unsafe { sys::mrb_class_get_under(mrb, sys::mrb_module_get(mrb, cstr!("Docopt")), cstr!("Options")) }; unsafe { sys::mrb_obj_value(mrb_obj.data_object_alloc::<docopt::ArgvMap>(kl ass, args, &docopt_option_type)) } }, Err(e) => unsafe { println!("ERROR: {:?}", e); sys::nil() }, } }
  82. class CommandLine USAGE = <<USAGE Heroku Docker Ruby Util. Usage:

    ruby-util detect-ruby <gemfile-path> ruby-util detect-gem <profiled-path> <gem> ruby-util install-ruby <gemfile-path> <install-path> <profiled-path> Options: -h --help Show this message USAGE def run(argv) options = Docopt.parse(USAGE, argv) if options["detect-ruby"] puts CLI::DetectRuby.new(File.read(options["<gemfile-path>"])).ruby_ver sion
  83. mruby-docopt in the wild --- a/mrbgem.rake +++ b/mrbgem.rake @@ -12,6

    +12,7 @@ MRuby::Gem::Specification.new('heroku-docker-ruby-util') do |spec| spec.add_dependency 'mruby-stringio', github: 'ksss/mruby-stringio' spec.add_dependency 'mruby-process', github: 'hone/mruby-process', branch: 'header' spec.add_dependency 'mruby-io', github: 'hone/mruby-io', branch: 'popen_status' + spec.add_dependency 'mruby-docopt', github: 'hone/mruby-docopt', branch: 'rust' # test deps spec.add_dependency 'mruby-time', core: 'mruby-time'
  84. What's Next?

  85. mferuby • Cover more mruby C API with Rust binding

    • Create higher level abstractions • Using mferuby shouldn't require "unsafe"
  86. mrbgems around low level rust crates

  87. tokio.rs

  88. File I/O

  89. Integrate with Helix

  90. Ruby hasn't had a good Windows history

  91. mruby doesn't have to repeat this

  92. mruby+rust can create a Windows friendly ecosystem

  93. Let's make ruby better for everyone

  94. Thank You @hone02 https://github.com/hone/mferuby https://github.com/hone/mruby-docopt