Ruby Kaigi 2017 - C how to supercharge your Ruby with Rubex - Speaker Deck

Tweet

Tweet

Slide 1

Slide 1 text

Namaste! こにちは！

Slide 2

Slide 2 text

Sameer Deshmukh @v0dro @v0dro

Slide 3

Slide 3 text

India | Pune

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Master’s Degree Student(HPC) Tokyo Institute of Technology

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Ruby Science Foundation www.sciruby.com @sciruby @sciruby

Slide 9

Slide 9 text

Rubex: Highly productive C extensions.

Slide 10

Slide 10 text

Ruby is an awesome language, but it is slow.

Slide 11

Slide 11 text

Ruby speed reliability C

Slide 12

Slide 12 text

Ruby speed reliability C Nokogiri Nokogiri::XML() fast_blank String#blank? libxml Handwritten C

Slide 13

Slide 13 text

C extensions have BIG problems

Slide 14

Slide 14 text

Difficult and irritating to write. Steep learning curve. Lots of scaffolding code.

Slide 15

Slide 15 text

Debugging is time consuming.

Slide 16

Slide 16 text

Manually bootstrap the extension with Ruby.

Slide 17

Slide 17 text

Need to care about small things™*. *Matz at RDRC 2016

Slide 18

Slide 18 text

Various solutions exist (partly) ● Ruby inline. – Doesn’t scale. ● FFI. – Reductive and manual compilation. ● SWIG. – Evil, unreadable wrappers. ● Helix. – Entirely new language/paradigm.

Slide 19

Slide 19 text

Ideal solution: Super-fast (and nice) Ruby.

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

Rubex: A language with the elegance of Ruby and the power of C.

Slide 22

Slide 22 text

Ruby vs. Rubex Ruby program Rubex program def add(int a,int b) return a + b end def add(a, b) return a + b end

Slide 23

Slide 23 text

Rubex code C code CRuby runtime Language which looks like Ruby. C Code ready to interface with Ruby VM. Code actually runs here.

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

["a", "b", "see", "d" ... ]

Slide 27

Slide 27 text

{ "a" => 0, "b" => 1, "see" => 2, "d" => 4 ... }

Slide 28

Slide 28 text

array.each_with_index.to_h

Slide 29

Slide 29 text

class Array2Hash def self.convert(arr a) long int i = a.size, j = 0 hsh result = {} while j < i do result[a[j]] = j j += 1 end return result end end

Slide 30

Slide 30 text

class Array2Hash def self.convert(arr a) long int i = a.size, j = 0 hsh result = {} while j < i do result[a[j]] = j j += 1 end return result end end

Slide 31

Slide 31 text

class Array2Hash def self.convert(arr a) long int i = a.size, j = 0 hsh result = {} while j < i do result[a[j]] = j j += 1 end return result end end

Slide 32

Slide 32 text

class Array2Hash def self.convert(arr a) long int i = a.size, j = 0 hsh result = {} while j < i do result[a[j]] = j j += 1 end return result end end

Slide 33

Slide 33 text

class Array2Hash def self.convert(arr a) long int i = a.size, j = 0 hsh result = {} while j < i do result[a[j]] = j j += 1 end return result end end

Slide 34

Slide 34 text

require 'array2hash.so' Array2Hash.convert array

Slide 35

Slide 35 text

Benchmarks Warming up -------------------------------------- convert 368.000 i/100ms each_with_index.to_h 236.000 i/100ms Calculating ------------------------------------- convert 3.488k (± 9.8%) i/s - 17.296k in 5.012260s each_with_index.to_h 2.192k (± 8.3%) i/s - 11.092k in 5.097432s Comparison: convert: 3487.8 i/s each_with_index.to_h: 2192.3 i/s - 1.59x slower

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

I’m cold. I need a struct blanket.

Slide 39

Slide 39 text

struct blanket { int warmth_factor; char* owner; float len, breadth; };

Slide 40

Slide 40 text

● GC marking of Ruby objects. ● Memory deallocation. ● Write an extconf.rb. ● struct rb_data_type_t. ● TypedData_Make_Struct(). ● TypedData_Get_Struct(). ● rb_define_instance_method(). ● rb_define_class(). ● rb_define_alloc_func().

Slide 41

Slide 41 text

struct blanket int warmth_factor char* owner float len, breadth end

Slide 42

Slide 42 text

class BlanketWrapper attach blanket def initialize(warmth_factor, owner, len, breadth) data$.blanket.warmth_factor = warmth_factor data$.blanket.owner = owner data$.blanket.len = len data$.blanket.breadth = breadth end def warmth_factor return data$.blanket.warmth_factor end # ... more code for blanket interface. end

Slide 43

Slide 43 text

class BlanketWrapper attach blanket def initialize(warmth_factor, owner, len, breadth) data$.blanket.warmth_factor = warmth_factor data$.blanket.owner = owner data$.blanket.len = len data$.blanket.breadth = breadth end def warmth_factor return data$.blanket.warmth_factor end # ... more code for blanket interface. end

Slide 44

Slide 44 text

class BlanketWrapper attach blanket def initialize(warmth_factor, owner, len, breadth) data$.blanket.warmth_factor = warmth_factor data$.blanket.owner = owner data$.blanket.len = len data$.blanket.breadth = breadth end def warmth_factor return data$.blanket.warmth_factor end # ... more code for blanket interface. end

Slide 45

Slide 45 text

class BlanketWrapper attach blanket def initialize(warmth_factor, owner, len, breadth) data$.blanket.warmth_factor = warmth_factor data$.blanket.owner = owner data$.blanket.len = len data$.blanket.breadth = breadth end def warmth_factor return data$.blanket.warmth_factor end # ... more code for blanket interface. end

Slide 46

Slide 46 text

class BlanketWrapper attach blanket def initialize(warmth_factor, owner, len, breadth) data$.blanket.warmth_factor = warmth_factor data$.blanket.owner = owner data$.blanket.len = len data$.blanket.breadth = breadth end def warmth_factor return data$.blanket.warmth_factor end # ... more code for blanket interface. end

Slide 47

Slide 47 text

class BlanketWrapper attach blanket def initialize(warmth_factor, owner, len, breadth) data$.blanket.warmth_factor = warmth_factor data$.blanket.owner = owner data$.blanket.len = len data$.blanket.breadth = breadth end def warmth_factor return data$.blanket.warmth_factor end # ... more code for blanket interface. end

Slide 48

Slide 48 text

class BlanketWrapper attach blanket def initialize(warmth_factor, owner, len, breadth) data$.blanket.warmth_factor = warmth_factor data$.blanket.owner = owner data$.blanket.len = len data$.blanket.breadth = breadth end def warmth_factor return data$.blanket.warmth_factor end # ... more code for blanket interface. end

Slide 49

Slide 49 text

class BlanketWrapper attach blanket def initialize(warmth_factor, owner, len, breadth) data$.blanket.warmth_factor = warmth_factor data$.blanket.owner = owner data$.blanket.len = len data$.blanket.breadth = breadth end def warmth_factor return data$.blanket.warmth_factor end # ... more code for blanket interface. end

Slide 50

Slide 50 text

Rubex struct wrapping ● ~3x reduction in LoC written. ● Friendly, elegant Ruby-like interface. ● No compromise in speed. ● No C code!

Slide 51

Slide 51 text

Exception Handling

Slide 52

Slide 52 text

Many C functions need to be used ● rb_raise() for raising error. ● rb_rescue(), rb_rescue2(), rb_protect(), rb_ensure() for rescue and ensure blocks. ● rb_errinfo() for getting the last error raised. ● rb_set_errinfo(Qnil) for resetting error information.

Slide 53

Slide 53 text

Workflow becomes complex ● Almost zero compliance with begin-ensure block workflow. ● Create C function callbacks. ● Manually catch and rescue exceptions. ● Inflexibility in sending data to callbacks.

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

int i = accept_number() begin raise(ArgumentError) if i == 3 raise(FooBarError) if i == 5 rescue ArgumentError i += 1 rescue FooBarError i += 2 ensure i += 10 end

Slide 56

Slide 56 text

Interfacing 3rd party C libraries using Rubex.

Slide 57

Slide 57 text

Building a Rubex wrapper for libcsv – a C library for parsing CSV files.

Slide 58

Slide 58 text

3 steps to write libcsv wrapper 1. Tell Rubex about the functions /types/constants and header files that you will be using. 2. Use functions in normal Rubex code. 3. Compile and call in your Ruby script.

Slide 59

Slide 59 text

lib "csv.h", link: "csv" struct csv_parser; end # more types ... int CSV_STRICT_FINI # more macros ... int csv_init(csv_parser, unsigned char) size_t csv_parse( csv_parser *p, void *, size_t, void (*cb1)(void *, size_t, void *), void (*cb2)(int, void *), void * ) end

Slide 60

Slide 60 text

lib "csv.h", link: "csv" struct csv_parser; end # more types ... int CSV_STRICT_FINI # more macros ... int csv_init(csv_parser, unsigned char) size_t csv_parse( csv_parser *, void *, size_t, void (*cb1)(void *, size_t, void *), void (*cb2)(int, void *), void * ) end

Slide 61

Slide 61 text

lib "csv.h", link: "csv" struct csv_parser; end # more types ... int CSV_STRICT_FINI # more macros ... int csv_init(csv_parser, unsigned char) size_t csv_parse( csv_parser *, void *, size_t, void (*cb1)(void *, size_t, void *), void (*cb2)(int, void *), void * ) end

Slide 62

Slide 62 text

lib "csv.h", link: "csv" struct csv_parser; end # more types ... int CSV_STRICT_FINI # more macros ... int csv_init(csv_parser, unsigned char) size_t csv_parse( csv_parser *, void *, size_t, void (*cb1)(void *, size_t, void *), void (*cb2)(int, void *), void * ) end

Slide 63

Slide 63 text

lib "csv.h", link: "csv" struct csv_parser; end # more types ... int CSV_STRICT_FINI # more macros ... int csv_init(csv_parser, unsigned char) size_t csv_parse( csv_parser *, void *, size_t, void (*cb1)(void *, size_t, void *), void (*cb2)(int, void *), void * ) end

Slide 64

Slide 64 text

lib "csv.h", link: "csv" struct csv_parser; end # more types ... int CSV_STRICT_FINI # more macros ... int csv_init(csv_parser, unsigned char) size_t csv_parse( csv_parser *, void *, size_t, void (*cb1)(void *, size_t, void *), void (*cb2)(int, void *), void * ) end

Slide 65

Slide 65 text

lib "csv.h", link: "csv" struct csv_parser; end # more types ... int CSV_STRICT_FINI # more macros ... int csv_init(csv_parser, unsigned char) size_t csv_parse( csv_parser *, void *, size_t, void (*cb1)(void *, size_t, void *), void (*cb2)(int, void *), void * ) end

Slide 66

Slide 66 text

# Store internal state struct rcsv_metadata size_t current_col size_t current_row object last_entry object result end

Slide 67

Slide 67 text

class LibCSVWrapper def self.parse(file_name, opts) # allocate memory, initialize variables ... begin if str_len != csv_parse(&cp, string, str_len, &eof_callback, &eol_callback, &meta) # check and raise errors end ensure # free allocated data end # return computed result end end

Slide 68

Slide 68 text

class LibCSVWrapper def self.parse(file_name, opts) # allocate memory, initialize variables ... begin if str_len != csv_parse(&cp, string, str_len, &eof_callback, &eol_callback, &meta) # check and raise errors end ensure # free allocated data end # return computed result end end

Slide 69

Slide 69 text

class LibCSVWrapper def self.parse(file_name, opts) # allocate memory, initialize variables ... begin if str_len != csv_parse(&cp, string, str_len, &eof_callback, &eol_callback, &meta) # check and raise errors end ensure # free allocated data end # return computed result end end

Slide 70

Slide 70 text

class LibCSVWrapper def self.parse(file_name, opts) # allocate memory, initialize variables ... begin if str_len != csv_parse(&cp, string, str_len, &eof_callback, &eol_callback, &meta) # check and raise errors end ensure # free allocated data end # return computed result end end

Slide 71

Slide 71 text

https://github.com/sciruby/rubex

Slide 72

Slide 72 text

Notable Rubex examples ● Rubex repo examples/ folder. – Fully functional libcsv wrapper for reading CSV files written entirely in Rubex. ● Array2Hash gem – https://github.com/v0dro/array2hash

Slide 73

Slide 73 text

Detailed Docs and Tutorial ● REFERENCE.md. – Complete specification of the entire language. ● TUTORIAL.md. – Quick, easy to use explanation with code samples.

Slide 74

Slide 74 text

Conclusion ● Rubex is a fast and productive way of writing Ruby C extensions. ● Provides users with the elegance of Ruby and the power of C while following the principle of least surprise. ● Future work will involve ability to release the GIL, interface with GPUs and Rubex APIs for gems.

Slide 75

Slide 75 text

Acknowledgements ● Ruby Association Grant 2016. ● Kenta Murata and Koichi Sasada for their support and mentorship. ● Fukuoka Ruby Award 2016.

Slide 76

Slide 76 text

I haz SciRuby stickers. ^_^

Slide 77

Slide 77 text

THANK YOU! どうもありがとうございます！