Extending Ruby

Transcript

Extending
Ruby
Frederick Cheung
CTO, dressipi.com
@fglc2 / spacevatican.org
Saturday, 28 September 13

Why?
• Performance
• Platform speciﬁc functionality
• Access to best in class libraries

Downsides
• Slower to write - ruby is very concise and
expressive
• Generally harder
• Memory management
• ruby c api
• Crashes harder when things go wrong

View Slide

Options
• Native java code
• “Classic” C-extension
• SWIG
• RubyInline
• RICE
• FFI

View Slide

Calling java from JRuby
• Just do it - stupidly easy
• JRuby handles most of the type conversion, method
name conversion
require 'java'
java_import java.lang.System
version = System.getProperties.get("java.runtime.version")
version = System.properties["java.runtime.version"]

View Slide

• Can subclass java classes in ruby
• Can implement a java interface in ruby
• Rescue/raise java exceptions from ruby
• Blocks handled nicely
java.lang.Thread.new do
puts 'hi'
end

View Slide

Ruby Motion / MacRuby
• Uses OS X BridgeSupport mechanism for extracting
available methods/signatures
• No manual work required
• Only works on OS X

View Slide

C extensions
• C code written using the C Ruby api - like the
internals of ruby itself
• Intricacies of the API not well documented
(README.EXT, headers)
• C api can change between versions (but then so can
the ruby API)

View Slide

Compatibility
• C-Ruby
• Rubinius (compatibility with MRI C api is a design
goal)
• partial compatibility with jruby (disabled by default)

View Slide

What’s in a C extension?

View Slide

static VALUE rb_keychain_find(int argc, VALUE *argv, VALUE self){
...
}
void Init_keychain(){
rb_cKeychain = rb_const_get(rb_cObject, rb_intern("Keychain"));
rb_define_singleton_method(rb_cKeychain, "find",
RUBY_METHOD_FUNC(rb_keychain_find),
-1);
}
C functions attached to ruby objects

View Slide

...
}
-1);
}

View Slide

...
}
-1);
}

View Slide

Native data structures
Person *person = ...
VALUE ruby_object = Data_Wrap_Struct(rb_cPerson,
NULL,
free_person,
person);
Person *person=NULL;
Data_Get_Struct(ruby_object, Person, person);

View Slide

NULL,
free_person,
person);

View Slide

NULL,
free_person,
person);

View Slide

Convert/check types
• rb_float_new / RFLOAT_VALUE
• CheckType(foo, T_STRING)
• StringValueCStr(foo)
• FIX2INT, INT2FIX, LL2NUM, ...
• NIL_P
• RTEST

View Slide

def find(first_or_all, kind, options={})
if options[:some_option]
...
end
end
Gets verbose quickly

View Slide

VALUE kind;
VALUE options;
VALUE first_or_all;
rb_scan_args(argc, argv, "2:", &first_or_all, &kind, &options);
Check_Type(first_or_all, T_SYMBOL);
Check_Type(kind, T_STRING);
if(options){
if(RTEST(rb_hash_aref(options,
ID2SYM(rb_intern("some_option"))))) {
...
}
}
}

View Slide

VALUE kind;
VALUE options;
VALUE first_or_all;
if(options){
...
}
}
}

View Slide

VALUE kind;
VALUE options;
VALUE first_or_all;
if(options){
...
}
}
}

View Slide

VALUE kind;
VALUE options;
VALUE first_or_all;
if(options){
...
}
}
}

View Slide

static VALUE doSomething(VALUE yielded_object,
VALUE rb_context,int argc, VALUE *argv){
Context *context = NULL;
Data_Get_Struct( rb_context, Context, context);
...
return Qnil;
}
Context *context = ...
VALUE wrapped_struct = Data_Wrap_Struct(rb_cFindContext,
NULL,NULL,context);
rb_block_call(rb_cUser, rb_intern("find_each"), 0, NULL,
RUBY_METHOD_FUNC(doSomething), wrapped_struct);
Call a method with a block

View Slide

...
return Qnil;
}
NULL,NULL,context);

View Slide

...
return Qnil;
}
NULL,NULL,context);

View Slide

• Writing apis that are rubylike is difﬁcult
• Anything is possible but it can be quite laborious
• Not much typesafety - nearly everything is a VALUE
• Sometimes unhelpful api naming: rb_str_new,
rb_str_new2, rb_str_new3, rb_str_new4, rb_str_new5

View Slide

• Generates code for C extensions automatically from
a marked up header ﬁle
• Can target many languages (ruby, python, php,
ocaml, perl, ...)
• Generated code is pretty illegible
• Interfaces often unnatural - need wrapping
SWIG

View Slide

RubyInline
• Write C extensions without some of the hassle
• generates, compiles and loads extension at runtime
• doesn’t work in irb

View Slide

require 'inline'
class Factorial
inline :C do |builder|
builder.c_singleton <<-SRC
long calculate(long n){
long result = 1;
for(long i = 2; i<=n; i++){
result *= i;
}
return result;
}
SRC
end
end

View Slide

require 'inline'
class Factorial
long result = 1;
result *= i;
}
return result;
}
SRC
end
end

View Slide

require 'inline'
class Factorial
long result = 1;
result *= i;
}
return result;
}
SRC
end
end

View Slide

static VALUE calculate(VALUE self, VALUE _n) {
long n = NUM2LONG(_n);
long result = 1;
result *= i;
}
return LONG2NUM(result);
}
#ifdef __cplusplus
extern "C" {
#endif
void Init_Inline_Factorial_7b051bd7f8f35cf0422f7470() {
VALUE c = rb_cObject;
c = rb_const_get(c, rb_intern("Factorial"));
rb_define_singleton_method(c, "calculate",
(VALUE(*)(ANYARGS))calculate, 1);
}
#ifdef __cplusplus
}
#endif

View Slide

long result = 1;
result *= i;
}
}
#ifdef __cplusplus
extern "C" {
#endif
}
#ifdef __cplusplus
}
#endif

View Slide

long result = 1;
result *= i;
}
}
#ifdef __cplusplus
extern "C" {
#endif
}
#ifdef __cplusplus
}
#endif

View Slide

module Foo
inline do |builder|
builder.add_compile_flags '-x c++', '-lstdc++',
'-Wall', '-Werror'
builder.prefix <<-SRC
# line #{__LINE__ + 1} "#{__FILE__}"
class foo {
...
}
SRC
builder.c_raw <<-SRC
VALUE attr_1(VALUE self){
foo *data = NULL;
Data_Get_Struct( self, foo, data);
return INT2FIX(data->get_attr_1());
}
SRC
end
end

View Slide

module Foo
inline do |builder|
'-Wall', '-Werror'
# line #{__LINE__ + 1} "#{__FILE__}"
class foo {
...
}
SRC
foo *data = NULL;
}
SRC
end
end

View Slide

module Foo
inline do |builder|
'-Wall', '-Werror'
# line #{__LINE__ + 1} "#{__FILE__}"
class foo {
...
}
SRC
foo *data = NULL;
}
SRC
end
end

View Slide

module Foo
inline do |builder|
'-Wall', '-Werror'
# line #{__LINE__ + 1} "#{__FILE__}"
class foo {
...
}
SRC
foo *data = NULL;
}
SRC
end
end

View Slide

Rubyinline - conclusion
• Easy to deploy - no extra steps required
• great for small hotspots / simple interfaces
• raises CompilationError on failure - easy to include
pure ruby & native version
• Advantages diminish as size grows, eg in general no
syntax highlighting, code completion.

View Slide

RICE
• C++ library wrapping the ruby c api
• automates some of the conversion boilerplate via
templates
• Can still call raw C api if needed

View Slide

long calculate_fibonacci(Object self, long n){
long result = 1, previous=0;
for(long i=0; i< n-1;i++){
long old_previous = previous;
previous = result;
result = old_previous + result;
}
return result;
}
extern "C"
void Init_fib(){
Class rb_cTest = define_class("Fibonnaci").
define_singleton_method("value",
&calculate_fibonacci);
}

View Slide

previous = result;
}
return result;
}
extern "C"
void Init_fib(){
}

View Slide

previous = result;
}
return result;
}
extern "C"
void Init_fib(){
}

View Slide

void some_method(String name, Object options_or_nil){
Hash options;
if(!options_or_nil.is_nil()){
options = options_or_nil;
}
...
}
extern "C" void Init_ext(){
Class rb_cTest = define_class("Test").
define_method("some_method",(
Arg("name"),
Arg("options_or_nil")=Nil));
}
Optional arguments

View Slide

Hash options;
}
...
}
Arg("name"),
}
Optional arguments

View Slide

Hash options;
}
...
}
Arg("name"),
}
Optional arguments

View Slide

Hash options;
}
...
}
Arg("name"),
}
Optional arguments

View Slide

Type conversions
Hash options = ...
int value = options["amount"];
VALUE amount_ = rb_hash_aref(options,
rb_str_new2("amount"));
int amount = FIX2INT(amount_);
OR

View Slide

Type conversions
Hash options = ...
OR

View Slide

Type conversions
Hash options = ...
OR

View Slide

Wrapping C++ objects
extern "C"
void Init_person(){
Data_Type rb_cPerson =
define_class("Person").
define_constructor(Constructor()).
define_method("name", &Person::get_name).
define_method("name=",&Person::set_name).
define_method("assimilate",&Person::assimilate).
}

View Slide

extern "C"
void Init_person(){
}

View Slide

extern "C"
void Init_person(){
}

View Slide

class Person {
public:
Person():m_name(""){}
const std::string& get_name() const{
return m_name;
}
void set_name(const std::string& new_name){
m_name = new_name;
}
void assimilate(Person other_person){
...
}
private:
std::string m_name;
};

View Slide

class Person {
public:
return m_name;
}
m_name = new_name;
}
...
}
private:
std::string m_name;
};

View Slide

class Person {
public:
return m_name;
}
m_name = new_name;
}
...
}
private:
std::string m_name;
};

View Slide

class Person {
public:
return m_name;
}
m_name = new_name;
}
...
}
private:
std::string m_name;
};

View Slide

• bridges C++ and ruby exceptions
• implements STL iterators for Array, Hash
• Loads of details at http://jasonroelofs.com/
2010/02/23/how-rice-works/
RICE

View Slide

And now, for something
completely different

View Slide

FFI
• based on libfﬁ
• Ruby DSL for binding C functions / global variables
• No compiler toolchain required, no C code to write
• Supported by jruby, mri, rubinius

View Slide

require 'ffi'
module C
extend FFI::Library
ffi_lib 'c'
attach_function 'puts', [:string], :int
end
C.puts "Hello world"

View Slide

require 'ffi'
module C
extend FFI::Library
ffi_lib 'c'
end

View Slide

require 'ffi'
module C
extend FFI::Library
ffi_lib 'c'
end

View Slide

require 'ffi'
module C
extend FFI::Library
ffi_lib 'c'
end

View Slide

module LibC
extend FFI::Library
ffi_lib FFI::Library::LIBC
callback :qsort_cmp, [:pointer, :pointer ], :int
attach_function :qsort,
[:pointer, :ulong, :ulong, :qsort_cmp],
:int
end
def sort(array_of_ints)
p = FFI::MemoryPointer.new(:int32, array_of_ints.size)
p.put_array_of_int32(0, array_of_ints)
LibC.qsort(p, array_of_ints.size, 4) do |p1, p2|
i1 = p1.get_int32(0)
i1 <=> i2
end
p.get_array_of_int32(0, array_of_ints.size)
end

View Slide

module LibC
extend FFI::Library
callback :qsort_cmp, [:pointer, :pointer ], :int
:int
end
i1 <=> i2
end
end

View Slide

module LibC
extend FFI::Library
callback :qsort_cmp, [:pointer, :pointer], :int
:int
end
i1 <=> i2
end
end

View Slide

module LibC
extend FFI::Library
callback :qsort_cmp, [:pointer, :pointer], :int
:int
end
i1 <=> i2
end
end

View Slide

module LibC
extend FFI::Library
class Timezone < FFI::Struct
layout :tz_minuteswest, :int,
:tz_dsttime, :int
end
class Timeval < FFI::Struct
layout :tv_sec, :time_t,
:tv_usec, :suseconds_t
end
attach_function :gettimeofday, [Timeval, Timezone], :int
end
out = LibC::Timeval.new
LibC.gettimeofday(out, nil)
puts "the time according to libc is #{out[:tv_sec]}"

View Slide

module LibC
extend FFI::Library
:tz_dsttime, :int
end
end
end

View Slide

module LibC
extend FFI::Library
:tz_dsttime, :int
end
end
end

View Slide

module LibC
extend FFI::Library
:tz_dsttime, :int
end
end
end

View Slide

Beyond toy examples
• OS X Keychain wrapper written using C api, FFI, RICE
• https://github.com/fcheung/keychain ~ 1000 lines of ruby, ~
600 lines of real code
• https://github.com/fcheung/keychain_c/ ~ 700 lines of C + a
small amount of ruby to autogenerate accessors
• https://github.com/fcheung/keychain_rice ~ 800 lines of C++
(no auto generated accessors)
• All versions pass the same set of specs

View Slide

def unlock! password=nil
if password
password = password.encode(Encoding::UTF_8)
status = Sec.SecKeychainUnlock self,
password.bytesize,
password,
1
else
status = Sec.SecKeychainUnlock self, 0, nil, 0
end
Sec.check_osstatus status
end
FFI

View Slide

RICE
void Keychain::unlock(const Object password_or_nil){
OSStatus result = 0;
if(password_or_nil.test()){
String password = password_or_nil.call("encode","UTF-8");
result = SecKeychainUnlock(m_keychain,
(UInt32)password.length(),
(UInt8*)password.c_str(),
true);
}
else{
result = SecKeychainUnlock(m_keychain,0,NULL,false);
}
CheckOSStatusOrRaise(result);
}

View Slide

C
static VALUE rb_keychain_unlock(int argc, VALUE *argv, VALUE self){
OSStatus result = noErr;
VALUE password;
SecKeychainRef keychain=NULL;
Data_Get_Struct(self, struct OpaqueSecKeychainRef, keychain);
rb_scan_args(argc, argv, "01", &password);
if(password){
StringValue(password);
password = rb_str_export_to_enc(password, rb_utf8_encoding());
result = SecKeychainUnlock(keychain, (UInt32)RSTRING_LEN(password),
(UInt8*)RSTRING_PTR(password), true);
}else{
result = SecKeychainUnlock(keychain,0,NULL,false);
}
CheckOSStatusOrRaise(result);
return Qnil;
}

View Slide

• Great when you have a C library to wrap
• Small performance penalty over a C extension
(often irrelevant)
• Much easier/nicer (in my experience) than writing C
extensions
• Handles many difﬁcult edge cases
FFI summary

View Slide

The good stuff
• No knowledge of MRI C api required
• Better compatibility across implementations
• All glue code is ruby - easier to write and easier to
be ruby-like
• less typechecking/wrapping/unwrapping boilerplate
• FFI yielded more reusable components
(core_foundation gem)

View Slide

But...
• Takes a little getting used to
• Constants / enums / #deﬁnes are a pain
• you still need to understand C, memory management
• Some incompatibilities between FFI implementations
• harder to create narrow slices of functionality
• Still crashes just as hard if you mess it up

View Slide

Other languages
• C is the lingua franca
• Wrap with C, then use previous techniques

View Slide

Objective-C
• Not handled by fﬁ out of the box
• Can write wrapper library exposing C entry points

View Slide

Or...
• Objective-C method calls are just calls to
objc_msgSend
• Objective-C supports run time introspection of
classes, methods etc.
• Can we use some ruby metaprogramming to bridge
the two?

View Slide

Thank You!
• [email protected]
• @fglc2
• github.com/fcheung
• http://spacevatican.org

View Slide

Extending Ruby

Extending Ruby

Frederick Cheung

More Decks by Frederick Cheung

Other Decks in Programming

Featured

Transcript