Slide 1
Slide 1 text
Road to Go gem
RubyKaigi 2025 / #rubykaigi #rubykaigiC
pixiv Inc.
sue445
2025.04.18
Slide 2
Slide 2 text
2
Hello!
Slide 3
Slide 3 text
3
My name is Go
The Go gopher was designed by Renee French. (http://reneefrench.blogspot.com/) The design is licensed under the
Creative Commons 4.0 Attributions license. Read this article for more details: https://blog.golang.org/gopher
Slide 4
Slide 4 text
4
About sue445
● Go Sueyoshi (a.k.a. sue445, sue-san)
● https://x.com/sue445
● https://github.com/sue445
● Gopher since 1982
● Shibuya.rb
● Tokyu.rb
sue445
Slide 5
Slide 5 text
5
Day job at pixiv.Inc
● Infrastructure Unit
● AWS and Google Cloud
○ Organization Owner
○ Solution Architect
● Certifications possessed
○ AWS Certified Solutions Architect –
Professional (SAP-C01)
○ Google Cloud Professional Cloud
Architect
sue445
Slide 6
Slide 6 text
6
Day job at pixiv.Inc
● on-premises Kubernetes
● Maintainer of In-house tools (GitLab, Sentry
etc)
● etc etc etc!!!!!!!!!!
sue445
Slide 7
Slide 7 text
7
CM. About pixiv Inc.
https://www.pixiv.co.jp/service/
10years+ 10years+ 10years+
NEW!
Slide 8
Slide 8 text
https://conference.pixiv.co.jp/2025/rubymusicmixin
8
CM. RubyMusicMixin 2025
Slide 9
Slide 9 text
9
CM. RubyMusicMixin 2025
RubyMusicMixin
2025
We are here
Slide 10
Slide 10 text
10
Riverside (川) #rubyriver
Slide 11
Slide 11 text
● All about native extension with Go
● Tips for creating native extensions in a new language
11
Goal in this talk
Slide 12
Slide 12 text
● Let's recall RubyKaigi 2015
● Getting started with Go gem
● About go-gem-wrapper
● Road to go-gem-wrapper
● Use “ruby.h” in go building
● Add --ext=go to bundle gem command
● Pros/Cons of Go gem
● Real use case
12
Agenda
Slide 13
Slide 13 text
● Using https://github.com/ruby-go-gem/go-gem-wrapper makes it very easy
to make native extension with Go
● We can write Ruby gem in Go in areas where Ruby is not good at (Go is very
good at)
● goroutine is very good!
13
Conclusion
Slide 14
Slide 14 text
https://rubykaigi.org/2015/presentations/mmasaki/
14
Let’s recall RubyKaigi 2015
Slide 15
Slide 15 text
● cgo (go build -buildmode=c-shared) was introduced at Go 1.5
● cgo creates shared object file (*.so) from Go
● Use this technology to create a native Ruby extension with Go
○ a.k.a. Go gem
15
tl;dr;
Slide 16
Slide 16 text
1. The code from 10 years ago (Go 1.5) doesn't work now (Go 1.24)
○ a later mention
2. To make a gem in Go, you need to copy and paste a lot of patches.
○ e.g.
■ ext/gem_name/extconf.rb
■ ext/gem_name/my_gem.go
■ ext/gem_name/go.mod
■ bindings
■ etc…
16
Past issues
Slide 17
Slide 17 text
https://github.com/mmasaki/memberlist/blob/master/ext/memberlist/wrapper.go
17
e.g. 10 years ago Go gem
Slide 18
Slide 18 text
● Technically possible. But it takes a lot of work.
18
Past issues
Slide 19
Slide 19 text
● Library of patches that needed to be done manually
● There was no standard package manager for Go at 10 years ago
● But today, we have Go module
19
github.com/ruby-go-gem/go-gem-wrapper
Slide 20
Slide 20 text
1. bundle gem –ext=c
2. Run “patch_for_go_gem.rb”
3. go get -u github.com/ruby-go-gem/go-gem-wrapper@latest
4. Write gem using Go!
20
Getting started with Go gem
Slide 21
Slide 21 text
$ bundle gem example --ext=c
Creating gem 'example'...
create example/Gemfile
create example/lib/example.rb
create example/lib/example/version.rb
(snip)
create example/ext/example/extconf.rb
create example/ext/example/example.h
create example/ext/example/example.c
Gem 'gem_name' was successfully created. For more information on making a
RubyGem visit https://bundler.io/guides/creating_gem.html
bundle gem –ext=c
21
Slide 22
Slide 22 text
● https://github.com/ruby-go-gem/go-gem-wrapper/tree/main/_tools/patch_
for_go_gem
● Patch to make a gem into a Go gem right after bundle gem
● This is a temporary tool until bundle gem --ext=go is added to the bundler
○ c.f. https://github.com/rubygems/rubygems/pull/8183 (a later mention)
22
Run “patch_for_go_gem.rb”
Slide 23
Slide 23 text
$ ruby patch_for_go_gem.rb --file example/example.gemspec
[INFO] /path/to/example/ext/example/example.go is created
[INFO] /path/to/example/ext/example/go.mod is created
[INFO] /path/to/example/ext/example/example.c is updated
[INFO] /path/to/example/ext/example/extconf.rb is updated
[INFO] example/example.gemspec is updated
Run “patch_for_go_gem.rb”
23
Slide 24
Slide 24 text
package main
/*
#include "example.h"
*/
import "C"
import (
"github.com/ruby-go-gem/go-gem-wrapper/ruby"
)
//export Init_example
func Init_example() {
rb_mExample := ruby.RbDefineModule("Example")
}
func main() {
}
ext//＜gem_name>.go
24
Slide 25
Slide 25 text
#include "my_gem.h"
#include "_cgo_export.h" // Added
ext//.c
25
Slide 26
Slide 26 text
require "mkmf"
require "go_gem/mkmf" # Added
append_cflags("-fvisibility=hidden")
# create_makefile("example/example")
create_go_makefile("example/example") # Added
ext//extconf.rb
26
Slide 27
Slide 27 text
Gem::Specification.new do |spec|
spec.add_dependency "go_gem" # Added
end
.gemspec
27
Slide 28
Slide 28 text
module Example
def self.sum(a, b)
a + b
end
end
e.g. Impl Example#sum
28
Slide 29
Slide 29 text
//export rb_example_sum
func rb_example_sum(_ C.VALUE, a C.VALUE, b C.VALUE) C.VALUE {
aLong := ruby.NUM2LONG(ruby.VALUE(a))
bLong := ruby.NUM2LONG(ruby.VALUE(b))
sum := aLong + bLong
return C.VALUE(ruby.LONG2NUM(sum))
}
ext//＜gem_name>.go
29
Slide 30
Slide 30 text
package main
/*
#include "example.h"
// Append this
VALUE rb_example_sum(VALUE self, VALUE a, VALUE b);
*/
import "C"
ext//＜gem_name>.go
30
Slide 31
Slide 31 text
//export Init_example
func Init_example() {
rb_mExample := ruby.RbDefineModule("Example")
// Append this
ruby.RbDefineSingletonMethod(rb_mExample, "sum", C.rb_example_sum, 2)
}
ext//＜gem_name>.go
31
Slide 32
Slide 32 text
● We can write a gem in Go just as you write a gem in C!
32
Easy, right?
Slide 33
Slide 33 text
● Go module: github.com/ruby-go-gem/go-gem-wrapper
● Ruby gem: https://rubygems.org/gems/go_gem
● Requiremets
○ Go 1.23+
○ Ruby 3.3+
○ glibc (a later mention)
33
About go-gem-wrapper
Slide 34
Slide 34 text
github.com/ruby-go-gem/go-gem-wrapper is Go module for Go gem
34
About github.com/ruby-go-gem/go-gem-wrapper
// ext/example/go.mod
module github.com/sue445/example
go 1.24
require (
github.com/ruby-go-gem/go-gem-wrapper v0.7.1
)
Slide 35
Slide 35 text
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/ruby/function_ruby_3_3_generated.go#L2687-L2697
35
e.g. RbDefineSingletonMethod
func RbDefineSingletonMethod(obj VALUE, mid string, arg3 unsafe.Pointer,
arity int) {
char, clean := string2Char(mid)
defer clean()
C.rb_define_singleton_method(C.VALUE(obj), char, toCFunctionPointer(arg3),
C.int(arity))
}
Slide 36
Slide 36 text
36
There are 1,100 functions in “ruby.h”
$ ls ruby/*_generated.go | xargs wc -l
59 ruby/enum_ruby_3_3_generated.go
57 ruby/enum_ruby_3_4_generated.go
9264 ruby/function_ruby_3_3_generated.go
9198 ruby/function_ruby_3_4_generated.go
105 ruby/type_ruby_3_3_generated.go
103 ruby/type_ruby_3_4_generated.go
18786 total
https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/ruby
Slide 37
Slide 37 text
● Helpers for compiling Go extensions for ruby
○ https://rubygems.org/gems/go_gem
○ https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/_gem
● Similar to https://rubygems.org/gems/rb_sys in Rust gem
37
About go_gem
Slide 38
Slide 38 text
require "mkmf"
require "go_gem/mkmf"
create_go_makefile("example/example")
# Pass debug flags to `go build`
create_go_makefile("example/example", go_build_args: "-gcflags='all=-N -l'")
create_go_makefile
Add a task and hacks to build Go against the “Makefile” generated by
create_makefile. (similar to create_rust_makeflile)
38
Slide 39
Slide 39 text
39
create_go_makefile
def create_go_makefile(target, srcprefix: nil, go_build_args: nil)
# (snip)
File.open("Makefile", "a") do |f|
f.write <<~MAKEFILE.gsub(/^ {8}/, "\t")
$(DLLIB): Makefile $(srcdir)/*.go
cd $(srcdir); \
CGO_CFLAGS='$(INCFLAGS)' CGO_LDFLAGS='#{ldflags}' GOFLAGS='#{goflags}' \
go build -p 4 -buildmode=c-shared -o #{current_dir}/$(DLLIB) #{go_build_args}
MAKEFILE
end
end
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/_gem/lib/go_gem/mkmf.rb
Slide 40
Slide 40 text
# Rakefile
require "go_gem/rake_task"
GoGem::RakeTask.new("example")
GoGem::RakeTask
Provides rake tasks for go test with CRuby. (a later mention)
40
Slide 41
Slide 41 text
● I had to go through many hardships to create go-gem-wrapper
● I will now introduce them
41
Road to go-gem-wrapper
Slide 42
Slide 42 text
1. cgo compatibility
2. Generate Go’s bindings from “ruby.h”
○ Type mismatch between outside and inside of module
○ C pointer
○ deprecated functions
○ Removed functions
○ Supports https://pkg.go.dev/
3. Go with CRuby
42
Road to go-gem-wrapper
Slide 43
Slide 43 text
● This technique was introduced in RubyKaigi 2015
● No problem at Go 1.5, but SEGV at Go 1.24
43
1. cgo compatibility
https://www.slideshare.net/slideshow/ruby-meets-go/56074507
Slide 44
Slide 44 text
● Current Go's string isn't a null-terminated string, so using
*(*[]byte)(unsafe.Pointer(&str)) will return a non-null-terminated byte array
and can't be *C.char
● Go 1.7+ are spec'd this way
44
Why?
Slide 45
Slide 45 text
45
e.g. Print Go string via C
package main
/*
#include
*/
import "C"
func Print(str string) {
cstr := GOSTRING_PTR(str)
C.fputs(cstr, (*C.FILE)(C.stdout))
}
func main() {
Print("ABCD")
}
Slide 46
Slide 46 text
46
OMG!
$ go run main.go
ABCDtrueallgallprootitabsbrkidledeadsync is LEAFbase of ) = <==GOGC] = pc=+Inf-Inf:
p=cas1cas2cas3cas4cas5cas6 at
m= sp= sp: lr: fp= gp= mp=) m=boolint8uintchanfuncopenstatfile in
sha1sha2timeJuneJuly/etcfalsedefersweeptestRtestWexecWhchanexecRschedsudogtimergscanm
heaptracepanicsleeparm64 cnt=gcing MB, got= ...
max=scav ptr ] = (init ms, fault and tab= top=[...],
fp:int16int32int64uint8arrayslicewriteclosegetwdlstatGreekpmullcrc32cpuidfcntlMarchAp
rilLocalerrno stringsysmontimersefenceselect, not GOROOTobject next= jobs= goid sweep
B -> % util alloc free span= prev= list=, i = code= addr=], sp= m->p= p->m=SCHED
curg= ctxt: min= max= bad ts(...)
Slide 47
Slide 47 text
47
Solution
func string2Char(str string) (*C.char, func()) {
cstr := C.CString(str)
clean := func() {
C.free(unsafe.Pointer(cstr))
}
return cstr, clean
}
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/ruby/wrapper.go#L26-L48
Slide 48
Slide 48 text
48
Solution
func RbDefineSingletonMethod(obj VALUE, mid string, arg3 unsafe.Pointer,
arity int) {
char, clean := string2Char(mid)
defer clean()
C.rb_define_singleton_method(C.VALUE(obj), char, toCFunctionPointer(arg3),
C.int(arity))
}
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/ruby/function_ruby_3_3_ge
nerated.go#L2687-L2697
Slide 49
Slide 49 text
● Go, similar to Ruby, is a very backward-compatible language.
● However, cgo has characteristics that depend on Go's internal
implementation and can be broken by Go version upgrades
● c.f. “Why Go's old code almost never stops working“
○ https://zenn.dev/catatsuy/articles/fda1e42acad421 (ja)
49
Tips
Slide 50
Slide 50 text
50
2. Generate Go’s bindings from “ruby.h”
Slide 51
Slide 51 text
● In this talk, binding is a Go function that corresponds to a CRuby function as
follows
51
2. Generate Go’s bindings from “ruby.h”
func RbDefineSingletonMethod(obj VALUE, mid string, arg3 unsafe.Pointer,
arity int) {
char, clean := string2Char(mid)
defer clean()
C.rb_define_singleton_method(C.VALUE(obj), char, toCFunctionPointer(arg3),
C.int(arity))
}
Slide 52
Slide 52 text
● Just on https://docs.ruby-lang.org/ja/latest/function/index.html , there are
less than 200
● There are about 2,000+ C functions defined in ruby.h
● About 1,100 functions can be included and actually used
● At first (about 30 functions), I wrote them by hand, but I decided to
generate them automatically from ruby.h because it's just too hard.
52
2. Generate Go’s bindings from “ruby.h”
Slide 53
Slide 53 text
1. Use existing OSS
2. Create it myself
53
Choices
Slide 54
Slide 54 text
Automatic C-Go Bindings Generator for Go Programming Language
54
https://github.com/xlab/c-for-go
Slide 55
Slide 55 text
Fixed an error when passing “ruby.h”
55
https://github.com/xlab/c-for-go/pull/171
Slide 56
Slide 56 text
● Bindings generated by c-for-go was abandoned due to lack of portability
○ e.g. Bindings generated on MacOS isn’t usable on Ubuntu
56
Result
Slide 57
Slide 57 text
● It may be due to the fact that there are many #ifdef branches in “ruby.h” for
each OS and build environment.
● e.g.
57
Result
https://github.com/ruby/ruby/blob/v3_4_2/internal/process.h#L14-L20
#ifdef HAVE_SYS_TYPES_H
# include /* for mode_t */
#endif
#ifdef _WIN32
# include "ruby/win32.h" /* for mode_t */
#endif
Slide 58
Slide 58 text
● I can change the source code for each environment we build by creating a file
similar to “ruby_linux_amd64.go” or “ruby_darwin_arm64.go” (Build
constraints)
● But this increases the amount of code, and CI also needs to prepare all build
constraints
● I gave up on it because it would be too hard maintain
○ https://github.com/ruby-go-gem/go-gem-wrapper/pull/117#issuecomm
ent-2336583098
58
Result
Slide 59
Slide 59 text
● However, I couldn't figure out how to create it…
59
Create it myself
Slide 60
Slide 60 text
60
Hint: @joker1007 ’s Rust gem talk
https://speakerdeck.com/joker1007/rustdezuo-rutree-sitterpasanorubybaindeingu?slide=16
Slide 61
Slide 61 text
● In a nutshell, I implemented what Rust gem does in Go gem
● I decided to generate Go source code automatically from ruby.h
61
tl;dr;
Slide 62
Slide 62 text
● https://github.com/rust-lang/rust-bindgen
○ Auto-generate C/C++ code from Rust FFI bindings
● https://github.com/oxidize-rb/rb-sys
○ Auto-generate Rust code from C API using bindgen
○ https://rubygems.org/gems/rb_sys : Helpers for compiling Rust
extensions for ruby
● https://github.com/matsadler/magnus
○ High level Ruby bindings for Rust
○ depends on rb-sys
62
Architecture of Rust gem
Slide 63
Slide 63 text
● rust-bindgen (Generate bindings) + rb-sys (Low layer API) = go-gem-wrapper
● go-gem-wrapper doesn’t include high layer API similar to magnus (maybe…)
● I created go-gem-wrapper by myself
63
What is go-gem-wrapper
Slide 64
Slide 64 text
64
Generate Go code from “ruby.h”
Slide 65
Slide 65 text
65
Generate Go code from “ruby.h”
ruby.h PORO
Go
Parser
Generator
Slide 66
Slide 66 text
● Parser for “ruby.h”
● Wrapper for https://github.com/universal-ctags/ctags
66
https://github.com/ruby-go-gem/ruby_header_parser
Slide 67
Slide 67 text
● https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/_tools/ruby
_h_to_go
● Generate Go code from “ruby.h” using ruby_header_parser
67
ruby_h_to_go
Slide 68
Slide 68 text
● Issue 1. Type mismatch between outside and inside of module
● Issue 2. C pointer
● Issue 3. deprecated functions
● Issue 4. Removed functions
● Issue 5. Supports https://pkg.go.dev/
68
Many many issues…
Slide 69
Slide 69 text
69
Issue 1. Type mismatch between outside and
inside of module
Slide 70
Slide 70 text
package main
// snip
import (
"github.com/ruby-go-gem/go-gem-wrapper"
)
//export rb_dummy_sum
func rb_dummy_sum(_ C.VALUE, a C.VALUE, b C.VALUE) C.VALUE {
aLong := ruby.RbNum2Long(a)
bLong := ruby.RbNum2Long(b)
sum := aLong + bLong
return ruby.RbLong2NumInline(sum)
}
PoC (1/2)
70
Slide 71
Slide 71 text
// in github.com/ruby-go-gem/go-gem-wrapper module
package ruby
/*
#include "ruby.h"
*/
import "C"
func RbNum2Long(n C.VALUE) C.long {
return C.rb_num2long(n)
}
PoC (2/2)
71
Slide 72
Slide 72 text
./dummy.go:16:35: cannot use a (variable of type _Ctype_ulong) as
ruby._Ctype_ulong value in argument to ruby.RbNum2Long
./dummy.go:17:35: cannot use b (variable of type _Ctype_ulong) as
ruby._Ctype_ulong value in argument to ruby.RbNum2Long
./dummy.go:21:39: cannot use sum (variable of type _Ctype_ulong) as
ruby.Long value in argument to ruby.RbLong2NumInline
make: *** [dummy.bundle] Error 1
Build error!!!!!
72
Slide 73
Slide 73 text
https://pkg.go.dev/cmd/cgo says
> Cgo translates C types into equivalent unexported Go types. Because the
translations are unexported, a Go package should not expose C types in its
exported API: a C type used in one Go package is different from the same C type
used in another.
ref. https://github.com/golang/go/issues/13467
73
Answer. cgo specification
Slide 74
Slide 74 text
package ruby
/*
#include "ruby.h"
*/
import "C"
type Long C.long
type VALUE C.VALUE
func RbNum2long(n VALUE) Long {
return Long(C.rb_num2long(C.VALUE(n)))
}
Workarround
74
Slide 75
Slide 75 text
75
Issue 2. C pointer
Slide 76
Slide 76 text
● C lang's pointers are very difficult for programming beginners
● C lang's pointers are very difficult for C lang's parser and Go generator too!
76
Issue 2. C pointer
Slide 77
Slide 77 text
● Please answer Go type that you would expect given C type
● e.g. C type: int -> Go type: int
77
Quiz. Convert C type to Go type
Slide 78
Slide 78 text
78
Q1. C type char* -> Go type ???
Slide 79
Slide 79 text
● A. Case by case
○ string (in many cases)
○ char* (when RSTRING_PTR and RSTRING_END returns)
79
Q1. C type char* -> Go type ???
https://github.com/ruby/ruby/blob/v3_4_2/include/ruby/internal/core/rstring.h#L408-L416
/**
* Queries the contents pointer of the string.
*
* @param[in] str String in question.
* @return Pointer to its contents.
* @pre `str` must be an instance of ::RString.
*/
static inline char * RSTRING_PTR(VALUE str)
Slide 80
Slide 80 text
80
Q2. C type VALUE* -> Go type ???
Slide 81
Slide 81 text
● A. Case by case
○ Array
○ Pointer for input/output
○ Pointer for output only
○ Pointer for input only
81
Q2. C type VALUE* -> Go type ???
Slide 82
Slide 82 text
82
e.g. Use VALUE* as Array
// RbFuncallv calls `rb_funcallv` in C
//
// Original definition is following
//
// VALUE rb_funcallv(VALUE recv, ID mid, int argc, const VALUE *argv)
func RbFuncallv(recv VALUE, mid ID, argc int, argv []VALUE) VALUE {
return VALUE(C.rb_funcallv(C.VALUE(recv), C.ID(mid), C.int(argc),
toCArray[VALUE, C.VALUE](argv)))
}
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L3885-L3892
Slide 83
Slide 83 text
83
e.g. Use VALUE* as in/out or out only pointer
// RbCvarFind calls `rb_cvar_find` in C
//
// Original definition is following
//
// VALUE rb_cvar_find(VALUE klass, ID name, VALUE *front)
func RbCvarFind(klass VALUE, name ID, front *VALUE) VALUE {
var cFront C.VALUE
ret := VALUE(C.rb_cvar_find(C.VALUE(klass), C.ID(name), &cFront))
*front = VALUE(cFront)
return ret
}
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L2278-L2288
Slide 84
Slide 84 text
84
e.g. Use VALUE* as input only pointer
// RbDefineVariable calls `rb_define_variable` in C
//
// Original definition is following
//
// void rb_define_variable(const char *name, VALUE *var)
func RbDefineVariable(name string, v *VALUE) {
char, clean := string2Char(name)
defer clean()
C.rb_define_variable(char, (*C.VALUE)(v))
}
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L2688-L2698
Slide 85
Slide 85 text
85
Q3. C type xxx** -> Go type ???
Slide 86
Slide 86 text
● A. Case by case
○ Multiple pointer
○ Array of pointer
○ Array of String
86
Q3. C type xxx** -> Go type ???
Slide 87
Slide 87 text
● Normal pointer
● Input only pointer
● Special one for multiple pointer
● Array
● Array of pointer
● Array of string
● Function pointer
87
Pointer list in “ruby.h”
Slide 88
Slide 88 text
● C type to Go type not uniquely defined
● Case by case!!!!!
88
Problem. Case by case
Slide 89
Slide 89 text
● All were handled individually
● About 37 / 1,100 functions
89
Solution
Slide 90
Slide 90 text
90
ruby_header_parser/config/default.yml.erb
function:
pointer_hint:
RSTRING_END:
self: raw
RSTRING_PTR:
self: raw
rb_data_object_make:
4: sref
rb_data_typed_object_make:
3: sref
https://github.com/ruby-go-gem/ruby_header_parser/blob/main/config/default.yml.erb#L83-L95
Slide 91
Slide 91 text
● Configuration file specification
○ https://github.com/ruby-go-gem/ruby_header_parser/blob/v0.4.2/CON
FIG.md
● Hint
○ https://github.com/xlab/c-for-go/wiki/Translator-config-section
91
ruby_header_parser/config/default.yml.erb
Slide 92
Slide 92 text
● Some functions in “ruby.h” are deprecated
● Deprecated function bindings should not be generated
92
Issue 3. deprecated functions
$ bundle exec rake build_all
/Users/sue445/.rbenv/versions/3.4.1/include/ruby-3.4.0/ruby/internal/core/rdata.h:314
:1: note: 'rb_data_object_get_warning' has been explicitly marked deprecated here
cgo-gcc-prolog:4427:11: warning: 'rb_data_object_wrap_warning' is deprecated: by
TypedData [-Wdeprecated-declarations]
Slide 93
Slide 93 text
93
Solution.
ruby_header_parser/config/default.yml.erb
function:
exclude_name:
# deprecated functions
- !ruby/regexp /^rb_check_safe_str$/i
- !ruby/regexp /^rb_clear_constant_cache$/i
- !ruby/regexp /^rb_clone_setup$/i
- !ruby/regexp /^rb_complex_polar$/i
- !ruby/regexp /^rb_data_object_alloc$/i
- !ruby/regexp /^rb_data_object_get_warning$/i
- !ruby/regexp /^rb_data_object_wrap_warning$/i
- !ruby/regexp /^rb_data_typed_object_alloc$/i
https://github.com/ruby-go-gem/ruby_header_parser/blob/v0.4.2/config/default.yml.erb#L7
Slide 94
Slide 94 text
94
Solution.
ruby_header_parser/config/default.yml.erb
function:
exclude_name:
# internal macros
- !ruby/regexp /^rb_define_global_function_(m?[0-9]+|notimpl)$/i
- !ruby/regexp /^rb_define_method_(m?[0-9]+|notimpl)$/i
- !ruby/regexp /^rb_define_method_id_(m?[0-9]+|notimpl)$/i
- !ruby/regexp /^rb_define_module_function_(m?[0-9]+|notimpl)$/i
- !ruby/regexp /^rb_define_private_method_(m?[0-9]+|notimpl)$/i
- !ruby/regexp /^rb_define_protected_method_(m?[0-9]+|notimpl)$/i
- !ruby/regexp /^rb_define_singleton_method_(m?[0-9]+|notimpl)$/i
https://github.com/ruby-go-gem/ruby_header_parser/blob/v0.4.2/config/default.yml.erb#L33
Slide 95
Slide 95 text
● Some functions were removed in Ruby 3.4
○ e.g. rb_data_object_make
○ https://bugs.ruby-lang.org/issues/20265
○ https://bugs.ruby-lang.org/issues/18290
● Calling a non-existent C function from within Go code will result in an error
at build time
95
Issue 4. Removed functions
Slide 96
Slide 96 text
● Even if these are removed in Ruby 3.4+, Ruby 3.3 bindings need to be able to
use it as these are
● It was necessary to use different Go bindings depending on the version of
Ruby at the time of build
96
Issue 4. Removed functions
Slide 97
Slide 97 text
97
Solution. Split binding files
$ ls -1 ruby/*_generated.go
ruby/enum_ruby_3_3_generated.go
ruby/enum_ruby_3_4_generated.go
ruby/function_ruby_3_3_generated.go
ruby/function_ruby_3_4_generated.go
ruby/type_ruby_3_3_generated.go
ruby/type_ruby_3_4_generated.go
https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/ruby
Slide 98
Slide 98 text
● Ability to change files to be built according to conditions
○ e.g. “ruby_linux_amd64.go”, “ruby_darwin_arm64.go”
● https://pkg.go.dev/go/build#hdr-Build_Constraints
● https://pkg.go.dev/cmd/go#hdr-Build_constraints
98
Build Constraints (a.k.a. Build tag)
Slide 99
Slide 99 text
99
Custom Build Constraints
// THE AUTOGENERATED LICENSE. ALL THE RIGHTS ARE RESERVED BY ROBOTS.
// WARNING: This file has automatically been generated
// Code generated by ruby_h_to_go. DO NOT EDIT.
//go:build ruby_3_4
package ruby
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L1-L8
custom build tag
Slide 100
Slide 100 text
100
GOFLAGS=’-tags=ruby_x_x’ go
build
def create_go_makefile(target, srcprefix: nil, go_build_args: nil)
(snip)
goflags = "-tags=#{GoGem::Util.ruby_minor_version_build_tag}"
File.open("Makefile", "a") do |f|
f.write <<~MAKEFILE.gsub(/^ {8}/, "\t")
$(DLLIB): Makefile $(srcdir)/*.go
cd $(srcdir); \
CGO_CFLAGS='$(INCFLAGS)' CGO_LDFLAGS='#{ldflags}' GOFLAGS='#{goflags}' \
go build -p 4 -buildmode=c-shared -o #{current_dir}/$(DLLIB) #{go_build_args}
MAKEFILE
end
end
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/_gem/lib/go_gem/mkmf.rb#L38-L47
Slide 101
Slide 101 text
● https://pkg.go.dev/ is the official reference site for Go standard libraries and
modules
○ e.g. https://pkg.go.dev/github.com/ruby-go-gem/go-gem-wrapper
○ Similar to https://docs.ruby-lang.org/ + https://rubydoc.info/
101
Issue 5. Supports https://pkg.go.dev/
Slide 102
Slide 102 text
● Before Custom Build Constraints was introduced, reference was displayed
correctly on the https://pkg.go.dev/
● But after, broken…
102
Issue 5. Supports https://pkg.go.dev/
Slide 103
Slide 103 text
Good
103
All go bindings were missing
Bad
Slide 104
Slide 104 text
● https://pkg.go.dev/ doesn't support Custom Build Constraints
● I don't want 1,100 bindings not showing up on https://pkg.go.dev/
● I thought about building this myself and hosting this on GitHub Pages, but I
thought this would be confusing to have Go module references published
outside of https://pkg.go.dev/
104
Why?
Slide 105
Slide 105 text
● I want to fallback to ruby_3_3 if there is no Custom Build Constraints
● However, Go's current Build Constraints doesn’t allow such a condition to be
specified. (maybe)
105
Ideal and Reality
Slide 106
Slide 106 text
106
Solution: ruby_h_to_go/config.yml
default_tag: ruby_3_3
available_tags:
- ruby_3_3
- ruby_3_4
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/_tools/ruby_h_to_go/config.yml
Slide 107
Slide 107 text
107
ruby_h_to_go/lib/ruby_h_to_go/go_util.rb
ruby_build_tag = GoGem::Util.ruby_minor_version_build_tag
header <<
if ruby_build_tag == RubyHToGo.config.default_tag
other_tags = RubyHToGo.config.available_tags - [RubyHToGo.config.default_tag]
condition = other_tags.map { |tag| "!#{tag}" }.join(" && ")
<<~GO
// FIXME: https://pkg.go.dev/ doesn't support custom build tag.
// Therefore, if no build tag is passed, treat it as the default tag
//go:build #{ruby_build_tag} || (#{condition})
GO
else
<<~GO
//go:build #{ruby_build_tag}
GO
end
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/_tools/ruby_h_to_go/lib/ruby_h_to_go/go_util.rb#L27-L45
Slide 108
Slide 108 text
108
function_ruby_3_3_generated.go
// THE AUTOGENERATED LICENSE. ALL THE RIGHTS ARE RESERVED BY ROBOTS.
// WARNING: This file has automatically been generated
// Code generated by ruby_h_to_go. DO NOT EDIT.
// FIXME: https://pkg.go.dev/ doesn't support custom build tag.
// Therefore, if no build tag is passed, treat it as the default tag
//go:build ruby_3_3 || !ruby_3_4
package ruby
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_3_generated.go
Slide 109
Slide 109 text
109
After ruby 3.5 is released (maybe)
// THE AUTOGENERATED LICENSE. ALL THE RIGHTS ARE RESERVED BY ROBOTS.
// WARNING: This file has automatically been generated
// Code generated by ruby_h_to_go. DO NOT EDIT.
// FIXME: https://pkg.go.dev/ doesn't support custom build tag.
// Therefore, if no build tag is passed, treat it as the default tag
//go:build ruby_3_3 || (!ruby_3_4 && !ruby_3_5)
package ruby
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_3_generated.go
Slide 110
Slide 110 text
110
Tips. Propagate Ruby Reference to Go
Slide 111
Slide 111 text
111
RubyKaigi 2015
https://www.slideshare.net/slideshow/ruby-meets-go/56074507#32
Slide 112
Slide 112 text
112
RubyKaigi 2015
https://www.slideshare.net/slideshow/ruby-meets-go/56074507#33
Slide 113
Slide 113 text
● go-gem-wrapper provides a mechanism for managing Ruby reference
counters in Go
113
Solution
Slide 114
Slide 114 text
114
Example (1/2)
//export rb_example_go_struct_set
func rb_example_go_struct_set(self C.VALUE, x C.VALUE, y C.VALUE) {
data := (*GoStruct)(ruby.GetGoStruct(ruby.VALUE(self)))
data.x = ruby.NUM2INT(ruby.VALUE(x))
data.y = ruby.NUM2INT(ruby.VALUE(y))
}
//export rb_example_go_struct_get
func rb_example_go_struct_get(self C.VALUE) C.VALUE {
data := (*GoStruct)(ruby.GetGoStruct(ruby.VALUE(self)))
ret := []ruby.VALUE{
ruby.INT2NUM(data.x),
ruby.INT2NUM(data.y),
}
return C.VALUE(ruby.Slice2rbAry(ret))
}
https://pkg.go.dev/github.com/ruby-go-gem/go-gem-wrapper/ruby#NewGoStruct
Slide 115
Slide 115 text
115
Example (2/2)
//export go_struct_alloc
func go_struct_alloc(klass C.VALUE) C.VALUE {
data := GoStruct{}
return C.VALUE(ruby.NewGoStruct(ruby.VALUE(klass), unsafe.Pointer(&data)))
}
//export Init_example
func Init_example() {
rb_mExample := ruby.RbDefineModule("Example")
// Create Example::GoStruct class
rb_cGoStruct := ruby.RbDefineClassUnder(rb_mExample, "GoStruct", ruby.VALUE(C.rb_cObject))
ruby.RbDefineAllocFunc(rb_cGoStruct, C.go_struct_alloc)
ruby.RbDefineMethod(rb_cGoStruct, "set", C.rb_example_go_struct_set, 2)
ruby.RbDefineMethod(rb_cGoStruct, "get", C.rb_example_go_struct_get, 0)
}
https://pkg.go.dev/github.com/ruby-go-gem/go-gem-wrapper/ruby#NewGoStruct
Slide 116
Slide 116 text
Use go build in Ruby native extension building
116
Up to here
CRuby Go
Slide 117
Slide 117 text
Use “ruby.h” in go building
117
From here
CRuby Go
Slide 118
Slide 118 text
118
Use “ruby.h” in go building
Slide 119
Slide 119 text
1. go test
2. golangci-lint
119
Use case
Slide 120
Slide 120 text
● go test is standard testing command for Go
● Functions created in native extension (not just Go) should be tested from
Ruby
○ Because this is called from Ruby
● Functions created in pure-Go should be tested from Go
○ Because this is called from Go
120
Use case 1. go test
Slide 121
Slide 121 text
郷に入りては郷に従え
Go ni irite wa Go ni shitagae
121
When in Rome, do as the Romans do
Slide 122
Slide 122 text
● In order to execute go test, go build must be able to execute
● “ruby.h” is needed to execute go build in Go gem
● Various environment variables are required to build with “ruby.h”
122
Problem: go test requires building with
“ruby.h”
Slide 123
Slide 123 text
123
Minimum required environment variables for go build
ENV["CGO_CFLAGS"] = "#{RbConfig::CONFIG["CFLAGS"]}" \
" -I#{RbConfig::CONFIG["rubyarchhdrdir"]}" \
" -I#{RbConfig::CONFIG["rubyhdrdir"]}"
ENV["CGO_LDFLAGS"] = "-L#{RbConfig::CONFIG["libdir"]}" \
" -l#{RbConfig::CONFIG["RUBY_SO_NAME"]}"
ENV["LD_LIBRARY_PATH"] = "#{RbConfig::CONFIG["libdir"]}"
Slide 124
Slide 124 text
124
go test full command
GOFLAGS=-tags=ruby_3_4 CGO_CFLAGS=-fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2
-fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wextra-tokens -Wdeprecated-declarations
-Wdivision-by-zero -Wdiv-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wpointer-arith
-Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn
-Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long
-Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality
-Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value
-Wunused-variable -Wmisleading-indentation -Wundef -fno-common -pipe
-I/Users/sue445/.rbenv/versions/3.4.1/include/ruby-3.4.0/arm64-darwin24
-I/Users/sue445/.rbenv/versions/3.4.1/include/ruby-3.4.0
CGO_LDFLAGS=-L/Users/sue445/.rbenv/versions/3.4.1/lib -lruby.3.4 -undefined dynamic_lookup
LD_LIBRARY_PATH=/Users/sue445/.rbenv/versions/3.4.1/lib go test -mod=readonly -count=1 ./...
Slide 125
Slide 125 text
# Rakefile
require "go_gem/rake_task"
GoGem::RakeTask.new("example")
Solution GoGem::RakeTask
Provides rake tasks for go test with CRuby
125
Slide 126
Slide 126 text
$ bundle exec rake -T | grep "go:"
rake go:build_envs[env_name] # Print build envs for `go build`
rake go:build_tag # Print build tag
rake go:fmt # Run go fmt
rake go:test # Run go test
rake go:testrace # Run go test -race
rake go tasks
126
Slide 127
Slide 127 text
● With Go, in addition to the official linter, there are various 3rd party tools
for each linter
● In Ruby, it's similar to the way gem is created for each RuboCop’s Cop
127
Go’s linter situation
Slide 128
Slide 128 text
● golangci-lint is linters runner for Go
● https://github.com/golangci/golangci-lint
● This is 3rd party tool but widely used in the Go community
128
Use case 2. golangci-lint
Slide 129
Slide 129 text
# Rakefile
require "go_gem/rake_task"
go_task = GoGem::RakeTask.new("gem_name")
namespace :go do
desc "Run golangci-lint"
task :lint do
go_task.within_target_dir do
sh "which golangci-lint" do |ok, _|
raise "golangci-lint isn't installed. See. https://golangci-lint.run/welcome/install/" unless ok
end
build_tag = GoGem::Util.ruby_minor_version_build_tag
sh GoGem::RakeTask.build_env_vars, "golangci-lint run --build-tags #{build_tag} --modules-download-mode=readonly"
end
end
end
e.g. Use golangci-lint in Go gem
129
https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.3/_gem#example-add-additional-tasks
Slide 130
Slide 130 text
● “patch_for_go_gem” is needed to make Go gem
● I want to use bundler gem similar to --ext=c and --ext=rust
130
Add --ext=go to bundle gem command
Slide 131
Slide 131 text
131
https://github.com/orgs/rubygems/discussions/8128
Slide 132
Slide 132 text
● I waited for a while and got no response
● So I had hsbt triage it on ruby-jp slack
132
Feature Proposal
Slide 133
Slide 133 text
133
LGTM
Slide 134
Slide 134 text
134
https://github.com/rubygems/rubygems/pull/8183
Slide 135
Slide 135 text
135
Pros/Cons of Go gem
Slide 136
Slide 136 text
● We can use Go modules in Ruby
136
Pros 1
Slide 137
Slide 137 text
● We can write in Go in areas where Ruby is not good at (and Go is very good
at)
● Go's features not found in other languages
○ goroutine: lightweight thread
■ https://go.dev/tour/concurrency/1
○ channel: Messaging between multiple goroutine
■ https://go.dev/tour/concurrency/2
137
Pros 2
Slide 138
Slide 138 text
● Benchmark with tak function
○ https://www.ruby-lang.org/en/news/2020/12/25/ruby-3-0-0-released/
● Multi thread and multi process are not eligible
○ Because the results depend on the number of CPU cores on the machine
being benchmarked
○ This is not fair to compare these to goroutine
138
Ractor vs Fiber vs goroutine
Slide 139
Slide 139 text
139
goroutine is 30x faster than Ractor
$ ruby tarai.rb
go version go1.24.1 darwin/arm64
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [arm64-darwin24]
Comparison:
Go: goroutine: 1.6 i/s
Go: sequential: 0.5 i/s - 3.65x slower
Ruby: Ractor: 0.1 i/s - 31.58x slower
Ruby: sequential: 0.0 i/s - 92.85x slower
Ruby: Fiber: 0.0 i/s - 93.08x slower
https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.3/_benchmark
Slide 140
Slide 140 text
140
@osyoyu says
https://speakerdeck.com/osyoyu/zigdeckuo-zhang-wozuo-ru-2024-edition?slide=34
Slide 141
Slide 141 text
141
Native extension with Zig
https://speakerdeck.com/osyoyu/zigdeckuo-zhang-wozuo-ru-2024-edition
Slide 142
Slide 142 text
● go-gem-wrapper is a low-layer wrapper, so not much different than making
Native extension with C
○ Similar to writing in Go where you would write in C
○ Need to be aware of both CRuby and Go
142
Cons 1
Slide 143
Slide 143 text
● Go's variable-length arguments couldn't be passed directly to C
○ CRuby provides both variable length argument and non-variable length
argument versions of its functions.
○ If we only have functions with variable-length arguments, it is a bit of a
problem
■ e.g. rb_raise
143
Cons 2
Slide 144
Slide 144 text
144
Workaround for rb_raise
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/ruby_internal_error.go
/*
#include "ruby.h"
void __rb_raise(VALUE exception, const char *str) {
rb_raise(exception, "%s", str);
}
*/
import "C"
func RbRaise(exc VALUE, format string, a ...interface{}) {
str := fmt.Sprintf(format, a...)
char, clean := string2Char(str)
defer clean()
C.__rb_raise(C.VALUE(exc), char)
}
Slide 145
Slide 145 text
● https://github.com/orgs/ruby-go-gem/discussions/236
145
I'm looking for good ideas and patches!
Slide 146
Slide 146 text
● Not intended for use with ruby-head, since binding is required for each Ruby
version
146
Cons 3
Slide 147
Slide 147 text
● SEGV when calling CRuby functions in goroutine
● c.f. https://github.com/ruby-go-gem/go-gem-wrapper/issues/286
147
Cons 4
Slide 148
Slide 148 text
This doesn’t work… (SEGV!!!!)
148
e.g. Call yield inside goroutine
//export rb_example_call_proc_inside_goroutine
func rb_example_call_yield_inside_goroutine(self C.VALUE) {
if ruby.RbBlockGivenP() == 0 {
ruby.RbRaise(C.rb_eArgError, "Block not given")
}
go func() {
ruby.RbYield(C.Qnil)
}()
}
Slide 149
Slide 149 text
● glibc is required
○ Not supported for use with musl (e.g. alpine)
○ c.f. https://github.com/ruby-go-gem/go-gem-wrapper/issues/253
○ Windows (maybe)
149
Cons 5
Slide 150
Slide 150 text
150
Real use case
Slide 151
Slide 151 text
● Perform HTTP requests in parallel with goroutine
● Unlike thread and process, we don't have to specify concurrency
○ e.g. max threads, max processes
151
https://github.com/sue445/funnel_http
Slide 152
Slide 152 text
152
Usage
require "funnel_http"
client = FunnelHttp::Client.new
requests = [
{
method: :get,
url: "https://example.com/api/user/1",
},
]
responses = client.perform(requests)
# => [
# { status_code: 200, body: "Response of /api/user/1",
# header: { "Content-Type" => ["text/plain;charset=utf-8"]} }
# ]
Slide 153
Slide 153 text
NOTE: open-uri and net/http doesn’t work in Ractor…
153
benchmark (Fiber vs goroutine)
$ bundle exec ruby benchmark.rb
go version go1.24.1 darwin/arm64
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24]
Comparison:
FunnelHttp::Client#perform: 22.7 i/s
sequential: 7.0 i/s - 3.27x slower
Parallel with Fiber: 3.7 i/s - 6.14x slower
https://github.com/sue445/funnel_http/tree/v0.3.2/benchmark
Slide 154
Slide 154 text
● Using https://github.com/ruby-go-gem/go-gem-wrapper makes native
extension with Go very easy to make
● We can write Ruby gem in Go in areas where Ruby is not good at (Go is very
good at)
● goroutine is very good!
154
Conclusion
Slide 155
Slide 155 text
goroutine is very good!
155
Conclusion
Slide 156
Slide 156 text
goroutine is very good!
156
Conclusion