Slide 1

Slide 1 text

Road to Go gem RubyKaigi 2025 / #rubykaigi #rubykaigiC pixiv Inc. sue445 2025.04.18

Slide 2

Slide 2 text

2 Hello!

Slide 3

Slide 3 text

3 My name is Go The Go gopher was designed by Renee French. (http://reneefrench.blogspot.com/) The design is licensed under the Creative Commons 4.0 Attributions license. Read this article for more details: https://blog.golang.org/gopher

Slide 4

Slide 4 text

4 About sue445 ● Go Sueyoshi (a.k.a. sue445, sue-san) ● https://x.com/sue445 ● https://github.com/sue445 ● Gopher since 1982 ● Shibuya.rb ● Tokyu.rb sue445

Slide 5

Slide 5 text

5 Day job at pixiv.Inc ● Infrastructure Unit ● AWS and Google Cloud ○ Organization Owner ○ Solution Architect ● Certifications possessed ○ AWS Certified Solutions Architect – Professional (SAP-C01) ○ Google Cloud Professional Cloud Architect sue445

Slide 6

Slide 6 text

6 Day job at pixiv.Inc ● on-premises Kubernetes ● Maintainer of In-house tools (GitLab, Sentry etc) ● etc etc etc!!!!!!!!!! sue445

Slide 7

Slide 7 text

7 CM. About pixiv Inc. https://www.pixiv.co.jp/service/ 10years+ 10years+ 10years+ NEW!

Slide 8

Slide 8 text

https://conference.pixiv.co.jp/2025/rubymusicmixin 8 CM. RubyMusicMixin 2025

Slide 9

Slide 9 text

9 CM. RubyMusicMixin 2025 RubyMusicMixin 2025 We are here

Slide 10

Slide 10 text

10 Riverside (川) #rubyriver

Slide 11

Slide 11 text

● All about native extension with Go ● Tips for creating native extensions in a new language 11 Goal in this talk

Slide 12

Slide 12 text

● Let's recall RubyKaigi 2015 ● Getting started with Go gem ● About go-gem-wrapper ● Road to go-gem-wrapper ● Use “ruby.h” in go building ● Add --ext=go to bundle gem command ● Pros/Cons of Go gem ● Real use case 12 Agenda

Slide 13

Slide 13 text

● Using https://github.com/ruby-go-gem/go-gem-wrapper makes it very easy to make native extension with Go ● We can write Ruby gem in Go in areas where Ruby is not good at (Go is very good at) ● goroutine is very good! 13 Conclusion

Slide 14

Slide 14 text

https://rubykaigi.org/2015/presentations/mmasaki/ 14 Let’s recall RubyKaigi 2015

Slide 15

Slide 15 text

● cgo (go build -buildmode=c-shared) was introduced at Go 1.5 ● cgo creates shared object file (*.so) from Go ● Use this technology to create a native Ruby extension with Go ○ a.k.a. Go gem 15 tl;dr;

Slide 16

Slide 16 text

1. The code from 10 years ago (Go 1.5) doesn't work now (Go 1.24) ○ a later mention 2. To make a gem in Go, you need to copy and paste a lot of patches. ○ e.g. ■ ext/gem_name/extconf.rb ■ ext/gem_name/my_gem.go ■ ext/gem_name/go.mod ■ bindings ■ etc… 16 Past issues

Slide 17

Slide 17 text

https://github.com/mmasaki/memberlist/blob/master/ext/memberlist/wrapper.go 17 e.g. 10 years ago Go gem

Slide 18

Slide 18 text

● Technically possible. But it takes a lot of work. 18 Past issues

Slide 19

Slide 19 text

● Library of patches that needed to be done manually ● There was no standard package manager for Go at 10 years ago ● But today, we have Go module 19 github.com/ruby-go-gem/go-gem-wrapper

Slide 20

Slide 20 text

1. bundle gem –ext=c 2. Run “patch_for_go_gem.rb” 3. go get -u github.com/ruby-go-gem/go-gem-wrapper@latest 4. Write gem using Go! 20 Getting started with Go gem

Slide 21

Slide 21 text

$ bundle gem example --ext=c Creating gem 'example'... create example/Gemfile create example/lib/example.rb create example/lib/example/version.rb (snip) create example/ext/example/extconf.rb create example/ext/example/example.h create example/ext/example/example.c Gem 'gem_name' was successfully created. For more information on making a RubyGem visit https://bundler.io/guides/creating_gem.html bundle gem –ext=c 21

Slide 22

Slide 22 text

● https://github.com/ruby-go-gem/go-gem-wrapper/tree/main/_tools/patch_ for_go_gem ● Patch to make a gem into a Go gem right after bundle gem ● This is a temporary tool until bundle gem --ext=go is added to the bundler ○ c.f. https://github.com/rubygems/rubygems/pull/8183 (a later mention) 22 Run “patch_for_go_gem.rb”

Slide 23

Slide 23 text

$ ruby patch_for_go_gem.rb --file example/example.gemspec [INFO] /path/to/example/ext/example/example.go is created [INFO] /path/to/example/ext/example/go.mod is created [INFO] /path/to/example/ext/example/example.c is updated [INFO] /path/to/example/ext/example/extconf.rb is updated [INFO] example/example.gemspec is updated Run “patch_for_go_gem.rb” 23

Slide 24

Slide 24 text

package main /* #include "example.h" */ import "C" import ( "github.com/ruby-go-gem/go-gem-wrapper/ruby" ) //export Init_example func Init_example() { rb_mExample := ruby.RbDefineModule("Example") } func main() { } ext//<gem_name>.go 24

Slide 25

Slide 25 text

#include "my_gem.h" #include "_cgo_export.h" // Added ext//.c 25

Slide 26

Slide 26 text

require "mkmf" require "go_gem/mkmf" # Added append_cflags("-fvisibility=hidden") # create_makefile("example/example") create_go_makefile("example/example") # Added ext//extconf.rb 26

Slide 27

Slide 27 text

Gem::Specification.new do |spec| spec.add_dependency "go_gem" # Added end .gemspec 27

Slide 28

Slide 28 text

module Example def self.sum(a, b) a + b end end e.g. Impl Example#sum 28

Slide 29

Slide 29 text

//export rb_example_sum func rb_example_sum(_ C.VALUE, a C.VALUE, b C.VALUE) C.VALUE { aLong := ruby.NUM2LONG(ruby.VALUE(a)) bLong := ruby.NUM2LONG(ruby.VALUE(b)) sum := aLong + bLong return C.VALUE(ruby.LONG2NUM(sum)) } ext//<gem_name>.go 29

Slide 30

Slide 30 text

package main /* #include "example.h" // Append this VALUE rb_example_sum(VALUE self, VALUE a, VALUE b); */ import "C" ext//<gem_name>.go 30

Slide 31

Slide 31 text

//export Init_example func Init_example() { rb_mExample := ruby.RbDefineModule("Example") // Append this ruby.RbDefineSingletonMethod(rb_mExample, "sum", C.rb_example_sum, 2) } ext//<gem_name>.go 31

Slide 32

Slide 32 text

● We can write a gem in Go just as you write a gem in C! 32 Easy, right?

Slide 33

Slide 33 text

● Go module: github.com/ruby-go-gem/go-gem-wrapper ● Ruby gem: https://rubygems.org/gems/go_gem ● Requiremets ○ Go 1.23+ ○ Ruby 3.3+ ○ glibc (a later mention) 33 About go-gem-wrapper

Slide 34

Slide 34 text

github.com/ruby-go-gem/go-gem-wrapper is Go module for Go gem 34 About github.com/ruby-go-gem/go-gem-wrapper // ext/example/go.mod module github.com/sue445/example go 1.24 require ( github.com/ruby-go-gem/go-gem-wrapper v0.7.1 )

Slide 35

Slide 35 text

https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/ruby/function_ruby_3_3_generated.go#L2687-L2697 35 e.g. RbDefineSingletonMethod func RbDefineSingletonMethod(obj VALUE, mid string, arg3 unsafe.Pointer, arity int) { char, clean := string2Char(mid) defer clean() C.rb_define_singleton_method(C.VALUE(obj), char, toCFunctionPointer(arg3), C.int(arity)) }

Slide 36

Slide 36 text

36 There are 1,100 functions in “ruby.h” $ ls ruby/*_generated.go | xargs wc -l 59 ruby/enum_ruby_3_3_generated.go 57 ruby/enum_ruby_3_4_generated.go 9264 ruby/function_ruby_3_3_generated.go 9198 ruby/function_ruby_3_4_generated.go 105 ruby/type_ruby_3_3_generated.go 103 ruby/type_ruby_3_4_generated.go 18786 total https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/ruby

Slide 37

Slide 37 text

● Helpers for compiling Go extensions for ruby ○ https://rubygems.org/gems/go_gem ○ https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/_gem ● Similar to https://rubygems.org/gems/rb_sys in Rust gem 37 About go_gem

Slide 38

Slide 38 text

require "mkmf" require "go_gem/mkmf" create_go_makefile("example/example") # Pass debug flags to `go build` create_go_makefile("example/example", go_build_args: "-gcflags='all=-N -l'") create_go_makefile Add a task and hacks to build Go against the “Makefile” generated by create_makefile. (similar to create_rust_makeflile) 38

Slide 39

Slide 39 text

39 create_go_makefile def create_go_makefile(target, srcprefix: nil, go_build_args: nil) # (snip) File.open("Makefile", "a") do |f| f.write <<~MAKEFILE.gsub(/^ {8}/, "\t") $(DLLIB): Makefile $(srcdir)/*.go cd $(srcdir); \ CGO_CFLAGS='$(INCFLAGS)' CGO_LDFLAGS='#{ldflags}' GOFLAGS='#{goflags}' \ go build -p 4 -buildmode=c-shared -o #{current_dir}/$(DLLIB) #{go_build_args} MAKEFILE end end https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/_gem/lib/go_gem/mkmf.rb

Slide 40

Slide 40 text

# Rakefile require "go_gem/rake_task" GoGem::RakeTask.new("example") GoGem::RakeTask Provides rake tasks for go test with CRuby. (a later mention) 40

Slide 41

Slide 41 text

● I had to go through many hardships to create go-gem-wrapper ● I will now introduce them 41 Road to go-gem-wrapper

Slide 42

Slide 42 text

1. cgo compatibility 2. Generate Go’s bindings from “ruby.h” ○ Type mismatch between outside and inside of module ○ C pointer ○ deprecated functions ○ Removed functions ○ Supports https://pkg.go.dev/ 3. Go with CRuby 42 Road to go-gem-wrapper

Slide 43

Slide 43 text

● This technique was introduced in RubyKaigi 2015 ● No problem at Go 1.5, but SEGV at Go 1.24 43 1. cgo compatibility https://www.slideshare.net/slideshow/ruby-meets-go/56074507

Slide 44

Slide 44 text

● Current Go's string isn't a null-terminated string, so using *(*[]byte)(unsafe.Pointer(&str)) will return a non-null-terminated byte array and can't be *C.char ● Go 1.7+ are spec'd this way 44 Why?

Slide 45

Slide 45 text

45 e.g. Print Go string via C package main /* #include */ import "C" func Print(str string) { cstr := GOSTRING_PTR(str) C.fputs(cstr, (*C.FILE)(C.stdout)) } func main() { Print("ABCD") }

Slide 46

Slide 46 text

46 OMG! $ go run main.go ABCDtrueallgallprootitabsbrkidledeadsync is LEAFbase of ) = <==GOGC] = pc=+Inf-Inf: p=cas1cas2cas3cas4cas5cas6 at m= sp= sp: lr: fp= gp= mp=) m=boolint8uintchanfuncopenstatfile in sha1sha2timeJuneJuly/etcfalsedefersweeptestRtestWexecWhchanexecRschedsudogtimergscanm heaptracepanicsleeparm64 cnt=gcing MB, got= ... max=scav ptr ] = (init ms, fault and tab= top=[...], fp:int16int32int64uint8arrayslicewriteclosegetwdlstatGreekpmullcrc32cpuidfcntlMarchAp rilLocalerrno stringsysmontimersefenceselect, not GOROOTobject next= jobs= goid sweep B -> % util alloc free span= prev= list=, i = code= addr=], sp= m->p= p->m=SCHED curg= ctxt: min= max= bad ts(...)

Slide 47

Slide 47 text

47 Solution func string2Char(str string) (*C.char, func()) { cstr := C.CString(str) clean := func() { C.free(unsafe.Pointer(cstr)) } return cstr, clean } https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/ruby/wrapper.go#L26-L48

Slide 48

Slide 48 text

48 Solution func RbDefineSingletonMethod(obj VALUE, mid string, arg3 unsafe.Pointer, arity int) { char, clean := string2Char(mid) defer clean() C.rb_define_singleton_method(C.VALUE(obj), char, toCFunctionPointer(arg3), C.int(arity)) } https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/ruby/function_ruby_3_3_ge nerated.go#L2687-L2697

Slide 49

Slide 49 text

● Go, similar to Ruby, is a very backward-compatible language. ● However, cgo has characteristics that depend on Go's internal implementation and can be broken by Go version upgrades ● c.f. “Why Go's old code almost never stops working“ ○ https://zenn.dev/catatsuy/articles/fda1e42acad421 (ja) 49 Tips

Slide 50

Slide 50 text

50 2. Generate Go’s bindings from “ruby.h”

Slide 51

Slide 51 text

● In this talk, binding is a Go function that corresponds to a CRuby function as follows 51 2. Generate Go’s bindings from “ruby.h” func RbDefineSingletonMethod(obj VALUE, mid string, arg3 unsafe.Pointer, arity int) { char, clean := string2Char(mid) defer clean() C.rb_define_singleton_method(C.VALUE(obj), char, toCFunctionPointer(arg3), C.int(arity)) }

Slide 52

Slide 52 text

● Just on https://docs.ruby-lang.org/ja/latest/function/index.html , there are less than 200 ● There are about 2,000+ C functions defined in ruby.h ● About 1,100 functions can be included and actually used ● At first (about 30 functions), I wrote them by hand, but I decided to generate them automatically from ruby.h because it's just too hard. 52 2. Generate Go’s bindings from “ruby.h”

Slide 53

Slide 53 text

1. Use existing OSS 2. Create it myself 53 Choices

Slide 54

Slide 54 text

Automatic C-Go Bindings Generator for Go Programming Language 54 https://github.com/xlab/c-for-go

Slide 55

Slide 55 text

Fixed an error when passing “ruby.h” 55 https://github.com/xlab/c-for-go/pull/171

Slide 56

Slide 56 text

● Bindings generated by c-for-go was abandoned due to lack of portability ○ e.g. Bindings generated on MacOS isn’t usable on Ubuntu 56 Result

Slide 57

Slide 57 text

● It may be due to the fact that there are many #ifdef branches in “ruby.h” for each OS and build environment. ● e.g. 57 Result https://github.com/ruby/ruby/blob/v3_4_2/internal/process.h#L14-L20 #ifdef HAVE_SYS_TYPES_H # include /* for mode_t */ #endif #ifdef _WIN32 # include "ruby/win32.h" /* for mode_t */ #endif

Slide 58

Slide 58 text

● I can change the source code for each environment we build by creating a file similar to “ruby_linux_amd64.go” or “ruby_darwin_arm64.go” (Build constraints) ● But this increases the amount of code, and CI also needs to prepare all build constraints ● I gave up on it because it would be too hard maintain ○ https://github.com/ruby-go-gem/go-gem-wrapper/pull/117#issuecomm ent-2336583098 58 Result

Slide 59

Slide 59 text

● However, I couldn't figure out how to create it… 59 Create it myself

Slide 60

Slide 60 text

60 Hint: @joker1007 ’s Rust gem talk https://speakerdeck.com/joker1007/rustdezuo-rutree-sitterpasanorubybaindeingu?slide=16

Slide 61

Slide 61 text

● In a nutshell, I implemented what Rust gem does in Go gem ● I decided to generate Go source code automatically from ruby.h 61 tl;dr;

Slide 62

Slide 62 text

● https://github.com/rust-lang/rust-bindgen ○ Auto-generate C/C++ code from Rust FFI bindings ● https://github.com/oxidize-rb/rb-sys ○ Auto-generate Rust code from C API using bindgen ○ https://rubygems.org/gems/rb_sys : Helpers for compiling Rust extensions for ruby ● https://github.com/matsadler/magnus ○ High level Ruby bindings for Rust ○ depends on rb-sys 62 Architecture of Rust gem

Slide 63

Slide 63 text

● rust-bindgen (Generate bindings) + rb-sys (Low layer API) = go-gem-wrapper ● go-gem-wrapper doesn’t include high layer API similar to magnus (maybe…) ● I created go-gem-wrapper by myself 63 What is go-gem-wrapper

Slide 64

Slide 64 text

64 Generate Go code from “ruby.h”

Slide 65

Slide 65 text

65 Generate Go code from “ruby.h” ruby.h PORO Go Parser Generator

Slide 66

Slide 66 text

● Parser for “ruby.h” ● Wrapper for https://github.com/universal-ctags/ctags 66 https://github.com/ruby-go-gem/ruby_header_parser

Slide 67

Slide 67 text

● https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/_tools/ruby _h_to_go ● Generate Go code from “ruby.h” using ruby_header_parser 67 ruby_h_to_go

Slide 68

Slide 68 text

● Issue 1. Type mismatch between outside and inside of module ● Issue 2. C pointer ● Issue 3. deprecated functions ● Issue 4. Removed functions ● Issue 5. Supports https://pkg.go.dev/ 68 Many many issues…

Slide 69

Slide 69 text

69 Issue 1. Type mismatch between outside and inside of module

Slide 70

Slide 70 text

package main // snip import ( "github.com/ruby-go-gem/go-gem-wrapper" ) //export rb_dummy_sum func rb_dummy_sum(_ C.VALUE, a C.VALUE, b C.VALUE) C.VALUE { aLong := ruby.RbNum2Long(a) bLong := ruby.RbNum2Long(b) sum := aLong + bLong return ruby.RbLong2NumInline(sum) } PoC (1/2) 70

Slide 71

Slide 71 text

// in github.com/ruby-go-gem/go-gem-wrapper module package ruby /* #include "ruby.h" */ import "C" func RbNum2Long(n C.VALUE) C.long { return C.rb_num2long(n) } PoC (2/2) 71

Slide 72

Slide 72 text

./dummy.go:16:35: cannot use a (variable of type _Ctype_ulong) as ruby._Ctype_ulong value in argument to ruby.RbNum2Long ./dummy.go:17:35: cannot use b (variable of type _Ctype_ulong) as ruby._Ctype_ulong value in argument to ruby.RbNum2Long ./dummy.go:21:39: cannot use sum (variable of type _Ctype_ulong) as ruby.Long value in argument to ruby.RbLong2NumInline make: *** [dummy.bundle] Error 1 Build error!!!!! 72

Slide 73

Slide 73 text

https://pkg.go.dev/cmd/cgo says > Cgo translates C types into equivalent unexported Go types. Because the translations are unexported, a Go package should not expose C types in its exported API: a C type used in one Go package is different from the same C type used in another. ref. https://github.com/golang/go/issues/13467 73 Answer. cgo specification

Slide 74

Slide 74 text

package ruby /* #include "ruby.h" */ import "C" type Long C.long type VALUE C.VALUE func RbNum2long(n VALUE) Long { return Long(C.rb_num2long(C.VALUE(n))) } Workarround 74

Slide 75

Slide 75 text

75 Issue 2. C pointer

Slide 76

Slide 76 text

● C lang's pointers are very difficult for programming beginners ● C lang's pointers are very difficult for C lang's parser and Go generator too! 76 Issue 2. C pointer

Slide 77

Slide 77 text

● Please answer Go type that you would expect given C type ● e.g. C type: int -> Go type: int 77 Quiz. Convert C type to Go type

Slide 78

Slide 78 text

78 Q1. C type char* -> Go type ???

Slide 79

Slide 79 text

● A. Case by case ○ string (in many cases) ○ char* (when RSTRING_PTR and RSTRING_END returns) 79 Q1. C type char* -> Go type ??? https://github.com/ruby/ruby/blob/v3_4_2/include/ruby/internal/core/rstring.h#L408-L416 /** * Queries the contents pointer of the string. * * @param[in] str String in question. * @return Pointer to its contents. * @pre `str` must be an instance of ::RString. */ static inline char * RSTRING_PTR(VALUE str)

Slide 80

Slide 80 text

80 Q2. C type VALUE* -> Go type ???

Slide 81

Slide 81 text

● A. Case by case ○ Array ○ Pointer for input/output ○ Pointer for output only ○ Pointer for input only 81 Q2. C type VALUE* -> Go type ???

Slide 82

Slide 82 text

82 e.g. Use VALUE* as Array // RbFuncallv calls `rb_funcallv` in C // // Original definition is following // // VALUE rb_funcallv(VALUE recv, ID mid, int argc, const VALUE *argv) func RbFuncallv(recv VALUE, mid ID, argc int, argv []VALUE) VALUE { return VALUE(C.rb_funcallv(C.VALUE(recv), C.ID(mid), C.int(argc), toCArray[VALUE, C.VALUE](argv))) } https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L3885-L3892

Slide 83

Slide 83 text

83 e.g. Use VALUE* as in/out or out only pointer // RbCvarFind calls `rb_cvar_find` in C // // Original definition is following // // VALUE rb_cvar_find(VALUE klass, ID name, VALUE *front) func RbCvarFind(klass VALUE, name ID, front *VALUE) VALUE { var cFront C.VALUE ret := VALUE(C.rb_cvar_find(C.VALUE(klass), C.ID(name), &cFront)) *front = VALUE(cFront) return ret } https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L2278-L2288

Slide 84

Slide 84 text

84 e.g. Use VALUE* as input only pointer // RbDefineVariable calls `rb_define_variable` in C // // Original definition is following // // void rb_define_variable(const char *name, VALUE *var) func RbDefineVariable(name string, v *VALUE) { char, clean := string2Char(name) defer clean() C.rb_define_variable(char, (*C.VALUE)(v)) } https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L2688-L2698

Slide 85

Slide 85 text

85 Q3. C type xxx** -> Go type ???

Slide 86

Slide 86 text

● A. Case by case ○ Multiple pointer ○ Array of pointer ○ Array of String 86 Q3. C type xxx** -> Go type ???

Slide 87

Slide 87 text

● Normal pointer ● Input only pointer ● Special one for multiple pointer ● Array ● Array of pointer ● Array of string ● Function pointer 87 Pointer list in “ruby.h”

Slide 88

Slide 88 text

● C type to Go type not uniquely defined ● Case by case!!!!! 88 Problem. Case by case

Slide 89

Slide 89 text

● All were handled individually ● About 37 / 1,100 functions 89 Solution

Slide 90

Slide 90 text

90 ruby_header_parser/config/default.yml.erb function: pointer_hint: RSTRING_END: self: raw RSTRING_PTR: self: raw rb_data_object_make: 4: sref rb_data_typed_object_make: 3: sref https://github.com/ruby-go-gem/ruby_header_parser/blob/main/config/default.yml.erb#L83-L95

Slide 91

Slide 91 text

● Configuration file specification ○ https://github.com/ruby-go-gem/ruby_header_parser/blob/v0.4.2/CON FIG.md ● Hint ○ https://github.com/xlab/c-for-go/wiki/Translator-config-section 91 ruby_header_parser/config/default.yml.erb

Slide 92

Slide 92 text

● Some functions in “ruby.h” are deprecated ● Deprecated function bindings should not be generated 92 Issue 3. deprecated functions $ bundle exec rake build_all /Users/sue445/.rbenv/versions/3.4.1/include/ruby-3.4.0/ruby/internal/core/rdata.h:314 :1: note: 'rb_data_object_get_warning' has been explicitly marked deprecated here cgo-gcc-prolog:4427:11: warning: 'rb_data_object_wrap_warning' is deprecated: by TypedData [-Wdeprecated-declarations]

Slide 93

Slide 93 text

93 Solution. ruby_header_parser/config/default.yml.erb function: exclude_name: # deprecated functions - !ruby/regexp /^rb_check_safe_str$/i - !ruby/regexp /^rb_clear_constant_cache$/i - !ruby/regexp /^rb_clone_setup$/i - !ruby/regexp /^rb_complex_polar$/i - !ruby/regexp /^rb_data_object_alloc$/i - !ruby/regexp /^rb_data_object_get_warning$/i - !ruby/regexp /^rb_data_object_wrap_warning$/i - !ruby/regexp /^rb_data_typed_object_alloc$/i https://github.com/ruby-go-gem/ruby_header_parser/blob/v0.4.2/config/default.yml.erb#L7

Slide 94

Slide 94 text

94 Solution. ruby_header_parser/config/default.yml.erb function: exclude_name: # internal macros - !ruby/regexp /^rb_define_global_function_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_method_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_method_id_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_module_function_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_private_method_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_protected_method_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_singleton_method_(m?[0-9]+|notimpl)$/i https://github.com/ruby-go-gem/ruby_header_parser/blob/v0.4.2/config/default.yml.erb#L33

Slide 95

Slide 95 text

● Some functions were removed in Ruby 3.4 ○ e.g. rb_data_object_make ○ https://bugs.ruby-lang.org/issues/20265 ○ https://bugs.ruby-lang.org/issues/18290 ● Calling a non-existent C function from within Go code will result in an error at build time 95 Issue 4. Removed functions

Slide 96

Slide 96 text

● Even if these are removed in Ruby 3.4+, Ruby 3.3 bindings need to be able to use it as these are ● It was necessary to use different Go bindings depending on the version of Ruby at the time of build 96 Issue 4. Removed functions

Slide 97

Slide 97 text

97 Solution. Split binding files $ ls -1 ruby/*_generated.go ruby/enum_ruby_3_3_generated.go ruby/enum_ruby_3_4_generated.go ruby/function_ruby_3_3_generated.go ruby/function_ruby_3_4_generated.go ruby/type_ruby_3_3_generated.go ruby/type_ruby_3_4_generated.go https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/ruby

Slide 98

Slide 98 text

● Ability to change files to be built according to conditions ○ e.g. “ruby_linux_amd64.go”, “ruby_darwin_arm64.go” ● https://pkg.go.dev/go/build#hdr-Build_Constraints ● https://pkg.go.dev/cmd/go#hdr-Build_constraints 98 Build Constraints (a.k.a. Build tag)

Slide 99

Slide 99 text

99 Custom Build Constraints // THE AUTOGENERATED LICENSE. ALL THE RIGHTS ARE RESERVED BY ROBOTS. // WARNING: This file has automatically been generated // Code generated by ruby_h_to_go. DO NOT EDIT. //go:build ruby_3_4 package ruby https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L1-L8 custom build tag

Slide 100

Slide 100 text

100 GOFLAGS=’-tags=ruby_x_x’ go build def create_go_makefile(target, srcprefix: nil, go_build_args: nil) (snip) goflags = "-tags=#{GoGem::Util.ruby_minor_version_build_tag}" File.open("Makefile", "a") do |f| f.write <<~MAKEFILE.gsub(/^ {8}/, "\t") $(DLLIB): Makefile $(srcdir)/*.go cd $(srcdir); \ CGO_CFLAGS='$(INCFLAGS)' CGO_LDFLAGS='#{ldflags}' GOFLAGS='#{goflags}' \ go build -p 4 -buildmode=c-shared -o #{current_dir}/$(DLLIB) #{go_build_args} MAKEFILE end end https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/_gem/lib/go_gem/mkmf.rb#L38-L47

Slide 101

Slide 101 text

● https://pkg.go.dev/ is the official reference site for Go standard libraries and modules ○ e.g. https://pkg.go.dev/github.com/ruby-go-gem/go-gem-wrapper ○ Similar to https://docs.ruby-lang.org/ + https://rubydoc.info/ 101 Issue 5. Supports https://pkg.go.dev/

Slide 102

Slide 102 text

● Before Custom Build Constraints was introduced, reference was displayed correctly on the https://pkg.go.dev/ ● But after, broken… 102 Issue 5. Supports https://pkg.go.dev/

Slide 103

Slide 103 text

Good 103 All go bindings were missing Bad

Slide 104

Slide 104 text

● https://pkg.go.dev/ doesn't support Custom Build Constraints ● I don't want 1,100 bindings not showing up on https://pkg.go.dev/ ● I thought about building this myself and hosting this on GitHub Pages, but I thought this would be confusing to have Go module references published outside of https://pkg.go.dev/ 104 Why?

Slide 105

Slide 105 text

● I want to fallback to ruby_3_3 if there is no Custom Build Constraints ● However, Go's current Build Constraints doesn’t allow such a condition to be specified. (maybe) 105 Ideal and Reality

Slide 106

Slide 106 text

106 Solution: ruby_h_to_go/config.yml default_tag: ruby_3_3 available_tags: - ruby_3_3 - ruby_3_4 https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/_tools/ruby_h_to_go/config.yml

Slide 107

Slide 107 text

107 ruby_h_to_go/lib/ruby_h_to_go/go_util.rb ruby_build_tag = GoGem::Util.ruby_minor_version_build_tag header << if ruby_build_tag == RubyHToGo.config.default_tag other_tags = RubyHToGo.config.available_tags - [RubyHToGo.config.default_tag] condition = other_tags.map { |tag| "!#{tag}" }.join(" && ") <<~GO // FIXME: https://pkg.go.dev/ doesn't support custom build tag. // Therefore, if no build tag is passed, treat it as the default tag //go:build #{ruby_build_tag} || (#{condition}) GO else <<~GO //go:build #{ruby_build_tag} GO end https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/_tools/ruby_h_to_go/lib/ruby_h_to_go/go_util.rb#L27-L45

Slide 108

Slide 108 text

108 function_ruby_3_3_generated.go // THE AUTOGENERATED LICENSE. ALL THE RIGHTS ARE RESERVED BY ROBOTS. // WARNING: This file has automatically been generated // Code generated by ruby_h_to_go. DO NOT EDIT. // FIXME: https://pkg.go.dev/ doesn't support custom build tag. // Therefore, if no build tag is passed, treat it as the default tag //go:build ruby_3_3 || !ruby_3_4 package ruby https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_3_generated.go

Slide 109

Slide 109 text

109 After ruby 3.5 is released (maybe) // THE AUTOGENERATED LICENSE. ALL THE RIGHTS ARE RESERVED BY ROBOTS. // WARNING: This file has automatically been generated // Code generated by ruby_h_to_go. DO NOT EDIT. // FIXME: https://pkg.go.dev/ doesn't support custom build tag. // Therefore, if no build tag is passed, treat it as the default tag //go:build ruby_3_3 || (!ruby_3_4 && !ruby_3_5) package ruby https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_3_generated.go

Slide 110

Slide 110 text

110 Tips. Propagate Ruby Reference to Go

Slide 111

Slide 111 text

111 RubyKaigi 2015 https://www.slideshare.net/slideshow/ruby-meets-go/56074507#32

Slide 112

Slide 112 text

112 RubyKaigi 2015 https://www.slideshare.net/slideshow/ruby-meets-go/56074507#33

Slide 113

Slide 113 text

● go-gem-wrapper provides a mechanism for managing Ruby reference counters in Go 113 Solution

Slide 114

Slide 114 text

114 Example (1/2) //export rb_example_go_struct_set func rb_example_go_struct_set(self C.VALUE, x C.VALUE, y C.VALUE) { data := (*GoStruct)(ruby.GetGoStruct(ruby.VALUE(self))) data.x = ruby.NUM2INT(ruby.VALUE(x)) data.y = ruby.NUM2INT(ruby.VALUE(y)) } //export rb_example_go_struct_get func rb_example_go_struct_get(self C.VALUE) C.VALUE { data := (*GoStruct)(ruby.GetGoStruct(ruby.VALUE(self))) ret := []ruby.VALUE{ ruby.INT2NUM(data.x), ruby.INT2NUM(data.y), } return C.VALUE(ruby.Slice2rbAry(ret)) } https://pkg.go.dev/github.com/ruby-go-gem/go-gem-wrapper/ruby#NewGoStruct

Slide 115

Slide 115 text

115 Example (2/2) //export go_struct_alloc func go_struct_alloc(klass C.VALUE) C.VALUE { data := GoStruct{} return C.VALUE(ruby.NewGoStruct(ruby.VALUE(klass), unsafe.Pointer(&data))) } //export Init_example func Init_example() { rb_mExample := ruby.RbDefineModule("Example") // Create Example::GoStruct class rb_cGoStruct := ruby.RbDefineClassUnder(rb_mExample, "GoStruct", ruby.VALUE(C.rb_cObject)) ruby.RbDefineAllocFunc(rb_cGoStruct, C.go_struct_alloc) ruby.RbDefineMethod(rb_cGoStruct, "set", C.rb_example_go_struct_set, 2) ruby.RbDefineMethod(rb_cGoStruct, "get", C.rb_example_go_struct_get, 0) } https://pkg.go.dev/github.com/ruby-go-gem/go-gem-wrapper/ruby#NewGoStruct

Slide 116

Slide 116 text

Use go build in Ruby native extension building 116 Up to here CRuby Go

Slide 117

Slide 117 text

Use “ruby.h” in go building 117 From here CRuby Go

Slide 118

Slide 118 text

118 Use “ruby.h” in go building

Slide 119

Slide 119 text

1. go test 2. golangci-lint 119 Use case

Slide 120

Slide 120 text

● go test is standard testing command for Go ● Functions created in native extension (not just Go) should be tested from Ruby ○ Because this is called from Ruby ● Functions created in pure-Go should be tested from Go ○ Because this is called from Go 120 Use case 1. go test

Slide 121

Slide 121 text

郷に入りては郷に従え Go ni irite wa Go ni shitagae 121 When in Rome, do as the Romans do

Slide 122

Slide 122 text

● In order to execute go test, go build must be able to execute ● “ruby.h” is needed to execute go build in Go gem ● Various environment variables are required to build with “ruby.h” 122 Problem: go test requires building with “ruby.h”

Slide 123

Slide 123 text

123 Minimum required environment variables for go build ENV["CGO_CFLAGS"] = "#{RbConfig::CONFIG["CFLAGS"]}" \ " -I#{RbConfig::CONFIG["rubyarchhdrdir"]}" \ " -I#{RbConfig::CONFIG["rubyhdrdir"]}" ENV["CGO_LDFLAGS"] = "-L#{RbConfig::CONFIG["libdir"]}" \ " -l#{RbConfig::CONFIG["RUBY_SO_NAME"]}" ENV["LD_LIBRARY_PATH"] = "#{RbConfig::CONFIG["libdir"]}"

Slide 124

Slide 124 text

124 go test full command GOFLAGS=-tags=ruby_3_4 CGO_CFLAGS=-fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wextra-tokens -Wdeprecated-declarations -Wdivision-by-zero -Wdiv-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wmisleading-indentation -Wundef -fno-common -pipe -I/Users/sue445/.rbenv/versions/3.4.1/include/ruby-3.4.0/arm64-darwin24 -I/Users/sue445/.rbenv/versions/3.4.1/include/ruby-3.4.0 CGO_LDFLAGS=-L/Users/sue445/.rbenv/versions/3.4.1/lib -lruby.3.4 -undefined dynamic_lookup LD_LIBRARY_PATH=/Users/sue445/.rbenv/versions/3.4.1/lib go test -mod=readonly -count=1 ./...

Slide 125

Slide 125 text

# Rakefile require "go_gem/rake_task" GoGem::RakeTask.new("example") Solution GoGem::RakeTask Provides rake tasks for go test with CRuby 125

Slide 126

Slide 126 text

$ bundle exec rake -T | grep "go:" rake go:build_envs[env_name] # Print build envs for `go build` rake go:build_tag # Print build tag rake go:fmt # Run go fmt rake go:test # Run go test rake go:testrace # Run go test -race rake go tasks 126

Slide 127

Slide 127 text

● With Go, in addition to the official linter, there are various 3rd party tools for each linter ● In Ruby, it's similar to the way gem is created for each RuboCop’s Cop 127 Go’s linter situation

Slide 128

Slide 128 text

● golangci-lint is linters runner for Go ● https://github.com/golangci/golangci-lint ● This is 3rd party tool but widely used in the Go community 128 Use case 2. golangci-lint

Slide 129

Slide 129 text

# Rakefile require "go_gem/rake_task" go_task = GoGem::RakeTask.new("gem_name") namespace :go do desc "Run golangci-lint" task :lint do go_task.within_target_dir do sh "which golangci-lint" do |ok, _| raise "golangci-lint isn't installed. See. https://golangci-lint.run/welcome/install/" unless ok end build_tag = GoGem::Util.ruby_minor_version_build_tag sh GoGem::RakeTask.build_env_vars, "golangci-lint run --build-tags #{build_tag} --modules-download-mode=readonly" end end end e.g. Use golangci-lint in Go gem 129 https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.3/_gem#example-add-additional-tasks

Slide 130

Slide 130 text

● “patch_for_go_gem” is needed to make Go gem ● I want to use bundler gem similar to --ext=c and --ext=rust 130 Add --ext=go to bundle gem command

Slide 131

Slide 131 text

131 https://github.com/orgs/rubygems/discussions/8128

Slide 132

Slide 132 text

● I waited for a while and got no response ● So I had hsbt triage it on ruby-jp slack 132 Feature Proposal

Slide 133

Slide 133 text

133 LGTM

Slide 134

Slide 134 text

134 https://github.com/rubygems/rubygems/pull/8183

Slide 135

Slide 135 text

135 Pros/Cons of Go gem

Slide 136

Slide 136 text

● We can use Go modules in Ruby 136 Pros 1

Slide 137

Slide 137 text

● We can write in Go in areas where Ruby is not good at (and Go is very good at) ● Go's features not found in other languages ○ goroutine: lightweight thread ■ https://go.dev/tour/concurrency/1 ○ channel: Messaging between multiple goroutine ■ https://go.dev/tour/concurrency/2 137 Pros 2

Slide 138

Slide 138 text

● Benchmark with tak function ○ https://www.ruby-lang.org/en/news/2020/12/25/ruby-3-0-0-released/ ● Multi thread and multi process are not eligible ○ Because the results depend on the number of CPU cores on the machine being benchmarked ○ This is not fair to compare these to goroutine 138 Ractor vs Fiber vs goroutine

Slide 139

Slide 139 text

139 goroutine is 30x faster than Ractor $ ruby tarai.rb go version go1.24.1 darwin/arm64 ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [arm64-darwin24] Comparison: Go: goroutine: 1.6 i/s Go: sequential: 0.5 i/s - 3.65x slower Ruby: Ractor: 0.1 i/s - 31.58x slower Ruby: sequential: 0.0 i/s - 92.85x slower Ruby: Fiber: 0.0 i/s - 93.08x slower https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.3/_benchmark

Slide 140

Slide 140 text

140 @osyoyu says https://speakerdeck.com/osyoyu/zigdeckuo-zhang-wozuo-ru-2024-edition?slide=34

Slide 141

Slide 141 text

141 Native extension with Zig https://speakerdeck.com/osyoyu/zigdeckuo-zhang-wozuo-ru-2024-edition

Slide 142

Slide 142 text

● go-gem-wrapper is a low-layer wrapper, so not much different than making Native extension with C ○ Similar to writing in Go where you would write in C ○ Need to be aware of both CRuby and Go 142 Cons 1

Slide 143

Slide 143 text

● Go's variable-length arguments couldn't be passed directly to C ○ CRuby provides both variable length argument and non-variable length argument versions of its functions. ○ If we only have functions with variable-length arguments, it is a bit of a problem ■ e.g. rb_raise 143 Cons 2

Slide 144

Slide 144 text

144 Workaround for rb_raise https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/ruby_internal_error.go /* #include "ruby.h" void __rb_raise(VALUE exception, const char *str) { rb_raise(exception, "%s", str); } */ import "C" func RbRaise(exc VALUE, format string, a ...interface{}) { str := fmt.Sprintf(format, a...) char, clean := string2Char(str) defer clean() C.__rb_raise(C.VALUE(exc), char) }

Slide 145

Slide 145 text

● https://github.com/orgs/ruby-go-gem/discussions/236 145 I'm looking for good ideas and patches!

Slide 146

Slide 146 text

● Not intended for use with ruby-head, since binding is required for each Ruby version 146 Cons 3

Slide 147

Slide 147 text

● SEGV when calling CRuby functions in goroutine ● c.f. https://github.com/ruby-go-gem/go-gem-wrapper/issues/286 147 Cons 4

Slide 148

Slide 148 text

This doesn’t work… (SEGV!!!!) 148 e.g. Call yield inside goroutine //export rb_example_call_proc_inside_goroutine func rb_example_call_yield_inside_goroutine(self C.VALUE) { if ruby.RbBlockGivenP() == 0 { ruby.RbRaise(C.rb_eArgError, "Block not given") } go func() { ruby.RbYield(C.Qnil) }() }

Slide 149

Slide 149 text

● glibc is required ○ Not supported for use with musl (e.g. alpine) ○ c.f. https://github.com/ruby-go-gem/go-gem-wrapper/issues/253 ○ Windows (maybe) 149 Cons 5

Slide 150

Slide 150 text

150 Real use case

Slide 151

Slide 151 text

● Perform HTTP requests in parallel with goroutine ● Unlike thread and process, we don't have to specify concurrency ○ e.g. max threads, max processes 151 https://github.com/sue445/funnel_http

Slide 152

Slide 152 text

152 Usage require "funnel_http" client = FunnelHttp::Client.new requests = [ { method: :get, url: "https://example.com/api/user/1", }, ] responses = client.perform(requests) # => [ # { status_code: 200, body: "Response of /api/user/1", # header: { "Content-Type" => ["text/plain;charset=utf-8"]} } # ]

Slide 153

Slide 153 text

NOTE: open-uri and net/http doesn’t work in Ractor… 153 benchmark (Fiber vs goroutine) $ bundle exec ruby benchmark.rb go version go1.24.1 darwin/arm64 ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24] Comparison: FunnelHttp::Client#perform: 22.7 i/s sequential: 7.0 i/s - 3.27x slower Parallel with Fiber: 3.7 i/s - 6.14x slower https://github.com/sue445/funnel_http/tree/v0.3.2/benchmark

Slide 154

Slide 154 text

● Using https://github.com/ruby-go-gem/go-gem-wrapper makes native extension with Go very easy to make ● We can write Ruby gem in Go in areas where Ruby is not good at (Go is very good at) ● goroutine is very good! 154 Conclusion

Slide 155

Slide 155 text

goroutine is very good! 155 Conclusion

Slide 156

Slide 156 text

goroutine is very good! 156 Conclusion