Share
Embed
Transcript
156 slides
-
Road to Go gem RubyKaigi 2025 / #rubykaigi #rubykaigiC pixiv
Inc. sue445 2025.04.18 -
3 My name is Go The Go gopher was designed
by Renee French. (http://reneefrench.blogspot.com/) The design is licensed under the Creative Commons 4.0 Attributions license. Read this article for more details: https://blog.golang.org/gopher -
4 About sue445 • Go Sueyoshi (a.k.a. sue445, sue-san) •
https://x.com/sue445 • https://github.com/sue445 • Gopher since 1982 • Shibuya.rb • Tokyu.rb sue445 -
5 Day job at pixiv.Inc • Infrastructure Unit • AWS
and Google Cloud ◦ Organization Owner ◦ Solution Architect • Certifications possessed ◦ AWS Certified Solutions Architect – Professional (SAP-C01) ◦ Google Cloud Professional Cloud Architect sue445 -
6 Day job at pixiv.Inc • on-premises Kubernetes • Maintainer
of In-house tools (GitLab, Sentry etc) • etc etc etc!!!!!!!!!! sue445 -
• All about native extension with Go • Tips for
creating native extensions in a new language 11 Goal in this talk -
• Let's recall RubyKaigi 2015 • Getting started with Go
gem • About go-gem-wrapper • Road to go-gem-wrapper • Use “ruby.h” in go building • Add --ext=go to bundle gem command • Pros/Cons of Go gem • Real use case 12 Agenda -
• Using https://github.com/ruby-go-gem/go-gem-wrapper makes it very easy to make native
extension with Go • We can write Ruby gem in Go in areas where Ruby is not good at (Go is very good at) • goroutine is very good! 13 Conclusion -
• cgo (go build -buildmode=c-shared) was introduced at Go 1.5
• cgo creates shared object file (*.so) from Go • Use this technology to create a native Ruby extension with Go ◦ a.k.a. Go gem 15 tl;dr; -
1. The code from 10 years ago (Go 1.5) doesn't
work now (Go 1.24) ◦ a later mention 2. To make a gem in Go, you need to copy and paste a lot of patches. ◦ e.g. ▪ ext/gem_name/extconf.rb ▪ ext/gem_name/my_gem.go ▪ ext/gem_name/go.mod ▪ bindings ▪ etc… 16 Past issues -
• Technically possible. But it takes a lot of work.
18 Past issues -
• Library of patches that needed to be done manually
• There was no standard package manager for Go at 10 years ago • But today, we have Go module 19 github.com/ruby-go-gem/go-gem-wrapper -
1. bundle gem <gem_name> –ext=c 2. Run “patch_for_go_gem.rb” 3. go
get -u github.com/ruby-go-gem/go-gem-wrapper@latest 4. Write gem using Go! 20 Getting started with Go gem -
$ bundle gem example --ext=c Creating gem 'example'... create example/Gemfile
create example/lib/example.rb create example/lib/example/version.rb (snip) create example/ext/example/extconf.rb create example/ext/example/example.h create example/ext/example/example.c Gem 'gem_name' was successfully created. For more information on making a RubyGem visit https://bundler.io/guides/creating_gem.html bundle gem <gem_name> –ext=c 21 -
• https://github.com/ruby-go-gem/go-gem-wrapper/tree/main/_tools/patch_ for_go_gem • Patch to make a gem into
a Go gem right after bundle gem • This is a temporary tool until bundle gem --ext=go is added to the bundler ◦ c.f. https://github.com/rubygems/rubygems/pull/8183 (a later mention) 22 Run “patch_for_go_gem.rb” -
$ ruby patch_for_go_gem.rb --file example/example.gemspec [INFO] /path/to/example/ext/example/example.go is created [INFO]
/path/to/example/ext/example/go.mod is created [INFO] /path/to/example/ext/example/example.c is updated [INFO] /path/to/example/ext/example/extconf.rb is updated [INFO] example/example.gemspec is updated Run “patch_for_go_gem.rb” 23 -
package main /* #include "example.h" */ import "C" import (
"github.com/ruby-go-gem/go-gem-wrapper/ruby" ) //export Init_example func Init_example() { rb_mExample := ruby.RbDefineModule("Example") } func main() { } ext/<gem_name>/<gem_name>.go 24 -
module Example def self.sum(a, b) a + b end end
e.g. Impl Example#sum 28 -
//export rb_example_sum func rb_example_sum(_ C.VALUE, a C.VALUE, b C.VALUE) C.VALUE
{ aLong := ruby.NUM2LONG(ruby.VALUE(a)) bLong := ruby.NUM2LONG(ruby.VALUE(b)) sum := aLong + bLong return C.VALUE(ruby.LONG2NUM(sum)) } ext/<gem_name>/<gem_name>.go 29 -
package main /* #include "example.h" // Append this VALUE rb_example_sum(VALUE
self, VALUE a, VALUE b); */ import "C" ext/<gem_name>/<gem_name>.go 30 -
//export Init_example func Init_example() { rb_mExample := ruby.RbDefineModule("Example") // Append
this ruby.RbDefineSingletonMethod(rb_mExample, "sum", C.rb_example_sum, 2) } ext/<gem_name>/<gem_name>.go 31 -
• We can write a gem in Go just as
you write a gem in C! 32 Easy, right? -
• Go module: github.com/ruby-go-gem/go-gem-wrapper • Ruby gem: https://rubygems.org/gems/go_gem • Requiremets
◦ Go 1.23+ ◦ Ruby 3.3+ ◦ glibc (a later mention) 33 About go-gem-wrapper -
github.com/ruby-go-gem/go-gem-wrapper is Go module for Go gem 34 About github.com/ruby-go-gem/go-gem-wrapper
// ext/example/go.mod module github.com/sue445/example go 1.24 require ( github.com/ruby-go-gem/go-gem-wrapper v0.7.1 ) -
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/ruby/function_ruby_3_3_generated.go#L2687-L2697 35 e.g. RbDefineSingletonMethod func RbDefineSingletonMethod(obj VALUE, mid string, arg3
unsafe.Pointer, arity int) { char, clean := string2Char(mid) defer clean() C.rb_define_singleton_method(C.VALUE(obj), char, toCFunctionPointer(arg3), C.int(arity)) } -
36 There are 1,100 functions in “ruby.h” $ ls ruby/*_generated.go
| xargs wc -l 59 ruby/enum_ruby_3_3_generated.go 57 ruby/enum_ruby_3_4_generated.go 9264 ruby/function_ruby_3_3_generated.go 9198 ruby/function_ruby_3_4_generated.go 105 ruby/type_ruby_3_3_generated.go 103 ruby/type_ruby_3_4_generated.go 18786 total https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/ruby -
• Helpers for compiling Go extensions for ruby ◦ https://rubygems.org/gems/go_gem
◦ https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/_gem • Similar to https://rubygems.org/gems/rb_sys in Rust gem 37 About go_gem -
require "mkmf" require "go_gem/mkmf" create_go_makefile("example/example") # Pass debug flags to
`go build` create_go_makefile("example/example", go_build_args: "-gcflags='all=-N -l'") create_go_makefile Add a task and hacks to build Go against the “Makefile” generated by create_makefile. (similar to create_rust_makeflile) 38 -
39 create_go_makefile def create_go_makefile(target, srcprefix: nil, go_build_args: nil) # (snip)
File.open("Makefile", "a") do |f| f.write <<~MAKEFILE.gsub(/^ {8}/, "\t") $(DLLIB): Makefile $(srcdir)/*.go cd $(srcdir); \ CGO_CFLAGS='$(INCFLAGS)' CGO_LDFLAGS='#{ldflags}' GOFLAGS='#{goflags}' \ go build -p 4 -buildmode=c-shared -o #{current_dir}/$(DLLIB) #{go_build_args} MAKEFILE end end https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/_gem/lib/go_gem/mkmf.rb -
# Rakefile require "go_gem/rake_task" GoGem::RakeTask.new("example") GoGem::RakeTask Provides rake tasks for
go test with CRuby. (a later mention) 40 -
• I had to go through many hardships to create
go-gem-wrapper • I will now introduce them 41 Road to go-gem-wrapper -
1. cgo compatibility 2. Generate Go’s bindings from “ruby.h” ◦
Type mismatch between outside and inside of module ◦ C pointer ◦ deprecated functions ◦ Removed functions ◦ Supports https://pkg.go.dev/ 3. Go with CRuby 42 Road to go-gem-wrapper -
• This technique was introduced in RubyKaigi 2015 • No
problem at Go 1.5, but SEGV at Go 1.24 43 1. cgo compatibility https://www.slideshare.net/slideshow/ruby-meets-go/56074507 -
• Current Go's string isn't a null-terminated string, so using
*(*[]byte)(unsafe.Pointer(&str)) will return a non-null-terminated byte array and can't be *C.char • Go 1.7+ are spec'd this way 44 Why? -
45 e.g. Print Go string via C package main /*
#include <stdio.h> */ import "C" func Print(str string) { cstr := GOSTRING_PTR(str) C.fputs(cstr, (*C.FILE)(C.stdout)) } func main() { Print("ABCD") } -
46 OMG! $ go run main.go ABCDtrueallgallprootitabsbrkidledeadsync is LEAFbase of
) = <==GOGC] = pc=+Inf-Inf: p=cas1cas2cas3cas4cas5cas6 at m= sp= sp: lr: fp= gp= mp=) m=boolint8uintchanfuncopenstatfile in sha1sha2timeJuneJuly/etcfalsedefersweeptestRtestWexecWhchanexecRschedsudogtimergscanm heaptracepanicsleeparm64 cnt=gcing MB, got= ... max=scav ptr ] = (init ms, fault and tab= top=[...], fp:int16int32int64uint8arrayslicewriteclosegetwdlstatGreekpmullcrc32cpuidfcntlMarchAp rilLocalerrno stringsysmontimersefenceselect, not GOROOTobject next= jobs= goid sweep B -> % util alloc free span= prev= list=, i = code= addr=], sp= m->p= p->m=SCHED curg= ctxt: min= max= bad ts(...) -
47 Solution func string2Char(str string) (*C.char, func()) { cstr :=
C.CString(str) clean := func() { C.free(unsafe.Pointer(cstr)) } return cstr, clean } https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/ruby/wrapper.go#L26-L48 -
48 Solution func RbDefineSingletonMethod(obj VALUE, mid string, arg3 unsafe.Pointer, arity
int) { char, clean := string2Char(mid) defer clean() C.rb_define_singleton_method(C.VALUE(obj), char, toCFunctionPointer(arg3), C.int(arity)) } https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.1/ruby/function_ruby_3_3_ge nerated.go#L2687-L2697 -
• Go, similar to Ruby, is a very backward-compatible language.
• However, cgo has characteristics that depend on Go's internal implementation and can be broken by Go version upgrades • c.f. “Why Go's old code almost never stops working“ ◦ https://zenn.dev/catatsuy/articles/fda1e42acad421 (ja) 49 Tips -
• In this talk, binding is a Go function that
corresponds to a CRuby function as follows 51 2. Generate Go’s bindings from “ruby.h” func RbDefineSingletonMethod(obj VALUE, mid string, arg3 unsafe.Pointer, arity int) { char, clean := string2Char(mid) defer clean() C.rb_define_singleton_method(C.VALUE(obj), char, toCFunctionPointer(arg3), C.int(arity)) } -
• Just on https://docs.ruby-lang.org/ja/latest/function/index.html , there are less than 200
• There are about 2,000+ C functions defined in ruby.h • About 1,100 functions can be included and actually used • At first (about 30 functions), I wrote them by hand, but I decided to generate them automatically from ruby.h because it's just too hard. 52 2. Generate Go’s bindings from “ruby.h” -
• Bindings generated by c-for-go was abandoned due to lack
of portability ◦ e.g. Bindings generated on MacOS isn’t usable on Ubuntu 56 Result -
• It may be due to the fact that there
are many #ifdef branches in “ruby.h” for each OS and build environment. • e.g. 57 Result https://github.com/ruby/ruby/blob/v3_4_2/internal/process.h#L14-L20 #ifdef HAVE_SYS_TYPES_H # include <sys/types.h> /* for mode_t */ #endif #ifdef _WIN32 # include "ruby/win32.h" /* for mode_t */ #endif -
• I can change the source code for each environment
we build by creating a file similar to “ruby_linux_amd64.go” or “ruby_darwin_arm64.go” (Build constraints) • But this increases the amount of code, and CI also needs to prepare all build constraints • I gave up on it because it would be too hard maintain ◦ https://github.com/ruby-go-gem/go-gem-wrapper/pull/117#issuecomm ent-2336583098 58 Result -
• However, I couldn't figure out how to create it…
59 Create it myself -
• In a nutshell, I implemented what Rust gem does
in Go gem • I decided to generate Go source code automatically from ruby.h 61 tl;dr; -
• https://github.com/rust-lang/rust-bindgen ◦ Auto-generate C/C++ code from Rust FFI bindings
• https://github.com/oxidize-rb/rb-sys ◦ Auto-generate Rust code from C API using bindgen ◦ https://rubygems.org/gems/rb_sys : Helpers for compiling Rust extensions for ruby • https://github.com/matsadler/magnus ◦ High level Ruby bindings for Rust ◦ depends on rb-sys 62 Architecture of Rust gem -
• rust-bindgen (Generate bindings) + rb-sys (Low layer API) =
go-gem-wrapper • go-gem-wrapper doesn’t include high layer API similar to magnus (maybe…) • I created go-gem-wrapper by myself 63 What is go-gem-wrapper -
• https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/_tools/ruby _h_to_go • Generate Go code from “ruby.h” using
ruby_header_parser 67 ruby_h_to_go -
• Issue 1. Type mismatch between outside and inside of
module • Issue 2. C pointer • Issue 3. deprecated functions • Issue 4. Removed functions • Issue 5. Supports https://pkg.go.dev/ 68 Many many issues… -
package main // snip import ( "github.com/ruby-go-gem/go-gem-wrapper" ) //export rb_dummy_sum
func rb_dummy_sum(_ C.VALUE, a C.VALUE, b C.VALUE) C.VALUE { aLong := ruby.RbNum2Long(a) bLong := ruby.RbNum2Long(b) sum := aLong + bLong return ruby.RbLong2NumInline(sum) } PoC (1/2) 70 -
// in github.com/ruby-go-gem/go-gem-wrapper module package ruby /* #include "ruby.h" */
import "C" func RbNum2Long(n C.VALUE) C.long { return C.rb_num2long(n) } PoC (2/2) 71 -
./dummy.go:16:35: cannot use a (variable of type _Ctype_ulong) as ruby._Ctype_ulong
value in argument to ruby.RbNum2Long ./dummy.go:17:35: cannot use b (variable of type _Ctype_ulong) as ruby._Ctype_ulong value in argument to ruby.RbNum2Long ./dummy.go:21:39: cannot use sum (variable of type _Ctype_ulong) as ruby.Long value in argument to ruby.RbLong2NumInline make: *** [dummy.bundle] Error 1 Build error!!!!! 72 -
https://pkg.go.dev/cmd/cgo says > Cgo translates C types into equivalent unexported
Go types. Because the translations are unexported, a Go package should not expose C types in its exported API: a C type used in one Go package is different from the same C type used in another. ref. https://github.com/golang/go/issues/13467 73 Answer. cgo specification -
package ruby /* #include "ruby.h" */ import "C" type Long
C.long type VALUE C.VALUE func RbNum2long(n VALUE) Long { return Long(C.rb_num2long(C.VALUE(n))) } Workarround 74 -
• C lang's pointers are very difficult for programming beginners
• C lang's pointers are very difficult for C lang's parser and Go generator too! 76 Issue 2. C pointer -
• Please answer Go type that you would expect given
C type • e.g. C type: int -> Go type: int 77 Quiz. Convert C type to Go type -
• A. Case by case ◦ string (in many cases)
◦ char* (when RSTRING_PTR and RSTRING_END returns) 79 Q1. C type char* -> Go type ??? https://github.com/ruby/ruby/blob/v3_4_2/include/ruby/internal/core/rstring.h#L408-L416 /** * Queries the contents pointer of the string. * * @param[in] str String in question. * @return Pointer to its contents. * @pre `str` must be an instance of ::RString. */ static inline char * RSTRING_PTR(VALUE str) -
• A. Case by case ◦ Array ◦ Pointer for
input/output ◦ Pointer for output only ◦ Pointer for input only 81 Q2. C type VALUE* -> Go type ??? -
82 e.g. Use VALUE* as Array // RbFuncallv calls `rb_funcallv`
in C // // Original definition is following // // VALUE rb_funcallv(VALUE recv, ID mid, int argc, const VALUE *argv) func RbFuncallv(recv VALUE, mid ID, argc int, argv []VALUE) VALUE { return VALUE(C.rb_funcallv(C.VALUE(recv), C.ID(mid), C.int(argc), toCArray[VALUE, C.VALUE](argv))) } https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L3885-L3892 -
83 e.g. Use VALUE* as in/out or out only pointer
// RbCvarFind calls `rb_cvar_find` in C // // Original definition is following // // VALUE rb_cvar_find(VALUE klass, ID name, VALUE *front) func RbCvarFind(klass VALUE, name ID, front *VALUE) VALUE { var cFront C.VALUE ret := VALUE(C.rb_cvar_find(C.VALUE(klass), C.ID(name), &cFront)) *front = VALUE(cFront) return ret } https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L2278-L2288 -
84 e.g. Use VALUE* as input only pointer // RbDefineVariable
calls `rb_define_variable` in C // // Original definition is following // // void rb_define_variable(const char *name, VALUE *var) func RbDefineVariable(name string, v *VALUE) { char, clean := string2Char(name) defer clean() C.rb_define_variable(char, (*C.VALUE)(v)) } https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L2688-L2698 -
• A. Case by case ◦ Multiple pointer ◦ Array
of pointer ◦ Array of String 86 Q3. C type xxx** -> Go type ??? -
• Normal pointer • Input only pointer • Special one
for multiple pointer • Array • Array of pointer • Array of string • Function pointer 87 Pointer list in “ruby.h” -
• C type to Go type not uniquely defined •
Case by case!!!!! 88 Problem. Case by case -
• All were handled individually • About 37 / 1,100
functions 89 Solution -
90 ruby_header_parser/config/default.yml.erb function: pointer_hint: RSTRING_END: self: raw RSTRING_PTR: self: raw
rb_data_object_make: 4: sref rb_data_typed_object_make: 3: sref https://github.com/ruby-go-gem/ruby_header_parser/blob/main/config/default.yml.erb#L83-L95 -
• Configuration file specification ◦ https://github.com/ruby-go-gem/ruby_header_parser/blob/v0.4.2/CON FIG.md • Hint ◦
https://github.com/xlab/c-for-go/wiki/Translator-config-section 91 ruby_header_parser/config/default.yml.erb -
• Some functions in “ruby.h” are deprecated • Deprecated function
bindings should not be generated 92 Issue 3. deprecated functions $ bundle exec rake build_all /Users/sue445/.rbenv/versions/3.4.1/include/ruby-3.4.0/ruby/internal/core/rdata.h:314 :1: note: 'rb_data_object_get_warning' has been explicitly marked deprecated here cgo-gcc-prolog:4427:11: warning: 'rb_data_object_wrap_warning' is deprecated: by TypedData [-Wdeprecated-declarations] -
93 Solution. ruby_header_parser/config/default.yml.erb function: exclude_name: # deprecated functions - !ruby/regexp
/^rb_check_safe_str$/i - !ruby/regexp /^rb_clear_constant_cache$/i - !ruby/regexp /^rb_clone_setup$/i - !ruby/regexp /^rb_complex_polar$/i - !ruby/regexp /^rb_data_object_alloc$/i - !ruby/regexp /^rb_data_object_get_warning$/i - !ruby/regexp /^rb_data_object_wrap_warning$/i - !ruby/regexp /^rb_data_typed_object_alloc$/i https://github.com/ruby-go-gem/ruby_header_parser/blob/v0.4.2/config/default.yml.erb#L7 -
94 Solution. ruby_header_parser/config/default.yml.erb function: exclude_name: # internal macros - !ruby/regexp
/^rb_define_global_function_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_method_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_method_id_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_module_function_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_private_method_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_protected_method_(m?[0-9]+|notimpl)$/i - !ruby/regexp /^rb_define_singleton_method_(m?[0-9]+|notimpl)$/i https://github.com/ruby-go-gem/ruby_header_parser/blob/v0.4.2/config/default.yml.erb#L33 -
• Some functions were removed in Ruby 3.4 ◦ e.g.
rb_data_object_make ◦ https://bugs.ruby-lang.org/issues/20265 ◦ https://bugs.ruby-lang.org/issues/18290 • Calling a non-existent C function from within Go code will result in an error at build time 95 Issue 4. Removed functions -
• Even if these are removed in Ruby 3.4+, Ruby
3.3 bindings need to be able to use it as these are • It was necessary to use different Go bindings depending on the version of Ruby at the time of build 96 Issue 4. Removed functions -
97 Solution. Split binding files $ ls -1 ruby/*_generated.go ruby/enum_ruby_3_3_generated.go
ruby/enum_ruby_3_4_generated.go ruby/function_ruby_3_3_generated.go ruby/function_ruby_3_4_generated.go ruby/type_ruby_3_3_generated.go ruby/type_ruby_3_4_generated.go https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.1/ruby -
• Ability to change files to be built according to
conditions ◦ e.g. “ruby_linux_amd64.go”, “ruby_darwin_arm64.go” • https://pkg.go.dev/go/build#hdr-Build_Constraints • https://pkg.go.dev/cmd/go#hdr-Build_constraints 98 Build Constraints (a.k.a. Build tag) -
99 Custom Build Constraints // THE AUTOGENERATED LICENSE. ALL THE
RIGHTS ARE RESERVED BY ROBOTS. // WARNING: This file has automatically been generated // Code generated by ruby_h_to_go. DO NOT EDIT. //go:build ruby_3_4 package ruby https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_4_generated.go#L1-L8 custom build tag -
100 GOFLAGS=’-tags=ruby_x_x’ go build def create_go_makefile(target, srcprefix: nil, go_build_args: nil)
(snip) goflags = "-tags=#{GoGem::Util.ruby_minor_version_build_tag}" File.open("Makefile", "a") do |f| f.write <<~MAKEFILE.gsub(/^ {8}/, "\t") $(DLLIB): Makefile $(srcdir)/*.go cd $(srcdir); \ CGO_CFLAGS='$(INCFLAGS)' CGO_LDFLAGS='#{ldflags}' GOFLAGS='#{goflags}' \ go build -p 4 -buildmode=c-shared -o #{current_dir}/$(DLLIB) #{go_build_args} MAKEFILE end end https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/_gem/lib/go_gem/mkmf.rb#L38-L47 -
• https://pkg.go.dev/ is the official reference site for Go standard
libraries and modules ◦ e.g. https://pkg.go.dev/github.com/ruby-go-gem/go-gem-wrapper ◦ Similar to https://docs.ruby-lang.org/ + https://rubydoc.info/ 101 Issue 5. Supports https://pkg.go.dev/ -
• Before Custom Build Constraints was introduced, reference was displayed
correctly on the https://pkg.go.dev/ • But after, broken… 102 Issue 5. Supports https://pkg.go.dev/ -
• https://pkg.go.dev/ doesn't support Custom Build Constraints • I don't
want 1,100 bindings not showing up on https://pkg.go.dev/ • I thought about building this myself and hosting this on GitHub Pages, but I thought this would be confusing to have Go module references published outside of https://pkg.go.dev/ 104 Why? -
• I want to fallback to ruby_3_3 if there is
no Custom Build Constraints • However, Go's current Build Constraints doesn’t allow such a condition to be specified. (maybe) 105 Ideal and Reality -
106 Solution: ruby_h_to_go/config.yml default_tag: ruby_3_3 available_tags: - ruby_3_3 - ruby_3_4
https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/_tools/ruby_h_to_go/config.yml -
107 ruby_h_to_go/lib/ruby_h_to_go/go_util.rb ruby_build_tag = GoGem::Util.ruby_minor_version_build_tag header << if ruby_build_tag ==
RubyHToGo.config.default_tag other_tags = RubyHToGo.config.available_tags - [RubyHToGo.config.default_tag] condition = other_tags.map { |tag| "!#{tag}" }.join(" && ") <<~GO // FIXME: https://pkg.go.dev/ doesn't support custom build tag. // Therefore, if no build tag is passed, treat it as the default tag //go:build #{ruby_build_tag} || (#{condition}) GO else <<~GO //go:build #{ruby_build_tag} GO end https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/_tools/ruby_h_to_go/lib/ruby_h_to_go/go_util.rb#L27-L45 -
108 function_ruby_3_3_generated.go // THE AUTOGENERATED LICENSE. ALL THE RIGHTS ARE
RESERVED BY ROBOTS. // WARNING: This file has automatically been generated // Code generated by ruby_h_to_go. DO NOT EDIT. // FIXME: https://pkg.go.dev/ doesn't support custom build tag. // Therefore, if no build tag is passed, treat it as the default tag //go:build ruby_3_3 || !ruby_3_4 package ruby https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_3_generated.go -
109 After ruby 3.5 is released (maybe) // THE AUTOGENERATED
LICENSE. ALL THE RIGHTS ARE RESERVED BY ROBOTS. // WARNING: This file has automatically been generated // Code generated by ruby_h_to_go. DO NOT EDIT. // FIXME: https://pkg.go.dev/ doesn't support custom build tag. // Therefore, if no build tag is passed, treat it as the default tag //go:build ruby_3_3 || (!ruby_3_4 && !ruby_3_5) package ruby https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/function_ruby_3_3_generated.go -
• go-gem-wrapper provides a mechanism for managing Ruby reference counters
in Go 113 Solution -
114 Example (1/2) //export rb_example_go_struct_set func rb_example_go_struct_set(self C.VALUE, x C.VALUE,
y C.VALUE) { data := (*GoStruct)(ruby.GetGoStruct(ruby.VALUE(self))) data.x = ruby.NUM2INT(ruby.VALUE(x)) data.y = ruby.NUM2INT(ruby.VALUE(y)) } //export rb_example_go_struct_get func rb_example_go_struct_get(self C.VALUE) C.VALUE { data := (*GoStruct)(ruby.GetGoStruct(ruby.VALUE(self))) ret := []ruby.VALUE{ ruby.INT2NUM(data.x), ruby.INT2NUM(data.y), } return C.VALUE(ruby.Slice2rbAry(ret)) } https://pkg.go.dev/github.com/ruby-go-gem/go-gem-wrapper/ruby#NewGoStruct -
115 Example (2/2) //export go_struct_alloc func go_struct_alloc(klass C.VALUE) C.VALUE {
data := GoStruct{} return C.VALUE(ruby.NewGoStruct(ruby.VALUE(klass), unsafe.Pointer(&data))) } //export Init_example func Init_example() { rb_mExample := ruby.RbDefineModule("Example") // Create Example::GoStruct class rb_cGoStruct := ruby.RbDefineClassUnder(rb_mExample, "GoStruct", ruby.VALUE(C.rb_cObject)) ruby.RbDefineAllocFunc(rb_cGoStruct, C.go_struct_alloc) ruby.RbDefineMethod(rb_cGoStruct, "set", C.rb_example_go_struct_set, 2) ruby.RbDefineMethod(rb_cGoStruct, "get", C.rb_example_go_struct_get, 0) } https://pkg.go.dev/github.com/ruby-go-gem/go-gem-wrapper/ruby#NewGoStruct -
Use go build in Ruby native extension building 116 Up
to here CRuby Go -
• go test is standard testing command for Go •
Functions created in native extension (not just Go) should be tested from Ruby ◦ Because this is called from Ruby • Functions created in pure-Go should be tested from Go ◦ Because this is called from Go 120 Use case 1. go test -
郷に入りては郷に従え Go ni irite wa Go ni shitagae 121 When
in Rome, do as the Romans do -
• In order to execute go test, go build must
be able to execute • “ruby.h” is needed to execute go build in Go gem • Various environment variables are required to build with “ruby.h” 122 Problem: go test requires building with “ruby.h” -
123 Minimum required environment variables for go build ENV["CGO_CFLAGS"] =
"#{RbConfig::CONFIG["CFLAGS"]}" \ " -I#{RbConfig::CONFIG["rubyarchhdrdir"]}" \ " -I#{RbConfig::CONFIG["rubyhdrdir"]}" ENV["CGO_LDFLAGS"] = "-L#{RbConfig::CONFIG["libdir"]}" \ " -l#{RbConfig::CONFIG["RUBY_SO_NAME"]}" ENV["LD_LIBRARY_PATH"] = "#{RbConfig::CONFIG["libdir"]}" -
124 go test full command GOFLAGS=-tags=ruby_3_4 CGO_CFLAGS=-fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fdeclspec
-O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wextra-tokens -Wdeprecated-declarations -Wdivision-by-zero -Wdiv-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wmisleading-indentation -Wundef -fno-common -pipe -I/Users/sue445/.rbenv/versions/3.4.1/include/ruby-3.4.0/arm64-darwin24 -I/Users/sue445/.rbenv/versions/3.4.1/include/ruby-3.4.0 CGO_LDFLAGS=-L/Users/sue445/.rbenv/versions/3.4.1/lib -lruby.3.4 -undefined dynamic_lookup LD_LIBRARY_PATH=/Users/sue445/.rbenv/versions/3.4.1/lib go test -mod=readonly -count=1 ./... -
$ bundle exec rake -T | grep "go:" rake go:build_envs[env_name]
# Print build envs for `go build` rake go:build_tag # Print build tag rake go:fmt # Run go fmt rake go:test # Run go test rake go:testrace # Run go test -race rake go tasks 126 -
• With Go, in addition to the official linter, there
are various 3rd party tools for each linter • In Ruby, it's similar to the way gem is created for each RuboCop’s Cop 127 Go’s linter situation -
• golangci-lint is linters runner for Go • https://github.com/golangci/golangci-lint •
This is 3rd party tool but widely used in the Go community 128 Use case 2. golangci-lint -
# Rakefile require "go_gem/rake_task" go_task = GoGem::RakeTask.new("gem_name") namespace :go do
desc "Run golangci-lint" task :lint do go_task.within_target_dir do sh "which golangci-lint" do |ok, _| raise "golangci-lint isn't installed. See. https://golangci-lint.run/welcome/install/" unless ok end build_tag = GoGem::Util.ruby_minor_version_build_tag sh GoGem::RakeTask.build_env_vars, "golangci-lint run --build-tags #{build_tag} --modules-download-mode=readonly" end end end e.g. Use golangci-lint in Go gem 129 https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.3/_gem#example-add-additional-tasks -
• “patch_for_go_gem” is needed to make Go gem • I
want to use bundler gem similar to --ext=c and --ext=rust 130 Add --ext=go to bundle gem command -
• I waited for a while and got no response
• So I had hsbt triage it on ruby-jp slack 132 Feature Proposal -
• We can write in Go in areas where Ruby
is not good at (and Go is very good at) • Go's features not found in other languages ◦ goroutine: lightweight thread ▪ https://go.dev/tour/concurrency/1 ◦ channel: Messaging between multiple goroutine ▪ https://go.dev/tour/concurrency/2 137 Pros 2 -
• Benchmark with tak function ◦ https://www.ruby-lang.org/en/news/2020/12/25/ruby-3-0-0-released/ • Multi thread
and multi process are not eligible ◦ Because the results depend on the number of CPU cores on the machine being benchmarked ◦ This is not fair to compare these to goroutine 138 Ractor vs Fiber vs goroutine -
139 goroutine is 30x faster than Ractor $ ruby tarai.rb
go version go1.24.1 darwin/arm64 ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [arm64-darwin24] Comparison: Go: goroutine: 1.6 i/s Go: sequential: 0.5 i/s - 3.65x slower Ruby: Ractor: 0.1 i/s - 31.58x slower Ruby: sequential: 0.0 i/s - 92.85x slower Ruby: Fiber: 0.0 i/s - 93.08x slower https://github.com/ruby-go-gem/go-gem-wrapper/tree/v0.7.3/_benchmark -
• go-gem-wrapper is a low-layer wrapper, so not much different
than making Native extension with C ◦ Similar to writing in Go where you would write in C ◦ Need to be aware of both CRuby and Go 142 Cons 1 -
• Go's variable-length arguments couldn't be passed directly to C
◦ CRuby provides both variable length argument and non-variable length argument versions of its functions. ◦ If we only have functions with variable-length arguments, it is a bit of a problem ▪ e.g. rb_raise 143 Cons 2 -
144 Workaround for rb_raise https://github.com/ruby-go-gem/go-gem-wrapper/blob/v0.7.3/ruby/ruby_internal_error.go /* #include "ruby.h" void __rb_raise(VALUE
exception, const char *str) { rb_raise(exception, "%s", str); } */ import "C" func RbRaise(exc VALUE, format string, a ...interface{}) { str := fmt.Sprintf(format, a...) char, clean := string2Char(str) defer clean() C.__rb_raise(C.VALUE(exc), char) } -
• Not intended for use with ruby-head, since binding is
required for each Ruby version 146 Cons 3 -
• SEGV when calling CRuby functions in goroutine • c.f.
https://github.com/ruby-go-gem/go-gem-wrapper/issues/286 147 Cons 4 -
This doesn’t work… (SEGV!!!!) 148 e.g. Call yield inside goroutine
//export rb_example_call_proc_inside_goroutine func rb_example_call_yield_inside_goroutine(self C.VALUE) { if ruby.RbBlockGivenP() == 0 { ruby.RbRaise(C.rb_eArgError, "Block not given") } go func() { ruby.RbYield(C.Qnil) }() } -
• glibc is required ◦ Not supported for use with
musl (e.g. alpine) ◦ c.f. https://github.com/ruby-go-gem/go-gem-wrapper/issues/253 ◦ Windows (maybe) 149 Cons 5 -
• Perform HTTP requests in parallel with goroutine • Unlike
thread and process, we don't have to specify concurrency ◦ e.g. max threads, max processes 151 https://github.com/sue445/funnel_http -
152 Usage require "funnel_http" client = FunnelHttp::Client.new requests = [
{ method: :get, url: "https://example.com/api/user/1", }, ] responses = client.perform(requests) # => [ # { status_code: 200, body: "Response of /api/user/1", # header: { "Content-Type" => ["text/plain;charset=utf-8"]} } # ] -
NOTE: open-uri and net/http doesn’t work in Ractor… 153 benchmark
(Fiber vs goroutine) $ bundle exec ruby benchmark.rb go version go1.24.1 darwin/arm64 ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24] Comparison: FunnelHttp::Client#perform: 22.7 i/s sequential: 7.0 i/s - 3.27x slower Parallel with Fiber: 3.7 i/s - 6.14x slower https://github.com/sue445/funnel_http/tree/v0.3.2/benchmark -
• Using https://github.com/ruby-go-gem/go-gem-wrapper makes native extension with Go very easy
to make • We can write Ruby gem in Go in areas where Ruby is not good at (Go is very good at) • goroutine is very good! 154 Conclusion