State of Namespace Satoshi Tagomori (@tagomoris) 2025-04-16 RubyKaigi 2025 

@tagomoris Satoshi Tagomori Maintainer/Founder: OSS: Fluentd, MessagePack, Norikra, Woothee, …. Event: ISUCON Service: Pathtraq 

https://www. fl ickr.com/photos/takkanm/3978417669 Asakusa.rb 

Product Manager, SAKURA Cloud (2024-08-01 ~) 

SAKURA internet ࣾձΛࢧ͑Δ
ύϒϦοΫΫϥ΢υɾେن໛ܭࢉࢿݯΠϯϑϥΛ
Ұॹʹ࡞Γ·ͤΜ͔ʁ
ソフトウェア開発、 インフラ基盤から フロントエンドまで 採 用 強化中! さくらインターネットではエン ジ ニア採 用 を強化しています さくらインターネットは新たなアイ デ アの創出に強い熱意と情熱を持って挑戦するお客様を は じ め、私たちとつな が りのあるす べ ての 人 たちのために、未来のある べ き姿を想い描きな が ら ―「やりたいこと」を「 で きる」に変える ― あらゆるア プ ローチを “インターネット” を通 じ て提供します。 詳しくはWebサイトにて、カジュアル 面 談もやってます 👉 www.sakura.ad.jp/lp/recruit-engineer/ 

https://sakura-tokyo.connpass.com/event/348223/ 

✦ Namespace ✦ Timeline ✦ Class de fi nition ✦ Stories between Kaigis ✦ TODO Topics 

✦ Separating apps/libs into isolated spaces ✦ Load apps/libs in a space ✦ Hide changes from apps/libs in the space to others ✦ Run methods de fi ned in the space with the de fi nition Namespace 

All classes/modules are shared in the entire Ruby process Before Namespace: Global Only Ruby Process Application Code App::Func User Library Code DB::Client (v2) Library Code ActiveSupport (v7) 

Collision: 2 versions of a library cause errors Before Namespace: Global Only Ruby Process Library Code DB::Client (v3) 😵 Application Code App::Func User Library Code DB::Client (v2) Library Code ActiveSupport (v7) 

Load apps/libs in a space Namespace Ruby Process Namespace Application Code App::Func User Library Code DB::Client (v2) Library Code ActiveSupport (v7) 

Hide changes by apps/libs in the namespace from others Namespace Ruby Process Namespace Application Code App::Func User Library Code DB::Client (v2) Library Code ActiveSupport (v7) Library Code DB::Client (v3) 😀 

Run methods de fi ned in the space with the de fi nition Namespace Ruby Process Namespace Application Code App::Func User Library Code DB::Client (v2) Library Code ActiveSupport (v7) Namespace Library Code DB::Client (v3) Namespace Application Code App::Func2 User Library Code ActiveSupport (v6) Application Code call call call 

https://rubykaigi.org/2024/presentations/tagomoris.html#day1 Check the past Kaigi deck/video for the details! 

Timeline 

What did I do after RubyKaigi 2024? 

DEBUG 

RubyKaigi 2024 and Then (1) ✦ RubyKaigi 2024 ✦ Demo code ran or SEGV ✦ ‘make test-all’ stopped in the middle (SEGV) ✦ Aug, 2024 ✦ Demo code ran in success! ✦ ‘make test-all’ ran to the end w/ failures/errors 

Namespace is still WIP, Mostly about just 1 SEGV. Stay tuned til Ruby 3.4! (or …?) Aug, 2024 My Idea at that time 1 

✦ Oct, 2024 ✦ ‘make test-all’ passed without failures/errors ✦ ‘make check’ with failures/errors ✦ “I have to give up on merging Namespace for Ruby 3.4” ✦ Jan, 2025 ✦ ‘make check’ passed without failures/errors ✦ ‘make exam’ with failures/errors RubyKaigi 2024 and Then (2) 

–Matz “Satoshi is too busy recently.” 

✦ Apr, 2025 ✦ ‘make exam’ with failures/errors RubyKaigi 2025 

✦ Apr, 2025 ✦ ‘make exam’ with failures/errors ✦ Demo code dies with BUG 😱 RubyKaigi 2025 

Class de fi nition and Namespace 

✦ struct RClass ✦ struct ✦ VALUE(Class object) ✦ struct rb_classext_struct (rb_classext_t) ✦ struct ✦ many other class de fi nitions ✦ struct RClass_and_rb_classext_t ✦ RClass + classext Ruby’s Class de fi nition RClass + rb_classext_t RClass rb_classext_t RClass_and_rb_classext_t 

(Ruby 3.4) Information in RClass + rb_classext_t struct RClass { struct RBasic basic; VALUE super; struct rb_id_table *m_tbl; }; struct rb_classext_struct { VALUE *iv_ptr; struct rb_id_table *const_tbl; struct rb_id_table *callable_m_tbl; struct rb_id_table *cc_tbl; struct rb_id_table *cvc_tbl; size_t superclass_depth; VALUE *superclasses; struct rb_subclass_entry *subclasses; struct rb_subclass_entry *subclass_entry; struct rb_subclass_entry *module_subclass_entry; const VALUE origin_; const VALUE refined_class; union { class; singleton_class; } as; const VALUE includer; attr_index_t max_iv_count; unsigned char variation_count; bool permanent_classpath : 1; bool cloned : 1; VALUE classpath; }; 

(Namespace) Information in RClass + rb_classext_t struct RClass { struct RBasic basic; st_table *ns_classext_tbl; bool prime_classext_readable : 1; bool prime_classext_writable : 1; }; struct rb_classext_struct { const rb_namespace_t *ns; VALUE super; VALUE *iv_ptr; struct rb_id_table *m_tbl; struct rb_id_table *const_tbl; struct rb_id_table *callable_m_tbl; struct rb_id_table *cc_tbl; struct rb_id_table *cvc_tbl; size_t superclass_depth; VALUE *superclasses; struct rb_subclass_anchor *subclasses; rb_ns_subclasses_t *ns_super_subclasses; rb_ns_subclasses_t *ns_module_subclasses; const VALUE origin_; const VALUE refined_class; union { class; singleton_class; } as; const VALUE includer; attr_index_t max_iv_count; unsigned char variation_count; bool permanent_classpath : 1; bool cloned : 1; bool shared_const_tbl : 1; bool iclass_is_origin : 1; bool iclass_origin_shared_mtbl : 1; bool superclasses_owner : 1; bool superclasses_with_self : 1; VALUE classpath; }; 

(Namespace) Information in RClass + rb_classext_t struct RClass { struct RBasic basic; st_table *ns_classext_tbl; bool prime_classext_readable : 1; bool prime_classext_writable : 1; }; struct rb_classext_struct { const rb_namespace_t *ns; VALUE super; VALUE *iv_ptr; struct rb_id_table *m_tbl; struct rb_id_table *const_tbl; struct rb_id_table *callable_m_tbl; struct rb_id_table *cc_tbl; struct rb_id_table *cvc_tbl; size_t superclass_depth; VALUE *superclasses; struct rb_subclass_anchor *subclasses; rb_ns_subclasses_t *ns_super_subclasses; rb_ns_subclasses_t *ns_module_subclasses; const VALUE origin_; const VALUE refined_class; union { class; singleton_class; } as; const VALUE includer; attr_index_t max_iv_count; unsigned char variation_count; bool permanent_classpath : 1; bool cloned : 1; bool shared_const_tbl : 1; bool iclass_is_origin : 1; bool iclass_origin_shared_mtbl : 1; bool superclasses_owner : 1; bool superclasses_with_self : 1; VALUE classpath; }; NEW NEW Moved from RClass Changed data structure Moved from Object flag FL_USER* 

✦ Di ff erent Namespace, di ff erent class de fi nition ✦ classext per Namespace by shallow copying the values (at fi rst) Namespaces and classexts RClass ns_classext_tbl rb_classext_t for root/builtin namespace (prime) rb_classext_t for main namespace (non-prime) rb_classext_t for namespace X (non-prime) rb_classext for namespac Y (non-prime 

✦ Di ff erent Namespace, di ff erent class de fi nition ✦ classext per Namespace by shallow copying the values (at fi rst) Namespaces and classexts RClass ns_classext_tbl rb_classext_t for root/builtin namespace (prime) rb_classext_t for main namespace (non-prime) rb_classext_t for namespace X (non-prime) rb_classext for namespac Y (non-prime My Idea at that time 2 

Stories between Kaigis (random order) 

autoload autoload :Set, ‘set’ Set require ‘set’ Set 

autoload autoload :Set, ‘set’ Set require ‘set’ Set const_tbl iv_ptr (__autoload__) rb_funcall(‘require’) Nothing to be done especially for Namespace! My Idea at that time 3 

autoload with Namespaces autoload :Set, ‘set’ Set require ‘set’ Set in Namespace A in Namespace A in Namespace A in Namespace A 

autoloading in caller ns (1) autoload :Set, ‘set’ in root ns Set in main ns require ‘set’ Set in main ns in main ns autoload entry consumed 1 

autoloading in caller ns (2) autoload :Set, ‘set’ NameError Set in root ns Set in main ns in root ns in root ns require ‘set’ Set in main ns in main ns 1 2 

autoloading in callee ns (1) autoload :Set, ‘set’ in root ns Set in main ns autoload entry consumed require ‘set’ Set in root ns in root ns 

autoloading in callee ns (2) autoload :Set, ‘set’ in root ns Set in main ns NameError in main ns autoload entry consumed require ‘set’ Set in root ns in root ns 

autoload in the right way autoload :Set, ‘set’ Set require ‘set’ Set Set autoload :Set, ‘set’ Set require ‘set’ in main ns trigger deep copying autoload_table in main ns in main ns in main ns in root ns in root ns in root ns in root ns 

✦ Top level constant “Ruby” introduced in Ruby 3.5 (trunk) ✦ Ruby::VERSION == RUBY_VERSION ✦ Tests crashed with SEGV on referring Ruby::VERSION ✦ After deleting and re-setting Ruby::VERSION ✦ Why??? Ruby::VERSION 

✦ Copying classext with shallow copying values including const_tbl ✦ keys and values of const_tbl are not copied Shallow copying const_tbl RClass ns_classext_tbl rb_classext_t (prime) const_tbl [A -> va] rb_classext_t (non-prime) const_tbl [A -> va] table shallow copy My Idea at that time 4 

✦ Deleting the constant A causes xfree(va) ✦ And va also exists in the const_tbl in the prime classext → SEGV! ✦ ASAN showed the root cause Deleting/Re-setting Constant on non-prime classext RClass ns_classext_tbl rb_classext_t (prime) const_tbl [A -> va] rb_classext_t (non-prime) const_tbl [] https://rubykaigi.org/2024/presentations/KJTsanaktsidis.html 

✦ Copying classext with deep copying const_tbl ✦ Values (rb_const_entry_t) are copied deeply to eliminate side-e ff ects ✦ Ruby::VERSION has test cases to delete/add its values, and it identi fi ed the bug deep copying const_tbl RClass ns_classext_tbl rb_classext_t (prime) const_tbl [A -> va] rb_classext_t (non-prime) const_tbl [A -> va’] table deep copy 

✦ ‘make exam’ randomly(?) crashed with “Killed: 9” (SIGKILL) ✦ on macOS ✦ without crash reports, without backtrace ✦ On Linux, ‘make exam’ also crashed with SEGV ✦ with broken crash reports, with broken backtrace “Killed: 9” 

✦ Identi fi ed the test case fi le: test/erb/test_erb.rb ✦ Wrote many debug prints (100 or more) ✦ required ‘cgi/escape’ — OK ✦ required ‘erb/escape’ — Crashed! ✦ Both are extensions (.so fi les) “Killed: 9” The Silver Bullet - printf 🤪 

✦ Extensions (.so) should be loaded in namespaces ✦ But dlopen() opens a fi le just once ✦ Second call on a fi le is just ignored 🥺 Loading Extensions in Namespace (1) util.so Namespace A Namespace B dlopen(“util.so") util_abc dlopen(“util.so") 

✦ 💡🤩 Copy the .so fi le with its basename and Namespace ✦ Di ff erent namespaces load di ff erent .so fi les Loading Extensions in Namespace (2) util.so Namespace A Namespace B dlopen(“a_util.so”) util_abc dlopen(“b_util.so”) util_abc My Idea at that time 5 

✦ Copy the .so fi le with its basename and Namespace ✦ “cgi/escape” in a → “a_escape.so” ✦ “erb/escape” in a → “a_escape.so” 🥶 ✦ What happens? when: 1. Overwriting the dlopen-ed .so fi le with others 2. Then calling dlopen() the .so fi le ✦ macOS: “Killed: 9” ✦ Linux: Nothing happened — then SEGV randomly Loading Extensions in Namespace (3) 

✦ Copy the .so fi le with its full feature name and Namespace ✦ feature name (fname): “cgi/escape”, “erb/escape” Loading Extensions in Namespace: Use Full Path cgi/escape Namespace A dlopen(“a_cgi+escape.so”) cgi_escape erb/escape erb_escape dlopen(“a_erb+escape.so”) 

✦ Stacks (push/pop) for managing namespaces ✦ One day: Too many pops on the stack — but why? ✦ rb_ensure() guards push/pop operations ✦ Bug #20655 ✦ Continuation (callcc, cont.call) triggers rb_ensure callback wrongly Bugs of MRI (1) cont.call triggers ensure in C VALUE rb_namespace_exec( const rb_namespace_t *ns, namespace_exec_func *func, VALUE arg) { rb_thread_t *th = GET_THREAD(); namespace_push(th, ns); return rb_ensure(func, arg, namespace_pop, (VALUE)th); } https://bugs.ruby-lang.org/issues/20655 

✦ VM_ASSERT(cme->owner == T_CLASS) failed in clearing method caches ✦ cme: callable method entry ✦ method: require, de fi ned_class: Kernel, owner: Kernel ✦ Bug #20767, caused by “Implicit Re fi nements” for namespace implementation ✦ Reproduced by 1. re fi ne Kernel#require (then using it) 2. alias #require to #original_require 3. rede fi ning Kernel#require Bugs of MRI (2) Invalid cme owner with re fi nement 

✦ ‘make check’ failed with SystemStackError ✦ Debug prints💡 — prism called itself again and again ✦ Using “—parser=parse.y” solved this problem ✦ But why? ✦ I found a comment by ko1 in bootstraptest/test_ractor.rb ✦ “prism parser with -O0 build consumes a lot of machine stack” ✦ Compiler option “-O3” solved this problem… but why? Bugs? of MRI (3) Don’t use compiler option “-O0” 

✦ Controlling method caches/entries ✦ cc_tbl, callable_m_tbl, gccct (global cc cache table) ✦ Rearchitecting subclasses data structure ✦ Identifying Namespaces to load .so/.rb fi les ✦ “Implicit Re fi nements” ✦ https://speakerdeck.com/tagomoris/namespace-now-and-then Other Major Things 

✦ Rearchitect current namespace management ✦ From stacks and threads To control frames (or cref) ✦ Namespace GC ✦ Collecting classext for GCed namespaces ✦ Deleting temp .so fi les TODO 

TODO (cont.) Merging Namespace into Ruby 3.5 … or 4.0? 

TODO (cont.) My Idea at THIS time Merging Namespace into Ruby 3.5 … or 4.0? See you next time!