Practical TypeProf: Lessons from Analyzing Optcarrot

@ RubyKaigi 2026

https://rubykaigi.org/2026/presentations/mametter.html

Transcript

1. Practical TypeProf: Lessons from Analyzing Optcarrot Yusuke Endoh (@mame) STORES, Inc. RubyKaigi 2026
2. Yusuke Endoh / @mametter • A Ruby Committer working at STORES w/ @ko1 • STORES: Nursery and Scholarship Sponsor, and… https://ruby-quiz-2026.storesinc.tech/
3. PR: RubyKaigi 2026 STORES Quiz
4. PR: Ruby One-Liner Game https://ruby-quiz-2026.storesinc.tech/ Show the badge to STORES staff to get your prize!
5. PR: Buy my books at @. bookstore! https://ruby-quiz-2026.storesinc.tech/
6. Today's players TypeProf A Ruby type analyzer • Infers types with minimal annotations • Provides editor support OptCarrot An 8-bit machine simulator • Written in Ruby • A long-loved Ruby performance benchmark 5.ti| 1 + "str" TypeError Do you mean: 5.times
7. Today's talk • Applied TypeProf to OptCarrot • I share what I learned
8. Today's talk • What is TypeProf? • Practical TypeProf through Optcarrot • TypeProf in the AI Era 8
9. What is TypeProf? Ruby Editor support • Error report, go to definition, completion, etc. • With minimal type annotations! 5.ti| 1 + "str" TypeError Do you mean: 5.times
10. Features • Type inference, error/warning report • Go to definition • Completion • Go to references • Go to type references • Automatic rename (method, constant) • Inline RBS (→ rbs-inline)
11. New Features since RubyKaigi 2025 • Better narrowing • Better support for meta-programming • Native support for Struct.new / Data.define • # typeprof:ignore (by x-smasato) • argument forwarding (by pvcresin) • RBS Record Type (by sinsoku)
12. Today's talk • What is TypeProf? • Practical TypeProf through Optcarrot • TypeProf in the AI Era 12
13. What is OptCarrot? • An 8-bit machine simulator • ~6000 lines of Ruby • Why this target? • Medium size • I am the author → deep understanding • Rails app is the real target, but one step at a time
14. Just run TypeProf for OptCarrot as-is • Count all errors in batch mode • 669 errors! • All are "undefined method" errors • All should be phantom errors • We want zero errors on OptCarrot • The natural goal if you use TypeProf • What did it take to bring 669 errors down to zero? typeprof --show-stats --show-errors lib/optcarrot/**/*.rb Error type Count nil#[], nil#+, etc. 223 {false, true}#[], etc. 368 {ClassName}#... 183
15. Two strategies against false positives • Strategy 1: Annotate the "key points" • Add RBS to methods where data flow converges • Superficial but easy; good as prevention • Strategy 2: Find and fix the root cause • Find and fix what injects unexpected types • Harder but fundamental; satisfying when it works :-)
16. Strategy 1: Annotate the "key points" • TypeProf prefers declared RBS over inferred types • CPU#fetch is a bottleneck of data flow • Same goes to CPU#store and #peek16 • 669 → 177 errors（-74%） #: (Integer) -> Integer def fetch(addr) @fetch[addr][addr] end OptCarrot CPU GPU RAM ROM CPU#fetch
17. How to find the "key points"? • Where data flow converges • As OptCarrot's author, I knew CPU#fetch mattered • Big-picture knowledge of the codebase is required • New option: AI • Claude Code identified CPU#fetch as the key • With good tooling and harness, this could be promising
18. Strategy 2: Find and fix the root cause • The culprit: the memory allocation code • @ram[0] was not initialized • @ram[n] always returns an Integer • n is 1..3 (not 0) in OptCarrot • However, TypeProf thinks that @ram[n] returns Integer | nil • Fix: initialize all @ramelements with 0 • 669 → 175 errors @ram = [nil] * 4 @ram[1] = 0 @ram[2] = 1 @ram[3] = 2 @ram = [0] * 4 rewrite
19. Comparing the two strategies (1) RBS at key points (2) Fix the cause Changes 3 lines of RBS 1 line of Ruby Effect 669→177 669→175 AI Could find key points Too hard for now Note Also useful preventively A type-aware developer wouldn't write such code?
20. Mopping up the rest (1) • attr_reader in a loop • TypeProf cannot tell what readers are defined → Wrote RBS [:loglevel, :romfile, …].each do attr_reader it end class Optcarrot::Config def loglevel: -> Integer def romfile: -> String? ... end
21. Mopping up the rest (2) • Unsuppressable optional types → expr || raise • Unavoidable false positives → # typeprof:ignore "foo".scan(/(\w)/) {|x,| x.upcase } bytes.unpack1("v") & 0x1 # typeprof:ignore "foo".scan(/(\w)/) {|x,| (x||raise).upcase }
22. What it took to reach zero errors Type Lines Modify Ruby code 55 [nil] * N → [0] * N 3 Add "|| raise " 36 Add "# typeprof:ignore " 4 Others 12 Add RBS (e.g., attr_reader in a loop, etc.) 67 total 122
23. FYI: How to start TypeProf on your project • See my RubyKaigi 2025 talk :-) • "Writing Ruby Scripts with TypeProf" • https://rubykaigi.org/2025/presentations/mametter.html
24. Steep and Sorbet on OptCarrot • I let Claude Code type OptCarrot with Steep/Sorbet • Claude Code took ~1 hour to reach zero errors (!) • Plus, I spent a few hours to clean up sloppiest parts
25. Comparing TypeProf, Steep, and Sorbet TypeProf Steep Sorbet Error 0 0 0 Code change 122 1429 1248 method sig. 67 1058 550 logic change 55 273 408 Type check time 1.30s 1.96s 0.24s Runtime overhead 0% 0% 72% (checks on) 17% (checks off) ※ See the next slide for fairness!
26. A note on fairness • The efforts are never equivalent • TypeProf work: hand-tuned, improving TypeProf in parallel • Steep/Sorbet work: mostly Claude Code, less polish • The checks are never equivalent • Basically, Steep/Sorbet performs stricter verification • TypeProf work tolerates "untyped" inferred results
27. Today's talk • What is TypeProf? • Practical TypeProf through Optcarrot • TypeProf in the AI Era 27
28. AI coding agents are here • Is everyone using Claude Code, Codex, etc.? • Writing Ruby in an editor → way less often • Even TypeProf development: Claude Code writes, I review the git diff • The main propose of TypeProf was an editor support • The question: What role should TypeProf play in the AI era?
29. Ruby works well with AI even without types Ruby, Python, JS are fastest, cheapest, and most stable from my article "Which Programming Language Is Best for Claude Code?" https://dev.to/mame/which-programming-language-is-best-for-claude-code-508a Ruby with Steep is not so good…
30. TypeProf's position in the AI era • TypeProf barely changes your Ruby code • So, it shouldn't slow AI down • TypeProf helps AI agents with inferred types? • As an MCP server, maybe? But… • For now, I continue to improve TypeProf for human • Let the answer emerge from practice
31. Today's talk • What is TypeProf? • Practical TypeProf through Optcarrot • TypeProf in the AI Era • Conclusion 31
32. Conclusion • TypeProf achieved 0 errors on OptCarrot • with much fewer changes than other type checkers • Main changes • Write RBS only for "key points" • Add "|| raise " • Use "# typeprof:ignore " • Ruby without explicit types fits the AI era • TypeProf's "inference w/o annotations" may matter more than ever
33. Future work • Today's experiment is just a start • Editor UX: not yet evaluated • Checking rigor: likely insufficient • Apply to larger / different codebases • Also, TypeProf improvement is also required • Speed, precision, more DSL support • AI integration and experiment