Good first issues of TypeProf Yusuke Endoh (@mame) RubyKaigi 2024

Yusuke Endoh / @mametter • A Ruby committer working at STORES w/ @ko1 • My recent Ruby change: New error message format 2 test.rb:1:in `func': unhandled exception Old: test.rb:1:in 'Foo#func': unhandled exception New: Backtick → Single quote (markdown friendly!) Class name (not only method name)

Yusuke Endoh / @mametter • A Ruby committer working at STORES w/ @ko1 • My recent STORES work (?) 3 1. Ruby "enbugging" quiz https://ruby-quiz-2024.storesinc.tech 2. Ruby "Quine" paper craft A spiral Ruby code Come to the STORES booth!

Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues 4

Today's topic: TypeProf Ruby Editor support • Error report, go to definition, completion, etc. • Without (many) type annotations! 5.ti| 1 + "str" TypeError Do you mean: 5.times

Demo Features • Type inference, error report • Go to definition • Completion • (Inline RBS, flow analysis) New Features • Go to references • Go to type references • Automatic rename (method, constant)

No content

Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues 8

How to play with TypeProf: Overview 1. Install Ruby 3.3 2. Install VSCode extension 3. git clone TypeProf 4. Open the directory with VSCode Slide deck: https://speakerdeck.com/mame/good-first-issues-of-typeprof

How to play with TypeProf (1/2) • Install Ruby 3.3 • Install VSCode extension (search "typeprof") 1. search "typeprof" from Extensions 2. click

How to play with TypeProf (2/2) • git clone https://github.com/ruby/typeprof.git • Open the directory, and "lib/typeprof/core/ast.rb" 1. open a file in lib/typeprof/core 2. You'll see an inferred type signature

TypeProf with your code 1. Configure typeprof path 2. Create typeprof configuration file 3. Open your project directory and your file NOTE • It would fail as many Ruby constructs are unsupported yet • The details are tentative

TypeProf with your code (1/4) • Configure typeprof path 1. search "typeprof" from Settings 2. click

TypeProf with your code (2/4) • Configure typeprof path Write the absolute path to typeprof/bin/typeprof

TypeProf with your code (3/4) • Add typeprof.conf.json to your project 1. Create typeprof.conf.json 2. Write this

TypeProf with your code (4/4) • Open your Ruby file 1. Open app/test.rb 2. You'll be able to use TypeProf

Tips: If TypeProf fails... Keyword argument is not supported yet not supported yet: keyword_hash_node Tips 1. Fix the code 2. Press Ctrl+P 3. Choose "TypeProf: Restart"

How to play: Summary • Install VSCode extension • Clone the edge of TypeProf • Add typeprof.conf.json • Open a file (and :pray:) Please play with TypeProf! • Not practical yet, but it works for fun • If you find anything wrong, write a patch!

Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues 19

How to develop TypeProf 1. Write a test scenario 2. Implement! 3. Make the scenario pass

Case study: interpolated symbol literal :"symbol#{ 42 }" #=> :symbol42

Test scenario DSL ## update: test.rb def foo :"symbol#{ 42 }" end ## assert: test.rb class Object def foo: -> Symbol end scenario/misc/dsym.rb Input: If you write test.rb like this Expected output: The inferred type signature should be like this

Why "scenario"? → To describe the edit ## update: test.rb def foo = "symbol#{ 42 }" ## assert: test.rb class Object def foo: -> String end ## update: test.rb def foo = :"symbol#{ 42 }" ## assert: test.rb class Object def foo: -> Symbol end If you write this The inferred type should be String If you edit the file like this (The colon is added) The inferred type should be updated to Symbol

Run the test scenario • Use "tool/scenario_runner.rb" $ tool/scenario_runner.rb scenario/misc/dsym.rb Loaded suite tool/scenario_runner Started E =================================================================== Error: test: scenario/misc/dsym.rb(ScenarioCompiler::ScenarioTest): RuntimeError: not supported yet: interpolated_symbol_node … TypeProf does not support the construct yet

Implement! diff --git a/lib/typeprof/core/ast.rb b/lib/typeprof/core/ast.rb index 28073bb..b2a3d78 100644 --- a/lib/typeprof/core/ast.rb +++ b/lib/typeprof/core/ast.rb @@ -170,6 +170,7 @@ module TypeProf::Core when :integer_node then IntegerNode.new(raw_node, lenv) when :float_node then FloatNode.new(raw_node, lenv) when :symbol_node then SymbolNode.new(raw_node, lenv) + when :interpolated_symbol_node then InterpolatedSymbolNode.new(raw_node, lenv) when :string_node then StringNode.new(raw_node, lenv, raw_node.content) when :source_file_node then StringNode.new(raw_node, lenv, "") when :interpolated_string_node then InterpolatedStringNode.new(raw_node, lenv) diff --git a/lib/typeprof/core/ast/value.rb b/lib/typeprof/core/ast/value.rb index 8e1fe99..e5c8031 100644 --- a/lib/typeprof/core/ast/value.rb +++ b/lib/typeprof/core/ast/value.rb @@ -69,6 +69,34 @@ module TypeProf::Core def install0(genv) = Source.new(Type::Symbol.new(genv, @lit)) end + class InterpolatedSymbolNode < Node + def initialize(raw_node, lenv) + super(raw_node, lenv) + @parts = [] + raw_node.parts.each do |raw_part| + case raw_part.type + when :string_node + @parts << AST.create_node(raw_part, lenv) + when :embedded_statements_node + @parts << AST.create_node(raw_part.statements, lenv) + else + raise "unknown symbol part: #{ raw_part.type }" + end + end + end + + attr_reader :parts + + def subnodes = { parts: } + + def install0(genv) + @parts.each do |subnode| + subnode.install(genv) + end + Source.new(genv.symbol_type) + end + end + class StringNode < LiteralNode def initialize(raw_node, lenv, content) super(raw_node, lenv, content) https://github.com/ruby/typeprof/pull/170

Make the test scenario pass • Use "tool/scenario_runner.rb" $ tool/scenario_runner.rb scenario/misc/dsym.rb Loaded suite tool/scenario_runner Started Finished in 0.002024702 seconds. ------------------------------------------------------------------- 1 tests, 1 assertions, 0 failures, 0 errors, 0 pendings, 0 omission s, 0 notifications 100% passed ------------------------------------------------------------------- 493.90 tests/s, 493.90 assertions/s

How to development: Summary 1. Write a test scenario 2. Implement it! 3. Make the scenario pass • Next, explain how to implement Note: PR is welcome to add only test scenarios! • Please add it to scenario/known-issues/

How to implement • Read the code!

Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues 29

TypeProf internals (1): Dataflow analysis def foo(n) r = n.to_s r end x = 123 y = x z = foo(y) x y foo .to_s z n r int int int int 123 str str (toplevel) foo( ) int "Source" outputs fixed type(s) "Vertex" outputs input type(s) "Box" (Ruby call): passes args "Box" (RBS call): typechecks args str

TypeProf internals (2): Create dataflow graph y = x + 1 Graph AST ChangeSet code + x 1 = x based on Prism What to add to the graph + x y 1 CallBox: Edge: Edge: Edge: x y + + + + 1

TypeProf internals (3): Incremental update + x y 1 a.rb Add Box... Add Edge... b.rb Add Box... Add Edge... c.rb Add Box... Add Edge... Add Box... Add Edge... c.rb (2) edit revert this ChangeSet add a new ChangeSet Graph AST ChangeSet code

TypeProf source file structure • lib/typeprof/lsp: LSP server • lib/typeprof/core: Type Analyzer • service.rb: Endpoint APIs • ast/: Definitions of AST nodes • graph/: Dataflow graph (Source, Vertex, Box...) • type.rb: Definitions of types • env/: Environments (Class hierarchy, Method definitions, etc.)

Tips: How to read TypeProf • service.rb is a good start • Endpoint APIs to the analysis algorithm • Tweak this if you want to add/change LSP features • ast/ is (relatively) easy • Tweak this if you want to support Ruby/RBS constructs • env/ is hard • graph/ is lunatic • Challengers are welcome

Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues 35

Good first issues: Level 1 (easy) • Play with TypeProf • If you find anything weird, add a new test scenario into scenario/known-issues/ • Support more Ruby/RBS constructs • Easy one can be done by copying other similar nodes • (But difficult ones would be Level 3) • Make it type inference command-line tool • Essential feature is already in tool/scenario_runner.rb

Good first issues: Level 2 (medium) • Improve the error messages • Many errors and warnings have not implemented yet • There are many comments like "TODO: report ..." • Implement "go to definition" of variables • There are all the parts we need (probably) foo(42) foo(42) failed to resolve overloads expected: String, found: Integer

Good first issues: Level 3 (hard) • Make the completion smarter • Currently, "TriggerCharacter" is supported for method name • Need to support other triggers, variable names, … • Improve the flow analysis • Currently, very limited set is supported • "if var", "if var.is_a?(Foo), "if @var", "if @var.is_a?(Foo)" • Need to support "&&", "||", etc. • Make TypeProf a plugin for ruby-lsp • Support rbs-inline • Find tasks!

Today's talk • What is TypeProf? • How to play with TypeProf • How to develop TypeProf • TypeProf internals overview • Good first issues • Conclusion 39

Future work • This year • Support full Ruby/RBS syntax • Bundle the latest TypeProf with Ruby • Dog-fooding • Next year.. • Implement many features • Improve accuracy and speed of analysis • Start towards Rails apps

Conclusion • Introduced how to play with TypeProf • Introduced how to contribute to TypeProf • Please play with TypeProf first • Write a test scenario • If possible, try reading the code Slide deck: https://speakerdeck.com/mame/good-first-issues-of-typeprof