Slide 1

Slide 1 text

error_highlight: User-friendly Error Diagnostics Yusuke Endoh (@mametter) RubyKaigi 2022

Slide 2

Slide 2 text

2

Slide 3

Slide 3 text

Yusuke Endoh / @mametter • A Ruby committer working at Cookpad w/ @ko1 • My major Ruby contributions: • Keyword arguments • coverage.so • TypeProf • error_highlight  Today's topic 3 https://ruby-puzzles-2022.cookpad.tech/

Slide 4

Slide 4 text

error_highlight Ruby 3.1 reports the fine-grained error location 4 $ ruby test.rb test.rb:1:in `': undefined method `time' for 42:Integer (NoMethodError) 42.time { print "Hello" } ^^^^^ Did you mean? times error_highlight

Slide 5

Slide 5 text

json is nil ? Or json[:article] is nil ? undefined method `[]' for nil:NilClass (NoMethodError) How useful? To tell which of the calls failed in a line Typical case: 5 json[:article][:author] ^^^^^^^^^ undefined method `[]' for nil:NilClass (NoMethodError) json[:article][:author] ^^^^^^^^^^ json[:article][:author] json is nil json[:article] is nil

Slide 6

Slide 6 text

More useful than theory Developer experience is more improved than expected • Suggests "how to fix" • Helps you reach the wrong code in the editor 6 $ ruby test.rb test.rb:123:in `': undefined method `gsuub' for ... str.gsuub(/....../, "foobar") ^^^^^^ Did you mean? gsub! gsub

Slide 7

Slide 7 text

What will change in Ruby 3.2? • ArgumentError / TypeError • Ruby 3.1 supported only NameError / NoMethodError 7 $ ruby test.rb test.rb:1:in `+': nil can't be coerced into Integer (TypeError) 1 + nil ^^^ Today's topic: the road to achieve them • Rails' error page

Slide 8

Slide 8 text

Agenda • ➔Background • How error_highlight implemented in Ruby 3.1 • Problems and solutions • Conclusion 8

Slide 9

Slide 9 text

Implementation overview • 1. Record the code range in bytecode • 2. Determine where to underline • 3. Prints an error message 9 3.1

Slide 10

Slide 10 text

1. Record code ranges in byte code 10 compile foo.bar(1) 0..2 0..9 8..8 send "foo" put 1 send "bar" byte code source code code range 0..2 8..8 0..9

Slide 11

Slide 11 text

2. Determine where to underline 11 send "foo" put 1 send "bar" foo.bar(42) 0..9 foo.bar(42) 0..2 error error 0..2 8..8 0..9 This approach is too naïve! Why?

Slide 12

Slide 12 text

ary.map do … end.selct do … end Naïve code range is too wide • error_highlight uses AST analysis & Regexps (!) • AST does not retain a location of punctuation (e.g., period) • Kevin Newton's parser work may improve the situation (related to the next talk in this session!) 12 ary.map do … end.selct do … end expected ary.map do … end.selct do … end naïve Typo

Slide 13

Slide 13 text

Support various method calls 13 obj.foo += 1 obj.foo is not defined obj.foo += 1 obj.foo returned nil obj.foo += 1 obj.foo= is not defined prinnt(str) nil + 1 nil[1] obj&.foo obj. foo obj .foo Suggestion for improvement is welcome

Slide 14

Slide 14 text

3. Prints code with the underline • Overrides NameError#message • Ruby's error printer writes #message to stderr • Ruby 3.1 released this version, but 14 class NameError def message super + "¥n" + code + "¥n" + underline end end This approach had many problems!

Slide 15

Slide 15 text

Agenda • Background • ➔Problems and solutions • Less expandable • Wrong underline location • Poor ecosystem support • Conclusion 15

Slide 16

Slide 16 text

Problem 1: Less expandable • Only NameError / NoMethodError were supported • Why? • Because an error message has some usages • A. To show an error trace • B. To test error-handling code • C. To log an error • error_highlight breaks B and C 16

Slide 17

Slide 17 text

Problem 1-B: Test compatibility • Many tests in the wild checks an error message • Changing Exception#message is incompatible • Acceptable to change NameError#message since few test cases check its result 17 expect { … }.to raise_error("foobar")

Slide 18

Slide 18 text

Problem 1-C: Logging compatibility Many expect #message to return a one-line string •"error_highlight makes it difficult to parse a log file!" •"error_highlight makes so confusing!" 18 E, [2022-09-10T10:00:00.000000 #12345] ERROR -- : undefined method `time' for 42:Integer 42.time ^^^^^ Did you mean? times # p $!

Slide 19

Slide 19 text

Solution for Problem 1 Exception#detailed_message is introduced 19 $!.message undefined method `time' for 1:Integer $!.detailed_message undefined method `time' for 1:Integer (NoMethodError) 1.time ^^^^^ Did you mean? times

Slide 20

Slide 20 text

Request to framework developers • Frameworks may want to use #detailed_message • Instead of #message • Example: https://github.com/rack/rack/pull/1926 • Sentry and DataDog said they would support this • https://bugs.ruby-lang.org/issues/18438 • thanks to Stan Lo (Sentry) and Ivo Anjo (DataDog) 20

Slide 21

Slide 21 text

Problem 1 is solved ArgumentError / TypeError are now supported Without significant incompatibility 21 test.rb:1:in `+': nil can't be coerced into Integer (TypeError) 1 + nil ^^^

Slide 22

Slide 22 text

Problem 2: Wrong underline Ruby's error printer "escapes" the message 22 test.rb:1:in `': undefined method `gsuub' for " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥":String (NoMethodError) " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "") ^^^^^^ " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "") Wrong! Wrong too!

Slide 23

Slide 23 text

Solution to Problem 2 Stopped Ruby's error printer from escaping 23 This simple solution took about a month! Why?

Slide 24

Slide 24 text

Why did it escape the message? • Because of security consideration! • "TERMINAL EMULATOR SECURITY ISSUES" [1] • Some terminals had insecure escape sequences • Create a file • Input as if a user typed it 24 [1] https://marc.info/?l=bugtraq&m=104612710031920&w=2

Slide 25

Slide 25 text

Is this concern still valid? • [1] is very old (in 2003) and also says: • Surveyed the current situation • Rxvt / Eterm / XTerm disabled the dangerous features The terminals mentioned in [1] • Gnome Terminal, Windows Terminal and iTerm2 don't support the dangerous features 25 The responsibility should rest on the actual terminal emulator

Slide 26

Slide 26 text

Problem 2 is solved • We agreed for Ruby to stop escaping • Correct underline in Ruby 3.2 26 test.rb:1:in `': undefined method `gsuub' for " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥":String (NoMethodError) " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "") ^^^^^^ " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "")

Slide 27

Slide 27 text

Problem 3: Poor ecosystem support Rack error page 27 broken

Slide 28

Slide 28 text

Solution to Problem 3 Write a patch! https://github.com/rack/rack/pull/1925 28

              

Slide 29

Slide 29 text

Support Rails error page • Rails provides a more dedicated error page • error_highlight provides only Exception#message Parse #message …? 29

Slide 30

Slide 30 text

Export error_highlight API 30 ErrorHighlight.spot( $!, backtrace_location: $!.backtrace_locations[0] ) { :first_lineno=>2, :first_column=>3, :last_lineno=>2, :last_column=>8, :snippet=>" 1.time¥n", :script_lines=>[…] }

Slide 31

Slide 31 text

Problem 3 is solved Rails error page now shows error_highlight The patch is merged https://github.com/rails/rails/pull/45818 31

Slide 32

Slide 32 text

Agenda • Background • Problems and solutions • ➔Conclusion 32

Slide 33

Slide 33 text

Ruby 3.2's error_highlight • ArgumentError / TypeError Ruby 3.1 supported only NameError / NoMethodError 33 $ ruby test.rb test.rb:1:in `+': nil can't be coerced into Integer (TypeError) 1 + nil ^^^ • Rails' error page

Slide 34

Slide 34 text

Acknowledgments • @yui-knk made the early prototype of error_highlight • @ioquatix reported a lot of issues • Those who joined in the discussion on tickets, PRs, or the dev meeting 34

Slide 35

Slide 35 text

35

Slide 36

Slide 36 text

Memory overhead of bytecode • Approx. 3% (on scaffold Rails app) • Record AST node_id instead of raw linenos/columns • Simple compression 36

Slide 37

Slide 37 text

Future work / Known problems • Support CJK full-width characters • I don't want to do that! • Support more exception classes • User-defined exceptions • Show column number • test.rb:2:2 37

Slide 38

Slide 38 text

Does error_highlight use escape sequence? • It can, but I have no plan • Why? • Terminal messages are often copy/pasted as a text • Critical use of escape sequence will make people want to use screenshots • ➔ Searching by error message will be difficult 38

Slide 39

Slide 39 text

Lots of considerations and tasks • ANSI escape code (= terminal font style) • Gems • did_you_mean gem (thanks to Yuki Nishijima) • syntax_suggest gem (thanks to Richard Schneeman) 39 $!.detailed_message(highlight: true) undefined method `time' for 1:Integer (NoMethodError) 1.time ^^^^^ Did you mean? times