Slide 1
Slide 1 text
RNode with code locations
Jun 1, 2018 in RubyKaigi 2018
@yui-knk
Yuichiro Kaneko
Slide 2
Slide 2 text
Self-introduction
• Yuichiro Kaneko
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
Self-introduction
• Yuichiro Kaneko
• Asakusa.rb
• A CRuby Committer (2015/12~)
• GitHub (yui-knk)
• Twitter (spikeolaf)
Slide 5
Slide 5 text
I will join Treasure Data next
week!!!
Slide 6
Slide 6 text
Today's topic
• RNode
• Node of Abstract Syntax Tree
• Code location
• Location information of RNode
Slide 7
Slide 7 text
Run the code
• Run the code on …
• Ruby 2.4
• Ruby 2.5
$ ruby --dump=p -e '"str".upcase'
Slide 8
Slide 8 text
Ruby 2.4
$ ruby --dump=p -e '"str".upcase'
# @ NODE_SCOPE (line: 1)
# +- nd_tbl: (empty)
# +- nd_args:
# | (null node)
# +- nd_body:
# @ NODE_PRELUDE (line: 1)
# +- nd_head:
# | (null node)
# +- nd_body:
# | @ NODE_CALL (line: 1)
# | +- nd_mid: :upcase
# | +- nd_recv:
# | | @ NODE_STR (line: 1)
# | | +- nd_lit: "str"
# | +- nd_args:
# | (null node)
# +- nd_compile_option:
# +- coverage_enabled: false
Slide 9
Slide 9 text
$ ruby --dump=p -e '"str".upcase'
# @ NODE_SCOPE (line: 1, code_range: (1,0)-(1,12))
# +- nd_tbl: (empty)
# +- nd_args:
# | (null node)
# +- nd_body:
# @ NODE_PRELUDE (line: 1, code_range: (1,0)-(1,12))
# +- nd_head:
# | (null node)
# +- nd_body:
# | @ NODE_CALL (line: 1, code_range: (1,0)-(1,12))
# | +- nd_mid: :upcase
# | +- nd_recv:
# | | @ NODE_STR (line: 1, code_range: (1,0)-(1,5))
# | | +- nd_lit: "str"
# | +- nd_args:
# | (null node)
# +- nd_compile_option:
# +- coverage_enabled: false
Ruby 2.5
Slide 10
Slide 10 text
@@ -1,8 +1,8 @@
# +- nd_body:
-# | @ NODE_CALL (line: 1)
+# | @ NODE_CALL (line: 1, code_range: (1,0)-
(1,12))
# | +- nd_mid: :upcase
# | +- nd_recv:
-# | | @ NODE_STR (line: 1)
+# | | @ NODE_STR (line: 1, code_range:
(1,0)-(1,5))
# | | +- nd_lit: "str"
# | +- nd_args:
# | (null node)
Slide 11
Slide 11 text
@@ -1,8 +1,8 @@
# +- nd_body:
-# | @ NODE_CALL (line: 1)
+# | @ NODE_CALL (line: 1, code_range: (1,0)-
(1,12))
# | +- nd_mid: :upcase
# | +- nd_recv:
-# | | @ NODE_STR (line: 1)
+# | | @ NODE_STR (line: 1, code_range:
(1,0)-(1,5))
# | | +- nd_lit: "str"
# | +- nd_args:
# | (null node)
Today’s Topic
Slide 12
Slide 12 text
Agenda
• What code locations are
• Why code locations are needed
• Ruby crash course
• How to implement code locations
• The future plan of code locations feature
• Conclusion
Slide 13
Slide 13 text
What code locations are
Slide 14
Slide 14 text
Location information in
programming
• Location information of script is used in various
situations.
Slide 15
Slide 15 text
Exception
Traceback (most recent call last):
1: from src/exception.rb:5:in `'
src/exception.rb:2:in `a': undefined method `foo' for "":String
(NoMethodError)
Slide 16
Slide 16 text
Warning
src/warning.rb:2: warning: instance variable @a not initialized
Slide 17
Slide 17 text
Location information in
programming
• Location information of script is used in various
situations.
• "An exception is raised from line number
XX" (Exception)
• "Instance variable of line number XX not
initialized" (Warning)
• "No test cases for line number XX" (Coverage)
Slide 18
Slide 18 text
Is line number enough to
represent location?
Slide 19
Slide 19 text
Location and position
• Location is presented by 2 numbers:
• Line number (lineno)
• Distance from beginning of line (column)
Slide 20
Slide 20 text
Location and position
• 4 numbers are needed to represent “begin” and “end”.
• "Code position" is a pair of lineno and column.
• "Code location" is a pair of begin position and end
position.
1 + 2
^ ^ ^
| | +- @3 (1.4-1.5)
| +--- @2 (1.2-1.3)
+----- @1 (1.0-1.1)
@3 (1.4-1.5)
Code position (begin)
Code location
@3 (1.4-1.5)
Lineno (1)
Column (4)
Code position (end)
Slide 21
Slide 21 text
Location in Ruby
• Ruby holds *only* line numbers until Ruby 2.4.
• Ruby holds line numbers and columns since Ruby 2.5.
• Today’s main topic is “Column”.
Slide 22
Slide 22 text
Minor details about column
• 0-based / 1-based
• 0-based
• Vary according to programming languages and editors.
• From the beginning of line / file
• Line
Slide 23
Slide 23 text
Minor details about column
• Byte length / Character length
• Byte length
• “ߏจʹৄࡉͳҐஔใΛͨͤΔܭը”
• https://bugs.ruby-lang.org/projects/ruby-trunk/wiki/
Node-position-memo
Slide 24
Slide 24 text
Why code locations are
needed
Slide 25
Slide 25 text
For coverage features
• For branch coverage and method coverage (Ruby 2.5~).
• "An introduction and future of Ruby coverage library”
• http://rubykaigi.org/2017/presentations/mametter.html
(30:50-)
Slide 26
Slide 26 text
What is branch coverage
• "Branch coverage tells you which branches are executed,
and which not." (doc/NEWS-2.5.0)
(a == 2) ? :t : :f
Slide 27
Slide 27 text
What is branch coverage
• You may forget to write test codes for `then` cases.
• `n/m`
• `n`: How many times the “then clause” is executed.
• `m`: How many times the “else clause” is executed.
0/1: (a == 2) ? :t : :f
Slide 28
Slide 28 text
Use-case (1)
• Code locations can be used for visualizing branch
coverage results.
0/1: (a == 2) ? :t : :f
Slide 29
Slide 29 text
Use-case (1)
• Code locations can be used for visualizing branch
coverage results.
0/1: (a == 2) ? :t : :f
YOU SHOULD WRITE TEST !!!
Slide 30
Slide 30 text
Use-case (2)
• One line can contain one or more branches.
• In these case, we can't recognize which clause is
executed by only line numbers.
(a == b) ? ((c == d) ? :A : :B) : :C
obj&.foo? ? "a" : "b"
Slide 31
Slide 31 text
Use-case (2)
• One line can contain one or more branches.
• In these case, we can't recognize which clause is
executed by only line numbers.
(a == b) ? ((c == d) ? :A : :B) : :C
obj&.foo? ? "a" : "b"
Slide 32
Slide 32 text
Ruby crash course
Slide 33
Slide 33 text
How Ruby script is processed
4UFQ *OQVU 0VUQVU %FCVH 4PSVDF
5PLFOJ[BUJPO 3VCZTDSJQU 5PLFOT EVNQZ QBSTFZ
1BSTJOH 5PLFOT "45 EVNQQ QBSTFZ
$PNQJMF "45 #ZUFDPEF EVNQJ DPNQJMFD
Parsing ___
\
Ruby script -> Tokens -> AST -> Byte code (insns / ISeq)
__/ __/
Tokenization Compile
Slide 34
Slide 34 text
Ruby script
1 + 2
Slide 35
Slide 35 text
Tokenization
Parsing ___
\
Ruby script -> Tokens -> AST -> Byte code (insns / ISeq)
__/ __/
Tokenization Compile
Slide 36
Slide 36 text
Tokenization
• Each token has
• a token type (tINTEGER)
• a semantic value (1)
1 + 2
^ ^ ^^
| | |+--- '\n' / "end-of-input"
| | +---- tINTEGER (2)
| +------ '+'
+-------- tINTEGER (1)
Slide 37
Slide 37 text
Tokenization
$ ruby --dump=y -e '1 + 2' | grep Shifting
Shifting token tINTEGER (1.0-1.1: )
Shifting token '+' (1.2-1.3: )
Shifting token tINTEGER (1.4-1.5: )
Shifting token '\n' (1.5-1.5: )
Shifting token "end-of-input" (1.5-1.5: )
On Ruby 2.5.1
1
2
Slide 38
Slide 38 text
Tokenization
$ ruby --dump=y -e '1 + 2' | grep Shifting
On Ruby 2.6.0preview1
Shifting token tINTEGER (1.0-1.1: 1)
Shifting token '+' (1.2-1.3: )
Shifting token tINTEGER (1.4-1.5: 2)
Shifting token '\n' (1.5-1.5: )
Shifting token "end-of-input" (1.5-1.5: )
Slide 39
Slide 39 text
Tokenization
$ ruby --dump=y -e '1 + 2' | grep Shifting
On Ruby 2.6.0preview1
Shifting token tINTEGER (1.0-1.1: 1)
Shifting token '+' (1.2-1.3: )
Shifting token tINTEGER (1.4-1.5: 2)
Shifting token '\n' (1.5-1.5: )
Shifting token "end-of-input" (1.5-1.5: )
Slide 40
Slide 40 text
r61997 / 46e2fad
Slide 41
Slide 41 text
Parsing
Parsing ___
\
Ruby script -> Tokens -> AST -> Byte code (insns / ISeq)
__/ __/
Tokenization Compile
Slide 42
Slide 42 text
Parsing
• Analyzes tokens conforming to the rules of Ruby syntax.
• Builds AST.
Slide 43
Slide 43 text
Parsing
numeric : simple_numeric
| tUMINUS_NUM simple_numeric %prec tLOWEST
;
simple_numeric : tINTEGER
| tFLOAT
| tRATIONAL
| tIMAGINARY
;
%%
program : {
}
top_compstmt
parse.y
Slide 44
Slide 44 text
Parsing
numeric : simple_numeric
| tUMINUS_NUM simple_numeric %prec tLOWEST
;
simple_numeric : tINTEGER
| tFLOAT
| tRATIONAL
| tIMAGINARY
;
%%
program : {
}
top_compstmt
W
W
W
Rules
parse.y
Slide 45
Slide 45 text
Parsing
numeric : simple_numeric
| tUMINUS_NUM simple_numeric %prec tLOWEST
;
simple_numeric : tINTEGER
| tFLOAT
| tRATIONAL
| tIMAGINARY
;
%%
program : {
}
top_compstmt
parse.y
Slide 46
Slide 46 text
Parsing
numeric : simple_numeric
| tUMINUS_NUM simple_numeric %prec tLOWEST
;
simple_numeric : tINTEGER
| tFLOAT
| tRATIONAL
| tIMAGINARY
;
%%
program : {
}
top_compstmt
1
2.1
3r
4i
parse.y
Slide 47
Slide 47 text
Parsing
numeric : simple_numeric
| tUMINUS_NUM simple_numeric %prec tLOWEST
;
simple_numeric : tINTEGER
| tFLOAT
| tRATIONAL
| tIMAGINARY
;
%%
program : {
}
top_compstmt
parse.y
Slide 48
Slide 48 text
Parsing
numeric : simple_numeric
| tUMINUS_NUM simple_numeric %prec tLOWEST
;
simple_numeric : tINTEGER
| tFLOAT
| tRATIONAL
| tIMAGINARY
;
%%
program : {
}
top_compstmt
1
parse.y
Slide 49
Slide 49 text
Parsing
numeric : simple_numeric
| tUMINUS_NUM simple_numeric %prec tLOWEST
;
simple_numeric : tINTEGER
| tFLOAT
| tRATIONAL
| tIMAGINARY
;
%%
program : {
}
top_compstmt
1
parse.y
Slide 50
Slide 50 text
Parsing
numeric : simple_numeric
| tUMINUS_NUM simple_numeric %prec tLOWEST
;
simple_numeric : tINTEGER
| tFLOAT
| tRATIONAL
| tIMAGINARY
;
%%
program : {
}
top_compstmt
Goal
parse.y
Slide 51
Slide 51 text
Parsing
$ ruby --dump=y -e '1 + 2'
# Shifting token tINTEGER (1)
tINTEGER
simple_numeric
numeric
literal
primary
arg
Slide 52
Slide 52 text
Parsing
# Shifting token '+'
arg '+'
# Shifting token tINTEGER (2)
arg '+' tINTEGER
arg '+' simple_numeric
arg '+' numeric
arg '+' literal
arg '+' primary
arg '+' arg
arg
expr
stmt
top_stmt
top_stmts
Slide 53
Slide 53 text
Parsing
# Shifting token '\n'
top_stmts '\n'
top_stmts term
top_stmts terms
top_stmts opt_terms
top_compstmt
program # Completed
Slide 54
Slide 54 text
Build AST
$ ruby --dump=p -e '1 + 2'
NODE_SCOPE
NODE_PRELUDE
NODE_OPCALL (:+)
NODE_LIT (1)
NODE_ARRAY
NODE_LIT (2)
NODE_SCOPE
NODE_PRELUDE
NODE_OPCALL (:+)
NODE_LIT (1) NODE_ARRAY
NODE_LIT (2)
Slide 55
Slide 55 text
Build AST
typedef struct RNode {
VALUE flags;
union {
struct RNode *node;
...
} u1;
union {
struct RNode *node;
...
} u2;
union {
struct RNode *node;
...
} u3;
rb_code_location_t nd_loc;
} NODE;
Slide 56
Slide 56 text
Build AST
typedef struct RNode {
VALUE flags;
union {
struct RNode *node;
...
} u1;
union {
struct RNode *node;
...
} u2;
union {
struct RNode *node;
...
} u3;
rb_code_location_t nd_loc;
} NODE;
Contain node_type
Slide 57
Slide 57 text
Build AST
typedef struct RNode {
VALUE flags;
union {
struct RNode *node;
...
} u1;
union {
struct RNode *node;
...
} u2;
union {
struct RNode *node;
...
} u3;
rb_code_location_t nd_loc;
} NODE;
Contain node_type
Contain various data
Slide 58
Slide 58 text
Build AST
typedef struct RNode {
VALUE flags;
union {
struct RNode *node;
...
} u1;
union {
struct RNode *node;
...
} u2;
union {
struct RNode *node;
...
} u3;
rb_code_location_t nd_loc;
} NODE;
Contain node_type
Contain various data
Contain
Location information
Slide 59
Slide 59 text
Build AST
• Builds AST in actions.
• $1 stands for the value of the 1st component (`arg`).
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
NODE *expr;
value_expr(recv);
value_expr(arg1);
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
nd_set_line(expr, op_loc->beg_pos.lineno);
return expr;
}
Slide 60
Slide 60 text
Build AST
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
NODE *expr;
value_expr(recv);
value_expr(arg1);
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
nd_set_line(expr, op_loc->beg_pos.lineno);
return expr;
}
Action
Slide 61
Slide 61 text
Build AST
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
NODE *expr;
value_expr(recv);
value_expr(arg1);
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
nd_set_line(expr, op_loc->beg_pos.lineno);
return expr;
}
Slide 62
Slide 62 text
Build AST
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
NODE *expr;
value_expr(recv);
value_expr(arg1);
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
nd_set_line(expr, op_loc->beg_pos.lineno);
return expr;
}
Slide 63
Slide 63 text
Build AST
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
NODE *expr;
value_expr(recv);
value_expr(arg1);
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
nd_set_line(expr, op_loc->beg_pos.lineno);
return expr;
} Create NODE_OPCALL
Slide 64
Slide 64 text
Build AST
$ ruby --dump=p -e '1 + 2'
NODE_SCOPE
NODE_PRELUDE
NODE_OPCALL (:+)
NODE_LIT (1)
NODE_ARRAY
NODE_LIT (2)
Slide 65
Slide 65 text
Compile
Parsing ___
\
Ruby script -> Tokens -> AST -> Byte code (insns / ISeq)
__/ __/
Tokenization Compile
Slide 66
Slide 66 text
Compile
• Do compile.
• See “compile.c”.
$ ruby --dump=i -e '1 + 2'
== disasm: #@-e:1 (1,0)-
(1,5)>==============================
0000 putobject_OP_INT2FIX_O_1_C_
( 1)[Li]
0001 putobject 2
0003 opt_plus ,
0006 leave
Slide 67
Slide 67 text
Compile
• Do compile.
• See “compile.c”.
$ ruby --dump=i -e '1 + 2'
== disasm: #@-e:1 (1,0)-
(1,5)>==============================
0000 putobject_OP_INT2FIX_O_1_C_
( 1)[Li]
0001 putobject 2
0003 opt_plus ,
0006 leave
ISeq
4 insn(s)
Slide 68
Slide 68 text
References
• "Ruby Hacking Guide" (Part 2: Syntax analysis)
• https://ruby-hacking-guide.github.io/ [EN]
• http://i.loveruby.net/ja/rhg/book/ [JA]
• "Ruby Under a Microscope" / “Rubyͷ͘͠Έ"
Slide 69
Slide 69 text
How to implement code
locations
Slide 70
Slide 70 text
Goal
• Branch coverage
• To pass code locations to compile phase.
• Method coverage
• To store code locations on ISeq.
• What should we implement
• Embed code locations into each NODE.
Slide 71
Slide 71 text
Hint
• Original source of location information is Ruby script.
• If we want to use location information in "n"th step, we
should implement location information in "n-1"th step.
• In this case, it's need to pass location information from
"Tokenization" to "Compile" to use location information in
compile phase.
Parsing ___
\
Ruby script -> Tokens -> AST -> Byte code (insns / ISeq)
__/ __/
Tokenization Compile
Slide 72
Slide 72 text
parser_params crash course
• One on the main data structure of parser.
• Too Big!!!
struct parser_params {
rb_imemo_tmpbuf_t *heap;
YYSTYPE *lval;
struct {
rb_strterm_t *strterm;
VALUE (*gets)(struct parser_params*,VALUE);
VALUE input;
VALUE prevline;
VALUE lastline;
VALUE nextline;
const char *pbeg;
const char *pcur;
const char *pend;
const char *ptok;
long gets_ptr;
enum lex_state_e state;
/* track the nest level of any parens "()[]{}" */
int paren_nest;
/* keep p->lex.paren_nest at the beginning of lambda "->" to detect tLAMBEG and keyword_do_LAMBDA */
int lpar_beg;
/* track the nest level of only braces "{}" */
int brace_nest;
} lex;
stack_type cond_stack;
stack_type cmdarg_stack;
int tokidx;
int toksiz;
int tokline;
int heredoc_end;
int heredoc_indent;
int heredoc_line_indent;
char *tokenbuf;
struct local_vars *lvtbl;
int line_count;
int ruby_sourceline; /* current line no. */
char *ruby_sourcefile; /* current source file */
VALUE ruby_sourcefile_string;
rb_encoding *enc;
token_info *token_info;
VALUE compile_option;
VALUE debug_buffer;
VALUE debug_output;
ID cur_arg;
rb_ast_t *ast;
unsigned int command_start:1;
unsigned int eofp: 1;
unsigned int ruby__end__seen: 1;
unsigned int debug: 1;
unsigned int has_shebang: 1;
unsigned int in_defined: 1;
unsigned int in_main: 1;
unsigned int in_kwarg: 1;
unsigned int in_def: 1;
unsigned int in_class: 1;
unsigned int token_seen: 1;
unsigned int token_info_enabled: 1;
# if WARN_PAST_SCOPE
unsigned int past_scope_enabled: 1;
# endif
unsigned int error_p: 1;
unsigned int cr_seen: 1;
#ifndef RIPPER
/* Ruby core only */
unsigned int do_print: 1;
unsigned int do_loop: 1;
unsigned int do_chomp: 1;
unsigned int do_split: 1;
unsigned int warn_location: 1;
NODE *eval_tree_begin;
NODE *eval_tree;
VALUE error_buffer;
VALUE debug_lines;
VALUE coverage;
const struct rb_block *base_block;
#else
/* Ripper only */
VALUE delayed;
int delayed_line;
int delayed_col;
VALUE value;
VALUE result;
VALUE parsing_thread;
#endif
};
Slide 73
Slide 73 text
parser_params crash course
• It has struct for lexer (`lex`).
• Lexer processes input in units of lines.
• *Basically* processes from top to bottom.
struct parser_params {
...
struct {
...
VALUE prevline;
VALUE lastline;
VALUE nextline;
const char *pbeg;
const char *pcur;
const char *pend;
const char *ptok;
...
} lex;
...
};
Lines
W
W Pointers
Slide 74
Slide 74 text
What is column
/* parse.y */
/*
Structure of Lexer Buffer:
lex.pbeg lex.ptok lex.pcur lex.pend
| | | |
|------------+------------+------------|
|<---------->|
token
*/
Slide 75
Slide 75 text
What is column
• When token '+' is recognized (Left).
• When token tINTEGER (2) is recognized (Right).
1 + 2
^ ^^ ^
| || +--- lex.pend
| |+----- lex.pcur
| +------ lex.ptok
+-------- lex.pbeg
1 + 2
^ ^ ^
| | +--- lex.pcur, lex.pend
| +----- lex.ptok
+-------- lex.pbeg
Slide 76
Slide 76 text
What is column
• `lex.ptok - lex.pbeg` (begin) and `lex.pcur - lex.pbeg` (end).
• Column is a difference between pointers when a token is
recognized.
|--| lex.pcur - lex.pbeg (end)
|-| lex.ptok - lex.pbeg (begin)
1 + 2
^ ^^ ^
| || +--- lex.pend
| |+----- lex.pcur
| +------ lex.ptok
+-------- lex.pbeg
Slide 77
Slide 77 text
What is column
• We must store columns somewhere before next token is
recognized.
1 + 2
^ ^^ ^
| || +--- lex.pend
| |+----- lex.pcur
| +------ lex.ptok
+-------- lex.pbeg
1 + 2
^ ^ ^
| | +--- lex.pcur, lex.pend
| +----- lex.ptok
+-------- lex.pbeg
Slide 78
Slide 78 text
From Ruby script to tokens
• Copy location information to `YYLTYPE *yylloc` in `yylex`.
• The `yylloc` argument is newly added to `yylex`.
• Call `RUBY_SET_YYLLOC` to set `yylloc`.
Slide 79
Slide 79 text
static enum yytokentype
yylex(YYSTYPE *lval, YYLTYPE *yylloc, struct parser_params *p)
{
enum yytokentype t;
p->lval = lval;
lval->val = Qundef;
t = parser_yylex(p);
if (has_delayed_token(p))
dispatch_delayed_token(p, t);
else if (t != 0)
dispatch_scan_event(p, t);
if (p->lex.strterm && (p->lex.strterm->flags & STRTERM_HEREDOC))
RUBY_SET_YYLLOC_FROM_STRTERM_HEREDOC(*yylloc);
else
RUBY_SET_YYLLOC(*yylloc);
return t;
}
Slide 80
Slide 80 text
static enum yytokentype
yylex(YYSTYPE *lval, YYLTYPE *yylloc, struct parser_params *p)
{
enum yytokentype t;
p->lval = lval;
lval->val = Qundef;
t = parser_yylex(p);
if (has_delayed_token(p))
dispatch_delayed_token(p, t);
else if (t != 0)
dispatch_scan_event(p, t);
if (p->lex.strterm && (p->lex.strterm->flags & STRTERM_HEREDOC))
RUBY_SET_YYLLOC_FROM_STRTERM_HEREDOC(*yylloc);
else
RUBY_SET_YYLLOC(*yylloc);
return t;
}
New argument
Slide 81
Slide 81 text
static enum yytokentype
yylex(YYSTYPE *lval, YYLTYPE *yylloc, struct parser_params *p)
{
enum yytokentype t;
p->lval = lval;
lval->val = Qundef;
t = parser_yylex(p);
if (has_delayed_token(p))
dispatch_delayed_token(p, t);
else if (t != 0)
dispatch_scan_event(p, t);
if (p->lex.strterm && (p->lex.strterm->flags & STRTERM_HEREDOC))
RUBY_SET_YYLLOC_FROM_STRTERM_HEREDOC(*yylloc);
else
RUBY_SET_YYLLOC(*yylloc);
return t;
}
New argument
static enum yytokentype
yylex(YYSTYPE *lval, YYLTYPE *yylloc, struct parser_params *p)
{
enum yytokentype t;
p->lval = lval;
lval->val = Qundef;
t = parser_yylex(p);
if (has_delayed_token(p))
dispatch_delayed_token(p, t);
else if (t != 0)
dispatch_scan_event(p, t);
if (p->lex.strterm && (p->lex.strterm->flags & STRTERM_HEREDOC))
RUBY_SET_YYLLOC_FROM_STRTERM_HEREDOC(*yylloc);
else
RUBY_SET_YYLLOC(*yylloc);
return t;
}
Create token
Slide 82
Slide 82 text
static enum yytokentype
yylex(YYSTYPE *lval, YYLTYPE *yylloc, struct parser_params *p)
{
enum yytokentype t;
p->lval = lval;
lval->val = Qundef;
t = parser_yylex(p);
if (has_delayed_token(p))
dispatch_delayed_token(p, t);
else if (t != 0)
dispatch_scan_event(p, t);
if (p->lex.strterm && (p->lex.strterm->flags & STRTERM_HEREDOC))
RUBY_SET_YYLLOC_FROM_STRTERM_HEREDOC(*yylloc);
else
RUBY_SET_YYLLOC(*yylloc);
return t;
}
New argument
static enum yytokentype
yylex(YYSTYPE *lval, YYLTYPE *yylloc, struct parser_params *p)
{
enum yytokentype t;
p->lval = lval;
lval->val = Qundef;
t = parser_yylex(p);
if (has_delayed_token(p))
dispatch_delayed_token(p, t);
else if (t != 0)
dispatch_scan_event(p, t);
if (p->lex.strterm && (p->lex.strterm->flags & STRTERM_HEREDOC))
RUBY_SET_YYLLOC_FROM_STRTERM_HEREDOC(*yylloc);
else
RUBY_SET_YYLLOC(*yylloc);
return t;
}
Create token
Set `yylloc`
Slide 83
Slide 83 text
From tokens to Nodes
• Now we can use `@n` in each action.
• `@n` stands for the location of the nth component of the
right hand side.
• `@$` stands for the location of the left hand side grouping
(`YYLTYPE yyloc`).
• Set by `YYLLOC_DEFAULT`.
• https://www.gnu.org/software/bison/manual/html_node/
Tracking-Locations.html#Tracking-Locations
Slide 84
Slide 84 text
|---| @$ (1.0-1.5)
1 + 2
^ ^ ^
| | +- @3 (1.4-1.5)
| +--- @2 (1.2-1.3)
+----- @1 (1.0-1.1)
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
...
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
...
}
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
...
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
...
}
1 2
Slide 85
Slide 85 text
|---| @$ (1.0-1.5)
1 + 2
^ ^ ^
| | +- @3 (1.4-1.5)
| +--- @2 (1.2-1.3)
+----- @1 (1.0-1.1)
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
...
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
...
}
(1.0-1.5)
1 2
Slide 86
Slide 86 text
|---| @$ (1.0-1.5)
1 + 2
^ ^ ^
| | +- @3 (1.4-1.5)
| +--- @2 (1.2-1.3)
+----- @1 (1.0-1.1)
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
...
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
...
}
(1.0-1.5)
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
...
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
...
}
1 2
Slide 87
Slide 87 text
|---| @$ (1.0-1.5)
1 + 2
^ ^ ^
| | +- @3 (1.4-1.5)
| +--- @2 (1.2-1.3)
+----- @1 (1.0-1.1)
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
...
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
...
}
(1.0-1.5)
arg | arg '+' arg
{
$$ = call_bin_op(p, $1, '+', $3, &@2, &@$);
}
static NODE *
call_bin_op(struct parser_params *p, NODE *recv, ID id, NODE *arg1,
const YYLTYPE *op_loc, const YYLTYPE *loc)
{
...
expr = NEW_OPCALL(recv, id, NEW_LIST(arg1, &arg1->nd_loc), loc);
...
}
1 2
Slide 88
Slide 88 text
`NODE_OPCALL` is simple
• All location needed to NODE_OPCALL is supplied when
create NODE_OPCALL.
• NODE_LIT (1)
• mid (:+)
• NODE_ARRAY (2)
NODE_OPCALL 1 + 2
----------- ------------
arg : arg '+' arg
Slide 89
Slide 89 text
`NODE_*ASGN` family is not
simple
• `NODE_LASGN` (local variable assignment) or
`NODE_IASGN` (instance variable assignment).
@a = 1
NODE_SCOPE
NODE_IASGN (:@a)
NODE_LIT (1)
Slide 90
Slide 90 text
NODE_IASGN @a
---------- -------------
lhs : user_variable
{
/*%%%*/
$$ = assignable(p, $1, 0, &@$);
/*% %*/
}
NODE_IASGN NODE_IASGN = 1
---------- ------------------------
arg : lhs '=' arg_rhs
{
/*%%%*/
$$ = node_assign(p, $1, $3, &@$);
/*% %*/
/*% ripper: assign!($1, $3) %*/
}
Slide 91
Slide 91 text
NODE_IASGN @a
---------- -------------
lhs : user_variable
{
/*%%%*/
$$ = assignable(p, $1, 0, &@$);
/*% %*/
}
NODE_IASGN NODE_IASGN = 1
---------- ------------------------
arg : lhs '=' arg_rhs
{
/*%%%*/
$$ = node_assign(p, $1, $3, &@$);
/*% %*/
/*% ripper: assign!($1, $3) %*/
}
Create NODE_IASGN
Slide 92
Slide 92 text
NODE_IASGN @a
---------- -------------
lhs : user_variable
{
/*%%%*/
$$ = assignable(p, $1, 0, &@$);
/*% %*/
}
NODE_IASGN NODE_IASGN = 1
---------- ------------------------
arg : lhs '=' arg_rhs
{
/*%%%*/
$$ = node_assign(p, $1, $3, &@$);
/*% %*/
/*% ripper: assign!($1, $3) %*/
}
Create NODE_IASGN
Location is determined
Slide 93
Slide 93 text
NODE_IASGN @a
---------- -------------
lhs : user_variable
{
/*%%%*/
$$ = assignable(p, $1, 0, &@$);
/*% %*/
}
NODE_IASGN NODE_IASGN = 1
---------- ------------------------
arg : lhs '=' arg_rhs
{
/*%%%*/
$$ = node_assign(p, $1, $3, &@$);
/*% %*/
/*% ripper: assign!($1, $3) %*/
}
Create NODE_IASGN
Update location
Location is determined
Slide 94
Slide 94 text
`NODE_*ASGN` family is not
simple
• Create `NODE_*ASGN`.
• Assign right hand side.
• So it's needed to update location of `NODE_*ASGN` when
right hand side is assigned.
Slide 95
Slide 95 text
`NODE_ITER` is not simple
• `NODE_ITER` (method call with block).
3.times { foo }
NODE_ITER
NODE_CALL (:times)
NODE_LIT (3)
NODE_SCOPE
NODE_VCALL (:foo)
Slide 96
Slide 96 text
NODE_CALL 3 . times
----------- --------------------------------------------------
method_call | primary_value call_op operation2 opt_paren_args
{
}
NODE_ITER { NODE_ITER }
----------- ---------------------
brace_block : '{' brace_body '}'
{
}
NODE_ITER NODE_CALL NODE_ITER
--------- -----------------------
primary | method_call brace_block
{
/*%%%*/
block_dup_check(p, $1->nd_args, $2);
$$ = method_add_block(p, $1, $2, &@$);
/*% %*/
}
Slide 97
Slide 97 text
NODE_CALL 3 . times
----------- --------------------------------------------------
method_call | primary_value call_op operation2 opt_paren_args
{
}
NODE_ITER { NODE_ITER }
----------- ---------------------
brace_block : '{' brace_body '}'
{
}
NODE_ITER NODE_CALL NODE_ITER
--------- -----------------------
primary | method_call brace_block
{
/*%%%*/
block_dup_check(p, $1->nd_args, $2);
$$ = method_add_block(p, $1, $2, &@$);
/*% %*/
}
`3.times` `{ foo }`
Slide 98
Slide 98 text
NODE_CALL 3 . times
----------- --------------------------------------------------
method_call | primary_value call_op operation2 opt_paren_args
{
}
NODE_ITER { NODE_ITER }
----------- ---------------------
brace_block : '{' brace_body '}'
{
}
NODE_ITER NODE_CALL NODE_ITER
--------- -----------------------
primary | method_call brace_block
{
/*%%%*/
block_dup_check(p, $1->nd_args, $2);
$$ = method_add_block(p, $1, $2, &@$);
/*% %*/
}
`3.times` `{ foo }`
Update location
Slide 99
Slide 99 text
`NODE_ITER` is not simple
• `NODE_CALL` is created.
• `NODE_ITER` is created.
• `NODE_ITER` is added to `NODE_CALL`.
• It's needed to update location of `NODE_ITER` when it is
passed to `NODE_CALL`.
Slide 100
Slide 100 text
No content
Slide 101
Slide 101 text
$ git shortlog -s -n parse.y | head -10
XXX ????
362 matz
133 mame
88 yui-knk
55 ko1
38 aamine
37 akr
33 naruse
25 usa
7 normal
On "v2_6_0_preview1".
Slide 102
Slide 102 text
$ git shortlog -s -n parse.y | head -10
XXX ????
362 matz
133 mame
88 yui-knk
55 ko1
38 aamine
37 akr
33 naruse
25 usa
7 normal
On "v2_6_0_preview1".
Me
Slide 103
Slide 103 text
$ git shortlog -s -n parse.y | head -10
884 nobu
362 matz
133 mame
88 yui-knk
55 ko1
38 aamine
37 akr
33 naruse
25 usa
7 normal
On "v2_6_0_preview1".
Slide 104
Slide 104 text
$ git shortlog -s -n parse.y | head -10
884 nobu
362 matz
133 mame
88 yui-knk
55 ko1
38 aamine
37 akr
33 naruse
25 usa
7 normal
On "v2_6_0_preview1".
x 10
Slide 105
Slide 105 text
$ git shortlog -s -n parse.y | head -10
884 nobu
362 matz
133 mame
88 yui-knk
55 ko1
38 aamine
37 akr
33 naruse
25 usa
7 normal
On "v2_6_0_preview1".
x 10
@nobu is the lord of Demon Castle "parse.y".
Slide 106
Slide 106 text
How to test
• Define some rules and check all ruby files in "test"
directory follow the rules.
• Related files:
• “ext/-test-/ast/ast.c"
• "test/-ext-/ast/test_ast.rb"
On "v2_6_0_preview1".
Slide 107
Slide 107 text
"ext/-test-" and "test/-ext-"
• "ext/-test-" contains C extensions which are used in
Ruby's tests.
• "ext/-test-/ast/ast.c" defines `AST` module and
`AST::Node` class.
On "v2_6_0_preview1".
Slide 108
Slide 108 text
"ext/-test-" and "test/-ext-"
• "test/-ext-" contains test cases which depend “ext/-
test-".
On "v2_6_0_preview1".
Slide 109
Slide 109 text
Rule 1
• `lineno` is initialized with `0` and `column` with `-1`.
• Validate all node locations are update at least once.
NODE_IF (line: 1, location: (0,-1)-(0,-1))
Slide 110
Slide 110 text
Rule 1
• `lineno` is initialized with `0` and `column` with `-1`.
• Validate all node locations are update at least once.
NODE_IF (line: 1, location: (0,-1)-(0,-1))
Slide 111
Slide 111 text
Rule 2
• Validate children do not exceed a parent location.
3.times { foo }
NODE_ITER [1.0-1.15]
NODE_CALL (:times) [1.0-1.8]
NODE_LIT (3) [1.0-1.1]
NODE_SCOPE [1.8-1.15]
NODE_VCALL (:foo) [1.10-1.13]
Slide 112
Slide 112 text
Rule 2
• Validate children do not exceed a parent location.
3.times { foo }
NODE_ITER [1.0-1.15]
NODE_CALL (:times) [1.0-1.8]
NODE_LIT (3) [1.0-1.1]
NODE_SCOPE [1.8-1.15] -> covers [1.10-1.13]
NODE_VCALL (:foo) [1.10-1.13]
Slide 113
Slide 113 text
Rule 2
• Validate children do not exceed a parent location.
3.times { foo }
NODE_ITER [1.0-1.15]
NODE_CALL (:times) [1.0-1.8] -> covers [1.0-1.1]
NODE_LIT (3) [1.0-1.1]
NODE_SCOPE [1.8-1.15] -> covers [1.10-1.13]
NODE_VCALL (:foo) [1.10-1.13]
Slide 114
Slide 114 text
Rule 2
• Validate children do not exceed a parent location.
3.times { foo }
NODE_ITER [1.0-1.15] -> covers [1.0-1.8] and [1.8-1.15]
NODE_CALL (:times) [1.0-1.8] -> covers [1.0-1.1]
NODE_LIT (3) [1.0-1.1]
NODE_SCOPE [1.8-1.15] -> covers [1.10-1.13]
NODE_VCALL (:foo) [1.10-1.13]
Slide 115
Slide 115 text
Rule 2
• Validate children do not exceed a parent location.
3.times { foo }
NODE_ITER [1.0-1.15] -> covers [1.0-1.8] and [1.8-1.15]
NODE_CALL (:times) [1.0-1.8] -> covers [1.0-1.1]
NODE_LIT (3) [1.0-1.1]
NODE_SCOPE [1.8-1.15] -> covers [1.10-1.13]
NODE_VCALL (:foo) [1.10-1.13]
Slide 116
Slide 116 text
Dir.glob("test/**/*.rb", base: SRCDIR).each do |path|
define_method("test_ranges:#{path}") do
helper = Helper.new("#{SRCDIR}/#{path}")
helper.validate_range
assert_equal([], helper.errors)
end
end
test/-ext-/ast/test_ast.rb
Slide 117
Slide 117 text
Dir.glob("test/**/*.rb", base: SRCDIR).each do |path|
define_method("test_ranges:#{path}") do
helper = Helper.new("#{SRCDIR}/#{path}")
helper.validate_range
assert_equal([], helper.errors)
end
end
Check all ruby files in "test" directory
test/-ext-/ast/test_ast.rb
Slide 118
Slide 118 text
Dir.glob("test/**/*.rb", base: SRCDIR).each do |path|
define_method("test_ranges:#{path}") do
helper = Helper.new("#{SRCDIR}/#{path}")
helper.validate_range
assert_equal([], helper.errors)
end
end
Check all ruby files in "test" directory
Validate each file
test/-ext-/ast/test_ast.rb
Slide 119
Slide 119 text
def validate_range0(node)
beg_pos, end_pos = node.beg_pos, node.end_pos
children = node.children.compact
min = children.map(&:beg_pos).min
max = children.map(&:end_pos).max
unless beg_pos <= min
@errors << { type: :min_validation_error, min: min, beg_pos: beg_pos,
node: node }
end
unless max <= end_pos
@errors << { type: :max_validation_error, max: max, end_pos: end_pos,
node: node }
end
children.each do |child|
validate_range0(child)
end
end
ast = AST.parse_file(@path)
validate_not_cared0(ast) test/-ext-/ast/test_ast.rb
Slide 120
Slide 120 text
def validate_range0(node)
beg_pos, end_pos = node.beg_pos, node.end_pos
children = node.children.compact
min = children.map(&:beg_pos).min
max = children.map(&:end_pos).max
unless beg_pos <= min
@errors << { type: :min_validation_error, min: min, beg_pos: beg_pos,
node: node }
end
unless max <= end_pos
@errors << { type: :max_validation_error, max: max, end_pos: end_pos,
node: node }
end
children.each do |child|
validate_range0(child)
end
end
ast = AST.parse_file(@path)
validate_not_cared0(ast)
Generate AST
test/-ext-/ast/test_ast.rb
Slide 121
Slide 121 text
def validate_range0(node)
beg_pos, end_pos = node.beg_pos, node.end_pos
children = node.children.compact
min = children.map(&:beg_pos).min
max = children.map(&:end_pos).max
unless beg_pos <= min
@errors << { type: :min_validation_error, min: min, beg_pos: beg_pos,
node: node }
end
unless max <= end_pos
@errors << { type: :max_validation_error, max: max, end_pos: end_pos,
node: node }
end
children.each do |child|
validate_range0(child)
end
end
ast = AST.parse_file(@path)
validate_not_cared0(ast)
Generate AST
test/-ext-/ast/test_ast.rb
Check ranges
Slide 122
Slide 122 text
def validate_range0(node)
beg_pos, end_pos = node.beg_pos, node.end_pos
children = node.children.compact
min = children.map(&:beg_pos).min
max = children.map(&:end_pos).max
unless beg_pos <= min
@errors << { type: :min_validation_error, min: min, beg_pos: beg_pos,
node: node }
end
unless max <= end_pos
@errors << { type: :max_validation_error, max: max, end_pos: end_pos,
node: node }
end
children.each do |child|
validate_range0(child)
end
end
ast = AST.parse_file(@path)
validate_not_cared0(ast)
Generate AST
test/-ext-/ast/test_ast.rb
Check ranges
Check children
Slide 123
Slide 123 text
The future plan of code
locations feature
Slide 124
Slide 124 text
Case 1 (Proc/Method)
• Add new methods to Proc/Method which return their
code location.
def a(&block)
p block.code_location
end
a do
1 + 2
end
# => [[5, 2], [7, 3]]
p self.class.instance_method(:a).code_location
# => [[1, 0], [3, 3]]
https://github.com/yui-knk/ruby/tree/feature/rb_iseq_code_location
Slide 125
Slide 125 text
Case 1 (Proc/Method)
• Add new methods to Proc/Method which return their
code location.
def a(&block)
p block.code_location
end
a do
1 + 2
end
# => [[5, 2], [7, 3]]
p self.class.instance_method(:a).code_location
# => [[1, 0], [3, 3]]
https://github.com/yui-knk/ruby/tree/feature/rb_iseq_code_location
Slide 126
Slide 126 text
Case 1 (Proc/Method)
• Add new methods to Proc/Method which return their
code location.
def a(&block)
p block.code_location
end
a do
1 + 2
end
# => [[5, 2], [7, 3]]
p self.class.instance_method(:a).code_location
# => [[1, 0], [3, 3]]
https://github.com/yui-knk/ruby/tree/feature/rb_iseq_code_location
Slide 127
Slide 127 text
Case 1 (Proc/Method)
• Add new methods to Proc/Method which return their
code location.
def a(&block)
p block.code_location
end
a do
1 + 2
end
# => [[5, 2], [7, 3]]
p self.class.instance_method(:a).code_location
# => [[1, 0], [3, 3]]
https://github.com/yui-knk/ruby/tree/feature/rb_iseq_code_location
Slide 128
Slide 128 text
Case 2 (NoMethodError)
• Give `NoMethodError` more detailed message.
class A
def foo
nil
end
end
A.new.foo.foo
Traceback (most recent call last):
/tmp/test.rb:7:in `': undefined method `foo' for nil:NilClass
(NoMethodError)
A.new.foo.foo
^^^^
https://github.com/yui-knk/ruby/tree/feature/node_id
Slide 129
Slide 129 text
Case 3 (AST module)
AST.parse("1 + 2")
# => #
AST.parse("1 + 2").children[1]
# => #
AST.parse("1 + 2").children[1].children
# => [#,
#]
Slide 130
Slide 130 text
Case 3 (AST module)
AST.parse("1 + 2")
# => #
AST.parse("1 + 2").children[1]
# => #
AST.parse("1 + 2").children[1].children
# => [#,
#]
Slide 131
Slide 131 text
• We discussed this topic at Developers Meeting yesterday.
Slide 132
Slide 132 text
Committed
Slide 133
Slide 133 text
Conference Driven
Development !!!
Slide 134
Slide 134 text
No content
Slide 135
Slide 135 text
No content
Slide 136
Slide 136 text
Case 3 (AST module)
• We can get children nodes.
RubyVM::AST.parse("1 + 2")
# => #
RubyVM::AST.parse("1 + 2").children[1]
# => #
RubyVM::AST.parse("1 + 2").children[1].children
# => [#,
#]
Slide 137
Slide 137 text
Case 3 (AST module)
• We can get location information.
[RubyVM::AST.parse("1 + 2").first_lineno,
RubyVM::AST.parse("1 + 2").first_column]
# => [1, 0]
[RubyVM::AST.parse("1 + 2").last_lineno,
RubyVM::AST.parse("1 + 2").last_column]
# => [1, 5]
Slide 138
Slide 138 text
Enjoy programming with
Ruby 2.6.0-preview2!
Slide 139
Slide 139 text
Conclusion
Slide 140
Slide 140 text
Acknowledgments
• @mametter
• @nobu
• @ko1
• @shyouhei
• @takeshinoda
• @hkdnet
• @HaiTo
• @littlestarling
Slide 141
Slide 141 text
Conclusion
• AST Node has location information.
• Share the future plan of code locations feature.
• If you have any idea to use location information, please
let me know :)
• https://bugs.ruby-lang.org/
• You now get the map of Demon Castle "parse.y", let's
hack “parse.y" :)
Slide 142
Slide 142 text
Thank you!!!
Slide 143
Slide 143 text
Bonus track
Slide 144
Slide 144 text
How to implement more detailed
message of `NoMethodError`
Slide 145
Slide 145 text
Target code
class A
def foo
nil
end
end
A.new.foo.foo
Slide 146
Slide 146 text
# @ NODE_CALL (line: 7, location: (7,0)-(7,13))* 13
# +- nd_mid: :foo
# +- nd_recv:
# | @ NODE_CALL (line: 7, location: (7,0)-(7,9)) 12
# | +- nd_mid: :foo
# | +- nd_recv:
# | | @ NODE_CALL (line: 7, location: (7,0)-(7,5)) 11
# | | +- nd_mid: :new
# | | +- nd_recv:
# | | | @ NODE_CONST (line: 7, location: (7,0)-(7,1)) 10
# | | | +- nd_vid: :A
# | | +- nd_args:
# | | (null node)
# | +- nd_args:
# | (null node)
# +- nd_args:
# (null node)
node_id
• Add unique id (per file), “node_id”, to Node.
Slide 147
Slide 147 text
== disasm: #@src/no_method_error2.rb:1 (1,0)-(7,13)> (catch: FALSE)
0000 putspecialobject 3 ( 1)[ 0][Li]
0002 putnil [ 9]
0003 defineclass :A, , 0
0007 pop
0008 getinlinecache 15, ( 7)[ 10][Li]
0011 getconstant :A
0013 setinlinecache
0015 opt_send_without_block , [ 11]
0018 opt_send_without_block , [ 12]
0021 opt_send_without_block , [ 13]
0024 leave [ 10]
node_id
• Store node_id of insn on an ISeq.
Slide 148
Slide 148 text
• Store node_id of insn on an ISeq.
• We can distinguish between `#foo`s by node_id.
== disasm: #@src/no_method_error2.rb:1 (1,0)-(7,13)> (catch: FALSE)
0000 putspecialobject 3 ( 1)[ 0][Li]
0002 putnil [ 9]
0003 defineclass :A, , 0
0007 pop
0008 getinlinecache 15, ( 7)[ 10][Li]
0011 getconstant :A
0013 setinlinecache
0015 opt_send_without_block , [ 11]
0018 opt_send_without_block , [ 12]
0021 opt_send_without_block , [ 13]
0024 leave [ 10]
Slide 149
Slide 149 text
Exceptions
• Exceptions have an ISeq and a program counter (pc).
== disasm: #@src/no_method_error2.rb:1 (1,0)-(7,13)> (catch: FALSE)
0000 putspecialobject 3 ( 1)[ 0][Li]
0002 putnil [ 9]
0003 defineclass :A, , 0
0007 pop
0008 getinlinecache 15, ( 7)[ 10][Li]
0011 getconstant :A
0013 setinlinecache
0015 opt_send_without_block , [ 11]
0018 opt_send_without_block , [ 12]
0021 opt_send_without_block , [ 13]
0024 leave [ 10]
Slide 150
Slide 150 text
Exceptions
• Exceptions have an ISeq and a program counter (pc).
== disasm: #@src/no_method_error2.rb:1 (1,0)-(7,13)> (catch: FALSE)
0000 putspecialobject 3 ( 1)[ 0][Li]
0002 putnil [ 9]
0003 defineclass :A, , 0
0007 pop
0008 getinlinecache 15, ( 7)[ 10][Li]
0011 getconstant :A
0013 setinlinecache
0015 opt_send_without_block , [ 11]
0018 opt_send_without_block , [ 12]
0021 opt_send_without_block , [ 13]
0024 leave [ 10]
Exception
Slide 151
Slide 151 text
Exceptions
• Exceptions have an ISeq and a program counter (pc).
== disasm: #@src/no_method_error2.rb:1 (1,0)-(7,13)> (catch: FALSE)
0000 putspecialobject 3 ( 1)[ 0][Li]
0002 putnil [ 9]
0003 defineclass :A, , 0
0007 pop
0008 getinlinecache 15, ( 7)[ 10][Li]
0011 getconstant :A
0013 setinlinecache
0015 opt_send_without_block , [ 11]
0018 opt_send_without_block , [ 12]
0021 opt_send_without_block , [ 13]
0024 leave [ 10]
Exception
Slide 152
Slide 152 text
Exceptions
• Get node_id from an exception.
== disasm: #@src/no_method_error2.rb:1 (1,0)-(7,13)> (catch: FALSE)
0000 putspecialobject 3 ( 1)[ 0][Li]
0002 putnil [ 9]
0003 defineclass :A, , 0
0007 pop
0008 getinlinecache 15, ( 7)[ 10][Li]
0011 getconstant :A
0013 setinlinecache
0015 opt_send_without_block , [ 11]
0018 opt_send_without_block , [ 12]
0021 opt_send_without_block , [ 13]
0024 leave [ 10]
Exception Get node_id (13)
Slide 153
Slide 153 text
# @ NODE_CALL (line: 7, location: (7,0)-(7,13))* 13
# +- nd_mid: :foo
# +- nd_recv:
# | @ NODE_CALL (line: 7, location: (7,0)-(7,9)) 12
# | +- nd_mid: :foo
# | +- nd_recv:
# | | @ NODE_CALL (line: 7, location: (7,0)-(7,5)) 11
# | | +- nd_mid: :new
# | | +- nd_recv:
# | | | @ NODE_CONST (line: 7, location: (7,0)-(7,1)) 10
# | | | +- nd_vid: :A
# | | +- nd_args:
# | | (null node)
# | +- nd_args:
# | (null node)
# +- nd_args:
# (null node)
• Parse the source code file and find Node by node_id.
Slide 154
Slide 154 text
# @ NODE_CALL (line: 7, location: (7,0)-(7,13))* 13
# +- nd_mid: :foo
# +- nd_recv:
# | @ NODE_CALL (line: 7, location: (7,0)-(7,9)) 12
# | +- nd_mid: :foo
# | +- nd_recv:
# | | @ NODE_CALL (line: 7, location: (7,0)-(7,5)) 11
# | | +- nd_mid: :new
# | | +- nd_recv:
# | | | @ NODE_CONST (line: 7, location: (7,0)-(7,1)) 10
# | | | +- nd_vid: :A
# | | +- nd_args:
# | | (null node)
# | +- nd_args:
# | (null node)
# +- nd_args:
# (null node)
• Parse the source code file and find Node by node_id.
Slide 155
Slide 155 text
# @ NODE_CALL (line: 7, location: (7,0)-(7,13))* 13
# +- nd_mid: :foo
# +- nd_recv:
# | @ NODE_CALL (line: 7, location: (7,0)-(7,9)) 12
# | +- nd_mid: :foo
# | +- nd_recv:
# | | @ NODE_CALL (line: 7, location: (7,0)-(7,5)) 11
# | | +- nd_mid: :new
# | | +- nd_recv:
# | | | @ NODE_CONST (line: 7, location: (7,0)-(7,1)) 10
# | | | +- nd_vid: :A
# | | +- nd_args:
# | | (null node)
# | +- nd_args:
# | (null node)
# +- nd_args:
# (null node)
• Parse the source code file and find Node by node_id.
Slide 156
Slide 156 text
A.new.foo.foo
NODE_CALL (:foo) (line: 7, location: (7,0)-(7,13))* 13
NODE_CALL (:foo) (line: 7, location: (7,0)-(7,9)) 12
NODE_CALL (:new) (line: 7, location: (7,0)-(7,5)) 11
NODE_CONST (:A) (line: 7, location: (7,0)-(7,1)) 10
• Get location of Node.
Slide 157
Slide 157 text
A.new.foo.foo
NODE_CALL (:foo) (line: 7, location: (7,0)-(7,13))* 13
NODE_CALL (:foo) (line: 7, location: (7,0)-(7,9)) 12
NODE_CALL (:new) (line: 7, location: (7,0)-(7,5)) 11
NODE_CONST (:A) (line: 7, location: (7,0)-(7,1)) 10
• Get location of Node.
Slide 158
Slide 158 text
A.new.foo.foo
NODE_CALL (:foo) (line: 7, location: (7,0)-(7,13))* 13
NODE_CALL (:foo) (line: 7, location: (7,0)-(7,9)) 12
NODE_CALL (:new) (line: 7, location: (7,0)-(7,5)) 11
NODE_CONST (:A) (line: 7, location: (7,0)-(7,1)) 10
A.new.foo
• Get location of Node.
Slide 159
Slide 159 text
A.new.foo.foo
NODE_CALL (:foo) (line: 7, location: (7,0)-(7,13))* 13
NODE_CALL (:foo) (line: 7, location: (7,0)-(7,9)) 12
NODE_CALL (:new) (line: 7, location: (7,0)-(7,5)) 11
NODE_CONST (:A) (line: 7, location: (7,0)-(7,1)) 10
A.new.foo
.foo
• Build an error message.
Slide 160
Slide 160 text
A.new.foo.foo
NODE_CALL (:foo) (line: 7, location: (7,0)-(7,13))* 13
NODE_CALL (:foo) (line: 7, location: (7,0)-(7,9)) 12
NODE_CALL (:new) (line: 7, location: (7,0)-(7,5)) 11
NODE_CONST (:A) (line: 7, location: (7,0)-(7,1)) 10
A.new.foo
.foo
^^^^
• Build an error message.
Slide 161
Slide 161 text
How to implement more detailed
message of `NoMethodError`
• Add unique id (per file), “node_id”, to Node.
• Store node_id of insn on an ISeq.
• Get node_id from an exception.
• Parse the source code file and find Node by node_id.
• Get location of Node.
• Build an error message.
Slide 162
Slide 162 text
Thank you!!!