Coverage.so, or RNode with column

ۚࢠ༤Ұ࿠ !ZVJLOL
ϚωʔϑΥϫʔυ3VCZษڧձ
ਐԽ͢Δ$PWFSBHFTPɺ
͋Δ͍͸/0%&ͱDPMVNOͷ͸ͳ͠

View Slide

raise MissingBeer unless
1

View Slide

ࣗݾ঺հ
• Ҏલ͸ॏ޻ۀͳϝʔΧʔͰܦཧΛ͍ͯͨ͠
• ϚωʔϑΥʔϫʔυͰ͸ձܭνʔϜͰRailsΞϓϦΛॻ͍͍ͯΔ
• Rubyίϛολʔ 2015/12~
• Asakusa.rb ॴଐ
2

View Slide

• Ruby 2.5ͷશମతͳ࿩͸͜ͷ͋ͱ͋Δ͸ͣ
• ࣗ෼͕ࠓ೥ͱΓ͘Μͩ͜ͱͷ࿩
3

View Slide

ࣗ෼͕ࠓ೥ͱΓ͘Μͩ͜ͱ
$ git log --numstat --pretty="%H" --author="yui-knk" --since="2016-12-26" |\
awk 'NF==3 {plus+=$1; minus+=$2; printf("%s %d %d\n", $3, $1, $2)}' |\
ruby -e 'ary=[]; while gets(); ary << $_.chomp.split(" ");end; \
ary.each_with_object({}) {|(n,p,m),h| h[n]||= [0,0];\
h[n] = [h[n][0] + p.to_i, h[n][1] + m.to_i]}.sort_by\
{|(k,(p,m))| -(p + m) }.each {|(k,(p,m))| puts "#{k} +#{p}, -#{m}"}'
4

View Slide

parse.y +2256, -1687
compile.c +148, -78
test/coverage/test_coverage.rb +100, -55
node.h +34, -38
node.c +38, -18
ext/coverage/coverage.c +20, -12
thread.c +8, -6
ext/objspace/objspace.c +1, -5
test/ruby/test_iseq.rb +3, -1
test/ripper/test_lexer.rb +1, -1
test/ruby/test_string.rb +1, -1
insns.def +0, -2
ext/objspace/object_tracing.c +1, -1
doc/extension.ja.rdoc +1, -1
doc/extension.rdoc +1, -1
iseq.c +0, -1
5

View Slide

• (ओʹ)CoverageͷͨΊʹɺparserΛ͍͍ͬͯͨ͡
6

View Slide

Contents
• Coverage ͷ͸ͳ͠
• Coverage ͱ͸
• Ruby 2.5ʹ͓͚Δ Coverage.so
• Branch Coverage ͷ࢓૊Έ
• NODE ͱ column ͷ͸ͳ͠
• lexer ͱ parser
• column ৘ใͷऔಘͱอ؅
• ޻෉͕ඞཁͳ NODE
• test ʹ͍ͭͯ
• ͓·͚
7

View Slide

஫ҙ఺
• ϋογϡλά => #mf_ruby (શ෦খจࣈ)
• લ൒ => Ruby 2.5Λ࢖͏͏͑Ͱ໾ʹཱͭ
• ޙ൒ => Ruby 2.5Λ͍͡Δ͏͑Ͱ໾ʹཱͭ
8

View Slide

Coverage ͷ͸ͳ͠
9

View Slide

Coverage ͷ͸ͳ͠
• Coverage ͱ͸
• Ruby 2.5ʹ͓͚Δ Coverage.so
• Branch Coverage ͷ࢓૊Έ
10

View Slide

Coverage ͱ͸
• ςετίʔυΛධՁ͢Δͱ͖ʹࢀߟʹͳΔ৘ใ
• ৄ͘͠͸ԕ౻(@mametter)͞ΜͷൃදΛ͝ཡ͍ͩ͘͞
• "An introduction and future of Ruby coverage library" (http://
rubykaigi.org/2017/presentations/mametter.html)
11

View Slide

Coverage ͷ࣮ྫ
# Coverageऔಘର৅ͷίʔυ
def target(bool)
if bool
1
else
0
end
end
target(true)
# CoverageΛऔಘ͢ΔͨΊͷίʔυ
require "coverage"
Coverage.start
require "target_file" # ର৅ͷϑΝΠϧΛload
Coverage.result
12

View Slide

Coverage ͷ࣮ྫ
# Coverage.result
{
"src/coverage_example.rb" => [1, 1, 1, nil, 0, nil, nil, nil, 1]
}
# ͦΕͬΆ͘mapping
1: def target(bool)
1: if bool
1: 1
nil: else
0: 0
nil: end
nil: end
nil:
1: target(true)
13

View Slide

Ruby 2.5ʹ͓͚Δ Coverage.so
• Branch CoverageͱMethod Coverage͕৽ͨʹಋೖ͞ΕΔ
Ruby 2.4 Ruby 2.5
Line Coverage o o
Branch Coverage - o
Method Coverage - o
14

View Slide

Method Coverageͱ͸
• ϝιου͕Կճݺͼग़͞Ε͔͕ͨΘ͔ΔΑ͏ʹͳΔ
def target1
end
def target2
end
target2
0: def target1
: end
:
1: def target2
: end
:
: target2
15

View Slide

Branch Coverageͱ͸
• Ͳͷ෼ذ͕Կճ࣮ߦ͞Ε͔ͨΘ͔ΔΑ͏ʹͳΔ
a = 1
if a
1
else
2
end
: a = 1
:
: if a
1: 1
: else
0: 2
: end
16

View Slide

Line CoverageͩͱऔΕͳ͔ͬͨ৘ใ(1)
• ޙஔif
1: a = true
1: b = false
1: 1 if a
1: 2 if b # b͕ධՁ͞ΕΔ͔Βpassͨ͜͠ͱʹͳΔ
17

View Slide

Line CoverageͩͱऔΕͳ͔ͬͨ৘ใ(2)
• ࡾ߲ԋࢉ
1: a = true
1: b = false
1: a ? 1 : 2 # a͕ධՁ͞ΕΔ͔Βpassͨ͜͠ͱʹͳΔ
1: b ? 1 : 2 # b͕ධՁ͞ΕΔ͔Βpassͨ͜͠ͱʹͳΔ
18

View Slide

Branch Coverageͷྫ(if/unless)
• ޙஔif/unless΍ࡾ߲ԋࢉࢠʹ΋ରԠ
: a = 1
:
1/0: 10 if a == 1
0/1: 11 unless a == 1
0/1: (a == 2) ? :t : :f
n/m -> n͕thenɺm͕elseͷճ਺
19

View Slide

Branch Coverageͷྫ(while/until)
: x = 3
: while x > 0
3: x -= 1
: end
: until x == 10
10: x += 1
: end
: y = 3
3: y -= 1 while y > 0
10: y += 1 until y == 10
20

View Slide

Branch Coverageͷྫ(case)
: x = 0
:
: case x
: when 0
1: 0
: when 1
0: 1
: end
:
: case
: when x == 0
1: 0
: when x == 1
0: 1
: end
21

View Slide

Branch Coverageͷྫ(&.)
: a = 10
: b = nil
1/0: a&.abs
0/1: b&.hoo
n/m -> n͕then(ϝιουΛcall)ɺm͕elseͷճ਺
22

View Slide

Branch Coverage ͷ࢓૊Έ
• ؅ཧςʔϒϧΛ༻ҙ
• byte codeʹಛघͳ໋ྩΛຒΊࠐΉ
• ໋ྩ͕ɺ؅ཧςʔϒϧͷ֘౰ՕॴΛΠϯΫϦϝϯτ͢Δ
a = 1
if a
:A
else
:B
end
23

View Slide

Branch Coverage ͷ࢓૊Έ (؅ཧςʔϒϧ)
• ؅ཧςʔϒϧΛ༻ҙ
[structure, counters]
structure = [
[
:if , 2, 0, 6, 3, # type, first_lineno, first_column, last_lineno, last_column
:then, 3, 2, 3, 4, 0, # type, first_lineno, first_column, last_lineno, last_column, counter_index
:else, 5, 2, 5, 4, 1 # type, first_lineno, first_column, last_lineno, last_column, counter_index
], ...
]
counters = [0, 0] # [:then_counter, :else_counter]
24

View Slide

Branch Coverage ͷ࢓૊Έ (໋ྩຒΊࠐΈ)
• byte codeʹಛघͳ໋ྩΛຒΊࠐΉ
src = <<~SRC
a = 1
if a
:A
else
:B
end
SRC
# Coverage OFF
puts RubyVM::InstructionSequence.compile(src).disasm
require "coverage"
ENV["COVERAGE_EXPERIMENTAL_MODE"] = "true"
Coverage.start(branches: true)
# Coverage ON
puts RubyVM::InstructionSequence.compile(src).disasm
25

View Slide

• Coverage͕OFFͷͱ͖
== disasm: #@>================================
local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 1] a
0000 putobject_OP_INT2FIX_O_1_C_ ( 1)[Li]
0001 setlocal_OP__WC__0 a
0003 getlocal_OP__WC__0 a ( 2)[Li]
0005 branchunless 10
0007 putobject :A ( 3)[Li]
0009 leave ( 5)
0010 putobject :B[Li]
0012 leave
26

View Slide

• Coverage͕ONͷͱ͖
== disasm: #@>================================
[ 1] a
0007 trace2 131072, 1 ( 3)
0010 putobject :A[Li]
0012 leave ( 5)
0013 trace2 131072, 17
0018 leave
27

View Slide

• Coverage͕ONͷͱ͖
== disasm: #@>================================
[ 1] a
0007 trace2 131072, 1 ( 3)
0012 leave ( 5)
0013 trace2 131072, 17
0018 leave
28

View Slide

Branch Coverage ͷ࢓૊Έ (count up)
trace2 131072, 1 # => counter_idx = 0
trace2 131072, 17 # => counter_idx = 1
• ୈ2Φϖϥϯυ(1΍17)͕counter_idxΛද͢
• counter_idx * 16 + COVERAGE_INDEX_BRANCHES
• COVERAGE_INDEX_BRANCHES͸1
29

View Slide

== disasm: #@>================================
[ 1] a
0007 trace2 131072, 1 ( 3)
0012 leave ( 5)
0013 trace2 131072, 17
0018 leave
• a͕false΋͘͠͸nilͳΒ0013ʹͱͿ
• ͦΕҎ֎ͷͱ͖͸Կ΋ͤͣɺ࣍ͷ໋ྩ΁͍͘
30

View Slide

== disasm: #@>================================
[ 1] a
0007 trace2 131072, 1 ( 3)
0012 leave ( 5)
0013 trace2 131072, 17
0018 leave
• ࠓճͷέʔεͰ͸a = 1ͳͷͰɺtrace2 131072, 1͕࣮ߦ͞Ε
Δ -> :then_counter͕૿Ճ͢Δ
31

View Slide

Branch Coverage ͷ࢓૊Έ (݁Ռ)
[structure, counters]
structure = [[:if, 2, 0, 6, 3, :then, 3, 2, 3, 4, 0, :else, 5, 2, 5, 4, 1], ...]
counters = [1, 0] # [:then_counter, :else_counter]
=>
# ͦΕͬΆ͘mapping
: a = 1
: if a
1: :A
: else
0: :B
: end
32

View Slide

column৘ใ͕΄͍͠έʔε
• औಘͨ͠Coverage݁ՌΛՄࢹԽ͢Δͱ͖
• :Aͱ͔:BͷҐஔ͕Θ͔Δͱศར
a, b = 1, 1
(a == b) ? :A : :B
33

View Slide

• 1ߦʹෳ਺ͷ෼ذ͕ଘࡏ͢Δͱ͖
a, b, c, d = 1, 1, 3, 4
(a == b) ? ((c == d) ? :A : :B) : :C
{
[:if, 0, 2] => { # 2ߦ໨
[:then, 1, 2] => 0, # 2ߦ໨
[:else, 2, 2] => 1 # 2ߦ໨
},
[:if, 3, 2] => { # 2ߦ໨
[:then, 4, 2] => 1, # 2ߦ໨
[:else, 5, 2] => 0 # 2ߦ໨
}
}
34

View Slide

• 1ߦʹෳ਺ͷ෼ذ͕ଘࡏ͢Δͱ͖
a, b, c, d = 1, 1, 3, 4
(a == b) ? ((c == d) ? :A : :B) : :C
{
[:if, 0, 2, 12, 2, 30] => { # [2.12-2.30]
[:then, 1, 2, 23, 2, 25] => 0, # [2.23-2.25]
[:else, 2, 2, 28, 2, 30] => 1 # [2.28-2.30]
},
[:if, 3, 2, 0, 2, 36] => { # [2.0 -2.36]
[:then, 4, 2, 12, 2, 30] => 1, # [2.12-2.30]
[:else, 5, 2, 34, 2, 36] => 0 # [2.34-2.36]
}
}
35

View Slide

NODE ͱ column ͷ͸ͳ͠
36

View Slide

NODE ͱ column ͷ͸ͳ͠
• column ͱ͸ͳʹ͔
• column ৘ใͷऔಘͱอ؅
• lexer ͱ parser
• ޻෉͕ඞཁͳ NODE
• test ʹ͍ͭͯ
37

View Slide

column ͱ͸ͳʹ͔
• ࣜ΍จͷҐஔ৘ใ͸ྫ͑͹ҎԼͷ2ͭͰදݱͰ͖Δ
• ߦ൪߸ (lineno)
• ߦ಄͔Βͷoffset (column)
a = 1
if a # [3.0-7.3]
1 # [4.2-4.3]
else
10 # [6.2-6.4]
end
38

View Slide

column ͱ͸ͳʹ͔
• ߦͷઌ಄͔Β / ϑΝΠϧͷઌ಄͔Β
• ݱঢ়͸ߦͷઌ಄͔Β
• 0-based / 1-based
• ݴޠ΍ΤσΟλʹΑͬͯҟͳΔ
• ݱঢ়͸ 0-based
• byte਺ / จࣈ਺
• ݱঢ়͸byte਺
• "ߏจ໦ʹৄࡉͳҐஔ৘ใΛ΋ͨͤΔܭը" https://bugs.ruby-lang.org/projects/ruby-trunk/wiki/Node-
position-memo
39

View Slide

column ৘ใͷऔಘͱอ؅
• Ruby 2.4·Ͱ͸columnͷ৘ใΛอ؅͍ͯ͠ͳ͔ͬͨ(linenoͷΈ
Λอ͍࣋ͯ͠Δ)
• Ruby 2.5ͰରԠͨ͜͠ͱ
• columnΛه࿥͢ΔྖҬΛ(ԕ౻͞Μ͕)֬อ(লུ)
• ద੾ͳcolumn৘ใΛॻ͖ࠐΉ
40

View Slide

column ৘ใͷऔಘͱอ؅ (Ruby 2.5)
$ ruby --dump=p -e '1 + 2'
# @ NODE_SCOPE (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 5)
# +- nd_tbl: (empty)
# +- nd_args:
# | (null node)
# +- nd_body:
# @ NODE_PRELUDE (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 5)
# +- nd_head:
# | (null node)
# +- nd_body:
# | @ NODE_OPCALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 5)
# | +- nd_mid: :+
# | +- nd_recv:
# | | @ NODE_LIT (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 1)
# | | +- nd_lit: 1
# | +- nd_args:
# | @ NODE_ARRAY (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 5)
# | +- nd_alen: 1
# | +- nd_head:
# | | @ NODE_LIT (line: 1, first_lineno: 1, first_column: 4, last_lineno: 1, last_column: 5)
# | | +- nd_lit: 2
# | +- nd_next:
# | (null node)
# +- nd_compile_option:
# +- coverage_enabled: false
42

View Slide

lexer ͱ parser
• ೖྗ͞ΕͨจࣈྻΛద੾ʹτʔΫϯʹ෼ׂ͢Δ(ࣈ۟ղੳ)
• lexer (parse.y yylex)
• ෼ׂ͞ΕͨτʔΫϯ͔ΒߏจΛ૊Έ্͛Δ(ߏจղੳ)
• parser (parse.c yyparse)
• ૊Έ্͛Δͱ͖ʹɺAST(ந৅ߏจ໦)Λͭ͘Δ
43

View Slide

lexer ͱ parser
# e.g.
1 + 2\n
44

View Slide

lexer ͱ parser (ࣈ۟ղੳ)
• ೖྗ͞ΕͨจࣈྻΛద੾ʹτʔΫϯʹ෼ׂ͢Δ(ࣈ۟ղੳ)
$ ruby --dump=y -e '1 + 2'
tINTEGER : 1
'+' : '+'
tINTEGER : 2
'\n' : '\n'
45

View Slide

lexer ͱ parser (ߏจղੳ)
• ෼ׂ͞ΕͨτʔΫϯ͔ΒߏจΛ૊Έ্͛Δ(ߏจղੳ)
$ ruby --dump=y -e '1 + 2'
46

View Slide

# Shifting token tINTEGER (1)
tINTEGER
simple_numeric
numeric
literal
primary
arg
47

View Slide

# Shifting token '+'
arg '+'
# Shifting token tINTEGER (2)
arg '+' tINTEGER
arg '+' simple_numeric
arg '+' numeric
arg '+' literal
arg '+' primary
arg '+' arg
arg
expr
stmt
top_stmt
top_stmts
48

View Slide

# Shifting token '\n'
top_stmts '\n'
top_stmts term
top_stmts terms
top_stmts opt_terms
top_compstmt
program # ׬ྃ
49

View Slide

lexer ͱ parser
• ૊Έ্͛Δͱ͖ʹɺAST(ந৅ߏจ໦)Λͭ͘Δ
$ ruby --dump=p -e '1 + 2'
NODE_SCOPE
NODE_PRELUDE
NODE_OPCALL (:+)
NODE_LIT (1)
NODE_ARRAY
NODE_LIT (2)
50

View Slide

lexer ͱ parser
• ৄ͘͠͸ parse.y ΛಡΜͰ͍ͩ͘͞
• https://github.com/ruby/ruby/blob/trunk/parse.y
51

View Slide

column ͱ͸ͳʹ͔ (ϓϩάϥϜͷ಺෦͔ΒΈ
ͨͱ͖)
• ͋ΔtokenΛೝࣝͨ͠ͱ͖ͷɺlexerͷoffset৘ใ
• tokp - lex_pbeg͓Αͼlex_p - lex_pbeg
• ࠓͷtokenΛ੾ΓऔΓऴΘͬͯɺ࣍ͷtokenΛ੾ΓऔΔલʹͲ͜
͔ʹୀආ͢Δඞཁ͕͋Δ
52

View Slide

ͨͱ͖)
/* parse.y */
/*
Structure of Lexer Buffer:
lex_pbeg tokp lex_p lex_pend
| | | |
|-----------+--------------+------------|
|<------------>|
token
*/
53

View Slide

ͨͱ͖)
• 1 + 2Ͱ+ͷtokenΛೝࣝͨ͠ॠؒ
1 + 2
^ ^^ ^
| || +- lex_pend
| |+---- lex_p
| +----- tokp
+-------- lex_pbeg
54

View Slide

columnͷऔಘํ๏
• lexer͔Βparserʹlocation৘ใΛ౉͢
• parser͔ΒNODEʹlocation৘ใΛ౉͢
55

View Slide

columnͷऔಘํ๏
• Ґஔ৘ใ͸YYLTYPEͱ͍͏ߏ଄ମͰ؅ཧ͍ͯ͠Δ
• ։࢝(ﬁrst)ͱऴྃ(last)ͷ৘ใΛอ༗͍ͯ͠Δ
#define YYLTYPE rb_code_range_t
typedef struct rb_code_location_struct {
int lineno;
int column;
} rb_code_location_t;
typedef struct rb_code_range_struct {
rb_code_location_t first_loc;
rb_code_location_t last_loc;
} rb_code_range_t;
56

View Slide

columnͷऔಘํ๏ (lexer -> parser)
•
yylexͷͳ͔ͰҐஔ৘ใΛ౎౓ୀආ͢Δ(
YYLTYPE *yylloc
)
• Ҿ਺ʹYYLTYPE *yylloc͕௥Ճ͞Εͨ
•
RUBY_SET_YYLLOCΛݺͼग़ͯ͠yyllocΛઃఆ͢Δ
57

View Slide

-static int
-yylex(YYSTYPE *lval, struct parser_params *parser)
+static enum yytokentype
+yylex(YYSTYPE *lval, YYLTYPE *yylloc, struct parser_params *parser)
{
- int t;
+ enum yytokentype t;
parser->lval = lval;
lval->val = Qundef;
t = parser_yylex(parser);
if (has_delayed_token())
dispatch_delayed_token(t);
else if (t != 0)
dispatch_scan_event(t);
+ RUBY_SET_YYLLOC(*yylloc);
+
return t;
}
parser_yylex͕࣮ࡍʹtokenΛೝࣝ͢ΔͨΊͷؔ਺
58

View Slide

#define RUBY_SET_YYLLOC(Current) \
rb_parser_set_location(parser, &(Current))
void
rb_parser_set_location(struct parser_params *parser, YYLTYPE *yylloc)
{
yylloc->first_loc.lineno = ruby_sourceline;
yylloc->first_loc.column = (int)(parser->tokp - lex_pbeg);
yylloc->last_loc.lineno = ruby_sourceline;
yylloc->last_loc.column = (int)(lex_p - lex_pbeg);
}
59

View Slide

columnͷऔಘํ๏ (parser -> NODE)
• ֤ΞΫγϣϯͰ͸Bisonͷ@nΛར༻ͯ͠tokenͷҐஔ৘ใΛऔಘ
͢Δ (YYLTYPE *yylsp)
• ओʹ࢖͏ͷ͸@$ (YYLTYPE yyloc)
60

View Slide

|---| @$ (1.0-1.5)
1 + 2
^ ^ ^
| | +- @3 (1.4-1.5)
| +--- @2 (1.2-1.3)
+----- @1 (1.0-1.1)
| arg '+' arg
{
- $$ = call_bin_op($1, '+', $3);
+ $$ = call_bin_op($1, '+', $3, &@$);
}
61

View Slide

static NODE *
call_bin_op_gen(struct parser_params *parser, NODE *recv, ID id, NODE *arg1, const YYLTYPE *location)
{
NODE *expr;
value_expr(recv);
value_expr(arg1);
expr = NEW_OPCALL(recv, id, new_list(arg1, location)); /* NODEͷੜ੒ */
fixpos(expr, recv);
expr->nd_loc = *location; /* locationΛηοτ */
return expr;
}
62

View Slide

޻෉͕͍Βͳ͍ NODE (NODE_OPCALL)
• NODE_OPCALLΛ࡞Δͱ͖ʹඞཁͳ৘ใ͕શ෦ͦΖ͍ͬͯΔ
• NODE_LIT (1)
• mid (:+)
• NODE_ARRAY (2)
1 + 2
-----------
arg '+' arg
63

View Slide

޻෉͕ඞཁͳ NODE (NODE_*ASGNܥ)
•
NODE_LASGN
(ϩʔΧϧม਺୅ೖ)΍NODE_IASGN
(Πϯελϯεม਺
୅ೖ)
@a = 1
64

View Slide

• ୅ೖͷӈล͕ܾ·Δ·͑ʹNODE_*ASGNΛੜ੒͢ΔͷͰɺӈล͕
ܾ·Δͱ͖ʹߋ৽͕ඞཁ
NODE_IASGN @a
---------- -------------
lhs : user_variable
{
$$ = assignable(var_field($1), 0, &@$);
}
=>
NODE_IASGN NODE_IASGN = 1
---------- ------------------------
arg : lhs '=' arg_rhs
{
$$ = node_assign($1, $3, &@$);
}
65

View Slide

static NODE *
node_assign_gen(struct parser_params *parser, NODE *lhs, NODE *rhs, const YYLTYPE *location)
{
if (!lhs) return 0;
switch (nd_type(lhs)) {
case NODE_GASGN:
case NODE_IASGN:
...
lhs->nd_value = rhs;
lhs->nd_loc = *location;
break;
...
}
return lhs;
}
66

View Slide

޻෉͕ඞཁͳ NODE (NODE_ITER)
•
NODE_ITER
(ϒϩοΫ෇͖ϝιουݺͼग़͠)
3.times { foo }
NODE_ITER
NODE_CALL (:times)
NODE_LIT (3)
NODE_SCOPE
NODE_VCALL (:foo)
67

View Slide

޻෉͕ඞཁͳ NODE (NODE_ITER)
• ϒϩοΫͷ෦෼ͰNODE_ITERΛੜ੒͢ΔͷͰɺ͋ͱ͔ΒNODEͷ
։࢝ҐஔΛमਖ਼͢Δඞཁ͕͋Δ
68

View Slide

3.times { foo }
NODE_CALL 3 . times
----------- --------------------------------------------------
method_call | primary_value call_op operation2 {} opt_paren_args
{
$$ = new_qcall($2, $1, $3, $5, &@$);
nd_set_line($$, $4);
}
NODE_ITER { NODE_ITER }
----------- ---------------------
brace_block : '{' {} brace_body '}'
{
$$ = $3;
nd_set_line($$, $2);
}
NODE_ITER NODE_CALL NODE_ITER
--------- -----------------------
primary | method_call brace_block
{
block_dup_check($1->nd_args, $2);
$2->nd_iter = $1;
$2->nd_loc = @$;
$$ = $2;
}
69

View Slide

test ʹ͍ͭͯ
• 2017/11/27ݱࡏɺtrunkʹtest͸commit͞Ε͍ͯͳ͍
70

View Slide

test ʹ͍ͭͯ
• test͚࣌ͩ༗ޮͳ֦ுϥΠϒϥϦΛॻ͘
• ະॳظԽͷNODEΛݕग़͢Δ
• NODEͷ਌ࢠؔ܎ΛνΣοΫ͢Δ
• NODEͷܑఋؔ܎ΛνΣοΫ͢Δ(༧ఆ)
71

View Slide

test͚࣌ͩ༗ޮͳ֦ுϥΠϒϥϦΛॻ͘
node = AST.parse("1 + 2")
# => #
node.children[1].children[0]
# => #
node.children[1].children[0].children
# => [#, #]
72

View Slide

ະॳظԽͷNODEΛݕग़͢Δ
• linenoΛ0, columnΛ-1ͰॳظԽͯ͋͠Δ
NODE_IF (line: 1, first_lineno: 0, first_column: -1, last_lineno: 0, last_column: -1)
73

View Slide

ະॳظԽͷNODEΛݕग़͢Δ (ݱঢ়)
• r60918 (2017/11/27)
• "test/ruby/*.rb"Λର৅ ->
!
$ ruby ast_validation.rb
"Total 0 count errors."
74

View Slide

NODEͷ਌ࢠؔ܎ΛνΣοΫ͢Δ
3.times { foo }
NODE_ITER [1.0-1.15] -> [1.0-1.13]ΛؚΉ
NODE_CALL (:times) [1.0-1.9] -> [1.0-1.1]ΛؚΉ
NODE_LIT (3) [1.0-1.1]
NODE_SCOPE [1.8-1.15] -> [1.10-1.13]ΛؚΉ
NODE_VCALL (:foo) [1.10-1.13]
75

View Slide

NODEͷ਌ࢠؔ܎ΛνΣοΫ͢Δ (ݱঢ়)
• r60918 (2017/11/27)
• "test/ruby/*.rb"Λର৅ ->
!
$ ruby ast_validation.rb
"test/ruby/beginmainend.rb has 1 errors"
[["NODE_ARRAY", 1]]
"test/ruby/sentence.rb has 1 errors"
[["NODE_ARGS", 1]]
...
"test/ruby/test_yield.rb has 1 errors"
[["NODE_ARGS", 1]]
"Total 552 count errors."
76

View Slide

• "NODE_ARRAY"΍"NODE_DSTR"ͳͲ͕໰୊
• "NODE_ARRAY"
• ྫ͑͹ [1,2,3]
77

View Slide

• ຤ඌ௥هͷͨͼʹશ෦ͷNODEͷlast locationΛߋ৽͢Δʁ
78

View Slide

__________ ___________
|NODE_ARRAY| |NODE_LIT(1)|
|__________| ----> |___________|
79

View Slide

__________ ___________
|NODE_ARRAY| |NODE_LIT(1)| # Ґஔ৘ใͷߋ৽͕ඞཁ
|__________| ----> |___________|
|
_____v____ ___________
|__________| ----> |___________|
80

View Slide

__________ ___________
|__________| ----> |___________|
|
_____v____ ___________
|__________| ----> |___________|
|
_____v____ ___________
|__________| ----> |___________|
81

View Slide

• ݱঢ়͸node.cಉ༷ʹ్தͷNODE_ARRAYΛඈ͹͍ͯ͠Δ
node = AST.parse("[1, 2, 3]")
# => #
node.children[1].children[0]
# => #
node.children[1].children[0].children
# => [
# #,
# #,
# #, nil
# ]
82

View Slide

NODEͷܑఋؔ܎ΛνΣοΫ͢Δ(༧ఆ)
1 + 2
NODE_OPCALL (:+) [1.0-1.5] -> (1)ͱ(2)͕ඃ͍ͬͯͳ͍
NODE_LIT (1) [1.0-1.1] -> (1)
NODE_ARRAY [1.4-1.5] -> (2)
NODE_LIT (2) [1.4-1.5]
83

View Slide

• "NODE_OP_ASGN*"ܥ͕໰୊
• ྫ͑͹ @a ||= 1 ("NODE_OP_ASGN_OR")
84

View Slide

NODE_OP_ASGN_OR [1.0-1.8] -> (1)ͱ(2)͕ඃ͍ͬͯΔ
NODE_IVAR (@a) [1.0-1.2] -> (1)
NODE_IASGN (@a) [1.0-1.8] -> (2)
NODE_LIT (1) [1.7-1.8]
@a ||= 1
-- <-- NODE_IVAR
-------- <-- NODE_IASGN
85

View Slide

·ͱΊ
86

View Slide

·ͱΊ
• Ruby 2.5Ͱ͸Branch CoverageͱMethod Coverage͕৽ͨʹಋೖ
͞ΕΔ
• Ruby 2.5Ͱ͸NODE͕։࢝ɾऴྃҐஔΛ࣋ͭΑ͏ʹͳΔ
• Ruby 2.5ͷϦϦʔε·Ͱʹ͍Ζ͍Ζमਖ਼Λ͍ΕΔ
87

View Slide

ँࣙ
• @mametter
• @nobu
• @ko1
• @shyouhei
• @takeshinoda
• @hkdnet
• @HaiTo
• @littlestarling
88

View Slide

Ruby 2.5ϦϦʔε·ͰͷಓͷΓ
• testΛcommit͢Δ
• test͕௨ΔΑ͏ʹ͢Δ
• Method CoverageͰ΋column͕ͱΕΔΑ͏ʹ͢Δ
89

View Slide

ࠓޙͷ֦ு(ͷՄೳੑ)
• &&΍||ͷΧόϨοδ΋ͱΕΔΑ͏ʹ͢Δ
• NoMethodErrorͷ৘ใΛ૿΍͢
• power_assertͷ৘ใΛ૿΍͢
90

View Slide

ࢀߟ৘ใ
• "Add branch coverage" https://bugs.ruby-lang.org/issues/13901
• "ߏจ໦ʹৄࡉͳҐஔ৘ใΛ΋ͨͤΔܭը" https://bugs.ruby-
lang.org/projects/ruby-trunk/wiki/Node-position-memo
• https://www.gnu.org/software/bison/manual/html_node/Token-
Locations.html#Token-Locations
• http://i.loveruby.net/ja/rhg/book/ ͷ"ୈ 2 ෦ʮߏจղੳʯ"
91

View Slide

͓ΘΓ
92

View Slide

͓·͚
93

View Slide

heredocʹ͍ͭͯ
94

View Slide

heredocʹ͍ͭͯ
• heredocศརͰ͢ΑͶ
def upcase(s)
s.upcase
end
upcase(<abcde
STR1
# => "ABCDE\n"
95

View Slide

heredocʹ͍ͭͯ
• heredocศར...
def concat(s, t)
s + t
end
concat(<abcde
STR1
12345
STR2
# => "abcde\n12345\n"
96

View Slide

heredocʹ͍ͭͯ
• Ͳ͜·Ͱ͕heredocͷൣғʁ
def concat(s, t)
s + t
end
concat(<abcde # ͔͜͜Β
fghij # ͜͜·Ͱͱ
STR1
12345 # ͔͜͜Β
67890 # ͜͜·Ͱʁ
STR2
97

View Slide

ఏҊ
• heredocͷNODEͷൣғΛ <• test͕༰қʹͳΔ
• ࣮ࡍͷจࣈྻͷൣғ͸୅දҐஔ(ະ࣮૷)ͳͲͰද͢
concat(<abcde
fghij
STR1
12345
67890
STR2
98

View Slide

heredocͷ࢓૊Έ
• ·ͣී௨ʹconcat(·ͰಡΉ
concat(
99

View Slide

heredocͷ࢓૊Έ
• <concat(<100

View Slide

heredocͷ࢓૊Έ
• STR1·ͰಡΈࠐΜͰɺheredocϞʔυΛൈ͚Δ
concat(<abcde
fghij
STR1
101

View Slide

heredocͷ࢓૊Έ
• ,ΛಡΉ
concat(<abcde
fghij
STR1
102

View Slide

heredocͷ࢓૊Έ
• <concat(<abcde
fghij
STR1
103

View Slide

heredocͷ࢓૊Έ
• STR2·ͰಡΈࠐΜͰɺheredocϞʔυΛൈ͚Δ
concat(<abcde
fghij
STR1
12345
67890
STR2
104

View Slide

heredocͷ࢓૊Έ
• lex_strterm ͷঢ়ଶʹΑͬͯɺݱࡏheredocΛղੳ͍ͯ͠Δ͔൑
அ͍ͯ͠Δ
static enum yytokentype
parser_yylex(struct parser_params *parser)
{
...
if (lex_strterm) {
if (lex_strterm->flags & STRTERM_HEREDOC) {
return here_document(&lex_strterm->u.heredoc);
}
else {
token_flush(parser);
return parse_string(&lex_strterm->u.literal);
}
}
...
switch (c = nextc()) {
case '\0': /* NUL */
case '\004': /* ^D */
...
105

View Slide

heredocͷ࢓૊Έ
• lex_strterm Λઃఆ͍ͯ͠Δίʔυ
(parser_heredoc_identifier)
• lex_goto_eol Ͱߦ຤·Ͱjump͍ͯ͠Δ
• ͜ΕʹΑͬͯ࣍ͷtokenऔಘ࣌ʹ࣍ͷߦ͔Β͸͡ΊΔ͜ͱ͕Ͱ
͖Δ
• lex_strterm ʹheredocΛൈ͚ͨ࣌ʹඞཁͳ৘ใΛୀආ͢Δ
106

View Slide

heredocͷ࢓૊Έ
static enum yytokentype
parser_heredoc_identifier(struct parser_params *parser)
{
...
lex_goto_eol(parser); /* ߦ຤·Ͱjump͍ͯ͠Δ */
lex_strterm = (rb_strterm_t*)rb_imemo_new(imemo_parser_strterm,
STR_NEW(tok(), toklen()), /* term */
lex_lastline, /* lastline */
len, /* lastidx */
ruby_sourceline); /* ෮ؼޙʹඞཁͳ৘ใΛୀආ */
lex_strterm->flags |= STRTERM_HEREDOC;
token_flush(parser);
heredoc_indent = indent;
heredoc_line_indent = 0;
return token; /* ͨͱ͑͹ɺtSTRING_BEG */
}
107

View Slide

ఏҊ (࣮૷)
• lex_strterm ʹୀආͯ͋͠Δ৘ใΛ࢖͑͹ɺ <͔Δ͸ͣ
108

View Slide

#define YYLLOC_DEFAULT(Current, Rhs, N) \
do \
if (N) \
- { \
- (Current).first_loc = YYRHSLOC(Rhs, 1).first_loc; \
- (Current).last_loc = YYRHSLOC(Rhs, N).last_loc; \
- } \
+ if (lex_strterm && (lex_strterm->flags & STRTERM_HEREDOC)) \
+ { \
+ rb_strterm_heredoc_t here = lex_strterm->u.heredoc; \
+ (Current).first_loc.lineno = (int)here.sourceline; \
+ (Current).first_loc.column = (int)(here.u3.lastidx - RSTRING_LEN(here.term));\
+ (Current).last_loc.lineno = (int)here.sourceline; \
+ (Current).last_loc.column = (int)(here.u3.lastidx);\
+ } \
+ else \
+ { \
+ (Current).first_loc = YYRHSLOC(Rhs, 1).first_loc; \
+ (Current).last_loc = YYRHSLOC(Rhs, N).last_loc; \
+ } \
else \
RUBY_SET_YYLLOC(Current); \
while (0)
109

View Slide

ͩΊͦ͏ͳέʔε (1)
"#{<<~"A"}"
#{:b if :a}
f = 11
A
110

View Slide

ͩΊͦ͏ͳέʔε (2)
"#{<<~"A"}"
#{<<~B}
12
13
B
f = 11
A
111

View Slide

parse.y͸ѱຐ৓ͳͷ͔ʁ
• ҎԼͷ఺Λআ͚͹ɺͦ͜·Ͱѱຐ৓Ͱ͸ͳ͍ͷͰ͸?
• Ұ෦ripperͱϑΝΠϧΛڞ༗͍ͯ͠Δ
• ߦ਺͕ଟ͍
• parser_paramsʹ͍Ζ͍Ζͳϑϥά͕͋Δ
• lex_stateʹ͍Ζ͍Ζͳϑϥά͕͋Δ
• Rubyʹ͸heredoc͕͋Δ
112

View Slide

Coverage.so, or RNode with column

Coverage.so, or RNode with column

yui-knk

More Decks by yui-knk

Other Decks in Programming

Featured

Transcript