Slide 1
Slide 1 text
POWER ASSERT IN
RUBY
Kazuki Tsujimoto
Nomura Research Institute, LTD.
Slide 2
Slide 2 text
SELF INTRODUCTION
• A CRuby committer
• Fix VM, GC bugs
Slide 3
Slide 3 text
AGENDA
• Concept (13 slides)
• Implementation (39 slides)
Slide 4
Slide 4 text
PROPOSAL NOTES
• I'll give unduly deep explanation; not only of
Ruby-level but VM/C-level
Slide 5
Slide 5 text
CONCEPT
Slide 6
Slide 6 text
TEST FRAMEWORKS
unit spec
test-unit ✔️
minitest ✔️ ✔️
RSpec ✔️
Slide 7
Slide 7 text
TEST FRAMEWORKS
unit spec
test-unit ✔️
minitest ✔️ ✔️
RSpec ✔️
Slide 8
Slide 8 text
TESTING WITH UNIT
• Use assertions
i = 0
assert_equal 0, i
assert_include 3.times.to_a, 1
Slide 9
Slide 9 text
TESTING WITH UNIT
• What is the difference?
assert_include 3.times.to_a, 3
assert 3.times.to_a.include?(3)
Slide 10
Slide 10 text
TESTING WITH UNIT
• Understandability of the report
Failure:
<[0, 1, 2]> expected to include
<3>.
Failure:
expected but was
Slide 11
Slide 11 text
TESTING WITH UNIT
• A number of assertions
assert assert_alias_method assert_block assert_boolean
assert_compare assert_const_defined assert_empty assert_equal
a s s e r t _ f a i l _ a s s e r t i o n a s s e r t _ f a l s e a s s e r t _ i n _ d e l t a
assert_in_epsilon assert_include assert_instance_of assert_kind_of
assert_match assert_nil assert_no_match assert_not_const_defined
assert_not_empty assert_not_equal assert_not_in_delta
assert_not_in_epsilon assert_not_include assert_not_instance_of
a s s e r t _ n o t _ k i n d _ o f a s s e r t _ n o t _ m a t c h a s s e r t _ n o t _ n i l
assert_not_operator assert_not_predicate assert_not_respond_to
assert_not_same assert_not_send assert_nothing_raised
assert_nothing_thrown assert_operator assert_path_exist
a s s e r t _ p a t h _ n o t _ e x i s t a s s e r t _ p r e d i c a t e a s s e r t _ r a i s e
assert_raise_kind_of assert_raise_message assert_respond_to
assert_same assert_send assert_throw assert_true
Slide 12
Slide 12 text
TESTING WITH
POWER ASSERT
• Use assert with a block
• That's all
assert { 3.times.to_a.include?(3) }
Failure:
assert { 3.times.to_a.include?(3) }
| | |
| | false
| [0, 1, 2]
#
Slide 13
Slide 13 text
POWER ASSERT IN GROOVY
http://docs.codehaus.org/display/GROOVY/Groovy+1.7+release+notes
Slide 14
Slide 14 text
POWER_ASSERT FAMILY
• power_assert
• Requires CRuby 2.0.0 or later
• test-unit-power_assert
• test-unit 3(@ktou)
• minitest-power_assert(@hsbt)
• pry-power_assert(@spikeolaf)
Slide 15
Slide 15 text
DEBUGGING WITH
POWER ASSERT
• tap is frequently used for inspecting the
value in method chains
3.times.to_a.tap {|i| p i }.include?(3)
#=> [0, 1, 2]
Slide 16
Slide 16 text
DEBUGGING WITH
POWER ASSERT
• Extend p with power_assert to inspect all
values
p { 3.times.to_a.include?(3) }
p { 3.times.to_a.include?(3) }
| | |
| | false
| [0, 1, 2]
#
Slide 17
Slide 17 text
DEBUGGING WITH
POWER ASSERT
module PowerP
def p(*, &blk)
if blk
PowerAssert.start(blk, assertion_method: __callee__) do |pa|
pa.yield
puts pa.message_proc.call
end
else
super
end
end
end
Slide 18
Slide 18 text
IMPLEMENTATION
Slide 19
Slide 19 text
THE CHALLENGES OF
IMPLEMENTATION
• How to get values
• How to get position information
3.to_s.length
| |
| 1
"3"
to_s
“3”
length
1
Values
to_s
2
length
7
Position info
Slide 20
Slide 20 text
HOW TO GET
VALUES
Slide 21
Slide 21 text
TRACEPOINT
Slide 22
Slide 22 text
WHAT IS TRACEPOINT?
• Tracing API which aims to replace
set_trace_func
• Introduced in Ruby 2.0
Slide 23
Slide 23 text
TRACEPOINT
trace = TracePoint.new(:return, :c_return) {|tp|
p id: tp.method_id, return_value: tp.return_value
}
trace.enable {
3.times.to_a
}
{:id=>:times, :return_value=>#}
{:id=>:times, :return_value=>3}
{:id=>:each, :return_value=>3}
{:id=>:to_a, :return_value=>[0, 1, 2]}
Slide 24
Slide 24 text
TRACEPOINT
trace = TracePoint.new(:return, :c_return) {|tp|
p id: tp.method_id, return_value: tp.return_value
}
trace.enable {
3.times.to_a
}
{:id=>:times, :return_value=>#}
{:id=>:times, :return_value=>3}
{:id=>:each, :return_value=>3}
{:id=>:to_a, :return_value=>[0, 1, 2]}
New event info
• return_value
• raised_exception
!
Traditional event info
• method_id
• binding
• etc.
Slide 25
Slide 25 text
OFF TOPIC:
EXCEPTION#CAUSE
• Exception#cause returns the previous
exception at the time the exception was
raised
begin
begin
raise "A"
rescue
raise "B"
end
rescue => e
p [e, e.cause]
# => [#, #]
end
Slide 26
Slide 26 text
OFF TOPIC:
EXCEPTION#CAUSE
• Implement Exception#cause using TracePoint
class Exception
attr_accessor :cause
TracePoint.trace(:raise){|tp|
e = tp.raised_exception
e.cause = $!
}
end
https://twitter.com/_ko1/statuses/409267459414179840
Slide 27
Slide 27 text
FILTERING RETURN VALUES
• TracePoint captures return values of all
method calls
• We must filter them
trace = TracePoint.new(:return, :c_return) {|tp|
p id: tp.method_id, return_value: tp.return_value
}
trace.enable {
3.times.to_a
}
{:id=>:times, :return_value=>#}
{:id=>:times, :return_value=>3}
{:id=>:each, :return_value=>3}
{:id=>:to_a, :return_value=>[0, 1, 2]}
Slide 28
Slide 28 text
FILTERING RETURN VALUES
• TracePoint captures return values of all
method calls
• We must filter them
trace = TracePoint.new(:return, :c_return) {|tp|
p id: tp.method_id, return_value: tp.return_value
}
trace.enable {
3.times.to_a
}
{:id=>:times, :return_value=>#}
{:id=>:times, :return_value=>3}
{:id=>:each, :return_value=>3}
{:id=>:to_a, :return_value=>[0, 1, 2]}
Slide 29
Slide 29 text
FILTERING RETURN VALUES
• Use Binding#eval and
Kernel#caller_locations
• tp.binding.eval(‘caller_locations’).length
returns the execution stack length of each
method calls
Slide 30
Slide 30 text
FILTERING RETURN VALUES
{:id=>:times, :return_value=>#, :loc_len=>5}
{:id=>:times, :return_value=>3, :loc_len=>7}
{:id=>:each, :return_value=>3, :loc_len=>6}
{:id=>:to_a, :return_value=>[0, 1, 2], :loc_len=>5}
trace = TracePoint.new(:return, :c_return) {|tp|
p tp.event, id: tp.method_id, return_value: tp.return_value,
loc_len: tp.binding.eval('caller_locations').length
}
trace.enable {
3.times.to_a
}
Slide 31
Slide 31 text
FILTERING RETURN VALUES
{:id=>:times, :return_value=>#, :loc_len=>5}
{:id=>:times, :return_value=>3, :loc_len=>7}
{:id=>:each, :return_value=>3, :loc_len=>6}
{:id=>:to_a, :return_value=>[0, 1, 2], :loc_len=>5}
trace = TracePoint.new(:return, :c_return) {|tp|
p tp.event, id: tp.method_id, return_value: tp.return_value,
loc_len: tp.binding.eval('caller_locations').length
}
trace.enable {
3.times.to_a
}
Slide 32
Slide 32 text
FILTERING RETURN VALUES
{:id=>:times, :return_value=>#, :loc_len=>5}
{:id=>:times, :return_value=>3, :loc_len=>7}
{:id=>:each, :return_value=>3, :loc_len=>6}
{:id=>:to_a, :return_value=>[0, 1, 2], :loc_len=>5}
trace = TracePoint.new(:return, :c_return) {|tp|
p tp.event, id: tp.method_id, return_value: tp.return_value,
loc_len: tp.binding.eval('caller_locations').length
}
trace.enable {
3.times.to_a
}
Slide 33
Slide 33 text
GETTING THE VALUE OF THE
VARIABLE
• Binding#eval is also used to get the value of
the variable
ary = 3.times.to_a
assert { ary.include?(3) }
Failure:
assert { ary.include?(3) }
| |
| false
[0, 1, 2]
Slide 34
Slide 34 text
TRAP OF POWER_ASSERT
• The report does not contain the value of ==
require 'test-unit-power_assert'
...
assert { [0, 1, 2].find {|i| i.odd? } == 3 }
$ ruby ./test.rb
Failure:
assert { [0, 1, 2].find {|i| i.odd? } == 3 }
|
1
Slide 35
Slide 35 text
WHAT SHOULD WE DO?
• Require power_assert first
$ ruby -rpower_assert ./test.rb
Failure:
assert { [0, 1, 2].find {|i| i.odd? } == 3 }
| |
| false
1
Slide 36
Slide 36 text
WHAT HAPPENED?
• Minimum code for the problem
trace = TracePoint.new(:return, :c_return) {|tp|
p tp.return_value
}
trace.enable {
1 == 3
}
(Expected)
false
(Actual)
Slide 37
Slide 37 text
INVESTIGATING BYTECODE
• Use RubyVM::InstructionSequence.disasm
puts RubyVM::InstructionSequence.disasm(Proc.new {
[0, 1, 2].find {|i| i.odd? } == 3
})
0000 trace 256
0002 trace 1
0004 duparray [0, 1, 2]
0006 send
0008 putobject 3
0010 opt_eq
0012 trace 512
0014 leave
Slide 38
Slide 38 text
INVESTIGATING BYTECODE
• Use RubyVM::InstructionSequence.disasm
puts RubyVM::InstructionSequence.disasm(Proc.new {
[0, 1, 2].find {|i| i.odd? } == 3
})
0000 trace 256
0002 trace 1
0004 duparray [0, 1, 2]
0006 send
0008 putobject 3
0010 opt_eq
0012 trace 512
0014 leave
Slide 39
Slide 39 text
OPT_EQ
{
val = opt_eq_func(recv, obj, ci);
if (val == Qundef) {
/* other */
PUSH(recv);
PUSH(obj);
CALL_SIMPLE_METHOD(recv);
}
}
Slide 40
Slide 40 text
OPT_EQ_FUNC
VALUE
opt_eq_func(VALUE recv, VALUE obj, CALL_INFO ci)
{
if (FIXNUM_2_P(recv, obj) &&
BASIC_OP_UNREDEFINED_P(BOP_EQ,
FIXNUM_REDEFINED_OP_FLAG)) {
return (recv == obj) ? Qtrue : Qfalse;
}
...
Slide 41
Slide 41 text
SEND
{
ci->argc = ci->orig_argc;
ci->blockptr = 0;
vm_caller_setup_args(th, reg_cfp, ci);
vm_search_method(ci, ci->recv = TOPN(ci->argc));
CALL_METHOD(ci);
}
Slide 42
Slide 42 text
VM_CALL_CFUNC
static VALUE
vm_call_cfunc(rb_thread_t *th, rb_control_frame_t
*reg_cfp, rb_call_info_t *ci)
{
...
val = vm_call_cfunc_latter(th, reg_cfp, ci);
EXEC_EVENT_HOOK(th, RUBY_EVENT_C_RETURN, recv,
me->called_id, me->klass, val);
...
Slide 43
Slide 43 text
VM_CALL_CFUNC
static VALUE
vm_call_cfunc(rb_thread_t *th, rb_control_frame_t
*reg_cfp, rb_call_info_t *ci)
{
...
val = vm_call_cfunc_latter(th, reg_cfp, ci);
EXEC_EVENT_HOOK(th, RUBY_EVENT_C_RETURN, recv,
me->called_id, me->klass, val);
...
TracePoint is invoked by
EXEC_EVENT_HOOK
Slide 44
Slide 44 text
DISABLE VM OPTIMIZATION
• Set optimization flags using
RubyVM::InstructionSequence.compile_opti
on= method
• power_assert contains following code
RubyVM::InstructionSequence.compile_option = {
specialized_instruction: false
}
Slide 45
Slide 45 text
DISABLE VM OPTIMIZATION
• Only power_assert is compiled with
optimization
$ ruby -rpower_assert ./test.rb
Compile option
Test code
power_assert
Optimization: Enabled
Compile Evaluate
Disabled
Compile Evaluate
Slide 46
Slide 46 text
DISABLE VM OPTIMIZATION
• Both power_assert and test code are
compiled with optimization
$ ruby ./test.rb
Compile option
Test code
power_assert
Optimization: Enabled
Compile Evaluate
Compile Evaluate
Evaluate
Disabled
Slide 47
Slide 47 text
HOW TO GET POSITION
INFORMATION
Slide 48
Slide 48 text
RIPPER
Slide 49
Slide 49 text
WHAT IS RIPPER?
• A Ruby script parser
• Ripper.sexp create S-exp tree including
position information
# assert { [0, 1, 2].find {|i| i.odd? } == 3 }
[:program,
[[:method_add_block,
[:method_add_arg, [:fcall, [:@ident, "assert", [1, 0]]], []],
[:brace_block, nil,
[[:binary,
[:method_add_block,
[:call,
[:array,ɹ[[:@int, "0", [1, 10]], [:@int, "1", [1, 13]], [:@int, "2", [1, 16]]]], :".",
[:@ident, "find", [1, 19]]],
[:brace_block,
[:block_var, [:params, [[:@ident, "i", [1, 26]]], nil, nil, nil, nil, nil, nil], false],
[[:call, [:var_ref, [:@ident, "i", [1, 29]]], :”.", [:@ident, "odd?", [1, 31]]]]]],
:==,
[:@int, "3", [1, 41]]]]]]]]
Slide 50
Slide 50 text
WHAT IS RIPPER?
• A Ruby script parser
• Ripper.sexp create S-exp tree including
position information
# assert { [0, 1, 2].find {|i| i.odd? } == 3 }
[:program,
[[:method_add_block,
[:method_add_arg, [:fcall, [:@ident, "assert", [1, 0]]], []],
[:brace_block, nil,
[[:binary,
[:method_add_block,
[:call,
[:array,ɹ[[:@int, "0", [1, 10]], [:@int, "1", [1, 13]], [:@int, "2", [1, 16]]]], :".",
[:@ident, "find", [1, 19]]],
[:brace_block,
[:block_var, [:params, [[:@ident, "i", [1, 26]]], nil, nil, nil, nil, nil, nil], false],
[[:call, [:var_ref, [:@ident, "i", [1, 29]]], :”.", [:@ident, "odd?", [1, 31]]]]]],
:==,
[:@int, "3", [1, 41]]]]]]]]
"assert"
Slide 51
Slide 51 text
EXTRACT
POSITION INFORMATION
• Use multiple assignment
tag, * = sexp
case tag
when :program
_, ((tag0, (tag1, (tag2, (tag3, mname, _)), _),
(tag4, _, ss))) = sexp
if tag0 == :method_add_block and tag1 == :method_add_arg and
tag2 == :fcall and tag3 == :@ident and
mname == @assertion_method_name and
(tag4 == :brace_block or tag4 == :do_block)
ss.flat_map {|s| extract_idents(s) }
else
...
Slide 52
Slide 52 text
EXTRACT
POSITION INFORMATION
• In early versions, use pattern matching
instead
match(sexp) do
with(_[:program,
_[_[:method_add_block,
_[:method_add_arg,
_[:fcall, _[:@ident, assertion_method.to_s, _]], _],
_[Or(:brace_block, :do_block), _, ss]]]]) do
ss.flat_map {|s| extract_idents(s) }
end
...
Slide 53
Slide 53 text
OFF TOPIC:
PATTERN MATCHING IN RUBY
• Gems have been released one after another
lately
• k-tsj/pattern-match
• egison/egison-ruby
• todesking/patm
• I hope Ruby supports pattern matching as a
core feature
• Welcome great ideas
Slide 54
Slide 54 text
CORNER CASES/LIMITATIONS
# Case A
assert { obj.foo }
# Case B
assert {
obj.foo
}
# Case C
assert {
# comment
obj.foo
}
# Case D
assert {
obj.foo
obj.bar
}
# Case E
foo { assert { obj.bar } }
Which expressions will be
power asserted?
Slide 55
Slide 55 text
CORNER CASES/LIMITATIONS
# Case A
assert { obj.foo }
# Case B
assert {
obj.foo
}
# Case C
assert {
# comment
obj.foo
}
# Case D
assert {
obj.foo
obj.bar
}
# Case E
foo { assert { obj.bar } }
obj.foo
obj.foo
obj.foo
obj.foo
Which expressions will be
power asserted?
Slide 56
Slide 56 text
CORNER CASES/LIMITATIONS
• Reassignment of variables
• LHS is 0 and RHS is 1, but eval('i') is 1
!
!
• Branch
• TracePoint does not know which obj.foo is
called
assert { (i = 0) == (i = 1) }
assert { cond ? obj.foo : obj.foo }
Slide 57
Slide 57 text
CONCLUSION
Slide 58
Slide 58 text
CONCLUSION
• Enjoy programming with power_assert
• Enjoy programming with TracePoint