Power Assert in Ruby

POWER ASSERT IN RUBY Kazuki Tsujimoto Nomura Research Institute, LTD.

SELF INTRODUCTION • A CRuby committer • Fix VM, GC
bugs

AGENDA • Concept (13 slides) • Implementation (39 slides)

PROPOSAL NOTES • I'll give unduly deep explanation; not only
of Ruby-level but VM/C-level

CONCEPT

TEST FRAMEWORKS unit spec test-unit ✔️ minitest ✔️ ✔️ RSpec
✔️

TESTING WITH UNIT • Use assertions i = 0 assert_equal
0, i assert_include 3.times.to_a, 1

TESTING WITH UNIT • What is the difference? assert_include 3.times.to_a,
3 assert 3.times.to_a.include?(3)

TESTING WITH UNIT • Understandability of the report Failure: <[0,
1, 2]> expected to include <3>. Failure: <true> expected but was <false>

TESTING WITH UNIT • A number of assertions assert assert_alias_method
assert_block assert_boolean assert_compare assert_const_defined assert_empty assert_equal a s s e r t _ f a i l _ a s s e r t i o n a s s e r t _ f a l s e a s s e r t _ i n _ d e l t a assert_in_epsilon assert_include assert_instance_of assert_kind_of assert_match assert_nil assert_no_match assert_not_const_defined assert_not_empty assert_not_equal assert_not_in_delta assert_not_in_epsilon assert_not_include assert_not_instance_of a s s e r t _ n o t _ k i n d _ o f a s s e r t _ n o t _ m a t c h a s s e r t _ n o t _ n i l assert_not_operator assert_not_predicate assert_not_respond_to assert_not_same assert_not_send assert_nothing_raised assert_nothing_thrown assert_operator assert_path_exist a s s e r t _ p a t h _ n o t _ e x i s t a s s e r t _ p r e d i c a t e a s s e r t _ r a i s e assert_raise_kind_of assert_raise_message assert_respond_to assert_same assert_send assert_throw assert_true

TESTING WITH POWER ASSERT • Use assert with a block
• That's all assert { 3.times.to_a.include?(3) } Failure: assert { 3.times.to_a.include?(3) } | | | | | false | [0, 1, 2] #<Enumerator: 3:times>

POWER ASSERT IN GROOVY http://docs.codehaus.org/display/GROOVY/Groovy+1.7+release+notes

POWER_ASSERT FAMILY • power_assert • Requires CRuby 2.0.0 or later
• test-unit-power_assert • test-unit 3(@ktou) • minitest-power_assert(@hsbt) • pry-power_assert(@spikeolaf)

DEBUGGING WITH POWER ASSERT • tap is frequently used for
inspecting the value in method chains 3.times.to_a.tap {|i| p i }.include?(3) #=> [0, 1, 2]

DEBUGGING WITH POWER ASSERT • Extend p with power_assert to
inspect all values p { 3.times.to_a.include?(3) } p { 3.times.to_a.include?(3) } | | | | | false | [0, 1, 2] #<Enumerator: 3:times>

DEBUGGING WITH POWER ASSERT module PowerP def p(*, &blk) if
blk PowerAssert.start(blk, assertion_method: __callee__) do |pa| pa.yield puts pa.message_proc.call end else super end end end

IMPLEMENTATION

THE CHALLENGES OF IMPLEMENTATION • How to get values •
How to get position information 3.to_s.length | | | 1 "3" to_s “3” length 1 Values to_s 2 length 7 Position info

HOW TO GET VALUES

TRACEPOINT

WHAT IS TRACEPOINT? • Tracing API which aims to replace
set_trace_func • Introduced in Ruby 2.0

TRACEPOINT trace = TracePoint.new(:return, :c_return) {|tp| p id: tp.method_id, return_value:
tp.return_value } trace.enable { 3.times.to_a } {:id=>:times, :return_value=>#<Enumerator: 3:times>} {:id=>:times, :return_value=>3} {:id=>:each, :return_value=>3} {:id=>:to_a, :return_value=>[0, 1, 2]}

TRACEPOINT trace = TracePoint.new(:return, :c_return) {|tp| p id: tp.method_id, return_value:
tp.return_value } trace.enable { 3.times.to_a } {:id=>:times, :return_value=>#<Enumerator: 3:times>} {:id=>:times, :return_value=>3} {:id=>:each, :return_value=>3} {:id=>:to_a, :return_value=>[0, 1, 2]} New event info • return_value • raised_exception ! Traditional event info • method_id • binding • etc.

OFF TOPIC: EXCEPTION#CAUSE • Exception#cause returns the previous exception at
the time the exception was raised begin begin raise "A" rescue raise "B" end rescue => e p [e, e.cause] # => [#<RuntimeError: B>, #<RuntimeError: A>] end

OFF TOPIC: EXCEPTION#CAUSE • Implement Exception#cause using TracePoint class Exception
attr_accessor :cause TracePoint.trace(:raise){|tp| e = tp.raised_exception e.cause = $! } end https://twitter.com/_ko1/statuses/409267459414179840

FILTERING RETURN VALUES • TracePoint captures return values of all
method calls • We must ﬁlter them trace = TracePoint.new(:return, :c_return) {|tp| p id: tp.method_id, return_value: tp.return_value } trace.enable { 3.times.to_a } {:id=>:times, :return_value=>#<Enumerator: 3:times>} {:id=>:times, :return_value=>3} {:id=>:each, :return_value=>3} {:id=>:to_a, :return_value=>[0, 1, 2]}

FILTERING RETURN VALUES • Use Binding#eval and Kernel#caller_locations • tp.binding.eval(‘caller_locations’).length
returns the execution stack length of each method calls

FILTERING RETURN VALUES {:id=>:times, :return_value=>#<Enumerator: 3:times>, :loc_len=>5} {:id=>:times, :return_value=>3, :loc_len=>7}
{:id=>:each, :return_value=>3, :loc_len=>6} {:id=>:to_a, :return_value=>[0, 1, 2], :loc_len=>5} trace = TracePoint.new(:return, :c_return) {|tp| p tp.event, id: tp.method_id, return_value: tp.return_value, loc_len: tp.binding.eval('caller_locations').length } trace.enable { 3.times.to_a }

GETTING THE VALUE OF THE VARIABLE • Binding#eval is also
used to get the value of the variable ary = 3.times.to_a assert { ary.include?(3) } Failure: assert { ary.include?(3) } | | | false [0, 1, 2]

TRAP OF POWER_ASSERT • The report does not contain the
value of == require 'test-unit-power_assert' ... assert { [0, 1, 2].find {|i| i.odd? } == 3 } $ ruby ./test.rb Failure: assert { [0, 1, 2].find {|i| i.odd? } == 3 } | 1

WHAT SHOULD WE DO? • Require power_assert ﬁrst $ ruby
-rpower_assert ./test.rb Failure: assert { [0, 1, 2].find {|i| i.odd? } == 3 } | | | false 1

WHAT HAPPENED? • Minimum code for the problem trace =
TracePoint.new(:return, :c_return) {|tp| p tp.return_value } trace.enable { 1 == 3 } (Expected) false (Actual)

INVESTIGATING BYTECODE • Use RubyVM::InstructionSequence.disasm puts RubyVM::InstructionSequence.disasm(Proc.new { [0, 1,
2].find {|i| i.odd? } == 3 }) 0000 trace 256 0002 trace 1 0004 duparray [0, 1, 2] 0006 send <callinfo!mid:find, argc:0, block:block> 0008 putobject 3 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SKIP> 0012 trace 512 0014 leave

OPT_EQ { val = opt_eq_func(recv, obj, ci); if (val ==
Qundef) { /* other */ PUSH(recv); PUSH(obj); CALL_SIMPLE_METHOD(recv); } }

OPT_EQ_FUNC VALUE opt_eq_func(VALUE recv, VALUE obj, CALL_INFO ci) { if
(FIXNUM_2_P(recv, obj) && BASIC_OP_UNREDEFINED_P(BOP_EQ, FIXNUM_REDEFINED_OP_FLAG)) { return (recv == obj) ? Qtrue : Qfalse; } ...

SEND { ci->argc = ci->orig_argc; ci->blockptr = 0; vm_caller_setup_args(th, reg_cfp,
ci); vm_search_method(ci, ci->recv = TOPN(ci->argc)); CALL_METHOD(ci); }

VM_CALL_CFUNC static VALUE vm_call_cfunc(rb_thread_t *th, rb_control_frame_t *reg_cfp, rb_call_info_t *ci) {
... val = vm_call_cfunc_latter(th, reg_cfp, ci); EXEC_EVENT_HOOK(th, RUBY_EVENT_C_RETURN, recv, me->called_id, me->klass, val); ...

VM_CALL_CFUNC static VALUE vm_call_cfunc(rb_thread_t *th, rb_control_frame_t *reg_cfp, rb_call_info_t *ci) {
... val = vm_call_cfunc_latter(th, reg_cfp, ci); EXEC_EVENT_HOOK(th, RUBY_EVENT_C_RETURN, recv, me->called_id, me->klass, val); ... TracePoint is invoked by EXEC_EVENT_HOOK

DISABLE VM OPTIMIZATION • Set optimization ﬂags using RubyVM::InstructionSequence.compile_opti on=
method • power_assert contains following code RubyVM::InstructionSequence.compile_option = { specialized_instruction: false }

DISABLE VM OPTIMIZATION • Only power_assert is compiled with optimization
$ ruby -rpower_assert ./test.rb Compile option Test code power_assert Optimization: Enabled Compile Evaluate Disabled Compile Evaluate

DISABLE VM OPTIMIZATION • Both power_assert and test code are
compiled with optimization $ ruby ./test.rb Compile option Test code power_assert Optimization: Enabled Compile Evaluate Compile Evaluate Evaluate Disabled

HOW TO GET POSITION INFORMATION

RIPPER

WHAT IS RIPPER? • A Ruby script parser • Ripper.sexp
create S-exp tree including position information # assert { [0, 1, 2].find {|i| i.odd? } == 3 } [:program, [[:method_add_block, [:method_add_arg, [:fcall, [:@ident, "assert", [1, 0]]], []], [:brace_block, nil, [[:binary, [:method_add_block, [:call, [:array,ɹ[[:@int, "0", [1, 10]], [:@int, "1", [1, 13]], [:@int, "2", [1, 16]]]], :".", [:@ident, "find", [1, 19]]], [:brace_block, [:block_var, [:params, [[:@ident, "i", [1, 26]]], nil, nil, nil, nil, nil, nil], false], [[:call, [:var_ref, [:@ident, "i", [1, 29]]], :”.", [:@ident, "odd?", [1, 31]]]]]], :==, [:@int, "3", [1, 41]]]]]]]]

WHAT IS RIPPER? • A Ruby script parser • Ripper.sexp
create S-exp tree including position information # assert { [0, 1, 2].find {|i| i.odd? } == 3 } [:program, [[:method_add_block, [:method_add_arg, [:fcall, [:@ident, "assert", [1, 0]]], []], [:brace_block, nil, [[:binary, [:method_add_block, [:call, [:array,ɹ[[:@int, "0", [1, 10]], [:@int, "1", [1, 13]], [:@int, "2", [1, 16]]]], :".", [:@ident, "find", [1, 19]]], [:brace_block, [:block_var, [:params, [[:@ident, "i", [1, 26]]], nil, nil, nil, nil, nil, nil], false], [[:call, [:var_ref, [:@ident, "i", [1, 29]]], :”.", [:@ident, "odd?", [1, 31]]]]]], :==, [:@int, "3", [1, 41]]]]]]]] "assert"

EXTRACT POSITION INFORMATION • Use multiple assignment tag, * =
sexp case tag when :program _, ((tag0, (tag1, (tag2, (tag3, mname, _)), _), (tag4, _, ss))) = sexp if tag0 == :method_add_block and tag1 == :method_add_arg and tag2 == :fcall and tag3 == :@ident and mname == @assertion_method_name and (tag4 == :brace_block or tag4 == :do_block) ss.flat_map {|s| extract_idents(s) } else ...

EXTRACT POSITION INFORMATION • In early versions, use pattern matching
instead match(sexp) do with(_[:program, _[_[:method_add_block, _[:method_add_arg, _[:fcall, _[:@ident, assertion_method.to_s, _]], _], _[Or(:brace_block, :do_block), _, ss]]]]) do ss.flat_map {|s| extract_idents(s) } end ...

OFF TOPIC: PATTERN MATCHING IN RUBY • Gems have been
released one after another lately • k-tsj/pattern-match • egison/egison-ruby • todesking/patm • I hope Ruby supports pattern matching as a core feature • Welcome great ideas

CORNER CASES/LIMITATIONS # Case A assert { obj.foo } #
Case B assert { obj.foo } # Case C assert { # comment obj.foo } # Case D assert { obj.foo obj.bar } # Case E foo { assert { obj.bar } } Which expressions will be power asserted?

CORNER CASES/LIMITATIONS # Case A assert { obj.foo } #
Case B assert { obj.foo } # Case C assert { # comment obj.foo } # Case D assert { obj.foo obj.bar } # Case E foo { assert { obj.bar } } obj.foo obj.foo obj.foo obj.foo Which expressions will be power asserted?

CORNER CASES/LIMITATIONS • Reassignment of variables • LHS is 0
and RHS is 1, but eval('i') is 1 ! ! • Branch • TracePoint does not know which obj.foo is called assert { (i = 0) == (i = 1) } assert { cond ? obj.foo : obj.foo }

CONCLUSION

CONCLUSION • Enjoy programming with power_assert • Enjoy programming with
TracePoint

Power Assert in Ruby

Power Assert in Ruby

More Decks by k_tsj

Other Decks in Programming

Featured

Transcript