Ruby Objects, A Walkabout

In Ruby, it's easy to create classes, methods, and objects. For instance, did you know that a hello world sinatra app uses 43 classes, 155 methods, and dispatches 548 methods for a single request. In this talk, we're going use TracePoint API to look at how you can find out this information among other things. Additionally, now that we have all these objects from our Ruby app, the evolving ObjectSpace API we can glean information about rough object size in memory. Let's take a walk through objects in Ruby.

Transcript

2. Ruby Objects A Walkabout
4. Terence Lee @hone02
5. Austin, TX
11. @schneems
13. zzak (@_zzak)
16. • TracePoint • ObjectSpace • Allocation Tracer Gem
17. TracePoint
18. Kernel#set_func_trace
19. enable trace set_trace_func proc {|event, file, line, id, binding, klass| printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, klass }
20. code to trace set_trace_func proc {|event, file, line, id, binding, klass| printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, klass } class Foo def bar "baz" end end Foo.new.bar
21. disable trace set_trace_func proc {|event, file, line, id, binding, klass| printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, klass } class Foo def bar "baz" end end Foo.new.bar set_trace_func(nil) Foo.new
22. c-return trace.rb:1 set_trace_func Kernel line trace.rb:5 c-call trace.rb:5 inherited Class c-return trace.rb:5 inherited Class class trace.rb:5 line trace.rb:6 c-call trace.rb:6 method_added Module c-return trace.rb:6 method_added Module end trace.rb:9 line trace.rb:11 c-call trace.rb:11 new Class c-call trace.rb:11 initialize BasicObject c-return trace.rb:11 initialize BasicObject c-return trace.rb:11 new Class call trace.rb:6 bar Foo line trace.rb:7 bar Foo return trace.rb:8 bar Foo line trace.rb:13 c-call trace.rb:13 set_trace_func Kernel
23. c-return trace.rb:1 set_trace_func Kernel line trace.rb:5 c-call trace.rb:5 inherited Class c-return trace.rb:5 inherited Class class trace.rb:5 line trace.rb:6 c-call trace.rb:6 method_added Module c-return trace.rb:6 method_added Module end trace.rb:9 line trace.rb:11 c-call trace.rb:11 new Class c-call trace.rb:11 initialize BasicObject c-return trace.rb:11 initialize BasicObject c-return trace.rb:11 new Class call trace.rb:6 bar Foo line trace.rb:7 bar Foo return trace.rb:8 bar Foo line trace.rb:13 c-call trace.rb:13 set_trace_func Kernel
24. set_trace_func proc {|event, file, line, id, binding, klass| printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, klass } class Foo def bar "baz" end end Foo.new.bar set_trace_func(nil) Foo.new
25. c-return trace.rb:1 set_trace_func Kernel line trace.rb:5 c-call trace.rb:5 inherited Class c-return trace.rb:5 inherited Class class trace.rb:5 line trace.rb:6 c-call trace.rb:6 method_added Module c-return trace.rb:6 method_added Module end trace.rb:9 line trace.rb:11 c-call trace.rb:11 new Class c-call trace.rb:11 initialize BasicObject c-return trace.rb:11 initialize BasicObject c-return trace.rb:11 new Class call trace.rb:6 bar Foo line trace.rb:7 bar Foo return trace.rb:8 bar Foo line trace.rb:13 c-call trace.rb:13 set_trace_func Kernel
26. set_trace_func proc {|event, file, line, id, binding, klass| printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, klass } class Foo def bar "baz" end end Foo.new.bar set_trace_func(nil) Foo.new
27. c-return trace.rb:1 set_trace_func Kernel line trace.rb:5 c-call trace.rb:5 inherited Class c-return trace.rb:5 inherited Class class trace.rb:5 line trace.rb:6 c-call trace.rb:6 method_added Module c-return trace.rb:6 method_added Module end trace.rb:9 line trace.rb:11 c-call trace.rb:11 new Class c-call trace.rb:11 initialize BasicObject c-return trace.rb:11 initialize BasicObject c-return trace.rb:11 new Class call trace.rb:6 bar Foo line trace.rb:7 bar Foo return trace.rb:8 bar Foo line trace.rb:13 c-call trace.rb:13 set_trace_func Kernel
28. set_trace_func proc {|event, file, line, id, binding, klass| printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, klass } class Foo def bar "baz" end end Foo.new.bar set_trace_func(nil) Foo.new
29. c-return trace.rb:1 set_trace_func Kernel line trace.rb:5 c-call trace.rb:5 inherited Class c-return trace.rb:5 inherited Class class trace.rb:5 line trace.rb:6 c-call trace.rb:6 method_added Module c-return trace.rb:6 method_added Module end trace.rb:9 line trace.rb:11 c-call trace.rb:11 new Class c-call trace.rb:11 initialize BasicObject c-return trace.rb:11 initialize BasicObject c-return trace.rb:11 new Class call trace.rb:6 bar Foo line trace.rb:7 bar Foo return trace.rb:8 bar Foo line trace.rb:13 c-call trace.rb:13 set_trace_func Kernel
30. c-return trace.rb:1 set_trace_func Kernel line trace.rb:5 c-call trace.rb:5 inherited Class c-return trace.rb:5 inherited Class class trace.rb:5 line trace.rb:6 c-call trace.rb:6 method_added Module c-return trace.rb:6 method_added Module end trace.rb:9 line trace.rb:11 c-call trace.rb:11 new Class c-call trace.rb:11 initialize BasicObject c-return trace.rb:11 initialize BasicObject c-return trace.rb:11 new Class call trace.rb:6 bar Foo line trace.rb:7 bar Foo return trace.rb:8 bar Foo line trace.rb:13 c-call trace.rb:13 set_trace_func Kernel
31. set_trace_func proc {|event, file, line, id, binding, klass| printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, klass } class Foo def bar "baz" end end Foo.new.bar set_trace_func(nil) Foo.new
32. c-return trace.rb:1 set_trace_func Kernel line trace.rb:5 c-call trace.rb:5 inherited Class c-return trace.rb:5 inherited Class class trace.rb:5 line trace.rb:6 c-call trace.rb:6 method_added Module c-return trace.rb:6 method_added Module end trace.rb:9 line trace.rb:11 c-call trace.rb:11 new Class c-call trace.rb:11 initialize BasicObject c-return trace.rb:11 initialize BasicObject c-return trace.rb:11 new Class call trace.rb:6 bar Foo line trace.rb:7 bar Foo return trace.rb:8 bar Foo line trace.rb:13 c-call trace.rb:13 set_trace_func Kernel
33. set_trace_func proc {|event, file, line, id, binding, klass| printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, klass } class Foo def bar "baz" end end Foo.new.bar set_trace_func(nil) Foo.new
34. c-return trace.rb:1 set_trace_func Kernel line trace.rb:5 c-call trace.rb:5 inherited Class c-return trace.rb:5 inherited Class class trace.rb:5 line trace.rb:6 c-call trace.rb:6 method_added Module c-return trace.rb:6 method_added Module end trace.rb:9 line trace.rb:11 c-call trace.rb:11 new Class c-call trace.rb:11 initialize BasicObject c-return trace.rb:11 initialize BasicObject c-return trace.rb:11 new Class call trace.rb:6 bar Foo line trace.rb:7 bar Foo return trace.rb:8 bar Foo line trace.rb:13 c-call trace.rb:13 set_trace_func Kernel
35. set_trace_func proc {|event, file, line, id, binding, klass| printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, klass } class Foo def bar "baz" end end Foo.new.bar set_trace_func(nil) Foo.new
36. events • c-call • c-return • call • return • class • end • line • raise
37. TracePoint
38. events • b_call • b_return • thread_begin • thread_end
39. tracepoint = TracePoint.new do |tp| printf "%8s %s:%-2d %10s %8s\n", tp.event, tp.path, tp.lineno, tp.method_id, tp.defined_class end tracepoint.enable do class Foo def bar "baz" end end Foo.new.bar end Foo.new
40. tracepoint = TracePoint.new do |tp| printf "%8s %s:%-2d %10s %8s\n", tp.event, tp.path, tp.lineno, tp.method_id, tp.defined_class end tracepoint.enable do class Foo def bar "baz" end end Foo.new.bar end Foo.new
41. tracepoint = TracePoint.new do |tp| printf "%8s %s:%-2d %10s %8s\n", tp.event, tp.path, tp.lineno, tp.method_id, tp.defined_class end tracepoint.enable do class Foo def bar "baz" end end Foo.new.bar end Foo.new
42. b_call tracepoint.rb:5 line tracepoint.rb:6 c_call tracepoint.rb:6 inherited Class c_return tracepoint.rb:6 inherited Class class tracepoint.rb:6 line tracepoint.rb:7 c_call tracepoint.rb:7 method_added Module c_return tracepoint.rb:7 method_added Module end tracepoint.rb:10 line tracepoint.rb:12 c_call tracepoint.rb:12 new Class c_call tracepoint.rb:12 initialize BasicObject c_return tracepoint.rb:12 initialize BasicObject c_return tracepoint.rb:12 new Class call tracepoint.rb:7 bar Foo line tracepoint.rb:8 bar Foo return tracepoint.rb:9 bar Foo b_return tracepoint.rb:13
43. TracePoint Middleware def call(env) stats = {} response = nil trace = TracePoint.new(:call) do |tp| stats[tp.defined_class] ||= {} stats[tp.defined_class][tp.method_id] ||= 0 stats[tp.defined_class][tp.method_id] += 1 end trace.enable { response = @app.call(env) } ... end * https://gist.github.com/mattetti/5097206
44. TracePoint Middleware def call(env) ... puts env['PATH_INFO'] puts "#{stats.keys.size} classes used" puts "#{stats.map{|k,v| v.keys}.flatten.size} methods used" puts "#{stats.map{|k,v| v.values}.flatten.inject(0) {|sum, num| sum + num }} methods dispatched" File.open("tmp/#{env['PATH_INFO'].gsub('/', '_')}_req_stats. json", "w"){|f| f << stats.to_json } puts "" response end * https://gist.github.com/mattetti/5097206
45. rack require 'rack/tracepoint' class RubyVersion def self.version `ruby -v` end end use Rack::TracePoint::Middleware run Proc.new {|env| [200, {"Content-Type" => "text/plain"}, [RubyVersion.version] ] }
46. 1 classes used 1 methods used 1 methods dispatched
47. sinatra require 'sinatra' require 'rack/tracepoint' use Rack::Tracepoint::Middleware get "/" do `ruby -v` end run Sinatra::Application
48. 43 classes used 155 methods used 548 methods dispatched
49. rails # app/controllers/welcome_controller.rb class WelcomeController < ApplicationController def index @ruby_version = `ruby -v` end end # app/views/welcome/index.html.erb <%= @ruby_version %>
50. 361 classes used 1193 methods used 8621 methods dispatched
51. { "Sinatra::Base": { "initialize": 1, "call": 1, "call!": 1, "indifferent_params": 1, "indifferent_hash": 1, "settings": 9, "force_encoding": 1, "invoke": 3, "dispatch!": 1, "filter!": 4, "route!": 1, "process_route": 1, "route_eval": 1, "error_block!": 1 } }
52. { "ERB::Compiler::Buffer#push": 133, "ERB::Compiler#add_put_cmd": 66, "ERB::Compiler#content_dump": 66, "ERB::Compiler#add_insert_cmd": 34, "Rack::Builder#use": 13, "#<Class:Sinatra::Base>#settings": 10, "Sinatra::Base#settings": 9, "Rack::Utils::HeaderHash#[]": 9, "#<Class:Sinatra::Application>#app_file": 8, "Rack::Utils::HeaderHash#[]=": 7, "Rack::Protection::Base#initialize": 6, "Rack::Protection::Base#default_options": 6, "#<Class:Sinatra::Base>#environment": 4, "#<Class:Sinatra::Base>#root": 4, "Sinatra::Base#filter!": 4, "Sinatra::Base#invoke": 3 ...
53. ObjectSpace
54. ObjectSpace.count_objects { "TOTAL": 30161, "FREE": 677, "T_OBJECT": 70, "T_CLASS": 616, "T_MODULE": 37, "T_FLOAT": 7, "T_STRING": 10950, "T_REGEXP": 77, "T_ARRAY": 1629, "T_HASH": 63, "T_STRUCT": 9, "T_BIGNUM": 2, "T_FILE": 29, "T_DATA": 983, "T_MATCH": 26, ... }
55. ObjectSpace#each_object require 'json' types_count = Hash.new(0) ObjectSpace.each_object do |o| types_count[o.class] += 1 end puts types_count.to_a.sort {|a, b| b[1] <=> a[1] }.to_h.to_json
56. { "String": 8199, "RubyVM::InstructionSequence": 750, "Array": 511, "Class": 288, "Encoding": 100, "Regexp": 77, "Hash": 44, "Module": 37, "MatchData": 20, "File": 18, "Gem::Requirement": 15, "Gem::Version": 15, "Gem::StubSpecification": 11, "Proc": 10, "Range": 9, "Gem::StubSpecification::StubLine": 8, "Float": 7
57. objspace.so
58. allocation tracer
59. Object Introspection require 'objspace' os = ObjectSpace o = nil os.trace_object_allocations do class Foo def bar "baz" end end o = Foo.new.bar end printf "%s:%-2d %10s %8s\n", os.allocation_sourcefile(o), os. allocation_sourceline(o), os.allocation_method_id(o), os. allocation_class_path(o)
60. require 'objspace' os = ObjectSpace o = nil os.trace_object_allocations do class Foo def bar "baz" end end o = Foo.new.bar end printf "%s:%-2d %10s %8s\n", os.allocation_sourcefile(o), os. allocation_sourceline(o), os.allocation_method_id(o), os. allocation_class_path(o)
61. require 'objspace' os = ObjectSpace o = nil os.trace_object_allocations do class Foo def bar "baz" end end o = Foo.new.bar end printf "%s:%-2d %10s %8s\n", os.allocation_sourcefile(o), os. allocation_sourceline(o), os.allocation_method_id(o), os. allocation_class_path(o)
62. require 'objspace' os = ObjectSpace o = nil os.trace_object_allocations do class Foo def bar "baz" end end o = Foo.new.bar end printf "%s:%-2d %10s %8s\n", os.allocation_sourcefile(o), os. allocation_sourceline(o), os.allocation_method_id(o), os. allocation_class_path(o)
63. objectspace.rb:8 bar Foo
64. Object Size Counting require 'objspace' Metadata = Struct.new(:class, :sourcefile, :sourceline, : class_path, :method_id, :memsize) objects = [] result = {} def walk_objects(result) ObjectSpace.each_object do |o| result[o.object_id] = Metadata.new( ObjectSpace.allocation_class_path(o), ObjectSpace.allocation_sourcefile(o), ObjectSpace.allocation_sourceline(o), ObjectSpace.allocation_class_path(o), ObjectSpace.allocation_method_id(o), ObjectSpace.memsize_of(o) ) end end
65. Object Size Counting ObjectSpace.trace_object_allocations do a = Foo.new.bar b = Baz.new.foo walk_objects(result) end result.to_a.sort {|a,b| b[1].memsize <=> a[1].memsize }. to_h.each do |object_id, s| next if s.class_path.nil? printf "%s:%-2d %5s#%s %d\n", s.sourcefile, s.sourceline, s. class_path, s.method_id, s.memsize end
66. memory_profiler.rb:28 Baz#foo 40 memory_profiler.rb:22 Foo#bar 40
67. $ gem install memory_profiler
68. Allocation Tracer Gem
69. $ gem install allocation_tracer
70. Made Up Algorithm def swap_elements(a) pivot = a.length/2 (0..(pivot-1)).each do |i| a[i], a[i + pivot] = a[i + pivot], a[i] end end N = 10000; arr = N.times.map{|x| (N-x).to_s } swap_elements(arr)
71. ObjectSpace::AllocationTracer.trace def swap_elements(a) pivot = a.length/2 (0..(pivot-1)).each do |i| a[i], a[i + pivot] = a[i + pivot], a[i] end end pp ObjectSpace::AllocationTracer.trace { N = 10000; arr = N.times.map{|x| (N-x).to_s } swap_elements(arr) }
72. {["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0], ["algorithm.rb", 6]=>[1, 0, 0, 0, 0, 0], ["algorithm.rb", 7]=>[5000, 0, 0, 0, 0, 0]}
73. File Info ["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0]
74. Objects Created ["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0]
75. Old Objects ["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0]
76. Total Age ["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0]
77. Min Age ["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0]
78. Max Age ["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0]
79. Memsize without RVALUE ["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0]
80. {["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0], ["algorithm.rb", 6]=>[1, 0, 0, 0, 0, 0], ["algorithm.rb", 7]=>[5000, 0, 0, 0, 0, 0]}
81. def swap_elements(a) pivot = a.length/2 (0..(pivot-1)).each do |i| a[i], a[i + pivot] = a[i + pivot], a[i] end end pp ObjectSpace::AllocationTracer.trace { N = 10000; arr = N.times.map{|x| (N-x).to_s } swap_elements(arr) }
82. {["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0], ["algorithm.rb", 6]=>[1, 0, 0, 0, 0, 0], ["algorithm.rb", 7]=>[5000, 0, 0, 0, 0, 0]}
83. def swap_elements(a) pivot = a.length/2 (0..(pivot-1)).each do |i| a[i], a[i + pivot] = a[i + pivot], a[i] end end pp ObjectSpace::AllocationTracer.trace { N = 10000; arr = N.times.map{|x| (N-x).to_s } swap_elements(arr) }
84. def swap_elements(a) pivot = a.length/2 (0..(pivot-1)).each do |i| a[i], a[i + pivot] = a[i + pivot], a[i] end end ObjectSpace::AllocationTracer.setup(%i{path line type}) pp ObjectSpace::AllocationTracer.trace { N = 10000; arr = N.times.map{|x| (N-x).to_s } swap_elements(arr) }
85. {["algorithm.rb", 15, :T_NODE]=>[2, 0, 2, 1, 1, 40], ["algorithm.rb", 7, :T_ARRAY]=>[5000, 0, 2511, 0, 1, 45560], ["algorithm.rb", 15, :T_DATA]=>[1, 0, 1, 1, 1, 0], ["algorithm.rb", 15, :T_ARRAY]=>[1, 0, 1, 1, 1, 0], ["algorithm.rb", 15, :T_STRING]=>[10000, 0, 10000, 1, 1, 0], ["algorithm.rb", 6, :T_STRUCT]=>[1, 0, 1, 1, 1, 0]}
86. {["algorithm.rb", 15, :T_NODE]=>[2, 0, 2, 1, 1, 40], ["algorithm.rb", 7, :T_ARRAY]=>[5000, 0, 2511, 0, 1, 45560], ["algorithm.rb", 15, :T_DATA]=>[1, 0, 1, 1, 1, 0], ["algorithm.rb", 15, :T_ARRAY]=>[1, 0, 1, 1, 1, 0], ["algorithm.rb", 15, :T_STRING]=>[10000, 0, 10000, 1, 1, 0], ["algorithm.rb", 6, :T_STRUCT]=>[1, 0, 1, 1, 1, 0]}
87. {["algorithm.rb", 15, :T_NODE]=>[2, 0, 2, 1, 1, 40], ["algorithm.rb", 7, :T_ARRAY]=>[5000, 0, 2511, 0, 1, 45560], ["algorithm.rb", 15, :T_DATA]=>[1, 0, 1, 1, 1, 0], ["algorithm.rb", 15, :T_ARRAY]=>[1, 0, 1, 1, 1, 0], ["algorithm.rb", 15, :T_STRING]=>[10000, 0, 10000, 1, 1, 0], ["algorithm.rb", 6, :T_STRUCT]=>[1, 0, 1, 1, 1, 0]}
88. def swap_elements(a) pivot = a.length/2 (0..(pivot-1)).each do |i| a[i], a[i + pivot] = a[i + pivot], a[i] end end pp ObjectSpace::AllocationTracer.trace { N = 10000; arr = N.times.map{|x| (N-x).to_s } swap_elements(arr) }
89. def swap_elements(a) pivot = a.length/2 (0..(pivot-1)).each do |i| x, y = a[i + pivot], a[i] a[i] = x a[i + pivot] = y end end pp ObjectSpace::AllocationTracer.trace { N = 10000; arr = N.times.map{|x| (N-x).to_s } swap_elements(arr) }
90. {["algorithm.rb", 15]=>[10004, 0, 0, 0, 0, 0], ["algorithm.rb", 6]=>[1, 0, 0, 0, 0, 0],
91. Conclusion
92. Thank You