RubyConf Brazil 2015: Deep Diving: How to Explore a New Code Base

As a developer, diving in a new code base is not uncommon: you've just been hired, you change of project, you want to help an open source project, the open source library your project depends on is buggy, etc. It's like entering in a underwater cave, you don't know the treasures or the monsters you'll find there, neither if the path is treacherous or if it's a true labyrinth where you'll get lost. You can prepare yourself, you can plan your visit, you can equip yourself, you can survive and find the gem you're looking for...

The gem trace_calls https://github.com/toch/trace_calls
The original snippet of code using TracePoint API https://gist.github.com/toch/82fd93ca449500d21f00

Transcript

1. Deep Diving: How to Explore a New Codebase Andy Spearing © 2009
2. _toch toch Hi, I'm Christophe
3. @matylda © 2011 Intro
4. @matylda © 2011 The most common task
5. continuous typing
7. but continuous reading Jono Witts © 2008
8. Each written line is read at least 1 time Gavin St. Ours © 20013
9. • 300 dev days • 14633 LOC • 634595 Changed lines • → 634595 / 14633 ~ 43 changed LOC / LOC
10. For 1 final line, I read at least 43 lines.
12. How to Dive into a project Stuart Hamilton © 2008
13. DO NOT Wander without a plan Felix Esteban © 2004
14. Journey Needs Plan Peter Southwood © 20011
15. Plan 1. Goal 2. Map 3. Equipment & Dive 4. Next? Peter Southwood © 20011
16. 1. Goal Peter Southwood © 20011
17. ➔ fix a bug ➔ implement a feature ➔ write some doc ➔ style the code ➔ refactor a small piece of code ➔ simply use it
19. http_proxy=http://127.0.0.1 \ ruby -e " require 'open-uri' open( 'http://google.com', proxy: nil ) "
20. ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED) from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `open' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `block in connect' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/timeout.rb:52:in `timeout' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:877:in `connect' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:862:in `do_start' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:851:in `start' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:313:in `open_http' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:708:in `buffer_open' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:210:in `block in open_loo from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:208:in `catch' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:208:in `open_loop' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:149:in `open_uri' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:688:in `open' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:34:in `open' from -e:1:in `<main>'
21. 2. Map Peter Southwood © 20011
22. Find the Map ➔ Repository URI ➔ Project URI ➔ Intro Doc: README
23. Find the Map • https://github.com/ruby/ruby • https://bugs.ruby-lang.org/ • https://github.com/ruby/ruby/blob/trunk/ README.md
24. Find the Legend
25. a Legend
26. Find the Legend ➔ CONTRIBUTING Guideline
27. Find the Legend ➔ CONTRIBUTING Guideline https://github.com/ruby/ruby/blob/trunk/ CONTRIBUTING.md
28. Find the Legend ➔ CONTRIBUTING Guideline https://github.com/ruby/ruby/blob/trunk/ CONTRIBUTING.md
29. Find the Legend ➔ CONTRIBUTING Guideline ➔ Directory Structure
30. Find the Legend ➔ CONTRIBUTING Guideline ➔ Directory Structure
31. Find the Legend ➔ CONTRIBUTING Guideline ➔ Directory Structure ➔ Talk to other contributors, maintainers
32. Find the Legend ➔ CONTRIBUTING Guideline ➔ Directory Structure ➔ Talk to other contributors, maintainers ➔ Participate to the dev meeting
33. If blocked? ➔ ASK!
34. Boris Dimitrov © 20009 3. Equipment & Dive
35. Get a Toolbelt ➔ Pick an Editor
36. xkcd
37. Get a Toolbelt ➔ Pick an Editor ➔ Bundle
38. $ cd GitLab $ bundle viz
39. $ cd GitLab $ bundle viz
40. $ cd GitLab $ bundle viz
41. Get a Toolbelt ➔ Pick an Editor ➔ Bundle ➔ Run Tests
42. Get a Toolbelt ➔ Pick an Editor ➔ Bundle ➔ Run Tests ➔ Others
43. Get a Toolbelt ➔ ST3 ➔ Apply a Patch & Use it ➔ MRI WIKI DeveloperHowto ➔ Sign up to Ruby Redmine
44. Use your Toolbelt ➔ Run the App & Use it
45. http_proxy=http://127.0.0.1 \ ruby -e " require 'open-uri' open( 'http://google.com', proxy: nil ) "
46. Use your Toolbelt ➔ Run the App & Use it ➔ Read the Code
47. $ wc -l *.rb 1001 common.rb 257 ftp.rb 1676 generic.rb 106 http.rb 22 https.rb 260 ldap.rb 20 ldaps.rb 280 mailto.rb 3622 total open-uri
48. ➔ Find proxy management ➔ Understand how open_http works ➔ Spot the bug URI
49. ➔ Trace the Calls Dive into URI
50. ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED) from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `open' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `block in connect' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/timeout.rb:52:in `timeout' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:877:in `connect' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:862:in `do_start' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:851:in `start' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:313:in `open_http' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:708:in `buffer_open' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:210:in `block in open_loo from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:208:in `catch' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:208:in `open_loop' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:149:in `open_uri' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:688:in `open' from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:34:in `open' from -e:1:in `<main>'
51. def get_call_graph_on scope = {} trace = TracePoint.new(:call, :line) do |tp| case tp.event when :call then puts "#{tp.path}:#{tp.lineno} #{tp.defined_class}::#{tp.method_id} called from " \ "#{scope[:path]}:#{scope[:lineno]} #{scope[:class]}::#{scope[:method_id]}" when :line then scope = { event: :line, lineno: tp.lineno, path: tp.path, class: tp.defined_class, method_id: tp.method_id } end end trace.enable yield trace.disable end
52. scope = {} trace = TracePoint.new(:call, :line) do |tp| case tp.event # ... end end
53. trace = TracePoint.new(:call, :line) do |tp| case tp.event when :call then puts "#{tp.path}:#{tp.lineno} ..." \ "#{scope[:path]}:#{scope[:lineno]} ..." # ... end end
54. # ... when :line then scope = { event: :line, lineno: tp.lineno, path: tp.path, class: tp.defined_class, method_id: tp.method_id }
55. trace.enable yield trace.disable
56. require 'open-uri' get_call_graph_on do open( 'http://google.com', proxy: nil ) end
58. require 'open-uri' require 'trace_calls' TraceCalls::on do open('http://google.com', proxy: nil) end puts TraceCalls::root
59. +-Kernel::open +-#<Class:URI>::parse called from Kernel::open at 33 | \-URI::Parser::parse called from #<Class:URI>::parse at 747 | | +-URI::Parser::split called from URI::Parser::parse at 211 | | +-#<Class:URI>::scheme_list called from URI::Parser::parse at 213 | | +-#<Class:URI>::scheme_list called from URI::Parser::parse at 214 | | \-URI::HTTP::initialize called from URI::Parser::parse at 660 | | | \-URI::Generic::initialize called from URI::HTTP::initialize at 84 | | | | +-URI::Generic::set_scheme called from URI::Generic::initialize at 203 | | | | +-URI::Generic::set_userinfo called from URI::Generic::initialize at 204 | | | | | \-URI::Generic::split_userinfo called from URI::Generic::set_userinfo at 525 | | | | +-URI::Generic::set_host called from URI::Generic::initialize at 205 | | | | +-URI::Generic::set_port called from URI::Generic::initialize at 206 | | | | +-URI::Generic::set_path called from URI::Generic::initialize at 207 | | | | +-URI::Generic::set_query called from URI::Generic::initialize at 208 | | | | +-URI::Generic::set_opaque called from URI::Generic::initialize at 209 | | | | +-URI::Generic::set_registry called from URI::Generic::initialize at 210 | | | | +-URI::Generic::set_fragment called from URI::Generic::initialize at 211 | | | | +-URI::Generic::default_port called from URI::Generic::initialize at 220 | | | | | \-#<Class:URI::Generic>::default_port called from URI::Generic::default_port at | | | | +-URI::Generic::default_port called from URI::Generic::initialize at 220 | | | | | \-#<Class:URI::Generic>::default_port called from URI::Generic::default_port at | | | | \-URI::Generic::set_port called from URI::Generic::initialize at 30
60. +-Kernel::open +-#<Class:URI>::parse called from Kernel::open at 33 | \-URI::Parser::parse called from #<Class:URI>::parse at 747 | | +-URI::Parser::split called from URI::Parser::parse at 211 | | +-#<Class:URI>::scheme_list called from URI::Parser::parse at 213 | | +-#<Class:URI>::scheme_list called from URI::Parser::parse at 214 | | \-URI::HTTP::initialize called from URI::Parser::parse at 660 | | | \-URI::Generic::initialize called from URI::HTTP::initialize at 84 | | | | +-URI::Generic::set_scheme called from URI::Generic::initialize at 203 | | | | +-URI::Generic::set_userinfo called from URI::Generic::initialize at 204 | | | | | \-URI::Generic::split_userinfo called from URI::Generic::set_userinfo at 525 | | | | +-URI::Generic::set_host called from URI::Generic::initialize at 205 | | | | +-URI::Generic::set_port called from URI::Generic::initialize at 206 | | | | +-URI::Generic::set_path called from URI::Generic::initialize at 207 | | | | +-URI::Generic::set_query called from URI::Generic::initialize at 208 | | | | +-URI::Generic::set_opaque called from URI::Generic::initialize at 209 | | | | +-URI::Generic::set_registry called from URI::Generic::initialize at 210 | | | | +-URI::Generic::set_fragment called from URI::Generic::initialize at 211 | | | | +-URI::Generic::default_port called from URI::Generic::initialize at 220 | | | | | \-#<Class:URI::Generic>::default_port called from URI::Generic::default_port at | | | | +-URI::Generic::default_port called from URI::Generic::initialize at 220 | | | | | \-#<Class:URI::Generic>::default_port called from URI::Generic::default_port at | | | | \-URI::Generic::set_port called from URI::Generic::initialize at 30
61. +-Kernel::open +-#<Class:URI>::parse called from Kernel::open at 33 | \-URI::Parser::parse called from #<Class:URI>::parse at 747 | | +-URI::Parser::split called from URI::Parser::parse at 211 | | +-#<Class:URI>::scheme_list called from URI::Parser::parse at 213 | | +-#<Class:URI>::scheme_list called from URI::Parser::parse at 214 | | \-URI::HTTP::initialize called from URI::Parser::parse at 660 | | | \-URI::Generic::initialize called from URI::HTTP::initialize at 84 | | | | +-URI::Generic::set_scheme called from URI::Generic::initialize at 203 | | | | +-URI::Generic::set_userinfo called from URI::Generic::initialize at 204 | | | | | \-URI::Generic::split_userinfo called from URI::Generic::set_userinfo at 525 | | | | +-URI::Generic::set_host called from URI::Generic::initialize at 205 | | | | +-URI::Generic::set_port called from URI::Generic::initialize at 206 | | | | +-URI::Generic::set_path called from URI::Generic::initialize at 207 | | | | +-URI::Generic::set_query called from URI::Generic::initialize at 208 | | | | +-URI::Generic::set_opaque called from URI::Generic::initialize at 209 | | | | +-URI::Generic::set_registry called from URI::Generic::initialize at 210 | | | | +-URI::Generic::set_fragment called from URI::Generic::initialize at 211 | | | | +-URI::Generic::default_port called from URI::Generic::initialize at 220 | | | | | \-#<Class:URI::Generic>::default_port called from URI::Generic::default_port at | | | | +-URI::Generic::default_port called from URI::Generic::initialize at 220 | | | | | \-#<Class:URI::Generic>::default_port called from URI::Generic::default_port at | | | | \-URI::Generic::set_port called from URI::Generic::initialize at 30
62. \-OpenURI::OpenRead::open called from Kernel::open at 34 | \-#<Class:OpenURI>::open_uri called from OpenURI::OpenRead::open at 688 | | +-#<Class:OpenURI>::scan_open_optional_arguments called from #<Class:OpenURI>::open_ur | | +-#<Class:OpenURI>::check_options called from #<Class:OpenURI>::open_uri at 136 | | \-#<Class:OpenURI>::open_loop called from #<Class:OpenURI>::open_uri at 149 | | | +-OpenURI::Buffer::initialize called from #<Class:OpenURI>::open_loop at 209 | | | +-URI::HTTP::buffer_open called from #<Class:OpenURI>::open_loop at 195 | | | | \-#<Class:OpenURI>::open_http called from URI::HTTP::buffer_open at 708 | | | | | +-URI::Generic::userinfo called from #<Class:OpenURI>::open_http at 259 | | | | | +-Kernel::require called from #<Class:OpenURI>::open_http at 267 | | | | | | +-MonitorMixin::mon_enter called from Kernel::require at 39 | | | | | | +-#<Class:Gem>::find_unresolved_default_spec called from Kernel::require at 43 | | | | | | | \-#<Class:Gem>::suffixes called from #<Class:Gem>::find_unresolved_default_s | | | | | | +-#<Class:Gem::Specification>::unresolved_deps called from Kernel::require at | | | | | | +-#<Class:Gem::Specification>::stubs called from Kernel::require at 63 | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64 | | | | | | | \-Gem::StubSpecification::name called from Gem::StubSpecification::activated | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64 | | | | | | | \-Gem::StubSpecification::name called from Gem::StubSpecification::activated | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64 | | | | | | | \-Gem::StubSpecification::name called from Gem::StubSpecification::activated | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64
63. \-OpenURI::OpenRead::open called from Kernel::open at 34 | \-#<Class:OpenURI>::open_uri called from OpenURI::OpenRead::open at 688 | | +-#<Class:OpenURI>::scan_open_optional_arguments called from #<Class:OpenURI>::open_ur | | +-#<Class:OpenURI>::check_options called from #<Class:OpenURI>::open_uri at 136 | | \-#<Class:OpenURI>::open_loop called from #<Class:OpenURI>::open_uri at 149 | | | +-OpenURI::Buffer::initialize called from #<Class:OpenURI>::open_loop at 209 | | | +-URI::HTTP::buffer_open called from #<Class:OpenURI>::open_loop at 195 | | | | \-#<Class:OpenURI>::open_http called from URI::HTTP::buffer_open at 708 | | | | | +-URI::Generic::userinfo called from #<Class:OpenURI>::open_http at 259 | | | | | +-Kernel::require called from #<Class:OpenURI>::open_http at 267 | | | | | | +-MonitorMixin::mon_enter called from Kernel::require at 39 | | | | | | +-#<Class:Gem>::find_unresolved_default_spec called from Kernel::require at 43 | | | | | | | \-#<Class:Gem>::suffixes called from #<Class:Gem>::find_unresolved_default_s | | | | | | +-#<Class:Gem::Specification>::unresolved_deps called from Kernel::require at | | | | | | +-#<Class:Gem::Specification>::stubs called from Kernel::require at 63 | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64 | | | | | | | \-Gem::StubSpecification::name called from Gem::StubSpecification::activated | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64 | | | | | | | \-Gem::StubSpecification::name called from Gem::StubSpecification::activated | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64 | | | | | | | \-Gem::StubSpecification::name called from Gem::StubSpecification::activated | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64
64. \-OpenURI::OpenRead::open called from Kernel::open at 34 | \-#<Class:OpenURI>::open_uri called from OpenURI::OpenRead::open at 688 | | +-#<Class:OpenURI>::scan_open_optional_arguments called from #<Class:OpenURI>::open_ur | | +-#<Class:OpenURI>::check_options called from #<Class:OpenURI>::open_uri at 136 | | \-#<Class:OpenURI>::open_loop called from #<Class:OpenURI>::open_uri at 149 | | | +-OpenURI::Buffer::initialize called from #<Class:OpenURI>::open_loop at 209 | | | +-URI::HTTP::buffer_open called from #<Class:OpenURI>::open_loop at 195 | | | | \-#<Class:OpenURI>::open_http called from URI::HTTP::buffer_open at 708 | | | | | +-URI::Generic::userinfo called from #<Class:OpenURI>::open_http at 259 | | | | | +-Kernel::require called from #<Class:OpenURI>::open_http at 267 | | | | | | +-MonitorMixin::mon_enter called from Kernel::require at 39 | | | | | | +-#<Class:Gem>::find_unresolved_default_spec called from Kernel::require at 43 | | | | | | | \-#<Class:Gem>::suffixes called from #<Class:Gem>::find_unresolved_default_s | | | | | | +-#<Class:Gem::Specification>::unresolved_deps called from Kernel::require at | | | | | | +-#<Class:Gem::Specification>::stubs called from Kernel::require at 63 | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64 | | | | | | | \-Gem::StubSpecification::name called from Gem::StubSpecification::activated | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64 | | | | | | | \-Gem::StubSpecification::name called from Gem::StubSpecification::activated | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64 | | | | | | | \-Gem::StubSpecification::name called from Gem::StubSpecification::activated | | | | | | +-Gem::StubSpecification::activated? called from Kernel::require at 64
65. | | | | | +-URI::Generic::hostname called from #<Class:OpenURI>::open_http at 278 | | | | | +-URI::HTTP::request_uri called from #<Class:OpenURI>::open_http at 280 | | | | | | \-URI::Generic::path_query called from URI::HTTP::request_uri at 96 | | | | | +-#<Class:Net::HTTP>::new called from #<Class:OpenURI>::open_http at 291 | | | | | | +-Net::HTTP::initialize called from #<Class:Net::HTTP>::new at 609 | | | | | | \-#<Class:Net::HTTP>::proxy_class? called from #<Class:Net::HTTP>::new at 611 | | | | | +-Net::HTTP::start called from #<Class:OpenURI>::open_http at 313 | | | | | | +-Net::HTTP::do_start called from Net::HTTP::start at 851 | | | | | | | \-Net::HTTP::connect called from Net::HTTP::do_start at 862 | | | | | | | | +-Net::HTTP::proxy? called from Net::HTTP::connect at 868 | | | | | | | | | \-Net::HTTP::proxy_uri called from Net::HTTP::proxy? at 1014 | | | | | | | | | | +-Kernel::URI called from Net::HTTP::proxy_uri at 1027 | | | | | | | | | | | \-#<Class:URI>::parse called from Kernel::URI at 994 | | | | | | | | | | | | \-URI::Parser::parse called from #<Class:URI>::parse at 747 | | | | | | | | | | | | | +-URI::Parser::split called from URI::Parser::parse at 211 | | | | | | | | | | | | | +-#<Class:URI>::scheme_list called from URI::Parser::parse at 21 | | | | | | | | | | | | | +-#<Class:URI>::scheme_list called from URI::Parser::parse at 21 | | | | | | | | | | | | | \-URI::HTTP::initialize called from URI::Parser::parse at 660 | | | | | | | | | | | | | | \-URI::Generic::initialize called from URI::HTTP::initialize a | | | | | | | | | | | | | | | +-URI::Generic::set_scheme called from URI::Generic::initial | | | | | | | | | | | | | | | +-URI::Generic::set_userinfo called from URI::Generic::initi
66. http = klass.new( target_host, target_port )
67. ➔ Trace the Calls ➔ Search the Source Dive into URI
68. Search the Source $ grep -nr "open_http". ./open-uri.rb:253: def OpenURI.open_http(buf, target, proxy, option ./open-uri.rb:708: OpenURI.open_http(buf, self, proxy, options) ./open-uri.rb:717: OpenURI.open_http(buf, self, proxy, option
69. Search the Source $ grep -rn "class HTTP " . ./net/http.rb:384: class HTTP < Protocol ./uri/http.rb:22: class HTTP < Generic
70. Search the Source $ EDITOR=subl \ bundle open gitlab_omniauth-ldap
71. ➔ Trace the Calls ➔ Search the Source ➔ Get Context with a Test Dive into URI
72. Get Context with a Test context "without params['markdown_img']" do it "returns an error" do TraceCalls::on do post :upload_image, id: project.to_param, format: :json end puts TraceCalls::root expect(response.status).to eq(422) end end
73. Use your Toolbelt ➔ Run the App & Use it ➔ Read the Code ➔ Edit the Code
74. http = proxy ? klass.new(target_host, target_port) : klass.new(target_host, target_port, nil)
75. Use your Toolbelt ➔ Run the App & Use it ➔ Read & Edit the Code ➔ Report a bug ➔ Submit a contribution
77. If blocked? ➔ ASK!
78. 4. Next? Peter Southwood © 20011
79. How long? ➔ Keep it reasonable at first ➔ From ½ to 1 day
80. Find Next Goal ➔ Fix a bug ➔ Develop a feature ➔ Run Static Analysis Tools ➔ Review a PullRequest
81. Iterate ➔ :) Jean-Marc Kuffer © 2007
82. If blocked? ➔ ASK!
83. Outro 1. Goal 2. Map 3. Equipment & Dive
84. ? _toch toch