Source-Diving for Fun and Profit

Ever spent hours pouring over a gem's documentation trying to figure out how to make it work? Dug through dozens of blog posts trying to understand why a library's not working? Well what if I promised you an end to all that?!

Well, ok, I'd be lying. But maybe I can save you some hair-pulling some of the time! Let me introduce you to the joys of Reading the Code. Maybe it seems obvious to you, but one of the biggest leaps I made as a ruby dev was really getting comfortable jumping into a gem's source as a debugging technique.

In an effort to get you over that hump earlier than I did, let's talk tips and tricks for getting in and out of a library's codebase as efficiently as possible. It won't solve every problem, but sometimes 5 minutes on GitHub will save you hours on StackOverflow.

Transcript

1. @kkuchta @kkuchta
2. @kkuchta Me writing code @kkuchta
3. @kkuchta I found an bug /Users/kevin/.rvm/gems/ruby-2.6.0/ gems/net-sftp-2.1.2/lib/net/sftp/ operations/file.rb:85:in `gets': wrong number of arguments (given 2, expected 0..1)
4. @kkuchta I found an bug /Users/kevin/.rvm/gems/ruby-2.6.0/ gems/net-sftp-2.1.2/lib/net/sftp/ operations/file.rb:85:in `gets': wrong number of arguments (given 2, expected 0..1)
5. @kkuchta @kkuchta
6. @kkuchta @kkuchta
7. @kkuchta @kkuchta
8. @kkuchta @kkuchta
9. @kkuchta @kkuchta
10. @kkuchta @kkuchta
11. @kkuchta @kkuchta
12. @kkuchta The End @kkuchta
13. @kkuchta @kkuchta
14. @kkuchta @kkuchta
15. @kkuchta
16. @kkuchta
17. @kkuchta
18. @kkuchta
19. @kkuchta
20. @kkuchta
21. @kkuchta
22. @kkuchta
23. @kkuchta
24. @kkuchta
25. @kkuchta
26. @kkuchta
27. @kkuchta @kkuchta
28. @kkuchta
29. @kkuchta Source Diving for Fun and Profit Kevin Kuchta
30. @kkuchta
31. @kkuchta The Kevin Kuchta 3ish- step Process©™® For source code diving
32. @kkuchta 1. Take me to your leader
33. @kkuchta Start at the top
34. @kkuchta Start at the top
35. @kkuchta Start at the top
36. @kkuchta Start at the top
37. @kkuchta Start at the top
38. @kkuchta Find the biggest
39. @kkuchta Find the biggest $ find lib -type f | xargs wc -l | sort -r | head Find in ./lib Sort by line count Get the line count for each Just show the top 10
40. @kkuchta Find the biggest
41. @kkuchta Take me to your leader • Entry points • Files with lots of content • Biggest files
42. @kkuchta 2. If you see something, search something
43. @kkuchta 2. Search
44. @kkuchta 2. Search
45. @kkuchta Ag is cool $ ag 'something'
46. @kkuchta Ag is cool $ ag 'something' $ ag -i 'something'
47. @kkuchta Ag is cool $ ag 'something' $ ag -i 'something' $ ag 'some.*thing'
48. @kkuchta Ag is cool $ ag 'something' $ ag -i 'something' $ ag 'some.*thing' $ ag --ruby 'something'
49. @kkuchta Regex Search user_profile UserProfile # user profile
50. @kkuchta Regex Search user_profile UserProfile # user profile $ ag -i 'user.?profile'
51. @kkuchta Searching session.subreddit('all').comments.stream do |comment| puts comment end
52. @kkuchta Searching session.subreddit('all').comments.stream do |comment| puts comment end $ ag stream
53. @kkuchta Iterative Searching
54. @kkuchta Iterative Searching
55. @kkuchta Iterative Searching paginated_listing.rb
56. @kkuchta Iterative Searching paginated_listing.rb
57. @kkuchta Iterative Searching paginated_listing.rb
58. @kkuchta Iterative Searching $ ag --ruby paginated
59. @kkuchta Iterative Searching $ ag --ruby paginated
60. @kkuchta Iterative Searching $ ag --ruby paginated session.subreddit('all').comments.stream
61. @kkuchta Iterative Searching session.subreddit('all').comments.stream In subreddit.rb:
62. @kkuchta Iterative Searching session.subreddit('all').comments.stream In subreddit.rb:
63. @kkuchta • def stream • PaginatedListing • def _stream • dead end • Subreddit • def listing • define_method(:comments) Iterative Searching session.subreddit('all').comments.stream
64. @kkuchta Iterative Searching 1. See something interesting 2. Search for it 3. GOTO 1
65. @kkuchta Iterative Searching Gotcha: metaprogramming def add_one(x) x + 1 end
66. @kkuchta Iterative Searching Gotcha: metaprogramming def add_one(x) x + 1 end define_method("add_one") do |x| x + 1 end
67. @kkuchta Iterative Searching Gotcha: metaprogramming def add_one(x) x + 1 end define_method("add_one") do |x| x + 1 end method_name = "add" + "_" + "one" define_method(method_name) do |x| x + 1 end
68. @kkuchta Iterative Searching Gotcha: metaprogramming User.find_by_email('kevin@example.com')
69. @kkuchta Iterative Searching Gotcha: metaprogramming User.find_by_email('kevin@example.com')
70. @kkuchta Searching can?(:read, some_post)
71. @kkuchta Searching can?(:read, some_post)
72. @kkuchta Searching: Referral can?(:read, some_post)
73. @kkuchta Searching: Referral can?(:read, some_post) referral --exact-name 'can?'
74. @kkuchta Searching: Referral can?(:read, some_post) referral --exact-name 'can?'
75. @kkuchta Searching: Referral referral --exact-name 'can?' --type 'instance_method' --columns file,line,source
76. @kkuchta Searching: Referral • Search by full name, exact name, partial name • Filter by type (constant, instance_method, local var, etc) • Regexes everywhere • Sort by anything • Customize output • Filter by scope • github.com/testdouble/referral
77. @kkuchta 2. Searching • Tools • ag • referral gem • Iterative searching
78. @kkuchta 3. Tracing
79. @kkuchta 3. Trace It result = some_gem.do_thing(x, y, true)
80. @kkuchta 3. Trace It puts "x = #{x}, y = #{y}" puts "some_gem = #{some_gem.inspect}" result = some_gem.do_thing(x, y, true) puts "result = #{result}"
81. @kkuchta 3. Trace It
82. @kkuchta 3. Trace It $ echo $GEM_HOME
83. @kkuchta 3. Trace It /Users/kevin/.rvm/gems/ruby-2.0.0-p648 $ echo $GEM_HOME
84. @kkuchta 3. Trace It /Users/kevin/.rvm/gems/ruby-2.0.0-p648 $ echo $GEM_HOME $ ls $GEM_HOME/gems/
85. @kkuchta 3. Trace It /Users/kevin/.rvm/gems/ruby-2.0.0-p648 actionmailer-4.0.4 commander-4.1.3 jquery-rails-3.1.1 actionpack-4.0.4 diff-lcs-1.2.5 json-1.8.1 activemodel-4.0.4 directory_watcher-1.4.1 kgio-2.9.2 activerecord-4.0.4 erubis-2.7.0 kramdown-0.14.2 activerecord-deprecated_finders-1.0.3 execjs-2.2.1 liquid-2.5.0 activesupport-4.0.4 fast-stemmer-1.0.2 listen-1.0.3 arel-4.0.2 ffi-1.8.1 lumberjack-1.0.3 builder-3.1.4 formatador-0.2.4 mail-2.5.4 byebug-9.0.6 guard-1.8.0 maruku-0.6.1 classifier-1.3.3 guard-haml-0.5 method_source-0.8.1 coderay-1.0.9 haml-3.1.4 method_source-0.9.2 coderay-1.1.2 highline-1.6.18 mime-types-1.25.1 coffee-rails-4.0.1 hike-1.2.3 minitest-4.7.5 coffee-script-2.3.0 i18n-0.6.11 multi_json-1.10.1 coffee-script-source-1.7.1 jbuilder-1.5.3 pg-0.17.1 $ echo $GEM_HOME $ ls $GEM_HOME/gems/
86. @kkuchta Rake
87. @kkuchta Rake code $ vim $GEM_HOME/gems/rake-13.0.0
88. @kkuchta Rake code $ vim $GEM_HOME/gems/rake-13.0.0
89. @kkuchta Rake code $ vim $GEM_HOME/gems/rake-13.0.0
90. @kkuchta 3. Trace It
91. @kkuchta 3. Trace It stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a
92. @kkuchta 3. Trace It /Users/kevin/.rvm/gems/ruby-2.6.0/gems/net-sftp-2.1.2/lib/net/sftp/operations/ file.rb:85:in `gets': wrong number of arguments (given 2, expected 0..1) stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a
93. @kkuchta 3. Trace It stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a
94. @kkuchta 3. Trace It 1. from /Users/kevin/.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/csv/parser.rb:392:in `resolve_row_separator' 2. /Users/kevin/.rvm/gems/ruby-2.6.0/gems/net-sftp-2.1.2/lib/net/sftp/operations/ file.rb:85:in `gets': wrong number of arguments (given 2, expected 0..1) stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a
95. @kkuchta 3. Trace It 2. /Users/kevin/.rvm/gems/ruby-2.6.0/gems/net-sftp-2.1.2/lib/net/sftp/operations/ file.rb:85:in `gets': wrong number of arguments (given 2, expected 0..1) stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a 1. from /Users/kevin/.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/csv/parser.rb:392:in `resolve_row_separator'
96. @kkuchta 3. Trace It stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a from /Users/kevin/.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/csv/parser.rb:392:in `resolve_row_separator' 392
97. @kkuchta 3. Trace It from /Users/kevin/.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/csv/parser.rb:392:in `resolve_row_separator' stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a
98. @kkuchta 3. Trace It stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a from /Users/kevin/.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/csv/parser.rb:392:in `resolve_row_separator'
99. @kkuchta 3. Trace It stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a /Users/kevin/.rvm/gems/ruby-2.6.0/gems/net-sftp-2.1.2/lib/net/sftp/operations/ file.rb:85:in `gets': wrong number of arguments (given 2, expected 0..1) from /Users/kevin/.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/csv/parser.rb:392:in `resolve_row_separator'
100. @kkuchta 3. Trace It stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a /Users/kevin/.rvm/gems/ruby-2.6.0/gems/net-sftp-2.1.2/lib/net/sftp/operations/ file.rb:85:in `gets': wrong number of arguments (given 2, expected 0..1)
101. @kkuchta 3. Trace It stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a
102. @kkuchta 3. Trace It stream = sftp.file.open(filepath) csv = CSV.new(stream).to_a
103. @kkuchta 3. Trace It • Print debugging • Quick, easy • Rerun to make changes
104. @kkuchta 3. Trace It • Print debugging • Quick, easy • Rerun to make changes • Debugger debugging • Steeper learning curve • Exploratory tracing
105. @kkuchta 3. Trace It def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end first(7)
106. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It
107. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It
108. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It
109. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)>
110. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step
111. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step
112. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step
113. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step pry(main)>
114. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step pry(main)> x
115. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step pry(main)> x => 7 pry(main)>
116. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step pry(main)> x => 7 pry(main)> next pry(main)>
117. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step pry(main)> x => 7 pry(main)> next pry(main)> x > 5 => true pry(main)>
118. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step pry(main)> x => 7 pry(main)> next pry(main)> x > 5 => true pry(main)> next pry(main)>
119. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step pry(main)> x => 7 pry(main)> next pry(main)> x > 5 => true pry(main)> next pry(main)> step pry(main)>
120. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step pry(main)> x => 7 pry(main)> next pry(main)> x > 5 => true pry(main)> next pry(main)> step pry(main)> y => 10 pry(main)>
121. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step pry(main)> x => 7 pry(main)> next pry(main)> x > 5 => true pry(main)> next pry(main)> step pry(main)> y => 10 pry(main)> finish
122. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It pry(main)> step pry(main)> x => 7 pry(main)> next pry(main)> x > 5 => true pry(main)> next pry(main)> step pry(main)> y => 10 pry(main)> finish
123. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It
124. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It next => Go to the next line
125. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It next => Go to the next line step => step into this function
126. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It next => Go to the next line step => step into this function finish => finish this function
127. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It next => Go to the next line step => step into this function finish => finish this function
128. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It next => Go to the next line step => step into this function finish => finish this function continue => stop debugging + move on
129. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It next => Go to the next line step => step into this function finish => finish this function whereami => print the code around you continue => stop debugging + move on
130. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It next => Go to the next line step => step into this function finish => finish this function whereami => print the code around you backtrace => print the current stack trace continue => stop debugging + move on
131. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It next => Go to the next line step => step into this function finish => finish this function whereami => print the code around you backtrace => print the current stack trace !!! => exit the whole program continue => stop debugging + move on
132. @kkuchta require 'pry-byebug' def first(x) x = x + 3 if x > 5 x = second(x) end x end def second(y) y = y + 4 while y > 1 y = y / 2.0 end y end binding.pry first(7) 3. Trace It next => Go to the next line step => step into this function finish => finish this function whereami => print the code around you backtrace => print the current stack trace !!! => exit the whole program continue => stop debugging + move on help => exit the whole program
133. @kkuchta Debuggers are great
134. @kkuchta Debuggers are great • Debugger: pry-byebug
135. @kkuchta Debuggers are great • Debugger: pry-byebug • Rebugger: Kevin Kuchta
136. @kkuchta Debuggers are great • Debugger: pry-byebug • Rebugger: Kevin Kuchta • Start in your code and step into gem code
137. @kkuchta Debuggers are great stream = sftp.file.open(filepath) csv = CSV.new(stream) csv.to_a
138. @kkuchta Debuggers are great binding.pry stream = sftp.file.open(filepath) csv = CSV.new(stream) csv.to_a
139. @kkuchta Debuggers are great binding.pry stream = sftp.file.open(filepath) csv = CSV.new(stream) csv.to_a
140. @kkuchta Debuggers are great binding.pry stream = sftp.file.open(filepath) csv = CSV.new(stream) csv.to_a
141. @kkuchta Debuggers are great binding.pry stream = sftp.file.open(filepath) csv = CSV.new(stream) csv.to_a
142. @kkuchta Debuggers are great binding.pry stream = sftp.file.open(filepath) csv = CSV.new(stream) csv.to_a pry(main)> step
143. @kkuchta Debuggers are great stream = sftp.file.open(filepath) csv = CSV.new(stream) csv.to_a
144. @kkuchta Debuggers are great stream = sftp.file.open(filepath) csv = CSV.new(stream) csv.to_a
145. @kkuchta Debuggers are great
146. @kkuchta Debuggers are great `gets': wrong number of arguments (given 2, expected 0..1)
147. @kkuchta Tracing: Forwards Starting from: result = first(c)
148. @kkuchta Tracing: Forwards Starting from: result = first(c) def first(x) second(x - 3) end
149. @kkuchta Tracing: Forwards Starting from: result = first(c) def first(x) second(x - 3) end def second(y) third(y * 2) end
150. @kkuchta Tracing: Forwards Starting from: result = first(c) def first(x) second(x - 3) end def second(y) third(y * 2) end def third(z) z / z + 1 end
151. @kkuchta Tracing: Forwards • Debuggers • Step into each new level • Print debugging • See how far the code is getting in each level • Good when you have a function call that's not doing what you'd expect
152. @kkuchta Tracing: Backwards Starting from: result = foobar(c)
153. @kkuchta Tracing: Backwards Starting from: result = foobar(c) def foo(c) ...
154. @kkuchta Tracing: Backwards Starting from: result = foobar(c) def foo(c) ... foo(b + 27)
155. @kkuchta Tracing: Backwards Starting from: result = foobar(c) def foo(c) ... foo(b + 27) b = a / 15
156. @kkuchta Tracing: Backwards
157. @kkuchta • Start at your point of interest Tracing: Backwards
158. @kkuchta • Start at your point of interest • Good for exceptions Tracing: Backwards
159. @kkuchta • Start at your point of interest • Good for exceptions • Text search / referral Tracing: Backwards
160. @kkuchta Tracing
161. @kkuchta • Searching for references Tracing
162. @kkuchta • Searching for references • Check current file Tracing
163. @kkuchta • Searching for references • Check current file • Check includes / extends Tracing
164. @kkuchta • Searching for references • Check current file • Check includes / extends • Metaprogramming Tracing
165. @kkuchta • Searching for references • Check current file • Check includes / extends • Metaprogramming • Use ag to search the whole codebase Tracing
166. @kkuchta • Searching for references • Check current file • Check includes / extends • Metaprogramming • Use ag to search the whole codebase • Check gem files Tracing
167. @kkuchta Tracing: Gem Gems
168. @kkuchta Tracing: Gem Gems
169. @kkuchta Tracing: Gem Gems
170. @kkuchta Tracing: Gem Gems
171. @kkuchta Tracing: Gem Gems
172. @kkuchta Tracing: Gem Gems
173. @kkuchta Tracing: Gem Gems
174. @kkuchta Tracing: Gem Gems
175. @kkuchta Tracing: Gem Gems
176. @kkuchta Tracing: Gem Gems
177. @kkuchta The Kuchta 3ish-step Process©™® 1. Take me to your leader • Entry points • Big files
178. @kkuchta The Kuchta 3ish-step Process©™® 1. Take me to your leader • Entry points • Big files 2. Text search • Ag • Repeated searching
179. @kkuchta The Kuchta 3ish-step Process©™® 1. Take me to your leader • Entry points • Big files 2. Text search • Ag • Repeated searching 3. Trace • Puts statements • Debugger
180. @kkuchta @kkuchta
181. @kkuchta @kkuchta
182. @kkuchta Thanks! Kevin Kuchta @kkuchta @kkuchta@mastodon.social kevinkuchta.com