Life on the Edge

This was my presentation on edge cases for Ruby Midwest 2011.

Transcript

1. Life on the Edge Introducing “Edge Case Thinking”
2. Who Am I?
3. James Edward Gray II I’ve been in the community for long time now I’ve written code and docs for Ruby I’m a Ruby Rogue
4. Edge Cases
5. The ultimate edge case
6. Stargate Universe The ultimate edge case
7. Figuring Out How to Survive Flung to the far end of the universe, the SGU crew has to go back to the basics of survival
8. Edge Cases are the Currency of Programmers Most of our development effort is spent on them If we didn’t have them, programming would be easy (maybe even automatic) How many bugs aren’t edge cases? This is important stuff!
9. Numbers are Behind Letters What are the minimal cards you have to flip to prove there’s an even behind each vowel?
10. Numbers are Behind Letters What are the minimal cards you have to flip to prove there’s an even behind each vowel?
11. Numbers are Behind Letters What are the minimal cards you have to flip to prove there’s an even behind each vowel?
12. Numbers are Behind Letters What are the minimal cards you have to flip to prove there’s an even behind each vowel?
13. Reprogramming Their Brains The crew had to rework their thoughts to match their new circumstances
14. Edge Case Thinking Edge cases are the important stuff The rest of our work is at least easier Given that, I simply prefer to focus on the edge cases first, when possible
15. A Stargate Safari As Destiny travels, Stargates allow the crew to explore many worlds
16. nil: The Edge of Nothing
17. The Trouble With nil It’s an edge case, by definition Given that, it’s best avoided as much as possible
18. Returning nil This is an example of a common, but pointless edge case module GooglePlus module_function def has_any_circles?(user) !!check_api_for_any_circles?(user) end def circles(user) if has_any_circles? user get_circles_from_api(user) else nil end end # ... end
19. Using nil We are forced into an edge case at this end as well if circles = GooglePlus.circles(user) circles.each do |circle| # use with circle here... end end
20. My Favorite De-edger This unusual core Ruby method can guarantee an Array (Enumerable) result Array([1, 2, 3]) # => [1, 2, 3] Array(:not_an_array) # => [:not_an_array] Array(nil) # => []
21. Dodging the Edge Case Remove the edge case instead of work with it, when possible Array(GooglePlus.circles(user)).each do |circle| # use with circle here... end
22. Avoiding Returning nil This method has a much better contract: I’ll return to you a list of circles module GooglePlus module_function def circles(user) Array(get_circles_from_api(user)) end # ... end
23. Ideal Usage Getting the method contract right automatically fixed this code too GooglePlus.circles(user).each do |circle| # use with circle here... end
24. We Haven’t Lost Anything We can still handle the no circles case, if needed circles = GooglePlus.circles(user) if circles.empty? # handle no circles here... else circles.each do |circle| # use with circle here... end end
25. Slurping Your Food Data
26. Too Much Heat? When Destiny begins flying straight into a sun, the crew assumes they are about to burn up
27. Data as a Stream Avoid “slurping” a ton of data in whenever possible Big data is a scary edge case Draw only what you need at once Come back for more as needed This applies to all data, including database queries
28. No Slurping Some exceptions: you produce and consume, you ensure a reasonable size, etc. File.read(path).each_line do |line| # use line here... end
29. No Slurping Some exceptions: you produce and consume, you ensure a reasonable size, etc. File.read(path).each_line do |line| # use line here... end
30. A Better Way It’s rarely any harder to work with data in chunks File.foreach(path) do |line| # use line here... end
31. The Rails Version In Rails, all() without a where() is generally the same as read() @comments = @post.comments.all
32. The Rails Version In Rails, all() without a where() is generally the same as read() @comments = @post.comments.all
33. Better It’s OK to grab subsets of data @comments = @post.comments .where("created_at >= ?", 1.day.ago) .all # ... or ... @comments = @post.comments.paginate(params[:page])
34. Best You can even walk all of the data in subsets @post.comments.find_each do |comment| # use comment here... end
35. Edgy Tests
36. Risk Management Tests are about managing concerns Edge cases are the primary concerns Focus your efforts there
37. Test Fodder Given a trivial method… class Post < ActiveRecord::Base def editable? created_at >= 1.hour.ago end # ... end
38. Boring Tests We often test it like this describe Post do it "is editable within an hour of creation" do Post.new(created_at: 10.minutes.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: 1.day.ago).wont_be :editable? end end
39. Boring Tests We often test it like this describe Post do it "is editable within an hour of creation" do Post.new(created_at: 10.minutes.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: 1.day.ago).wont_be :editable? end end
40. Moving to the Edge Spend your paranoia on the edges instead describe Post do it "is editable within an hour of creation" do Post.new(created_at: 1.hour.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: (1.hour + 1.minute).ago).wont_be :editable? end end
41. Moving to the Edge Spend your paranoia on the edges instead describe Post do it "is editable within an hour of creation" do Post.new(created_at: 1.hour.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: (1.hour + 1.minute).ago).wont_be :editable? end end
42. Edges Everywhere There are so many edges in programming The relationships between objects Interactions with frameworks Networks and other systems So much more…
43. It Pays to be Prepared When investigating alien technology, the crew protects themselves from the environment
44. The Hardline Make edge cases impossible, when you can # guarantee we do not touch API's class Geokit::Geocoders::MultiGeocoder def self.geocode(*args) fail "Tried to contact Google" end end class Boomerang::APICall def initialize(*args) fail "Tried to contact Amazon" end end module SendGridDisabler def make_api_call(*args) fail "Tried to contact SendGrid" end private :make_api_call end SendGrid.extend(SendGridDisabler)
45. All at Once But in Order
46. Eli: An Edge Case Thinker As the crew’s boy genius, Eli is always thinking his way around their edge cases
47. Don’t Do Extra Work! I’m not advocating heavy planning here There are so many great answers to edge cases besides hard work: It won’t affect us We accept the limitations There’s a easier way Etc.
48. Linear Requests (Part 1) Faraday allows us to abstract away the process of making requests and use middleware require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter Faraday.default_adapter end # ...
49. Linear Requests (Part 1) Faraday allows us to abstract away the process of making requests and use middleware require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter Faraday.default_adapter end # ...
50. Linear Requests (Part 2) We request the list of repos, then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
51. Linear Requests (Part 2) We request the list of repos, then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
52. Linear Requests (Part 2) We request the list of repos, then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
53. Linear Requests (Part 2) We request the list of repos, then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
54. Linear Requests (Part 2) We request the list of repos, then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
55. The Request Tree After fetching the list of repos, the remaining requests are independent
56. Parallel Requests (Part 1) Thanks to the abstraction, switching HTTP clients is trivial require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter :typhoeus end # ...
57. Parallel Requests (Part 1) Thanks to the abstraction, switching HTTP clients is trivial require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter :typhoeus end # ...
58. Parallel Requests (Part 2) We can now fetch the list, then handle the repo requests in parallel # ... ua.in_parallel(Typhoeus::Hydra.hydra) do repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end end
59. Parallel Requests (Part 2) We can now fetch the list, then handle the repo requests in parallel # ... ua.in_parallel(Typhoeus::Hydra.hydra) do repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end end
60. The Parallel Advantage We’ve cut the time for these requests down by about 84% $ time ruby linear_requests.rb highline: 1 faster_csv: 1 real 0m11.087s user 0m0.466s sys 0m0.121s $ time ruby parallel_requests.rb highline: 1 faster_csv: 1 real 0m1.721s user 0m0.403s sys 0m0.104s
61. The Parallel Advantage We’ve cut the time for these requests down by about 84% $ time ruby linear_requests.rb highline: 1 faster_csv: 1 real 0m11.087s user 0m0.466s sys 0m0.121s $ time ruby parallel_requests.rb highline: 1 faster_csv: 1 real 0m1.721s user 0m0.403s sys 0m0.104s
62. Thanks