Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Life on the Edge

jeg2
November 06, 2011

Life on the Edge

This was my presentation on edge cases for Ruby Midwest 2011.

jeg2

November 06, 2011
Tweet

More Decks by jeg2

Other Decks in Programming

Transcript

  1. James Edward Gray II I’ve been in the community for

    long time now I’ve written code and docs for Ruby I’m a Ruby Rogue
  2. Figuring Out How to Survive Flung to the far end

    of the universe, the SGU crew has to go back to the basics of survival
  3. Edge Cases are the Currency of Programmers Most of our

    development effort is spent on them If we didn’t have them, programming would be easy (maybe even automatic) How many bugs aren’t edge cases? This is important stuff!
  4. Numbers are Behind Letters What are the minimal cards you

    have to flip to prove there’s an even behind each vowel?
  5. Numbers are Behind Letters What are the minimal cards you

    have to flip to prove there’s an even behind each vowel?
  6. Numbers are Behind Letters What are the minimal cards you

    have to flip to prove there’s an even behind each vowel?
  7. Numbers are Behind Letters What are the minimal cards you

    have to flip to prove there’s an even behind each vowel?
  8. Edge Case Thinking Edge cases are the important stuff The

    rest of our work is at least easier Given that, I simply prefer to focus on the edge cases first, when possible
  9. The Trouble With nil It’s an edge case, by definition

    Given that, it’s best avoided as much as possible
  10. Returning nil This is an example of a common, but

    pointless edge case module GooglePlus module_function def has_any_circles?(user) !!check_api_for_any_circles?(user) end def circles(user) if has_any_circles? user get_circles_from_api(user) else nil end end # ... end
  11. Using nil We are forced into an edge case at

    this end as well if circles = GooglePlus.circles(user) circles.each do |circle| # use with circle here... end end
  12. My Favorite De-edger This unusual core Ruby method can guarantee

    an Array (Enumerable) result Array([1, 2, 3]) # => [1, 2, 3] Array(:not_an_array) # => [:not_an_array] Array(nil) # => []
  13. Dodging the Edge Case Remove the edge case instead of

    work with it, when possible Array(GooglePlus.circles(user)).each do |circle| # use with circle here... end
  14. Avoiding Returning nil This method has a much better contract:

    I’ll return to you a list of circles module GooglePlus module_function def circles(user) Array(get_circles_from_api(user)) end # ... end
  15. Ideal Usage Getting the method contract right automatically fixed this

    code too GooglePlus.circles(user).each do |circle| # use with circle here... end
  16. We Haven’t Lost Anything We can still handle the no

    circles case, if needed circles = GooglePlus.circles(user) if circles.empty? # handle no circles here... else circles.each do |circle| # use with circle here... end end
  17. Too Much Heat? When Destiny begins flying straight into a

    sun, the crew assumes they are about to burn up
  18. Data as a Stream Avoid “slurping” a ton of data

    in whenever possible Big data is a scary edge case Draw only what you need at once Come back for more as needed This applies to all data, including database queries
  19. No Slurping Some exceptions: you produce and consume, you ensure

    a reasonable size, etc. File.read(path).each_line do |line| # use line here... end
  20. No Slurping Some exceptions: you produce and consume, you ensure

    a reasonable size, etc. File.read(path).each_line do |line| # use line here... end
  21. A Better Way It’s rarely any harder to work with

    data in chunks File.foreach(path) do |line| # use line here... end
  22. The Rails Version In Rails, all() without a where() is

    generally the same as read() @comments = @post.comments.all
  23. The Rails Version In Rails, all() without a where() is

    generally the same as read() @comments = @post.comments.all
  24. Better It’s OK to grab subsets of data @comments =

    @post.comments .where("created_at >= ?", 1.day.ago) .all # ... or ... @comments = @post.comments.paginate(params[:page])
  25. Best You can even walk all of the data in

    subsets @post.comments.find_each do |comment| # use comment here... end
  26. Risk Management Tests are about managing concerns Edge cases are

    the primary concerns Focus your efforts there
  27. Test Fodder Given a trivial method… class Post < ActiveRecord::Base

    def editable? created_at >= 1.hour.ago end # ... end
  28. Boring Tests We often test it like this describe Post

    do it "is editable within an hour of creation" do Post.new(created_at: 10.minutes.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: 1.day.ago).wont_be :editable? end end
  29. Boring Tests We often test it like this describe Post

    do it "is editable within an hour of creation" do Post.new(created_at: 10.minutes.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: 1.day.ago).wont_be :editable? end end
  30. Moving to the Edge Spend your paranoia on the edges

    instead describe Post do it "is editable within an hour of creation" do Post.new(created_at: 1.hour.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: (1.hour + 1.minute).ago).wont_be :editable? end end
  31. Moving to the Edge Spend your paranoia on the edges

    instead describe Post do it "is editable within an hour of creation" do Post.new(created_at: 1.hour.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: (1.hour + 1.minute).ago).wont_be :editable? end end
  32. Edges Everywhere There are so many edges in programming The

    relationships between objects Interactions with frameworks Networks and other systems So much more…
  33. It Pays to be Prepared When investigating alien technology, the

    crew protects themselves from the environment
  34. The Hardline Make edge cases impossible, when you can #

    guarantee we do not touch API's class Geokit::Geocoders::MultiGeocoder def self.geocode(*args) fail "Tried to contact Google" end end class Boomerang::APICall def initialize(*args) fail "Tried to contact Amazon" end end module SendGridDisabler def make_api_call(*args) fail "Tried to contact SendGrid" end private :make_api_call end SendGrid.extend(SendGridDisabler)
  35. Eli: An Edge Case Thinker As the crew’s boy genius,

    Eli is always thinking his way around their edge cases
  36. Don’t Do Extra Work! I’m not advocating heavy planning here

    There are so many great answers to edge cases besides hard work: It won’t affect us We accept the limitations There’s a easier way Etc.
  37. Linear Requests (Part 1) Faraday allows us to abstract away

    the process of making requests and use middleware require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter Faraday.default_adapter end # ...
  38. Linear Requests (Part 1) Faraday allows us to abstract away

    the process of making requests and use middleware require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter Faraday.default_adapter end # ...
  39. Linear Requests (Part 2) We request the list of repos,

    then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
  40. Linear Requests (Part 2) We request the list of repos,

    then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
  41. Linear Requests (Part 2) We request the list of repos,

    then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
  42. Linear Requests (Part 2) We request the list of repos,

    then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
  43. Linear Requests (Part 2) We request the list of repos,

    then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
  44. The Request Tree After fetching the list of repos, the

    remaining requests are independent
  45. Parallel Requests (Part 1) Thanks to the abstraction, switching HTTP

    clients is trivial require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter :typhoeus end # ...
  46. Parallel Requests (Part 1) Thanks to the abstraction, switching HTTP

    clients is trivial require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter :typhoeus end # ...
  47. Parallel Requests (Part 2) We can now fetch the list,

    then handle the repo requests in parallel # ... ua.in_parallel(Typhoeus::Hydra.hydra) do repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end end
  48. Parallel Requests (Part 2) We can now fetch the list,

    then handle the repo requests in parallel # ... ua.in_parallel(Typhoeus::Hydra.hydra) do repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end end
  49. The Parallel Advantage We’ve cut the time for these requests

    down by about 84% $ time ruby linear_requests.rb highline: 1 faster_csv: 1 real 0m11.087s user 0m0.466s sys 0m0.121s $ time ruby parallel_requests.rb highline: 1 faster_csv: 1 real 0m1.721s user 0m0.403s sys 0m0.104s
  50. The Parallel Advantage We’ve cut the time for these requests

    down by about 84% $ time ruby linear_requests.rb highline: 1 faster_csv: 1 real 0m11.087s user 0m0.466s sys 0m0.121s $ time ruby parallel_requests.rb highline: 1 faster_csv: 1 real 0m1.721s user 0m0.403s sys 0m0.104s