Life on the Edge

259f23c3b129f07b0c496b9f0495f07e?s=47 jeg2
November 06, 2011

Life on the Edge

This was my presentation on edge cases for Ruby Midwest 2011.

259f23c3b129f07b0c496b9f0495f07e?s=128

jeg2

November 06, 2011
Tweet

Transcript

  1. Life on the Edge Introducing “Edge Case Thinking”

  2. Who Am I?

  3. James Edward Gray II I’ve been in the community for

    long time now I’ve written code and docs for Ruby I’m a Ruby Rogue
  4. Edge Cases

  5. The ultimate edge case

  6. Stargate Universe The ultimate edge case

  7. Figuring Out How to Survive Flung to the far end

    of the universe, the SGU crew has to go back to the basics of survival
  8. Edge Cases are the Currency of Programmers Most of our

    development effort is spent on them If we didn’t have them, programming would be easy (maybe even automatic) How many bugs aren’t edge cases? This is important stuff!
  9. Numbers are Behind Letters What are the minimal cards you

    have to flip to prove there’s an even behind each vowel?
  10. Numbers are Behind Letters What are the minimal cards you

    have to flip to prove there’s an even behind each vowel?
  11. Numbers are Behind Letters What are the minimal cards you

    have to flip to prove there’s an even behind each vowel?
  12. Numbers are Behind Letters What are the minimal cards you

    have to flip to prove there’s an even behind each vowel?
  13. Reprogramming Their Brains The crew had to rework their thoughts

    to match their new circumstances
  14. Edge Case Thinking Edge cases are the important stuff The

    rest of our work is at least easier Given that, I simply prefer to focus on the edge cases first, when possible
  15. A Stargate Safari As Destiny travels, Stargates allow the crew

    to explore many worlds
  16. nil: The Edge of Nothing

  17. The Trouble With nil It’s an edge case, by definition

    Given that, it’s best avoided as much as possible
  18. Returning nil This is an example of a common, but

    pointless edge case module GooglePlus module_function def has_any_circles?(user) !!check_api_for_any_circles?(user) end def circles(user) if has_any_circles? user get_circles_from_api(user) else nil end end # ... end
  19. Using nil We are forced into an edge case at

    this end as well if circles = GooglePlus.circles(user) circles.each do |circle| # use with circle here... end end
  20. My Favorite De-edger This unusual core Ruby method can guarantee

    an Array (Enumerable) result Array([1, 2, 3]) # => [1, 2, 3] Array(:not_an_array) # => [:not_an_array] Array(nil) # => []
  21. Dodging the Edge Case Remove the edge case instead of

    work with it, when possible Array(GooglePlus.circles(user)).each do |circle| # use with circle here... end
  22. Avoiding Returning nil This method has a much better contract:

    I’ll return to you a list of circles module GooglePlus module_function def circles(user) Array(get_circles_from_api(user)) end # ... end
  23. Ideal Usage Getting the method contract right automatically fixed this

    code too GooglePlus.circles(user).each do |circle| # use with circle here... end
  24. We Haven’t Lost Anything We can still handle the no

    circles case, if needed circles = GooglePlus.circles(user) if circles.empty? # handle no circles here... else circles.each do |circle| # use with circle here... end end
  25. Slurping Your Food Data

  26. Too Much Heat? When Destiny begins flying straight into a

    sun, the crew assumes they are about to burn up
  27. Data as a Stream Avoid “slurping” a ton of data

    in whenever possible Big data is a scary edge case Draw only what you need at once Come back for more as needed This applies to all data, including database queries
  28. No Slurping Some exceptions: you produce and consume, you ensure

    a reasonable size, etc. File.read(path).each_line do |line| # use line here... end
  29. No Slurping Some exceptions: you produce and consume, you ensure

    a reasonable size, etc. File.read(path).each_line do |line| # use line here... end
  30. A Better Way It’s rarely any harder to work with

    data in chunks File.foreach(path) do |line| # use line here... end
  31. The Rails Version In Rails, all() without a where() is

    generally the same as read() @comments = @post.comments.all
  32. The Rails Version In Rails, all() without a where() is

    generally the same as read() @comments = @post.comments.all
  33. Better It’s OK to grab subsets of data @comments =

    @post.comments .where("created_at >= ?", 1.day.ago) .all # ... or ... @comments = @post.comments.paginate(params[:page])
  34. Best You can even walk all of the data in

    subsets @post.comments.find_each do |comment| # use comment here... end
  35. Edgy Tests

  36. Risk Management Tests are about managing concerns Edge cases are

    the primary concerns Focus your efforts there
  37. Test Fodder Given a trivial method… class Post < ActiveRecord::Base

    def editable? created_at >= 1.hour.ago end # ... end
  38. Boring Tests We often test it like this describe Post

    do it "is editable within an hour of creation" do Post.new(created_at: 10.minutes.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: 1.day.ago).wont_be :editable? end end
  39. Boring Tests We often test it like this describe Post

    do it "is editable within an hour of creation" do Post.new(created_at: 10.minutes.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: 1.day.ago).wont_be :editable? end end
  40. Moving to the Edge Spend your paranoia on the edges

    instead describe Post do it "is editable within an hour of creation" do Post.new(created_at: 1.hour.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: (1.hour + 1.minute).ago).wont_be :editable? end end
  41. Moving to the Edge Spend your paranoia on the edges

    instead describe Post do it "is editable within an hour of creation" do Post.new(created_at: 1.hour.ago).must_be :editable? end it "is not editable beyond an hour of creation" do Post.new(created_at: (1.hour + 1.minute).ago).wont_be :editable? end end
  42. Edges Everywhere There are so many edges in programming The

    relationships between objects Interactions with frameworks Networks and other systems So much more…
  43. It Pays to be Prepared When investigating alien technology, the

    crew protects themselves from the environment
  44. The Hardline Make edge cases impossible, when you can #

    guarantee we do not touch API's class Geokit::Geocoders::MultiGeocoder def self.geocode(*args) fail "Tried to contact Google" end end class Boomerang::APICall def initialize(*args) fail "Tried to contact Amazon" end end module SendGridDisabler def make_api_call(*args) fail "Tried to contact SendGrid" end private :make_api_call end SendGrid.extend(SendGridDisabler)
  45. All at Once But in Order

  46. Eli: An Edge Case Thinker As the crew’s boy genius,

    Eli is always thinking his way around their edge cases
  47. Don’t Do Extra Work! I’m not advocating heavy planning here

    There are so many great answers to edge cases besides hard work: It won’t affect us We accept the limitations There’s a easier way Etc.
  48. Linear Requests (Part 1) Faraday allows us to abstract away

    the process of making requests and use middleware require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter Faraday.default_adapter end # ...
  49. Linear Requests (Part 1) Faraday allows us to abstract away

    the process of making requests and use middleware require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter Faraday.default_adapter end # ...
  50. Linear Requests (Part 2) We request the list of repos,

    then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
  51. Linear Requests (Part 2) We request the list of repos,

    then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
  52. Linear Requests (Part 2) We request the list of repos,

    then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
  53. Linear Requests (Part 2) We request the list of repos,

    then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
  54. Linear Requests (Part 2) We request the list of repos,

    then fetch each repo in turn for an issue count # ... repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end
  55. The Request Tree After fetching the list of repos, the

    remaining requests are independent
  56. Parallel Requests (Part 1) Thanks to the abstraction, switching HTTP

    clients is trivial require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter :typhoeus end # ...
  57. Parallel Requests (Part 1) Thanks to the abstraction, switching HTTP

    clients is trivial require "faraday_middleware" # load Faraday plus extra middleware ua = Faraday.new("https://api.github.com/") do |builder| builder.use Faraday::Response::ParseJson # parse JSON responses builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses builder.adapter :typhoeus end # ...
  58. Parallel Requests (Part 2) We can now fetch the list,

    then handle the repo requests in parallel # ... ua.in_parallel(Typhoeus::Hydra.hydra) do repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end end
  59. Parallel Requests (Part 2) We can now fetch the list,

    then handle the repo requests in parallel # ... ua.in_parallel(Typhoeus::Hydra.hydra) do repos = ua.get("/users/JEG2/repos") repos.on_complete do repos.body.each do |repo| name = repo["name"] project = ua.get("/repos/JEG2/#{name}") project.on_complete do issues = project.body["open_issues"] puts "#{name}: #{issues}" if issues.nonzero? end end end end
  60. The Parallel Advantage We’ve cut the time for these requests

    down by about 84% $ time ruby linear_requests.rb highline: 1 faster_csv: 1 real 0m11.087s user 0m0.466s sys 0m0.121s $ time ruby parallel_requests.rb highline: 1 faster_csv: 1 real 0m1.721s user 0m0.403s sys 0m0.104s
  61. The Parallel Advantage We’ve cut the time for these requests

    down by about 84% $ time ruby linear_requests.rb highline: 1 faster_csv: 1 real 0m11.087s user 0m0.466s sys 0m0.121s $ time ruby parallel_requests.rb highline: 1 faster_csv: 1 real 0m1.721s user 0m0.403s sys 0m0.104s
  62. Thanks