Life on the Edge

This was my presentation on edge cases for Ruby Midwest 2011.

Transcript

1. Life on the Edge
   Introducing “Edge Case Thinking”
   

   View Slide

2. Who Am I?
   

   View Slide

3. James Edward Gray II
   I’ve been in the
   community for long
   time now
   I’ve written code and
   docs for Ruby
   I’m a Ruby Rogue
   

   View Slide

4. Edge Cases
   

   View Slide

5. The ultimate edge case
   

   View Slide

6. Stargate Universe
   The ultimate edge case
   

   View Slide

7. Figuring Out How to Survive
   Flung to the far end of the universe, the SGU
   crew has to go back to the basics of survival
   

   View Slide

8. Edge Cases are the
   Currency of Programmers
   Most of our development effort is spent on them
   If we didn’t have them, programming would be easy
   (maybe even automatic)
   How many bugs aren’t edge cases?
   This is important stuff!
   

   View Slide

9. Numbers are Behind Letters
   What are the minimal cards you have to flip
   to prove there’s an even behind each vowel?
   

   View Slide

10. Numbers are Behind Letters
    What are the minimal cards you have to flip
    to prove there’s an even behind each vowel?
    

    View Slide

11. Numbers are Behind Letters
    What are the minimal cards you have to flip
    to prove there’s an even behind each vowel?
    

    View Slide

12. Numbers are Behind Letters
    What are the minimal cards you have to flip
    to prove there’s an even behind each vowel?
    

    View Slide

13. Reprogramming Their Brains
    The crew had to rework their thoughts
    to match their new circumstances
    

    View Slide

14. Edge Case Thinking
    Edge cases are the important stuff
    The rest of our work is at least easier
    Given that, I simply prefer to focus on the edge cases
    first, when possible
    

    View Slide

15. A Stargate Safari
    As Destiny travels, Stargates allow the crew
    to explore many worlds
    

    View Slide

16. nil: The Edge of Nothing
    

    View Slide

17. The Trouble With nil
    It’s an edge case, by definition
    Given that, it’s best avoided as much as possible
    

    View Slide

18. Returning nil
    This is an example of a common,
    but pointless edge case
    module GooglePlus
    module_function
    def has_any_circles?(user)
    !!check_api_for_any_circles?(user)
    end
    def circles(user)
    if has_any_circles? user
    get_circles_from_api(user)
    else
    nil
    end
    end
    # ...
    end
    

    View Slide

19. Using nil
    We are forced into an edge case
    at this end as well
    if circles = GooglePlus.circles(user)
    circles.each do |circle|
    # use with circle here...
    end
    end
    

    View Slide

20. My Favorite De-edger
    This unusual core Ruby method can guarantee
    an Array (Enumerable) result
    Array([1, 2, 3]) # => [1, 2, 3]
    Array(:not_an_array) # => [:not_an_array]
    Array(nil) # => []
    

    View Slide

21. Dodging the Edge Case
    Remove the edge case instead of work with it,
    when possible
    Array(GooglePlus.circles(user)).each do |circle|
    # use with circle here...
    end
    

    View Slide

22. Avoiding Returning nil
    This method has a much better contract:
    I’ll return to you a list of circles
    module GooglePlus
    module_function
    def circles(user)
    Array(get_circles_from_api(user))
    end
    # ...
    end
    

    View Slide

23. Ideal Usage
    Getting the method contract right
    automatically fixed this code too
    GooglePlus.circles(user).each do |circle|
    # use with circle here...
    end
    

    View Slide

24. We Haven’t Lost Anything
    We can still handle the no circles case, if needed
    circles = GooglePlus.circles(user)
    if circles.empty?
    # handle no circles here...
    else
    circles.each do |circle|
    # use with circle here...
    end
    end
    

    View Slide

25. Slurping Your Food Data
    

    View Slide

26. Too Much Heat?
    When Destiny begins flying straight into a sun,
    the crew assumes they are about to burn up
    

    View Slide

27. Data as a Stream
    Avoid “slurping” a ton of data in whenever possible
    Big data is a scary edge case
    Draw only what you need at once
    Come back for more as needed
    This applies to all data, including database queries
    

    View Slide

28. No Slurping
    Some exceptions: you produce and consume,
    you ensure a reasonable size, etc.
    File.read(path).each_line do |line|
    # use line here...
    end
    

    View Slide

29. No Slurping
    Some exceptions: you produce and consume,
    you ensure a reasonable size, etc.
    File.read(path).each_line do |line|
    # use line here...
    end
    

    View Slide

30. A Better Way
    It’s rarely any harder to work with data in chunks
    File.foreach(path) do |line|
    # use line here...
    end
    

    View Slide

31. The Rails Version
    In Rails, all() without a where() is generally
    the same as read()
    @comments = @post.comments.all
    

    View Slide

32. The Rails Version
    In Rails, all() without a where() is generally
    the same as read()
    @comments = @post.comments.all
    

    View Slide

33. Better
    It’s OK to grab subsets of data
    @comments = @post.comments
    .where("created_at >= ?", 1.day.ago)
    .all
    # ... or ...
    @comments = @post.comments.paginate(params[:page])
    

    View Slide

34. Best
    You can even walk all of the data in subsets
    @post.comments.find_each do |comment|
    # use comment here...
    end
    

    View Slide

35. Edgy Tests
    

    View Slide

36. Risk Management
    Tests are about managing concerns
    Edge cases are the primary concerns
    Focus your efforts there
    

    View Slide

37. Test Fodder
    Given a trivial method…
    class Post < ActiveRecord::Base
    def editable?
    created_at >= 1.hour.ago
    end
    # ...
    end
    

    View Slide

38. Boring Tests
    We often test it like this
    describe Post do
    it "is editable within an hour of creation" do
    Post.new(created_at: 10.minutes.ago).must_be :editable?
    end
    it "is not editable beyond an hour of creation" do
    Post.new(created_at: 1.day.ago).wont_be :editable?
    end
    end
    

    View Slide

39. Boring Tests
    We often test it like this
    describe Post do
    it "is editable within an hour of creation" do
    Post.new(created_at: 10.minutes.ago).must_be :editable?
    end
    it "is not editable beyond an hour of creation" do
    Post.new(created_at: 1.day.ago).wont_be :editable?
    end
    end
    

    View Slide

40. Moving to the Edge
    Spend your paranoia on the edges instead
    describe Post do
    it "is editable within an hour of creation" do
    Post.new(created_at: 1.hour.ago).must_be :editable?
    end
    it "is not editable beyond an hour of creation" do
    Post.new(created_at: (1.hour + 1.minute).ago).wont_be :editable?
    end
    end
    

    View Slide

41. Moving to the Edge
    Spend your paranoia on the edges instead
    describe Post do
    it "is editable within an hour of creation" do
    Post.new(created_at: 1.hour.ago).must_be :editable?
    end
    it "is not editable beyond an hour of creation" do
    Post.new(created_at: (1.hour + 1.minute).ago).wont_be :editable?
    end
    end
    

    View Slide

42. Edges Everywhere
    There are so many edges in programming
    The relationships between objects
    Interactions with frameworks
    Networks and other systems
    So much more…
    

    View Slide

43. It Pays to be Prepared
    When investigating alien technology, the crew
    protects themselves from the environment
    

    View Slide

44. The Hardline
    Make edge cases
    impossible, when
    you can
    # guarantee we do not touch API's
    class Geokit::Geocoders::MultiGeocoder
    def self.geocode(*args)
    fail "Tried to contact Google"
    end
    end
    class Boomerang::APICall
    def initialize(*args)
    fail "Tried to contact Amazon"
    end
    end
    module SendGridDisabler
    def make_api_call(*args)
    fail "Tried to contact SendGrid"
    end
    private :make_api_call
    end
    SendGrid.extend(SendGridDisabler)
    

    View Slide

45. All at Once But in Order
    

    View Slide

46. Eli: An Edge Case Thinker
    As the crew’s boy genius, Eli is always
    thinking his way around their edge cases
    

    View Slide

47. Don’t Do Extra Work!
    I’m not advocating heavy planning here
    There are so many great answers to edge cases
    besides hard work:
    It won’t affect us
    We accept the limitations
    There’s a easier way
    Etc.
    

    View Slide

48. Linear Requests (Part 1)
    Faraday allows us to abstract away the
    process of making requests and use middleware
    require "faraday_middleware" # load Faraday plus extra middleware
    ua = Faraday.new("https://api.github.com/") do |builder|
    builder.use Faraday::Response::ParseJson # parse JSON responses
    builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses
    builder.adapter Faraday.default_adapter
    end
    # ...
    

    View Slide

49. Linear Requests (Part 1)
    Faraday allows us to abstract away the
    process of making requests and use middleware
    require "faraday_middleware" # load Faraday plus extra middleware
    ua = Faraday.new("https://api.github.com/") do |builder|
    builder.use Faraday::Response::ParseJson # parse JSON responses
    builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses
    builder.adapter Faraday.default_adapter
    end
    # ...
    

    View Slide

50. Linear Requests (Part 2)
    We request the list of repos, then fetch
    each repo in turn for an issue count
    # ...
    repos = ua.get("/users/JEG2/repos")
    repos.on_complete do
    repos.body.each do |repo|
    name = repo["name"]
    project = ua.get("/repos/JEG2/#{name}")
    project.on_complete do
    issues = project.body["open_issues"]
    puts "#{name}: #{issues}" if issues.nonzero?
    end
    end
    end
    

    View Slide

51. Linear Requests (Part 2)
    We request the list of repos, then fetch
    each repo in turn for an issue count
    # ...
    repos = ua.get("/users/JEG2/repos")
    repos.on_complete do
    repos.body.each do |repo|
    name = repo["name"]
    project = ua.get("/repos/JEG2/#{name}")
    project.on_complete do
    issues = project.body["open_issues"]
    puts "#{name}: #{issues}" if issues.nonzero?
    end
    end
    end
    

    View Slide

52. Linear Requests (Part 2)
    We request the list of repos, then fetch
    each repo in turn for an issue count
    # ...
    repos = ua.get("/users/JEG2/repos")
    repos.on_complete do
    repos.body.each do |repo|
    name = repo["name"]
    project = ua.get("/repos/JEG2/#{name}")
    project.on_complete do
    issues = project.body["open_issues"]
    puts "#{name}: #{issues}" if issues.nonzero?
    end
    end
    end
    

    View Slide

53. Linear Requests (Part 2)
    We request the list of repos, then fetch
    each repo in turn for an issue count
    # ...
    repos = ua.get("/users/JEG2/repos")
    repos.on_complete do
    repos.body.each do |repo|
    name = repo["name"]
    project = ua.get("/repos/JEG2/#{name}")
    project.on_complete do
    issues = project.body["open_issues"]
    puts "#{name}: #{issues}" if issues.nonzero?
    end
    end
    end
    

    View Slide

54. Linear Requests (Part 2)
    We request the list of repos, then fetch
    each repo in turn for an issue count
    # ...
    repos = ua.get("/users/JEG2/repos")
    repos.on_complete do
    repos.body.each do |repo|
    name = repo["name"]
    project = ua.get("/repos/JEG2/#{name}")
    project.on_complete do
    issues = project.body["open_issues"]
    puts "#{name}: #{issues}" if issues.nonzero?
    end
    end
    end
    

    View Slide

55. The Request Tree
    After fetching the list of repos,
    the remaining requests are independent
    

    View Slide

56. Parallel Requests (Part 1)
    Thanks to the abstraction,
    switching HTTP clients is trivial
    require "faraday_middleware" # load Faraday plus extra middleware
    ua = Faraday.new("https://api.github.com/") do |builder|
    builder.use Faraday::Response::ParseJson # parse JSON responses
    builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses
    builder.adapter :typhoeus
    end
    # ...
    

    View Slide

57. Parallel Requests (Part 1)
    Thanks to the abstraction,
    switching HTTP clients is trivial
    require "faraday_middleware" # load Faraday plus extra middleware
    ua = Faraday.new("https://api.github.com/") do |builder|
    builder.use Faraday::Response::ParseJson # parse JSON responses
    builder.use Faraday::Response::RaiseError # raise on 40x and 50x responses
    builder.adapter :typhoeus
    end
    # ...
    

    View Slide

58. Parallel Requests (Part 2)
    We can now fetch the list, then handle
    the repo requests in parallel
    # ...
    ua.in_parallel(Typhoeus::Hydra.hydra) do
    repos = ua.get("/users/JEG2/repos")
    repos.on_complete do
    repos.body.each do |repo|
    name = repo["name"]
    project = ua.get("/repos/JEG2/#{name}")
    project.on_complete do
    issues = project.body["open_issues"]
    puts "#{name}: #{issues}" if issues.nonzero?
    end
    end
    end
    end
    

    View Slide

59. Parallel Requests (Part 2)
    We can now fetch the list, then handle
    the repo requests in parallel
    # ...
    ua.in_parallel(Typhoeus::Hydra.hydra) do
    repos = ua.get("/users/JEG2/repos")
    repos.on_complete do
    repos.body.each do |repo|
    name = repo["name"]
    project = ua.get("/repos/JEG2/#{name}")
    project.on_complete do
    issues = project.body["open_issues"]
    puts "#{name}: #{issues}" if issues.nonzero?
    end
    end
    end
    end
    

    View Slide

60. The Parallel Advantage
    We’ve cut the time for these requests down
    by about 84%
    $ time ruby linear_requests.rb
    highline: 1
    faster_csv: 1
    real 0m11.087s
    user 0m0.466s
    sys 0m0.121s
    $ time ruby parallel_requests.rb
    highline: 1
    faster_csv: 1
    real 0m1.721s
    user 0m0.403s
    sys 0m0.104s
    

    View Slide

61. The Parallel Advantage
    We’ve cut the time for these requests down
    by about 84%
    $ time ruby linear_requests.rb
    highline: 1
    faster_csv: 1
    real 0m11.087s
    user 0m0.466s
    sys 0m0.121s
    $ time ruby parallel_requests.rb
    highline: 1
    faster_csv: 1
    real 0m1.721s
    user 0m0.403s
    sys 0m0.104s
    

    View Slide

62. Thanks
    

    View Slide