A Practical Taxonomy of Bugs and How to Squash Them-Keep Ruby Weird 2016

Catching software bugs is a mysterious magic, unknowable by science and untouchable by process.

False! Programming bugs, like real bugs, can be organized into a taxonomy. Come with me and I’ll show you how classification can help you build “programmer’s instinct” into a logical debugging process.

Transcript

1. A PRACTICAL
   TAXONOMY OF BUGS
   AND HOW TO SQUASH THEM
   

   View Slide

2. Instinctual Indications…2
   Research Methods…9
   Practical Taxonomy…12
   Bohrbug…13
   Schrödinbug…20
   Fractalbug…24
   Heisenbug…30
   Mandelbug…36
   Common Patterns…43
   Resources…50
   Table of Contents
   

   View Slide

3. Debugging Skills
   

   View Slide

4. “As you familiarize yourself with the
   application, you’ll build up some
   debugging instincts"
   

   View Slide

5. “Whenever I see something like this
   happening, the first thing I do is scan
   the logs to see if this process is
   completing or is sending a weird
   message.”
   

   View Slide

6. Debugging Instincts
   “ ”
   

   View Slide

7. View Slide

8. “Whenever I see #{x}, I always check
   #{y}”
   

   View Slide

9. Research Methods
   • containment sometimes takes priority
   over squashing
   • we can only work with facts
   • we can’t squash every bug in this
   talk
   

   View Slide

10. View Slide

11. Observable Attributes
    

    View Slide

12. Warning: Contrived
    Scenarios Ahead
    

    View Slide

13. A Practical Taxonomy of Bugs
    Upsettingly
    Observable
    Wildly
    Chaotic
    {
    

    View Slide

14. How to Squash Them
    

    View Slide

15. upsettingly
    observable bug #1
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

16. Observable Attributes
    is the bug observable in production?
    can it be reproduced locally?
    does it seem to be restricted to one area?
    

    View Slide

17. Bohrbug
    deterministic,
    highly
    reproducible
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

18. Bohrbug
    Commonly found in
    code,sometimes on
    server
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

19. Bohrbug
    likes to hide in
    complex branching in
    functions, classes or
    config
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

20. Bohrbug
    In the wild:
    validation
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

21. Reproduction & Resolution
    replicate locally and in test
    write the simple solution
    rewrite to be highly readable and
    extendable
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

22. Bohrbug
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

23. Bohrbug
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

24. upsettingly
    observable bug #2
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

25. Observable Attributes
    how does this work?
    does this work?
    wait, what is this even testing?
    did this ever work?
    

    View Slide

26. Schrödinbug
    stick-like body
    appendages
    look like
    twigs
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

27. Schrödinbug
    Likes to
    pretend to be
    working code.
    On close
    inspection,
    reveals itself
    to be a bug.
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

28. Schrödinbug
    In the wild:
    code that never
    worked
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

29. Schrödinbug
    In the wild:
    it didn’t
    work how you
    thought it
    did
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

30. Logging as Verification
    Tool
    

    View Slide

31. Git Bisect
    Tool
    

    View Slide

32. Reproduction & Resolution
    reproduce the “broken” state locally and
    in test
    add log statements until you can verify
    what causes the broken state.
    if the bug did work at some point, find
    the point at which it did work.
    write tests to represent the
    configuration and flow of the fixed
    state
    

    View Slide

33. Schrödinbug
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

34. Schrödinbug
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

35. wildly chaotic bug #1
    WILDLY
    CHAOTIC
    

    View Slide

36. Observable Attributes
    Does it appear non-deterministic?
    Does it seem to disappear once you
    observe or debug it?
    

    View Slide

37. Heisenbug
    “now you see it, now you don’t”
    WILDLY
    CHAOTIC
    

    View Slide

38. Heisenbug
    WILDLY
    CHAOTIC
    In the wild: a
    heisenbug that
    lives in code
    

    View Slide

39. Heisenbug
    WILDLY
    CHAOTIC
    In the wild: a
    heisenbug that
    lives in data
    

    View Slide

40. Profiling for Verification
    https://kcachegrind.github.io/html/CallgrindFormat.html
    Tool
    

    View Slide

41. FLAME GRAPHS
    http://www.brendangregg.com/FlameGraphs/cpu-mysql-updated.svg
    Tool
    

    View Slide

42. Reproduction & Resolution
    use profiling to find the trigger state
    use the app (not fixtures or DB
    manipulation) to get the data in this
    state
    recreate that state in test
    follow borhbug instruction
    

    View Slide

43. Heisenbug
    WILDLY
    CHAOTIC
    

    View Slide

44. Heisenbug
    WILDLY
    CHAOTIC
    

    View Slide

45. wildly chaotic bug #2
    WILDLY
    CHAOTIC
    

    View Slide

46. Observable Attributes
    is everything broken?
    all of it?
    send help??
    

    View Slide

47. Mandelbug WILDLY
    CHAOTIC
    

    View Slide

48. Mandelbug
    seems like
    everything
    is broken
    at once
    WILDLY
    CHAOTIC
    

    View Slide

49. Mandelbug
    people are
    very upset
    with you
    WILDLY
    CHAOTIC
    

    View Slide

50. Mandelbug
    likely an
    issue with
    your
    system, not
    code
    WILDLY
    CHAOTIC
    

    View Slide

51. “The bug is huge and everywhere at once.
    SQL: could not connect to server: Connection refused was
    bubbling up all over the place.
    Jobs won’t run, emails won’t send, every submit button on
    the site fatal errored.”
    on-call log 24 June 2014
    WILDLY
    CHAOTIC
    

    View Slide

52. Disk Usage
    Tool
    df -h
    

    View Slide

53. Reproduction & Resolution
    attempt to connect to server & view logs
    use df -h to find if all the storage is
    being used
    can that be restarted, rotated or killed
    at this time?
    

    View Slide

54. Mandelbug
    WILDLY
    CHAOTIC
    

    View Slide

55. Mandelbug
    WILDLY
    CHAOTIC
    

    View Slide

56. A Practical Taxonomy of Bugs
    Upsettingly
    Observable
    Wildly
    Chaotic
    {
    bohrbug
    schrödinbug
    mandelbug
    heisenbug
    

    View Slide

57. “Debugging Instincts”
    

    View Slide

58. “Debugging Instincts”
    

    View Slide

59. View Slide

60. Resources
    • “Linux Debugging Tools I Love”, Julia
    Evans
    • Systems Performance, Brendan Gregg
    • “Why Do Computers Stop and What Can
    Be Done About It?”, Jim Gray
    • “Debug Patterns for Efficient High-
    levelSystemC Debugging”, Frank Rogin,
    Erhard Fehlauer, Christian Haufe,
    Sebastian Ohnewald
    

    View Slide

61. View Slide

62. View Slide

63. View Slide

64. View Slide