A Practical Taxonomy of Bugs and How to Squash Them-SunshinePHP

Review my talk and give me feedback: https://joind.in/talk/3f670
Catching software bugs is a mysterious magic, unknowable by science and untouchable by process.

False! Programming bugs, like real bugs, can be organized into a taxonomy. Come with me and I'll show you how classification can help you build “programmer’s instinct” into a logical debugging process.

Debugging may be instinctual but those instincts come from seeing, identifying and diagnosing patterns. In this talk, we’ll use phenetic taxonomy to build a debugging decision making network based on attributes and behaviors of common bugs. Newer developers will learn how to apply systematic pattern matching to debugging while seasoned developers will learn how to use their instincts to teach debugging.

Transcript

1. A PRACTICAL
   TAXONOMY OF BUGS
   AND HOW TO SQUASH THEM
   https://joind.in/talk/3f670
   

   View Slide

2. Instinctual Indications…6
   Research Methods…9
   Practical Taxonomy…13
   Bohrbug…17
   Schrödinbug…25
   Fractalbug…24
   Heisenbug…35
   Mandelbug…44
   Resources…55
   Table of Contents
   

   View Slide

3. Debugging Skills
   

   View Slide

4. “As you familiarize yourself with the
   application, you’ll build up some
   debugging instincts"
   

   View Slide

5. “Whenever I see something like this
   happening, the first thing I do is scan
   the logs to see if this process is
   completing or is sending a weird
   message.”
   

   View Slide

6. View Slide

7. Debugging Instincts
   “ ”
   

   View Slide

8. View Slide

9. View Slide

10. “Whenever I see #{x}, I always check
    #{y}”
    

    View Slide

11. Instincts are just internalized
    rulesets
    

    View Slide

12. Research Methods
    • containment sometimes takes priority
    over squashing
    • we can only work with facts
    • we can’t squash every bug in this
    talk
    

    View Slide

13. Observable Attributes
    

    View Slide

14. Phenetics
    {
    ]
    

    View Slide

15. Warning: Contrived
    Scenarios Ahead
    

    View Slide

16. A PRACTICAL
    TAXONOMY OF BUGS
    AND HOW TO SQUASH THEM
    

    View Slide

17. A Practical Taxonomy of Bugs
    Upsettingly
    Observable
    Wildly
    Chaotic
    {
    

    View Slide

18. Upsettingly Observable
    Bugs
    

    View Slide

19. Wildly Chaotic
    Bugs
    

    View Slide

20. How to Squash Them
    

    View Slide

21. upsettingly
    observable bug #1
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

22. Observable Attributes
    is the bug observable in production?
    

    View Slide

23. Observable Attributes
    is the bug observable in production?
    can it be reproduced locally?
    

    View Slide

24. Observable Attributes
    is the bug observable in production?
    can it be reproduced locally?
    does it seem to be restricted to one area?
    

    View Slide

25. Bohrbug
    deterministic,
    highly
    reproducible
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

26. Bohrbug
    Commonly found in
    code,sometimes on
    server
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

27. Bohrbug
    likes to hide in
    complex branching in
    functions, classes or
    config
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

28. Bohrbug
    In the wild:
    validation
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

29. Bohrbug
    UPSETTINGLY
    OBSERVABLE
    Easy to squash
    

    View Slide

30. Reproduction & Resolution
    replicate locally and in test
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

31. Reproduction & Resolution
    replicate locally and in test
    write the simple solution
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

32. Reproduction & Resolution
    replicate locally and in test
    write the simple solution
    rewrite to be highly readable and
    extendable
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

33. Bohrbug
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

34. upsettingly
    observable bug #2
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

35. Observable Attributes
    how does this work?
    

    View Slide

36. Observable Attributes
    how does this work?
    does this work?
    

    View Slide

37. Observable Attributes
    how does this work?
    does this work?
    wait, what is this even testing?
    

    View Slide

38. Observable Attributes
    how does this work?
    does this work?
    wait, what is this even testing?
    did this ever work?
    

    View Slide

39. Schrödinbug
    stick-like body
    appendages
    look like
    twigs
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

40. Schrödinbug
    Likes to
    pretend to be
    working code.
    On close
    inspection,
    reveals itself
    to be a bug.
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

41. Schrödinbug
    UPSETTINGLY
    OBSERVABLE
    Type I. Type II.
    

    View Slide

42. Schrödinbug
    Type I.
    Code that never
    worked
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

43. Schrödinbug
    Type I.
    Reveal
    themselves via
    side-effects
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

44. Schrödinbug
    Type I.
    In the wild:
    UI shows update
    but database
    entry not
    updated.
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

45. Schrödinbug
    Type II.
    Code that
    doesn’t work
    how you
    thought
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

46. Schrödinbug
    Type II.
    In the wild:
    Same function
    being called
    multiple
    times
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

47. Basic
    Reproduction & Resolution
    replicate locally and in test
    write the simple solution
    rewrite to be highly readable and
    extendable
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

48. -You
    “How can I reproduce this without
    knowing exactly what is happening?”
    

    View Slide

49. Logging as Verification
    Tool
    

    View Slide

50. Git Bisect
    Tool
    

    View Slide

51. Reproduction & Resolution
    reproduce the “broken” state locally and
    in test
    

    View Slide

52. Reproduction & Resolution
    reproduce the “broken” state locally and
    in test
    add log statements until you can verify
    what causes the broken state.
    

    View Slide

53. Reproduction & Resolution
    reproduce the “broken” state locally and
    in test
    add log statements until you can verify
    what causes the broken state.
    if the bug did work at some point, find
    the point at which it did work.
    

    View Slide

54. Reproduction & Resolution
    reproduce the “broken” state locally and
    in test
    add log statements until you can verify
    what causes the broken state.
    if the bug did work at some point, find
    the point at which it did work.
    write tests to represent the
    configuration and flow of the fixed
    state
    

    View Slide

55. Schrödinbug
    UPSETTINGLY
    OBSERVABLE
    

    View Slide

56. wildly chaotic bug #1
    WILDLY
    CHAOTIC
    

    View Slide

57. Observable Attributes
    Does it appear non-deterministic?
    

    View Slide

58. Observable Attributes
    Does it appear non-deterministic?
    Does it seem to disappear once you
    observe or debug it?
    

    View Slide

59. Heisenbug
    “now you see it, now you don’t”
    WILDLY
    CHAOTIC
    

    View Slide

60. Heisenbug
    WILDLY
    CHAOTIC
    Type I.
    Type II.
    

    View Slide

61. Heisenbug
    Type I.
    WILDLY
    CHAOTIC
    Lives in code
    

    View Slide

62. Heisenbug
    Type II.
    WILDLY
    CHAOTIC
    Lives in data
    

    View Slide

63. -You
    “How can I reproduce this without
    testing on production?”
    

    View Slide

64. Profiling for Verification
    https://kcachegrind.github.io/html/CallgrindFormat.html
    Tool
    

    View Slide

65. FLAME GRAPHS
    http://www.brendangregg.com/FlameGraphs/cpu-mysql-updated.svg
    Tool
    

    View Slide

66. Reproduction & Resolution
    use profiling to find the trigger state
    

    View Slide

67. Reproduction & Resolution
    use profiling to find the trigger state
    use the app (not fixtures or DB
    manipulation) to get the data in this
    state
    

    View Slide

68. Reproduction & Resolution
    use profiling to find the trigger state
    use the app (not fixtures or DB
    manipulation) to get the data in this
    state
    recreate that state in test
    

    View Slide

69. Reproduction & Resolution
    use profiling to find the trigger state
    use the app (not fixtures or DB
    manipulation) to get the data in this
    state
    recreate that state in test
    follow borhbug instruction
    

    View Slide

70. Heisenbug
    WILDLY
    CHAOTIC
    

    View Slide

71. wildly chaotic bug #2
    WILDLY
    CHAOTIC
    

    View Slide

72. Observable Attributes
    is everything broken?
    

    View Slide

73. Observable Attributes
    is everything broken?
    all of it?
    

    View Slide

74. Observable Attributes
    is everything broken?
    all of it?
    send help??
    

    View Slide

75. Mandelbug WILDLY
    CHAOTIC
    

    View Slide

76. Mandelbug
    seems like
    everything
    is broken
    at once
    WILDLY
    CHAOTIC
    

    View Slide

77. Mandelbug
    people are
    very upset
    with you
    WILDLY
    CHAOTIC
    

    View Slide

78. Mandelbug
    likely an
    issue with
    your
    system, not
    code
    WILDLY
    CHAOTIC
    

    View Slide

79. “The bug is huge and everywhere at once.
    SQL: could not connect to server: Connection refused
    was bubbling up all over the place.
    Jobs won’t run, emails won’t send, every submit button
    on the site fatal errored.”
    on-call log
    WILDLY
    CHAOTIC
    

    View Slide

80. Disk Usage
    Tool
    df -h
    

    View Slide

81. Logging as Verification
    Tool
    

    View Slide

82. Reproduction & Resolution
    use df -h to find if all the storage is
    being use
    

    View Slide

83. Reproduction & Resolution
    use df -h to find if all the storage is
    being use
    attempt to connect to server & view logs
    

    View Slide

84. Reproduction & Resolution
    use df -h to find if all the storage is
    being used
    attempt to connect to server & view logs
    can that be restarted, rotated or killed
    at this time?
    

    View Slide

85. Mandelbug
    WILDLY
    CHAOTIC
    

    View Slide

86. A Practical Taxonomy of Bugs
    Upsettingly
    Observable
    Wildly
    Chaotic
    {
    bohrbug
    schrödinbug
    mandelbug
    heisenbug
    

    View Slide

87. “Debugging Instincts”
    

    View Slide

88. Debugging Skills
    

    View Slide

89. Observe & Classify
    

    View Slide

90. Verify with logging and time travel
    

    View Slide

91. Verify without
    changing state by
    profiling
    

    View Slide

92. Use server tools to observe entire process
    

    View Slide

93. Observe & Classify
    Verify with logging and time travel
    Verify without
    changing state by
    profiling
    Use server tools to observe entire process
    

    View Slide

94. Build Up Your Own
    Toolkit and Share it
    

    View Slide

95. Resources & Further Study
    • “Linux Debugging Tools I Love”, Julia
    Evans
    • Systems Performance, Brendan Gregg
    • “Why Do Computers Stop and What Can
    Be Done About It?”, Jim Gray
    • “Debug Patterns for Efficient High-
    levelSystemC Debugging”, Frank Rogin,
    Erhard Fehlauer, Christian Haufe,
    Sebastian Ohnewald
    

    View Slide

96. https://joind.in/talk/3f670
    

    View Slide