Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Practical Taxonomy of Bugs and How to Squash Them-We Rise 2017

A Practical Taxonomy of Bugs and How to Squash Them-We Rise 2017

These are the slides I used for We Rise Tech, Women Who Code Atlanta's first conference.

Kylie

June 23, 2017
Tweet

More Decks by Kylie

Other Decks in Programming

Transcript

  1. A COMMON
    TAXONOMY OF BUGS
    AND HOW TO SQUASH THEM

    View full-size slide

  2. Instinctual Indications…6
    Research Methods…9
    Practical Taxonomy…13
    Bohrbug…17
    Schrödinbug…25
    Fractalbug…24
    Heisenbug…35
    Mandelbug…44
    Resources…55
    Table of Contents

    View full-size slide

  3. Debugging Skills

    View full-size slide

  4. “As you familiarize yourself with the
    application, you’ll build up some
    debugging instincts"

    View full-size slide

  5. “Whenever I see something like this
    happening, the first thing I do is scan
    the logs to see if this process is
    completing or is sending a weird
    message.”

    View full-size slide

  6. Debugging Instincts
    “ ”

    View full-size slide

  7. “Whenever I see something like this
    happening, the first thing I do is scan
    the logs to see if this process is
    completing or is sending a weird
    message.”

    View full-size slide

  8. “Whenever I see #{x}, I always check
    #{y}”

    View full-size slide

  9. Instincts are just internalized
    rulesets

    View full-size slide

  10. Research Methods
    • containment sometimes takes priority
    over squashing

    View full-size slide

  11. Research Methods
    • containment sometimes takes priority
    over squashing
    • we can only work with facts

    View full-size slide

  12. Research Methods
    • containment sometimes takes priority
    over squashing
    • we can only work with facts
    • we can’t squash every bug in this
    talk

    View full-size slide

  13. Observable Attributes

    View full-size slide

  14. Warning: Contrived
    Scenarios Ahead

    View full-size slide

  15. A COMMON
    TAXONOMY OF BUGS
    AND HOW TO SQUASH THEM

    View full-size slide

  16. Taxonomy of Bugs
    Upsettingly
    Observable
    Wildly
    Chaotic
    {

    View full-size slide

  17. Upsettingly Observable
    Bugs

    View full-size slide

  18. Wildly Chaotic
    Bugs

    View full-size slide

  19. How to Squash Them

    View full-size slide

  20. upsettingly
    observable bug #1
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  21. Observable Attributes
    is the bug observable in production?

    View full-size slide

  22. Observable Attributes
    is the bug observable in production?
    can it be reproduced locally?

    View full-size slide

  23. Observable Attributes
    is the bug observable in production?
    can it be reproduced locally?
    does it seem to be restricted to one area?

    View full-size slide

  24. Bohrbug
    deterministic,
    highly
    reproducible
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  25. Bohrbug
    Commonly found in
    code,sometimes on
    server
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  26. Bohrbug
    likes to hide in
    complex branching in
    functions, classes or
    config
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  27. Bohrbug
    In the wild:
    validation
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  28. Bohrbug
    UPSETTINGLY
    OBSERVABLE
    Easy to squash

    View full-size slide

  29. Reproduction & Resolution
    replicate locally and in test
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  30. Reproduction & Resolution
    replicate locally and in test
    write the simple solution
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  31. Reproduction & Resolution
    replicate locally and in test
    write the simple solution
    rewrite to be highly readable and
    extendable
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  32. Bohrbug
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  33. Bohrbug
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  34. upsettingly
    observable bug #2
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  35. Observable Attributes
    how does this work?

    View full-size slide

  36. Observable Attributes
    how does this work?
    does this work?

    View full-size slide

  37. Observable Attributes
    how does this work?
    does this work?
    wait, what is this even testing?

    View full-size slide

  38. Observable Attributes
    how does this work?
    does this work?
    wait, what is this even testing?
    did this ever work?

    View full-size slide

  39. Schrödinbug
    stick-like body
    appendages
    look like
    twigs
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  40. Schrödinbug
    Likes to
    pretend to be
    working code.
    On close
    inspection,
    reveals itself
    to be a bug.
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  41. Schrödinbug
    UPSETTINGLY
    OBSERVABLE
    Type I. Type II.

    View full-size slide

  42. Schrödinbug
    Type I.
    Code that never
    worked
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  43. Schrödinbug
    Type I.
    Reveal
    themselves via
    side-effects
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  44. Schrödinbug
    Type I.
    In the wild:
    UI shows update
    but database
    entry not
    updated.
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  45. Schrödinbug
    Type II.
    Code that
    doesn’t work
    how you
    thought
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  46. Schrödinbug
    Type II.
    In the wild:
    Same function
    being called
    multiple
    times
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  47. Basic
    Reproduction & Resolution
    replicate locally and in test
    write the simple solution
    rewrite to be highly readable and
    extendable
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  48. -You
    “How can I reproduce this without
    knowing exactly what is happening?”

    View full-size slide

  49. Logging as Verification
    Tool

    View full-size slide

  50. Git Bisect
    Tool

    View full-size slide

  51. Reproduction & Resolution
    reproduce the “broken” state locally and
    in test

    View full-size slide

  52. Reproduction & Resolution
    reproduce the “broken” state locally and
    in test
    add log statements until you can verify
    what causes the broken state.

    View full-size slide

  53. Reproduction & Resolution
    reproduce the “broken” state locally and
    in test
    add log statements until you can verify
    what causes the broken state.
    if the bug did work at some point, find
    the point at which it did work.

    View full-size slide

  54. Reproduction & Resolution
    reproduce the “broken” state locally and
    in test
    add log statements until you can verify
    what causes the broken state.
    if the bug did work at some point, find
    the point at which it did work.
    write tests to represent the
    configuration and flow of the fixed
    state

    View full-size slide

  55. Schrödinbug
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  56. Schrödinbug
    UPSETTINGLY
    OBSERVABLE

    View full-size slide

  57. wildly chaotic bug #1
    WILDLY
    CHAOTIC

    View full-size slide

  58. Observable Attributes
    Does it appear non-deterministic?

    View full-size slide

  59. Observable Attributes
    Does it appear non-deterministic?
    Does it seem to disappear once you
    observe or debug it?

    View full-size slide

  60. Heisenbug
    “now you see it, now you don’t”
    WILDLY
    CHAOTIC

    View full-size slide

  61. Heisenbug
    WILDLY
    CHAOTIC
    Type I.
    Type II.

    View full-size slide

  62. Heisenbug
    Type I.
    WILDLY
    CHAOTIC
    Lives in code

    View full-size slide

  63. Heisenbug
    Type II.
    WILDLY
    CHAOTIC
    Lives in data

    View full-size slide

  64. -You
    “How can I reproduce this without
    testing on production?”

    View full-size slide

  65. Profiling for Verification
    https://kcachegrind.github.io/html/CallgrindFormat.html
    Tool

    View full-size slide

  66. FLAME GRAPHS
    http://www.brendangregg.com/FlameGraphs/cpu-mysql-updated.svg
    Tool

    View full-size slide

  67. Heisenbug
    Type I.
    WILDLY
    CHAOTIC
    Profiling can
    reveal what is
    being called and
    when

    View full-size slide

  68. Heisenbug
    Type II.
    WILDLY
    CHAOTIC
    Profiling can
    reveal how much
    time is being
    spent.

    View full-size slide

  69. Reproduction & Resolution
    use profiling to find the trigger state

    View full-size slide

  70. Reproduction & Resolution
    use profiling to find the trigger state
    use the app (not fixtures or DB
    manipulation) to get the data in this
    state

    View full-size slide

  71. Reproduction & Resolution
    use profiling to find the trigger state
    use the app (not fixtures or DB
    manipulation) to get the data in this
    state
    recreate that state in test

    View full-size slide

  72. Reproduction & Resolution
    use profiling to find the trigger state
    use the app (not fixtures or DB
    manipulation) to get the data in this
    state
    recreate that state in test
    follow borhbug instruction

    View full-size slide

  73. Heisenbug
    WILDLY
    CHAOTIC

    View full-size slide

  74. Heisenbug
    WILDLY
    CHAOTIC

    View full-size slide

  75. wildly chaotic bug #2
    WILDLY
    CHAOTIC

    View full-size slide

  76. Observable Attributes
    is everything broken?

    View full-size slide

  77. Observable Attributes
    is everything broken?
    all of it?

    View full-size slide

  78. Observable Attributes
    is everything broken?
    all of it?
    send help??

    View full-size slide

  79. Mandelbug WILDLY
    CHAOTIC

    View full-size slide

  80. Mandelbug
    seems like
    everything
    is broken
    at once
    WILDLY
    CHAOTIC

    View full-size slide

  81. Mandelbug
    people are
    very upset
    with you
    WILDLY
    CHAOTIC

    View full-size slide

  82. Mandelbug
    likely an
    issue with
    your
    system, not
    code
    WILDLY
    CHAOTIC

    View full-size slide

  83. “The bug is huge and everywhere at once.
    SQL: could not connect to server: Connection refused
    was bubbling up all over the place.
    Jobs won’t run, emails won’t send, every submit button
    on the site fatal errored.”
    on-call log
    WILDLY
    CHAOTIC

    View full-size slide

  84. Disk Usage
    Tool
    df -h

    View full-size slide

  85. Logging as Verification
    Tool

    View full-size slide

  86. Reproduction & Resolution
    use df -h to find if all the storage is
    being use

    View full-size slide

  87. Reproduction & Resolution
    use df -h to find if all the storage is
    being use
    attempt to connect to server & view logs

    View full-size slide

  88. Reproduction & Resolution
    use df -h to find if all the storage is
    being used
    attempt to connect to server & view logs
    can that be restarted, rotated or killed
    at this time?

    View full-size slide

  89. Mandelbug
    WILDLY
    CHAOTIC

    View full-size slide

  90. Mandelbug
    WILDLY
    CHAOTIC

    View full-size slide

  91. A Practical Taxonomy of Bugs
    Upsettingly
    Observable
    Wildly
    Chaotic
    {
    bohrbug
    schrödinbug
    mandelbug
    heisenbug

    View full-size slide

  92. “Debugging Instincts”

    View full-size slide

  93. “Debugging Instincts”

    View full-size slide

  94. Debugging Skills

    View full-size slide

  95. Observe & Classify

    View full-size slide

  96. Verify with logging and time travel

    View full-size slide

  97. Verify without
    changing state by
    profiling

    View full-size slide

  98. Use server tools to observe entire process

    View full-size slide

  99. Observe & Classify
    Verify with logging and time travel
    Verify without
    changing state by
    profiling
    Use server tools to observe entire process

    View full-size slide

  100. Build Up Your Own
    Toolkit and Share it

    View full-size slide

  101. Resources & Further Study
    • “Linux Debugging Tools I Love”, Julia
    Evans
    • Systems Performance, Brendan Gregg
    • “Why Do Computers Stop and What Can
    Be Done About It?”, Jim Gray
    • “Debug Patterns for Efficient High-
    levelSystemC Debugging”, Frank Rogin,
    Erhard Fehlauer, Christian Haufe,
    Sebastian Ohnewald

    View full-size slide