Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Practical Taxonomy of Bugs and How to Squash Them-Keep Ruby Weird 2016

Kylie
November 04, 2016

A Practical Taxonomy of Bugs and How to Squash Them-Keep Ruby Weird 2016

Catching software bugs is a mysterious magic, unknowable by science and untouchable by process.

False! Programming bugs, like real bugs, can be organized into a taxonomy. Come with me and I’ll show you how classification can help you build “programmer’s instinct” into a logical debugging process.

Kylie

November 04, 2016
Tweet

More Decks by Kylie

Other Decks in Programming

Transcript

  1. A PRACTICAL
    TAXONOMY OF BUGS
    AND HOW TO SQUASH THEM

    View Slide

  2. Instinctual Indications…2
    Research Methods…9
    Practical Taxonomy…12
    Bohrbug…13
    Schrödinbug…20
    Fractalbug…24
    Heisenbug…30
    Mandelbug…36
    Common Patterns…43
    Resources…50
    Table of Contents

    View Slide

  3. Debugging Skills

    View Slide

  4. “As you familiarize yourself with the
    application, you’ll build up some
    debugging instincts"

    View Slide

  5. “Whenever I see something like this
    happening, the first thing I do is scan
    the logs to see if this process is
    completing or is sending a weird
    message.”

    View Slide

  6. Debugging Instincts
    “ ”

    View Slide

  7. View Slide

  8. “Whenever I see #{x}, I always check
    #{y}”

    View Slide

  9. Research Methods
    • containment sometimes takes priority
    over squashing
    • we can only work with facts
    • we can’t squash every bug in this
    talk

    View Slide

  10. View Slide

  11. Observable Attributes

    View Slide

  12. Warning: Contrived
    Scenarios Ahead

    View Slide

  13. A Practical Taxonomy of Bugs
    Upsettingly
    Observable
    Wildly
    Chaotic
    {

    View Slide

  14. How to Squash Them

    View Slide

  15. upsettingly
    observable bug #1
    UPSETTINGLY
    OBSERVABLE

    View Slide

  16. Observable Attributes
    is the bug observable in production?
    can it be reproduced locally?
    does it seem to be restricted to one area?

    View Slide

  17. Bohrbug
    deterministic,
    highly
    reproducible
    UPSETTINGLY
    OBSERVABLE

    View Slide

  18. Bohrbug
    Commonly found in
    code,sometimes on
    server
    UPSETTINGLY
    OBSERVABLE

    View Slide

  19. Bohrbug
    likes to hide in
    complex branching in
    functions, classes or
    config
    UPSETTINGLY
    OBSERVABLE

    View Slide

  20. Bohrbug
    In the wild:
    validation
    UPSETTINGLY
    OBSERVABLE

    View Slide

  21. Reproduction & Resolution
    replicate locally and in test
    write the simple solution
    rewrite to be highly readable and
    extendable
    UPSETTINGLY
    OBSERVABLE

    View Slide

  22. Bohrbug
    UPSETTINGLY
    OBSERVABLE

    View Slide

  23. Bohrbug
    UPSETTINGLY
    OBSERVABLE

    View Slide

  24. upsettingly
    observable bug #2
    UPSETTINGLY
    OBSERVABLE

    View Slide

  25. Observable Attributes
    how does this work?
    does this work?
    wait, what is this even testing?
    did this ever work?

    View Slide

  26. Schrödinbug
    stick-like body
    appendages
    look like
    twigs
    UPSETTINGLY
    OBSERVABLE

    View Slide

  27. Schrödinbug
    Likes to
    pretend to be
    working code.
    On close
    inspection,
    reveals itself
    to be a bug.
    UPSETTINGLY
    OBSERVABLE

    View Slide

  28. Schrödinbug
    In the wild:
    code that never
    worked
    UPSETTINGLY
    OBSERVABLE

    View Slide

  29. Schrödinbug
    In the wild:
    it didn’t
    work how you
    thought it
    did
    UPSETTINGLY
    OBSERVABLE

    View Slide

  30. Logging as Verification
    Tool

    View Slide

  31. Git Bisect
    Tool

    View Slide

  32. Reproduction & Resolution
    reproduce the “broken” state locally and
    in test
    add log statements until you can verify
    what causes the broken state.
    if the bug did work at some point, find
    the point at which it did work.
    write tests to represent the
    configuration and flow of the fixed
    state

    View Slide

  33. Schrödinbug
    UPSETTINGLY
    OBSERVABLE

    View Slide

  34. Schrödinbug
    UPSETTINGLY
    OBSERVABLE

    View Slide

  35. wildly chaotic bug #1
    WILDLY
    CHAOTIC

    View Slide

  36. Observable Attributes
    Does it appear non-deterministic?
    Does it seem to disappear once you
    observe or debug it?

    View Slide

  37. Heisenbug
    “now you see it, now you don’t”
    WILDLY
    CHAOTIC

    View Slide

  38. Heisenbug
    WILDLY
    CHAOTIC
    In the wild: a
    heisenbug that
    lives in code

    View Slide

  39. Heisenbug
    WILDLY
    CHAOTIC
    In the wild: a
    heisenbug that
    lives in data

    View Slide

  40. Profiling for Verification
    https://kcachegrind.github.io/html/CallgrindFormat.html
    Tool

    View Slide

  41. FLAME GRAPHS
    http://www.brendangregg.com/FlameGraphs/cpu-mysql-updated.svg
    Tool

    View Slide

  42. Reproduction & Resolution
    use profiling to find the trigger state
    use the app (not fixtures or DB
    manipulation) to get the data in this
    state
    recreate that state in test
    follow borhbug instruction

    View Slide

  43. Heisenbug
    WILDLY
    CHAOTIC

    View Slide

  44. Heisenbug
    WILDLY
    CHAOTIC

    View Slide

  45. wildly chaotic bug #2
    WILDLY
    CHAOTIC

    View Slide

  46. Observable Attributes
    is everything broken?
    all of it?
    send help??

    View Slide

  47. Mandelbug WILDLY
    CHAOTIC

    View Slide

  48. Mandelbug
    seems like
    everything
    is broken
    at once
    WILDLY
    CHAOTIC

    View Slide

  49. Mandelbug
    people are
    very upset
    with you
    WILDLY
    CHAOTIC

    View Slide

  50. Mandelbug
    likely an
    issue with
    your
    system, not
    code
    WILDLY
    CHAOTIC

    View Slide

  51. “The bug is huge and everywhere at once.
    SQL: could not connect to server: Connection refused was
    bubbling up all over the place.
    Jobs won’t run, emails won’t send, every submit button on
    the site fatal errored.”
    on-call log 24 June 2014
    WILDLY
    CHAOTIC

    View Slide

  52. Disk Usage
    Tool
    df -h

    View Slide

  53. Reproduction & Resolution
    attempt to connect to server & view logs
    use df -h to find if all the storage is
    being used
    can that be restarted, rotated or killed
    at this time?

    View Slide

  54. Mandelbug
    WILDLY
    CHAOTIC

    View Slide

  55. Mandelbug
    WILDLY
    CHAOTIC

    View Slide

  56. A Practical Taxonomy of Bugs
    Upsettingly
    Observable
    Wildly
    Chaotic
    {
    bohrbug
    schrödinbug
    mandelbug
    heisenbug

    View Slide

  57. “Debugging Instincts”

    View Slide

  58. “Debugging Instincts”

    View Slide

  59. View Slide

  60. Resources
    • “Linux Debugging Tools I Love”, Julia
    Evans
    • Systems Performance, Brendan Gregg
    • “Why Do Computers Stop and What Can
    Be Done About It?”, Jim Gray
    • “Debug Patterns for Efficient High-
    levelSystemC Debugging”, Frank Rogin,
    Erhard Fehlauer, Christian Haufe,
    Sebastian Ohnewald

    View Slide

  61. View Slide

  62. View Slide

  63. View Slide

  64. View Slide