A Common Taxonomy of Bugs and How to Squash Them-RubyConfAU

2ed277a8bb1cc68c943fd84f3ce32782?s=47 Kylie
February 09, 2017

A Common Taxonomy of Bugs and How to Squash Them-RubyConfAU

Catching software bugs is a mysterious magic, unknowable by science and untouchable by process.

False! Programming bugs, like real bugs, can be organized into a taxonomy. Come with me and I’ll show you how classification can help you build “programmer’s instinct” into a logical debugging process.

2ed277a8bb1cc68c943fd84f3ce32782?s=128

Kylie

February 09, 2017
Tweet

Transcript

  1. A COMMON TAXONOMY OF BUGS AND HOW TO SQUASH THEM

  2. Use Your Imagination

  3. Instinctual Indications…6 Research Methods…9 Practical Taxonomy…13 Bohrbug…17 Schrödinbug…25 Fractalbug…24 Heisenbug…35

    Mandelbug…44 Resources…55 Table of Contents
  4. Debugging Skills

  5. “As you familiarize yourself with the application, you’ll build up

    some debugging instincts"
  6. “Whenever I see something like this happening, the first thing

    I do is scan the logs to see if this process is completing or is sending a weird message.”
  7. None
  8. Debugging Instincts “ ”

  9. None
  10. None
  11. “Whenever I see something like this happening, the first thing

    I do is scan the logs to see if this process is completing or is sending a weird message.”
  12. “Whenever I see #{x}, I always check #{y}”

  13. Instincts are just internalized rulesets

  14. Research Methods • containment sometimes takes priority over squashing

  15. Research Methods • containment sometimes takes priority over squashing •

    we can only work with facts
  16. Research Methods • containment sometimes takes priority over squashing •

    we can only work with facts • we can’t squash every bug in this talk
  17. Observable Attributes

  18. Phenetics { ]

  19. Warning: Contrived Scenarios Ahead

  20. A COMMON TAXONOMY OF BUGS AND HOW TO SQUASH THEM

  21. Taxonomy of Bugs Upsettingly Observable Wildly Chaotic {

  22. Upsettingly Observable Bugs

  23. Wildly Chaotic Bugs

  24. How to Squash Them

  25. upsettingly observable bug #1 UPSETTINGLY OBSERVABLE

  26. Observable Attributes is the bug observable in production?

  27. Observable Attributes is the bug observable in production? can it

    be reproduced locally?
  28. Observable Attributes is the bug observable in production? can it

    be reproduced locally? does it seem to be restricted to one area?
  29. Bohrbug deterministic, highly reproducible UPSETTINGLY OBSERVABLE

  30. Bohrbug Commonly found in code,sometimes on server UPSETTINGLY OBSERVABLE

  31. Bohrbug likes to hide in complex branching in functions, classes

    or config UPSETTINGLY OBSERVABLE
  32. Bohrbug In the wild: validation UPSETTINGLY OBSERVABLE

  33. Bohrbug UPSETTINGLY OBSERVABLE Easy to squash

  34. Reproduction & Resolution replicate locally and in test UPSETTINGLY OBSERVABLE

  35. Reproduction & Resolution replicate locally and in test write the

    simple solution UPSETTINGLY OBSERVABLE
  36. Reproduction & Resolution replicate locally and in test write the

    simple solution rewrite to be highly readable and extendable UPSETTINGLY OBSERVABLE
  37. Bohrbug UPSETTINGLY OBSERVABLE

  38. Bohrbug UPSETTINGLY OBSERVABLE

  39. upsettingly observable bug #2 UPSETTINGLY OBSERVABLE

  40. Observable Attributes how does this work?

  41. Observable Attributes how does this work? does this work?

  42. Observable Attributes how does this work? does this work? wait,

    what is this even testing?
  43. Observable Attributes how does this work? does this work? wait,

    what is this even testing? did this ever work?
  44. Schrödinbug stick-like body appendages look like twigs UPSETTINGLY OBSERVABLE

  45. Schrödinbug Likes to pretend to be working code. On close

    inspection, reveals itself to be a bug. UPSETTINGLY OBSERVABLE
  46. Schrödinbug UPSETTINGLY OBSERVABLE Type I. Type II.

  47. Schrödinbug Type I. Code that never worked UPSETTINGLY OBSERVABLE

  48. Schrödinbug Type I. Reveal themselves via side-effects UPSETTINGLY OBSERVABLE

  49. Schrödinbug Type I. In the wild: UI shows update but

    database entry not updated. UPSETTINGLY OBSERVABLE
  50. Schrödinbug Type II. Code that doesn’t work how you thought

    UPSETTINGLY OBSERVABLE
  51. Schrödinbug Type II. In the wild: Same function being called

    multiple times UPSETTINGLY OBSERVABLE
  52. Basic Reproduction & Resolution replicate locally and in test write

    the simple solution rewrite to be highly readable and extendable UPSETTINGLY OBSERVABLE
  53. -You “How can I reproduce this without knowing exactly what

    is happening?”
  54. Logging as Verification Tool

  55. Git Bisect Tool

  56. Reproduction & Resolution reproduce the “broken” state locally and in

    test
  57. Reproduction & Resolution reproduce the “broken” state locally and in

    test add log statements until you can verify what causes the broken state.
  58. Reproduction & Resolution reproduce the “broken” state locally and in

    test add log statements until you can verify what causes the broken state. if the bug did work at some point, find the point at which it did work.
  59. Reproduction & Resolution reproduce the “broken” state locally and in

    test add log statements until you can verify what causes the broken state. if the bug did work at some point, find the point at which it did work. write tests to represent the configuration and flow of the fixed state
  60. Schrödinbug UPSETTINGLY OBSERVABLE

  61. Schrödinbug UPSETTINGLY OBSERVABLE

  62. wildly chaotic bug #1 WILDLY CHAOTIC

  63. Observable Attributes Does it appear non-deterministic?

  64. Observable Attributes Does it appear non-deterministic? Does it seem to

    disappear once you observe or debug it?
  65. Heisenbug “now you see it, now you don’t” WILDLY CHAOTIC

  66. Heisenbug WILDLY CHAOTIC Type I. Type II.

  67. Heisenbug Type I. WILDLY CHAOTIC Lives in code

  68. Heisenbug Type II. WILDLY CHAOTIC Lives in data

  69. -You “How can I reproduce this without testing on production?”

  70. Profiling for Verification https://kcachegrind.github.io/html/CallgrindFormat.html Tool

  71. FLAME GRAPHS http://www.brendangregg.com/FlameGraphs/cpu-mysql-updated.svg Tool

  72. Heisenbug Type I. WILDLY CHAOTIC Profiling can reveal what is

    being called and when
  73. Heisenbug Type II. WILDLY CHAOTIC Profiling can reveal how much

    time is being spent.
  74. Reproduction & Resolution use profiling to find the trigger state

  75. Reproduction & Resolution use profiling to find the trigger state

    use the app (not fixtures or DB manipulation) to get the data in this state
  76. Reproduction & Resolution use profiling to find the trigger state

    use the app (not fixtures or DB manipulation) to get the data in this state recreate that state in test
  77. Reproduction & Resolution use profiling to find the trigger state

    use the app (not fixtures or DB manipulation) to get the data in this state recreate that state in test follow borhbug instruction
  78. Heisenbug WILDLY CHAOTIC

  79. Heisenbug WILDLY CHAOTIC

  80. wildly chaotic bug #2 WILDLY CHAOTIC

  81. Observable Attributes is everything broken?

  82. Observable Attributes is everything broken? all of it?

  83. Observable Attributes is everything broken? all of it? send help??

  84. Mandelbug WILDLY CHAOTIC

  85. Mandelbug seems like everything is broken at once WILDLY CHAOTIC

  86. Mandelbug people are very upset with you WILDLY CHAOTIC

  87. Mandelbug likely an issue with your system, not code WILDLY

    CHAOTIC
  88. “The bug is huge and everywhere at once. SQL: could

    not connect to server: Connection refused was bubbling up all over the place. Jobs won’t run, emails won’t send, every submit button on the site fatal errored.” on-call log WILDLY CHAOTIC
  89. Disk Usage Tool df -h

  90. Logging as Verification Tool

  91. Reproduction & Resolution use df -h to find if all

    the storage is being use
  92. Reproduction & Resolution use df -h to find if all

    the storage is being use attempt to connect to server & view logs
  93. Reproduction & Resolution use df -h to find if all

    the storage is being used attempt to connect to server & view logs can that be restarted, rotated or killed at this time?
  94. Mandelbug WILDLY CHAOTIC

  95. Mandelbug WILDLY CHAOTIC

  96. A Practical Taxonomy of Bugs Upsettingly Observable Wildly Chaotic {

    bohrbug schrödinbug mandelbug heisenbug
  97. “Debugging Instincts”

  98. “Debugging Instincts”

  99. Debugging Skills

  100. Observe & Classify

  101. Verify with logging and time travel

  102. Verify without changing state by profiling

  103. Use server tools to observe entire process

  104. Observe & Classify Verify with logging and time travel Verify

    without changing state by profiling Use server tools to observe entire process
  105. Build Up Your Own Toolkit and Share it

  106. Resources & Further Study • “Linux Debugging Tools I Love”,

    Julia Evans • Systems Performance, Brendan Gregg • “Why Do Computers Stop and What Can Be Done About It?”, Jim Gray • “Debug Patterns for Efficient High- levelSystemC Debugging”, Frank Rogin, Erhard Fehlauer, Christian Haufe, Sebastian Ohnewald
  107. None