A Practical Taxonomy of Bugs and How to Squash Them-RubyConf Italy 2016

The slides I used to accompany my presentation "A Practical Taxonomy of Bugs and How to Squash Them" at Ruby Conf Italy 2016. Happy squashing!

Transcript

1. A PRACTICAL TAXONOMY OF BUGS AND HOW TO SQUASH THEM
2. Instinctual Indications…6 Research Methods…9 Practical Taxonomy…13 Bohrbug…17 Schrödinbug…25 Fractalbug…24 Heisenbug…35 Mandelbug…44 Resources…55 Table of Contents
3. Debugging Skills
5. “As you familiarize yourself with the application, you’ll build up some debugging instincts"
6. “Whenever I see something like this happening, the first thing I do is scan the logs to see if this process is completing or is sending a weird message.”
7. Debugging Instincts “ ”
9. “Whenever I see #{x}, I always check #{y}”
10. Research Methods • containment sometimes takes priority over squashing • we can only work with facts • we can’t squash every bug in this talk
11. Observable Attributes
12. Phenetics { ]
13. Warning: Contrived Scenarios Ahead
14. A Practical Taxonomy of Bugs Upsettingly Observable Wildly Chaotic {
15. Upsettingly Observable
16. Wildly Chaotic
17. How to Squash Them
18. upsettingly observable bug #1 UPSETTINGLY OBSERVABLE
19. Observable Attributes is the bug observable in production? can it be reproduced locally? does it seem to be restricted to one area?
20. Bohrbug deterministic, highly reproducible UPSETTINGLY OBSERVABLE
21. Bohrbug Commonly found in code,sometimes on server UPSETTINGLY OBSERVABLE
22. Bohrbug likes to hide in complex branching in functions, classes or config UPSETTINGLY OBSERVABLE
23. Bohrbug In the wild: validation UPSETTINGLY OBSERVABLE
24. Reproduction & Resolution replicate locally and in test write the simple solution rewrite to be highly readable and extendable UPSETTINGLY OBSERVABLE
25. Bohrbug UPSETTINGLY OBSERVABLE
26. Bohrbug UPSETTINGLY OBSERVABLE
27. upsettingly observable bug #2 UPSETTINGLY OBSERVABLE
28. Observable Attributes how does this work? does this work? wait, what is this even testing? did this ever work?
29. Schrödinbug stick-like body appendages look like twigs UPSETTINGLY OBSERVABLE
30. Schrödinbug Likes to pretend to be working code. On close inspection, reveals itself to be a bug. UPSETTINGLY OBSERVABLE
31. Schrödinbug In the wild: code that never worked UPSETTINGLY OBSERVABLE
32. Schrödinbug In the wild: it didn’t work how you thought it did UPSETTINGLY OBSERVABLE
33. Logging as Verification Tool
34. Git Bisect Tool
35. Reproduction & Resolution reproduce the “broken” state locally and in test add log statements until you can verify what causes the broken state. if the bug did work at some point, find the point at which it did work. write tests to represent the configuration and flow of the fixed state
36. Schrödinbug UPSETTINGLY OBSERVABLE
37. Schrödinbug UPSETTINGLY OBSERVABLE
38. wildly chaotic bug #1 WILDLY CHAOTIC
39. Observable Attributes Does it appear non-deterministic? Does it seem to disappear once you observe or debug it?
40. Heisenbug “now you see it, now you don’t” WILDLY CHAOTIC
41. Heisenbug WILDLY CHAOTIC In the wild: a heisenbug that lives in code
42. Heisenbug WILDLY CHAOTIC In the wild: a heisenbug that lives in data
43. Profiling for Verification https://kcachegrind.github.io/html/CallgrindFormat.html Tool
44. FLAME GRAPHS http://www.brendangregg.com/FlameGraphs/cpu-mysql-updated.svg Tool
45. Reproduction & Resolution use profiling to find the trigger state use the app (not fixtures or DB manipulation) to get the data in this state recreate that state in test follow borhbug instruction
46. Heisenbug WILDLY CHAOTIC
47. Heisenbug WILDLY CHAOTIC
48. wildly chaotic bug #2 WILDLY CHAOTIC
49. Observable Attributes is everything broken? all of it? send help??
50. Mandelbug WILDLY CHAOTIC
51. Mandelbug seems like everything is broken at once WILDLY CHAOTIC
52. Mandelbug people are very upset with you WILDLY CHAOTIC
53. Mandelbug likely an issue with your system, not code WILDLY CHAOTIC
54. “The bug is huge and everywhere at once. SQL: could not connect to server: Connection refused was bubbling up all over the place. Jobs won’t run, emails won’t send, every submit button on the site fatal errored.” on-call log 24 June 2014 WILDLY CHAOTIC
55. Disk Usage Tool df -h
56. Reproduction & Resolution attempt to connect to server & view logs use df -h to find if all the storage is being used can that be restarted, rotated or killed at this time?
57. Mandelbug WILDLY CHAOTIC
58. Mandelbug WILDLY CHAOTIC
59. A Practical Taxonomy of Bugs Upsettingly Observable Wildly Chaotic { bohrbug schrödinbug mandelbug heisenbug
60. “Debugging Instincts”
61. “Debugging Instincts”
62. Debugging Skills
63. Observe & Classify
64. Verify with logging and time travel
65. Verify without changing state by profiling
66. Use linux server tools to observe entire process
67. Observe & Classify Verify with logging and time travel Verify without changing state by profiling Use linux server tools to observe entire process
68. Build Up Your Own Toolkit and Share it
69. Resources & Further Study • “Linux Debugging Tools I Love”, Julia Evans • Systems Performance, Brendan Gregg • Site Reliability Engineering, Betsy Beyer, Chris Jones, Jennifer Petoff, Niall Richard Murphy • “Why Do Computers Stop and What Can Be Done About It?”, Jim Gray • “Debug Patterns for Efficient High- levelSystemC Debugging”, Frank Rogin, Erhard Fehlauer, Christian Haufe, Sebastian Ohnewald