Pharmacist or a Doctor - What does your code base need?

http://www.confreaks.com/videos/2921-gardencityruby-pharmacist-or-a-doctor-what-does-your-code-base-need

"This talk was presented by @pavanks and @anandhak at Garden Citry Ruby Conf 2014".

You might know of every single code quality & metrics tool in the Ruby ecosystem and what they give you, but do you know:

Which metrics do you currently need?
Do you really need them?
How do you make your team members own them?
Wait, there was a metaphor in the title

While a pharmacist knows about pretty much every medicine out there and what it cures, its really a doctor who figures out what is required given the symptoms of a patient.

Just like the vitals recorded for healthy adults, infants, pregnant women or an accident patient in an ICU changes, your code base needs different metrics in different contexts to help you uncover problems.

Talk take aways

Through a lot of examples, the talk helps:

Identify the current state of your code base
Understand different metrics and what do they tell you about your code base
Drive your team towards continuously fixing problems uncovered by these metrics

Transcript

1. Pavan Sudarshan @pavanks Anandha Krishnan @anandhak (Jake) Pharmacist or a Doctor - What does your code base need?
2. Disclaimer This talk is mostly platform/language independent. But, we think the Ruby community is very aligned to most of what we will be talking about
3. We have made a lot of mistakes*... …all our life as hackers We want to talk about one such mistake with a couple of examples * No employers or paranoid androids were hurt in the process!
4. About 3 months back... I saw this guy walk into a pharmacy and ask for some medicine for a toothache
5. Between us, we have seen people ask...
6. Medicines for → Headache → Fever → Dysentery → Rashes → Cuts from a knife (I am still scared about this one!)
7. Do you see anything wrong with this?
8. A pharmacy... Potentially has a cure for pretty much most ailments
9. But the important thing is… To figure out which ailment you are actually suffering from?
10. If choosing the right medicine was that simple, the world would just have...
11. Pharmacists and not Doctors
12. Doctors treat attack symptoms by... → Coming up with a problem hypothesis → Using Diagnostic tests → Prescribing the right treatment → Validating that the symptoms vanish
13. Pharmacists understand... → The details of the medicines → They can reverse engineer to find the disease based on the medicines → Are not responsible for prescribing treatment
14. Now, with this context, lets get back to our mistake...
15. Our mistake... The way we dealt with bad symptoms in our code bases & projects
16. Anecdote - 1
17. Pesky regression bugs Every time we built something, a bunch of other things broke.
18. Goal Reduce the number of these bugs which were around 10 every iteration to 5
19. Some facts → We took over the project from another team → Our knowledge of the code base was bad → Test Coverage was extremely low
20. What would you do if you were to fix this?
21. Low coverage was staring us straight in the eyes
22. Fix coverage first → Regression bugs could be caught by tests → We focussed on increasing coverage → Started monitoring and improving coverage steadily over a period of 2 months
23. Our thought process was something like this...
24. Metric/Tool Symptom Solution Provokes Improves
25. And the result?
26. We ended up... → Improving coverage drastically → However, things got marginally better, but just that, marginal
27. While us devs were happy about the coverage, the project manager was extremely frustrated
28. This and a lot of similar mistakes... Across different projects, drove us to rethink our approach. We ended up realising something basic...
29. Never improve a metric. Solving a problem should automatically improve the metric.
30. Instead of this...
31. Metric/Tool Symptom Solution Provokes Improves Targets
32. Problem Symptom Solution Metrics Measures Problem Solution Metrics Problem Solution Metrics Identifies Fixes
33. We basically stopped doing this... → Crawl github to find a suitable gem → Setup it and start monitoring → Spend time and effort to improve what you are monitoring
34. Lessons learnt about Test Coverage
35. Measuring → Controller specs vs Model specs coverage → Just line coverage is not very useful* → Line → Branch → Unique path * The Single Metric Fallacy
36. Reporting → Coverage is inherently not red/green → Code climate does a very good job
37. Ratcheting → Double edged sword → Make sure people are not scared/hate coverage
38. Improving → Adding unit tests to old untested code bases is dangerous → Higher level tests (functional tests) are a great safety net for refactoring
39. Always keep in mind the problem you’re solving
40. Anecdote - 2 About 5 percent of requests take a very long time to return
41. Goal Find the root cause of the unpredictable response times and fix it.
42. Questions?