Slide 1
Slide 1 presenter notes
ALAN
Slide 1 text
Ola! Obrigado por nos hospedar! Bem-vindo!
Welcome! We call this talk “A Brewer’s Guide to Filtering out Complexity and Churn.” It’s also known as, “The Coffee Machine Talk.”
Our goal is to show you how to remove from your applications the bitterness caused by complexity and churn.
Over the next 30 minutes, we will show you:
- How complexity sneaks into a code base;
- How to recognize complexity before it becomes painful; and,
- How to remove it permanently.
Let’s introduce ourselves and get going…
Slide 2
Slide 2 presenter notes
ALAN & FITO
Slide 2 text
I’ll go first. Hello! I’m Alan. I have 13 years experience with Ruby. And, I’m from Seattle, so there’s coffee in my veins. My favorite is a sugar free vanilla oat milk latte. Mmm!
Hi! I’m Fito. I also have 13 years of experience in Ruby. I’m from Asunción, Paraguay. And, if there’s one thing I love as much as Ruby, it’s coffee! My favorite is a Ghirardelli dark chocolate mocha. Mmm!
Alan and I work together at a company you might not expect, given that this is a conference for Rails developers. We work for Cisco Meraki—the largest Rails shop you’ve never heard of. But, we’ve been friends for years. In fact, we’ve worked together at three different companies over the last ten years. And, we’ve seen a wide variety of code bases. Plus, we spend time together on the weekends, writing code and drinking coffee. Alan, you grew up around coffee, didn’t you?
Slide 3
Slide 3 presenter notes
ALAN
Slide 3 text
Yes, I did! In fact, back when I was a kid, nothing fascinated me more than the mechanical vending machines at my dad’s office. You dropped in a coin, listened for the clink, and made your selection. The machine would spring to life: hissing, clicking, and whirring. When the automatic ballet ended, the final sound was that glorious, aromatic, black liquid splashing into the cup. Ah! Magnifico!
These days, I’m more fascinated by the inner workings of software. (We both are.) And, like that coffee machine, there are all kinds of hidden complexity in code. But, software doesn’t start out complex.
Show of hands: how many of you have worked on a greenfield or brand new application? How did that feel? Ok. How about legacy applications? How many of you have worked on one? How did that feel?
In our experience, greenfield development is enjoyable. It feels fast. There’s no existing code to work around. But, developing in a legacy applications feels harder. Why is that? We believe it has to do with complexity. We think that at some point a code base crosses a complexity threshold, after which we have two choices:
Slide 4
Slide 4 presenter notes
ALAN
Slide 4 text
You can live with the complexity as it grows and development slows down more and more over time under the illusion that it will go away on its own. Or, you can pause temporarily to reorganize the code and accelerate development again.
We’ve seen organizations go down both paths. Invariably, when you take the path of living with the complexity, engineers end up frustrated.
Slide 5
Slide 5 presenter notes
ALAN
Slide 5 text
And, in our experience, they sometimes even begin to blame Ruby and start looking for alternatives. But, it’s not Ruby fault. It’s the complexity.
So, what we’re going to do is show you how to take that second path, remove the complexity, and fall back in love with Ruby.
Slide 6
Slide 6 presenter notes
ALAN
Slide 6 text
We’re going to build a little coffee machine. We’ll add several features, then go back and look at how complexity snuck into the program with each commit. Finally, we’ll reorganize the code to enable us to add new features without increasing complexity.
Ok, are we ready?
Slide 7
Slide 7 presenter notes
ALAN
Slide 7 text
Let’s get started! Fito, you wanna build a coffee machine?
Slide 8
Slide 8 presenter notes
FITO
Slide 8 text
Yes! Let’s do it!
So, how does complexity sneak into software? The answer, of course, is one commit at a time. Let’s take a look.
Now, I’m going to move through these slides pretty fast, just to show you the shape of the code as it grows. And, as the code gets longer, the font size will get smaller. Don’t worry about trying to understand the actual code, it’s made up anyway. Also, we’re skipping tests here for the sake of time. But, in reality, we would be doing this with tests.
Slide 9
Slide 9 presenter notes
FITO
Slide 9 text
Here is the first commit in our coffee machine. At this point the machine does one and only one thing. It serves coffee.
First it dispenses a cup. Then it heats the water, prepares the grounds, and dispenses the hot water. Finally, it disposes of the grounds.
It works great, but not everyone likes coffee. So, to increase our sales, let’s add tea…
Slide 10
Slide 10 presenter notes
FITO
Slide 10 text
Here we added a conditional to determine whether to serve coffee or tea. And in the process of doing that, we added some duplication.
The dispense_cup, heat_water, and dispense_water steps are all duplicated between the two beverages.
So, let’s DRY it up…
Slide 11
Slide 11 presenter notes
FITO
Slide 11 text
Here’s the DRY version of the code.
With both coffee and tea in production, we’re starting to get feedback. The most frequent request is to add sweetener. So, let’s do that…
Slide 12
Slide 12 presenter notes
FITO
Slide 12 text
Here, we’ve added sweetener just after dispensing the hot water. Of course, not everyone wants sugar, so let’s make it optional.
We pushed this out. Customers like it. Now, they want cream…
Slide 13
Slide 13 presenter notes
FITO
Slide 13 text
Since we already have a pattern for optional ingredients, let’s dispense cream right after we dispense sugar.
For our next feature, it turns out that some folks don’t like coffee or tea. So, let’s offer them something else, like cocoa…
Slide 14
Slide 14 presenter notes
FITO
Slide 14 text
Here, we followed the existing pattern and added cocoa to the main if statement. But, there’s no need to add milk or sugar, since cocoa is already sweet and creamy. So, let’s exclude those optional ingredients when the customer requests cocoa.
Finally, who doesn’t like whipped cream on their cocoa? Heck, I even like it on my coffee! So, let’s add it…
Slide 15
Slide 15 presenter notes
FITO
Slide 15 text
Ok. So, whipped cream is an optional ingredient that no one wants on their tea. So, let’s add it after the other optional ingredients and exclude tea.
So, here we are…
Slide 16
Slide 16 presenter notes
FITO
Slide 16 text
7 commits into this codebase, and we’ve already got 9 conditionals in one method. At this point, it’s still relatively simple to understand and work with, if you’re the only one working on it. But, if you are part of a team, procedural code like this won’t scale. Future developers will just keep adding more conditionals with each new feature, causing complexity to sky rocket. And…
Our little coffee machine has been so successful that it was just purchased by a BIG national soup chain. They want us to add soup to our machines. That’s going to add a lot of complexity to our code. So, let’s pause here and evaluate where we are before trying to add any more features. Alan, can you take us through it?
Slide 17
Slide 17 presenter notes
ALAN
Slide 17 text
Sure! So, we’ve reached an inflection point in the life of our little coffee machine. But, how can we tell? What is it that tells us to pause and restructure? Well, the first hint is that we had to start reducing the font size to display the whole method on one slide. Method length is definitely an indicator that things are getting complex. Sandi Metz — author of Practical Object Oriented Development in Ruby — has a rule about it. She says methods can only have five lines of code.
In addition to method length, we also look at method complexity. This is a quantitative measurement of how difficult it is to understand a piece of code. Our preferred metric is called the Assignments, Branches, and Conditionals (or ABC) metric. The higher the number, the harder the code is to understand. We use a gem called Flog by Ryan Davis to measure this for us. Flog calculates an ABC complexity score for each method in an application. But, how do we know a good score from a bad score?
Slide 18
Slide 18 presenter notes
ALAN
Slide 18 text
Well, all the way back in 2008, a guy named Jake Scruggs (who wrote the metric_fu gem) wrote down these numbers for the flog score of a single method. Over the years, we’ve used these numbers as our guide. And, they’re actually quite effective at helping us drive our code toward simpler solutions.
So, how does our little coffee machine fare?
Let’s go back through the commits and watch complexity over time.
Slide 19
Slide 19 presenter notes
ALAN
Slide 19 text
So, our first commit weighs in at a complexity of 5.5. That’s awesome.
The churn number is the total number of commits to this file. That’ll become important later.
Slide 20
Slide 20 presenter notes
ALAN
Slide 20 text
Adding the conditional with duplicated code really shot up the complexity. At 14.6, we’re no longer in the Awesome zone. But, we’re still below 20, so that’s good enough.
Slide 21
Slide 21 presenter notes
ALAN
Slide 21 text
After removing the duplication, the complexity drops back down to 10.9. This may seem like a good thing. But, this is actually where things really start to go wrong. Notice how we’ve intermingled the two algorithms in a way that makes it harder to see what it takes to brew coffee or steep tea. It also sets a precedent for future developers to extend this code by adding new beverages into the mix using more conditional logic.
But, from Flog’s perspective, complexity went down. It can’t tell that we intermingled two algorithms. It’s just doing the math.
This is a valuable lesson. There is no magic metric that can light the way in every situation. Rather, there are tools that can inform our decision making. So, pay attention to how hard it feels to add new features to your application. If your intuition is telling you that it’s getting slower, then you might want to pause to reflect on your design.
Slide 22
Slide 22 presenter notes
ALAN
Slide 22 text
Next, we added sweetener. This is our 4th change. And, complexity has risen to 13.5.
Slide 23
Slide 23 presenter notes
ALAN
Slide 23 text
Adding cream causes the complexity to rise again, to 16.0. Still in the “good enough” zone.
Slide 24
Slide 24 presenter notes
ALAN
Slide 24 text
Adding cocoa pushes complexity all the way up to 21.3. That’s just above the “good enough” line. Let’s see where one more feature puts us.
Slide 25
Slide 25 presenter notes
ALAN
Slide 25 text
To top things off, whipped cream pushes the complexity score up to 25.5.
Slide 26
Slide 26 presenter notes
ALAN
Slide 26 text
If we look at the trend line, we can see that the complexity has reached a point where the line is curving upward. Plus, we are over the “good enough” line of 20.
Slide 27
Slide 27 presenter notes
ALAN
Slide 27 text
So, there you go, three ways to know when it’s time to pause and reflect on a method:
1. Method length. Anything over 5 is forbidden by Sandi. So, keep it short.
2. Method complexity. Anything under 20 is good enough. Anything over 60 is getting dangerously complex.
3. And, how does it feel? If new feature development is slowing down, it might be time to pause and reflect on your design.
So, that’s how we knew that we’d reached an inflection point. It’s time to start thinking about reorganizing this method before we try to add more features to it. And, that’s what Fito is going to do right now…
Slide 28
Slide 28 presenter notes
FITO
Slide 28 text
So… We broke Sandi’s rule. We crossed over Jake’s good enough line. And, we intermingled three algorithms. Sounds pretty dire. But, is it? Can we turn this code around?
Yes. Absolutely. Let’s look at how…
Slide 29
Slide 29 presenter notes
FITO
Slide 29 text
Here’s the method as we left it a moment ago.
Code that is DRYed too early can lead us in the wrong direction. So, let’s un-DRY this code — or add back the duplication — to see if there are any missing abstractions hiding in plain sight. We call this practice “rehydration.”
Now, before I show you what that looks like, it’s important to note that you can’t do any of this without tests. For the sake of time, we won’t be writing them from scratch here, but ensuring there’s good test coverage is the first step towards reducing complexity.
Slide 30
Slide 30 presenter notes
FITO
Slide 30 text
We really like this tool called SimpleCov. We use it to ensure that we’ve tested every line and branch of code in our applications. As you can see here, we have 100% line coverage. That’s great! We also have 100% branch coverage, which means that we’re testing both sides of every conditional in the code. Getting to 100% branch coverage is really important before rehydrating code. It gives you confidence that you’re not inadvertently changing behavior in the process of refactoring the code.
Slide 31
Slide 31 presenter notes
FITO
Slide 31 text
Alright. So, this is where we left the code a moment ago. Since we’ve confirmed that our tests are backing us up, we’re now ready to rehydrate the code. That looks like this…
Slide 32
Slide 32 presenter notes
FITO
Slide 32 text
Obviously, this increases duplication. But, that’s what we need to do to find the missing abstractions.
Now we can clearly see each recipe. And, since there’s no overlap in the algorithms anymore, we can safely extract each one into separate, polymorphic classes, like this…
Slide 33
Slide 33 presenter notes
FITO
Slide 33 text
Now, as you can see, we moved each recipe into its own class: one for coffee, tea, and cocoa. This structure has a couple of big advantages:
Each algorithm is now separate from the others. That means if you should ever need to modify one of them (to fix a bug, for example), there’s a much lower risk of you introducing a regression in another one of the algorithms. Plus, the vend method is much simpler.
Slide 34
Slide 34 presenter notes
FITO
Slide 34 text
Now, you may have noticed that there’s duplication between the classes. The calls to “dispense_cup”, “heat_water”, and “dispense_water” are all present in every class. We actually want that duplication. It makes understanding the complete algorithms much easier since the whole algorithm for each beverage is present in each beverage class. So, we do not want to DRY up the algorithms, per se.
Slide 35
Slide 35 presenter notes
FITO
Slide 35 text
Rather, what we want to do is to ensure that there is only one implementation of each of those methods. That’s what’s meant by Don’t Repeat Yourself. It is perfectly ok to call a method multiple times. It is preferable to only implement that method once.
Ruby provides multiple options for doing this. We could include a module, use composition, or introduce inheritance and put the methods in a base class. In this case, because we’re using polymorphic classes, and because we’re unlikely to need these methods elsewhere in the application, we’d probably go with inheritance.
We’re almost done. There’s just two remaining problems. First, the vend method has multiple responsibilities. And, second it is not open/closed, meaning that you have to modify the code to extend it. Let’s take a look at its responsibilities first. Its only real responsibility should be preparing the beverage. But, right now, it’s also picking which class to instantiate. That’s the job of a factory. So, let’s introduce one.
Slide 36
Slide 36 presenter notes
FITO
Slide 36 text
Here, we’ve pulled class instantiation out into a factory class. It’s only job is to choose which class to build based on what drink was selected.
Now the vend method only has the one remaining responsibility — to prepare beverages. And, it is now open/closed as well, meaning that we’ll never have to modify the vend method again to extend the functionality of the coffee machine. The vend method is now open for extension, but closed for modification.
However, the open/closed problem just moved to the factory. And, we introduced another issue. The build method in the factory might return nil, causing the CoffeeMachine’s vend method to throw an undefined method error. We can solve that by introducing the Null Object pattern, like this…
Slide 37
Slide 37 presenter notes
FITO
Slide 37 text
As you can see, the factory returns a NullBeverage by default. The NullBeverage class is simply a class with a prepare method that does nothing.
As for the second problem with the factory, it’s still not open/closed. To add a new beverage, we’ll have to modify the factory class. We can solve that by using a different kind of factory.
Slide 38
Slide 38 presenter notes
FITO
Slide 38 text
So, here we’ve replaced the conditional-based factory with one based on a convention and a little bit of meta-programming. The advantage of this approach is that it is mostly open/closed. There might come a day when we’d want to introduce a class name that doesn’t follow this convention. At that point, you’d need to modify the convention, which would violate the open/closed principle.
There are other approaches that are open/closed. And, we’d be happy to discuss them offline. But, that’s beyond the scope of this talk. So, we’ll stick with our convention-based factory for now.
At this point, let’s take a look at where we are with complexity.
Slide 39
Slide 39 presenter notes
FITO
Slide 39 text
Let’s start by revisiting this graph. Here’s where we left off after adding whipped cream.
The next thing we did was to rehydrate the code.
Slide 40
Slide 40 presenter notes
FITO
Slide 40 text
Look at how much more complex that was than the DRY solution. But, remember, the DRY solution was hiding the fact that there was a missing abstraction.
Next, we pulled the algorithms out into their own polymorphic classes. This dropped complexity significantly.
Slide 41
Slide 41 presenter notes
FITO
Slide 41 text
And, finally, we extracted a factory object…
Slide 42
Slide 42 presenter notes
FITO
Slide 42 text
And, now, the complexity is lower than it’s ever been.
We’re now well back in the Awesome range. And, the vend method will never need to change again.
Slide 43
Slide 43 presenter notes
FITO
Slide 43 text
Now, let’s take a look at all the other classes in the app. There are six. CoffeeMachine has settled down to a very low complexity of 3.9. The factory is next at 6.2. And, the three beverages are all more complex. But, well within the “good enough” zone. So, there’s really nothing more to do here.
Slide 44
Slide 44 presenter notes
FITO
Slide 44 text
Except that there was a reason why we did all that. We needed to add soup to our coffee machine. And, we wanted to do so without making it more complex.
Slide 45
Slide 45 presenter notes
FITO
Slide 45 text
So, here’s where we left off.
Let’s add some soup!
Slide 46
Slide 46 presenter notes
FITO
Slide 46 text
There you go! Adding soup did not require us to change any of the existing files. It just works. As long as the Soup class is loaded into memory, it will be available to the factory. And, in the future, adding another beverage will be as simple as adding another class. (And, its tests.)
Let’s look at that again.
Slide 47
Slide 47 presenter notes
FITO
Slide 47 text
So, here’s the code without soup.
Slide 48
Slide 48 presenter notes
FITO
Slide 48 text
And, here’s the code with soup. Nothing else changed.
So, that’s a look at a very small, green field application. But, your applications are obviously a whole lot bigger and a whole lot more complex than that. So, how will you know where to start when you get back to work?
Slide 49
Slide 49 text
Yeah, but does it scale?
How to apply this to a large, complicated code base.
Slide 50
Slide 50 text
☕ Plotting complexity vs. churn
Churn
Complexity
Getting Empirical about Refactoring by Michael Feathers
Slide 51
Slide 51 text
☕ Plotting complexity vs. churn
Pain Free
Churn
Complexity
Slide 52
Slide 52 text
☕ Plotting complexity vs. churn
Pain Free
Painful
Churn
Complexity
Slide 53
Slide 53 presenter notes
ALAN
Slide 53 text
Down in the lower right are low complex files that change frequently.
It’s possible that these files are actually configuration data masquerading as code (e.g. JSON hiding in a .rb file). If so, try to move the ever changing configuration data out of the code and let the code read it from a file as needed.
Slide 54
Slide 54 presenter notes
ALAN
Slide 54 text
And, finally, the upper left are high complexity files that rarely change.
These are likely what Sandi Metz refers to as Omega Messes. An Omega Mess is a file with a big, scary algorithm that never needs to be changed. Sandi’s advice in this case is that you should leave these files alone. They’re not causing any continued pain. And, mucking about in them could lead to bugs. So, just leave ‘em be.
These quadrants are helpful to think about. But, reality actually looks like this…
Slide 55
Slide 55 presenter notes
ALAN
Slide 55 text
The red line represents the pain threshold. Anything in the pink zone will be resistant to change and prone to regression.
This file <> is in need of the most attention. It’s super complex. And, it is being modified all the time. Extracting hidden abstractions from this class will help simplify the entire application. But, there’s no need to tackle all that complexity at once. Try improving the code a little bit each time you touch the file.
Also, you may not want to start with that file. It’s super complex code. Maybe start with something over here. <> It’ll give you a chance to practice some of the techniques we showed you without the pressure of working on some of your most complex code.
Slide 56
Slide 56 presenter notes
ALAN
Slide 56 text
So, that’s the story of our little coffee machine. How complexity snuck in. How we recognized it. And, how we removed it.
Let’s wrap up with some take aways and a bit of homework…
Slide 57
Slide 57 presenter notes
ALAN
Slide 57 text
Here are the three things we’d like you to take from this talk:
Slide 58
Slide 58 presenter notes
ALAN
Slide 58 text
First, complexity WILL sneak into your code. It happens one commit at a time. So, be vigilant. Pay particular attention to conditionals in your code. They could represent objects trying to escape your method. And, remember that DRY is about method implementation, not invocation. As my friend Josh Clayton says, “Don’t make your code so DRY it chafes.”
Slide 59
Slide 59 presenter notes
ALAN
Slide 59 text
Second, you can recognize complexity before it becomes painful. Keep methods short. Watch your complexity. And, pay attention to your intuition. If your methods are longer than 5 lines, or your flog scores are over 20, or it just feels slower than it used to, then it’s probably time to pause and reflect on your design.
Slide 60
Slide 60 presenter notes
ALAN
Slide 60 text
And, third, you can back away from painful complexity. Leverage polymorphism and factories to enable you to add new features to your application without having to change any existing files. Rehydrate your code — or reintroduce some duplication — to help identify missing abstractions that can be refactored into polymorphic classes.
Slide 61
Slide 61 presenter notes
ALAN
Slide 61 text
So, those are the three take aways. Now, here’s what we want you to do with that information.
Slide 62
Slide 62 presenter notes
ALAN
Slide 62 text
First, find out what your average method complexity is using Flog
Slide 63
Slide 63 presenter notes
ALAN
Slide 63 text
Second, find out which file has the most churn in your application using the Churn gem
Slide 64
Slide 64 presenter notes
ALAN
Slide 64 text
Third, find out which class needs the most attention. That’s the file with the highest churn and complexity. We bet you already know which class that is. But, go ahead and confirm your suspicions.
And, finally, let us know what you find.
Slide 65
Slide 65 presenter notes
ALAN
Slide 65 text
Here’s how to reach us. Feel free to write, tweet, or give us a toot! (Actually, you’re more likely to get an answer if you toot. Neither one of us are on Twitter much anymore.) You could also subscribe to my blog, if you’re so inclined.
Also, if you want to walk your way through the little coffee machine application, you can find it (with tests!) on GitHub. Plus you can check out our other personal projects while you’re there.
Slide 66
Slide 66 presenter notes
ALAN
Slide 66 text
One of which is a VS Code extension that shows you the flog score for selected text, the current method, or the average score for an entire class.
Slide 67
Slide 67 presenter notes
ALAN
Slide 67 text
That’s it! Thank you so much for coming! Here’s a list of our references and influences. (Take a picture if you’d like!) If you have any questions, come find us or drop us a note.
Obrigado!