How to work with  Legacy Code - Management Summary

Slide 1

Slide 1 text

Railslove We love to build the web. How to work with  Legacy Code Marco Schaden  @donschado [email protected]  21.10.2016 Management Summary

Slide 2

Slide 2 text

Legacy Code 21.10.2016 Disclaimer The following is a summary of our own practical experiences enriched with opinions and knowledge of various esteemed experts and super heroes such as: Katrina Owen Sandi Metz Corey Haines Jay Fields and other respected people Kent Beck Martin Fowler Uncle Bob Martin Michael C. Feathers

Slide 3

Slide 3 text

Legacy Code 21.10.2016 Disclaimer Legacy is not about a speciﬁc technology or programming paradigm.  Nonetheless the focus of this talk are object oriented languages and web applications.

Slide 4

Slide 4 text

Legacy Code 21.10.2016 Disclaimer Slides are available online:   https://speakerdeck.com/donschado This is the extended version

Slide 5

Slide 5 text

Legacy Code 21.10.2016 Why does software become “legacy”?

Slide 6

Slide 6 text

Legacy Code 21.10.2016 Is it the customers* fault  by changing the requirements all the time?

Slide 7

Slide 7 text

Legacy Code 21.10.2016 Why couldn't the customer just be happy, with what they ﬁrst said they need…

Slide 8

Slide 8 text

google: IT+Project+Tree How the customer   explained it How the Project Leader   understood it How the Analyst designed it   How the Programmer   wrote it How the Business Consultant described it How the project was   documented What Operations  deployed How the customer was billed How it was supported What the customer really needed

Slide 9

Slide 9 text

Legacy Code 21.10.2016 Random developer:   “It was not designed to do this, we need to start from scratch..."

Slide 10

Slide 10 text

Legacy Code 21.10.2016 Surprise: Requirements Change

Slide 11

Slide 11 text

Legacy Code 21.10.2016 Surprise: Requirements Change, all the time!

Slide 12

Slide 12 text

Legacy Code 21.10.2016 Surprise: Requirements Change, all the time! Software that cannot tolerate this, is simply poor designed.

Slide 13

Slide 13 text

Hi, I’m Marco @donschado [email protected] Software Developer  Master of Computer Science

Slide 14

Slide 14 text

T We’re an agile team   and develop products for the web. Web applications are more than our daily business. We support our customers in close cooperation from the initial idea through to the launch - and beyond.

Slide 15

Slide 15 text

Legacy Code 21.10.2016

Slide 16

Slide 16 text

Legacy Code 21.10.2016 https://www.railslove.com/jobs

Slide 17

Slide 17 text

Legacy Code 21.10.2016 Legacy Code vs Technical Debt Change Dependencies Testing Refactoring vs Rewrite Metrics

Slide 18

Slide 18 text

Legacy Code 21.10.2016 Legacy is not exclusively a bad thing

Slide 19

Slide 19 text

Legacy Code 21.10.2016 Most of us see legacy as…

Slide 20

Slide 20 text

Legacy Code 21.10.2016 #vintage #oldtimer #classic Legacy?

Slide 21

Slide 21 text

Legacy Code 21.10.2016 many “legacy” systems have reliably served   the needs of their businesses generating value (usually it’s the main system and the source of income…) people and processes depend on it just because code is old   doesn’t mean it has to be thrown away Some people in our industry call it legacy while the codebase is less than 2 years old?! #vintage #oldtimer #classic

Slide 22

Slide 22 text

Legacy Code 21.10.2016 def legacy_code

Slide 23

Slide 23 text

Legacy Code 21.10.2016 slang for complex, diﬀicult to change code code with many dependencies mostly somebody else's code “old” code with deprecated syntax/features highly coupled code / “spaghetti" / ball of mud code you’re afraid to touch code without tests def legacy_code

Slide 24

Slide 24 text

Legacy Code 21.10.2016 The Michael C. Feathers deﬁnition: “code without tests” no matter how well written it is,   without tests you don't know   if a change makes it better or worse def legacy_code

Slide 25

Slide 25 text

Why does software become “legacy”?

Slide 26

Slide 26 text

Legacy Code 21.10.2016 legacy code vs technical debt

Slide 27

Slide 27 text

Legacy Code 21.10.2016 legacy code vs technical debt *sometimes required to move projects forward

Slide 28

Slide 28 text

Legacy Code 21.10.2016 legacy code vs technical debt referring to the eventual consequences of any system design • complexity • coupling • hard to change • anti patterns *sometimes required to move projects forward

Slide 29

Slide 29 text

Legacy Code 21.10.2016 legacy code vs technical debt accumulating interest referring to the eventual consequences of any system design • complexity • coupling • hard to change • anti patterns *sometimes required to move projects forward

Slide 30

Slide 30 text

Legacy Code 21.10.2016 legacy code vs technical debt accumulating interest referring to the eventual consequences of any system design the result of unpaid debt • complexity • coupling • hard to change • anti patterns *sometimes required to move projects forward

Slide 31

Slide 31 text

Legacy Code 21.10.2016 legacy code vs technical debt accumulating interest code smells • metaprogramming madness • long parameter list • shotgun surgery • duplicated code • feature envy • large class • long method • case statements • complex conditionals referring to the eventual consequences of any system design the result of unpaid debt • complexity • coupling • hard to change • anti patterns *sometimes required to move projects forward

Slide 32

Slide 32 text

Legacy Code 21.10.2016 legacy code vs technical debt accumulating interest code smells refactoring • metaprogramming madness • long parameter list • shotgun surgery • duplicated code • feature envy • large class • long method • case statements • complex conditionals referring to the eventual consequences of any system design the result of unpaid debt • complexity • coupling • hard to change • anti patterns *sometimes required to move projects forward

Slide 33

Slide 33 text

BLAME THE DEVELOPERS!?

Slide 34

Slide 34 text

Legacy Code 21.10.2016 missing (product) strategy conceptually poor features unfinished requirements needless complexity unnecessary / tough deadlines high fluctuation and poor on-boarding not enough training / further education doing agile wrong (yeah, we’re doing scrum/kanban we’re so agile!) The influence of management* on legacy: Even a development process that encourages code reviews and pairing  can yield bad technical decisions which effect the whole architecture:

Slide 35

Slide 35 text

Legacy Code 21.10.2016 You might miss the context, at the time the code was written

Slide 36

Slide 36 text

Legacy Code 21.10.2016 Uncle Bob: “every 5 years, double new programmers, with less than 5 years experience” http://blog.cleancoder.com/uncle-bob/2014/06/20/MyLawn.html

Slide 37

Slide 37 text

Legacy Code 21.10.2016 change

Slide 38

Slide 38 text

Legacy Code 21.10.2016 changing legacy code is a big deal

Slide 39

Slide 39 text

Legacy Code 21.10.2016 In legacy code, it is particularly hard to come up with estimates that are meaningful. even the simplest code changes take a long time to implement (or release) it seems like no amount of time will be enough to understand everything you need to do to make a change Expensive

Slide 40

Slide 40 text

Legacy Code 21.10.2016 How much change can you aﬀord if changes are risky? What changes do we have to make? How will we know that we've done them correctly and haven't broken anything? Risk Often we don't know how much of the existing behaviour is at risk   when we make our changes.

Slide 41

Slide 41 text

Legacy Code 21.10.2016

Slide 42

Slide 42 text

Legacy Code 21.10.2016 1. Identify change points 2. Find test points 3. Break dependencies 4. Write tests 5. Make your changes and refactor Algorithm: Michael C. Feathers

Slide 43

Slide 43 text

Legacy Code 21.10.2016 1. Identify change points 2. Find test points 3. Break dependencies 4. Write tests 5. Make your changes and refactor Algorithm: debugger, inspect, puts, tap, raise, console.log, … internal vs 3rd party,   most obvious impediment to testing this can be hard for side eﬀects  sometime requires creative and ugly technics Always leave the campground cleaner than you found it Michael C. Feathers

Slide 44

Slide 44 text

Legacy Code 21.10.2016 internal vs 3rd party,   most obvious impediment to testing 1. Identify change points 2. Find test points 3. Break dependencies 4. Write tests 5. Make your changes and refactor Algorithm: debugger, inspect, puts, tap, raise, console.log, … this can be hard for side eﬀects  sometime requires creative and ugly technics Always leave the campground cleaner than you found it Michael C. Feathers

Slide 45

Slide 45 text

Legacy Code 21.10.2016 Dependencies “They will try to kill you”

Slide 46

Slide 46 text

Legacy Code 21.10.2016 Whenever we bring up on our screens a nasty batch of tangled legacy code, we are experiencing the results of poor dependency management. Uncle Bob:

Slide 47

Slide 47 text

Legacy Code 21.10.2016 Uncle Bob: Poor dependency management leads to code that is hard to change, fragile, and non-reusable. Whenever we bring up on our screens a nasty batch of tangled legacy code, we are experiencing the results of poor dependency management.

Slide 48

Slide 48 text

Legacy Code 21.10.2016 Uncle Bob: Poor dependency management leads to code that is hard to change, fragile, and non-reusable. When classes depend directly on things that are hard to use in a test, they are hard to modify and hard to work with. Whenever we bring up on our screens a nasty batch of tangled legacy code, we are experiencing the results of poor dependency management.

Slide 49

Slide 49 text

Legacy Code 21.10.2016 https://cleancoders.com/videos

Slide 50

Slide 50 text

Legacy Code 21.10.2016 Testing To survive legacy code you need to maximise safety

Slide 51

Slide 51 text

Legacy Code 21.10.2016 2 unit tests. 0 integration tests.

Slide 52

Slide 52 text

Legacy Code 21.10.2016 How to In theory, writing a test   for a piece of functionality shouldn’t be too bad. We instantiate a class,   call its methods and check the results. What could go wrong? GOALS • complete • stable • fast • few

Slide 53

Slide 53 text

Legacy Code 21.10.2016 The method might not be accessible to the test.   It could be private or have some other accessibility problem. It might be hard to call the method because it is hard to construct the parameters we need to call it. We might need to understand first some other object that the method uses. The method might have (bad) side effects modifying a database send notification launching a cruise missile … How to GOALS • complete • stable • fast • few

Slide 54

Slide 54 text

Legacy Code 21.10.2016 Legacy Code is special Testing legacy code requires creative and (sometimes) ugly technics Break all rules if it saves $$$ during development

Slide 55

Slide 55 text

Legacy Code 21.10.2016 Rule of thumb for nearly every legacy system: What the system does is more important than what it is supposed to do.

Slide 56

Slide 56 text

Legacy Code 21.10.2016 The tests that we need when we want to preserve behaviour are called: characterization tests

Slide 57

Slide 57 text

Legacy Code 21.10.2016 Golden Master Testing wow, such fancy, very noble

Slide 58

Slide 58 text

Legacy Code 21.10.2016 def: Golden Master Testing refers to capturing the result of “a process”,  and then comparing future runs against the saved golden master version to discover unexpected changes. So you expect, that what it does now, is correct.

Slide 59

Slide 59 text

Legacy (Rails) Code 24.05.2016 <<< The idea is: you get some output and save the result (snapshot) and every time you run the test again you compare:    * the same: great the test pass, because everything works as before * different: the test fails and the human has to check the output again >>>

Slide 60

Slide 60 text

Legacy Code 21.10.2016 Golden Master Testing wow such dirt, very temporary

Slide 61

Slide 61 text

Legacy Code 21.10.2016 1. Use a piece of untested   undocumented code 2. Write an assertion   that you know will fail 3. Let the failure tell you   what the behaviour is 4. Change the test so that it   expects the behaviour that the code produces 5. Reason about the code   and exercise every branch Golden Master Testing: 6. Repeat until   the code is covered

Slide 62

Slide 62 text

Legacy Code 21.10.2016 This seems to be fundamentally wrong Are our tests really testing anything at all? What if you just characterized a bug?

Slide 63

Slide 63 text

Legacy Code 21.10.2016 Protection for changing things => conﬁdence to refactor We are trying to put in a mechanism to ﬁnd bugs later.

Slide 64

Slide 64 text

Legacy Code 21.10.2016 Protection for changing things => conﬁdence to refactor A characterization test is not a test you want to keep around. • You use the test to get coverage, • you refactor, • cleanup / write new specs • and then you throw it away! We are trying to put in a mechanism to ﬁnd bugs later.

Slide 65

Slide 65 text

Legacy Code 21.10.2016 Don’t keep these highly coupled tests (or they will cost you more money than they save you) Use tools for partially automating this process Anti Pattern ahead:

Slide 66

Slide 66 text

Legacy Code 21.10.2016

Slide 67

Slide 67 text

Legacy Code 21.10.2016 Test coverage analysis Lines of Code (no code is better than no code) Cyclomatic Complexity ABC Score (Code Smells) Churn Aﬀerent / Eﬀerent Coupling Mutation Coverage Metrics: Usually language agnostic and a helpful tool, but metrics should never be a goal:

Slide 68

Slide 68 text

Legacy Code 21.10.2016

Slide 69

Slide 69 text

Legacy Code 21.10.2016 Refactoring

Slide 70

Slide 70 text

Legacy Code 21.10.2016

Slide 71

Slide 71 text

Legacy Code 21.10.2016 the list of refactoring technics is just the beginning composing methods (extract method, replace method with object) moving features (move method, extract class) organizing data (replace hash with object, replace type code with polymorphism) simplifying conditional (decompose, recompose, null objects) making method calls simpler (separate query from modifier, replace constructor with factory) dealing with generalization (template method, extract module, inheritance) learn when to start, when to stop when you have “refactoring tickets” you're doing it wrong (it’s part of your work!) not having the time for refactoring (because of deadlines?) is usually a sign that you need to do some refactoring Refactoring: pay your bills!

Slide 72

Slide 72 text

Legacy Code 21.10.2016 You are likely to see new possibilities, REFACTOR ALL THE CODE to pursue truth and beauty...

Slide 73

Slide 73 text

Legacy Code 21.10.2016 DON’T.

Slide 74

Slide 74 text

Legacy Code 21.10.2016 A big refactoring is a recipe for disaster. As ugly as the mess looks now, focus on the real problems. When you need to add a new feature: add some specs to get confidence clean up add new code refactoring hat, feature hat Don't mix an unfinished refactoring with other new tasks. Your goal is to leave the code computing exactly the same, like you found it. Discipline Refactor when it’s necessary! (It will never be finished…)

Slide 75

Slide 75 text

Legacy Code 21.10.2016 Rewrite vs Refactoring

Slide 76

Slide 76 text

Legacy Code 21.10.2016 if it doesn't work at all (code has to work mostly correctly, before you refactor) if it is full of bugs and you cannot stabilize it if there are big changing requirements, the current app can't support if time and money are no constraints When is it easier to start from scratch, instead of refactoring a big mess? + + + + (For most developers this is the favourite choice, but it’s usually the most expensive)

Slide 77

Slide 77 text

Legacy Code 21.10.2016 Compromise! Refactor one large piece into components.  Favour piece by piece rewrite over big bang rewrite and rebuild incrementally. This strategy will let you: • focus only on the important and critical parts • recover knowledge which got lost in complex code • preserve investments which went into the code already (time for bugﬁxes, requirements…)

Slide 78

Slide 78 text

Legacy Code 21.10.2016 Compromise! Refactor one large piece into components.  Favour piece by piece rewrite over big bang rewrite and rebuild incrementally. What's the cost of debt? Which parts are less critical? Think of extra costs for maintenance and extension caused by overly complex code. This strategy will let you: • focus only on the important and critical parts • recover knowledge which got lost in complex code • preserve investments which went into the code already (time for bugﬁxes, requirements…)

Slide 79

Slide 79 text

Legacy Code 21.10.2016 TL;DR technical debt is the main reason for legacy code  treat your legacy and former developers with respect  the business always wins* (changing requirements)  keep your code well tested  break dependencies and decouple business logic  maintenance == refactoring (pay your debt!)   

Slide 80

Slide 80 text

Railslove We love to build the web. KTHXBYE Questions? Marco Schaden  @donschado [email protected]  21.10.2016

Slide 81

Slide 81 text

Legacy Code 21.10.2016 Refactoring - Ruby Edition http://www.amazon.com/Refactoring-Ruby-Addison-Wesley-Professional/dp/0321984137 TL;DR: http://ghendry.net/refactor.html Patterns of Enterprise Application Architecture http://www.amazon.com/Patterns-Enterprise-Application-Architecture-Martin/dp/0321127420/ Working Effectively with Legacy Code http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052  Golden Master Testing: Refactor Complicated Views Katrina Owen: July 09, 2014 http://www.sitepoint.com/golden-master-testing-refactor-complicated-views/ Rails Conf 2013 The Magic Tricks of Testing by Sandi Metz https://www.youtube.com/watch?v=URSWYvyc42M GORUCO 2009 - SOLID Object-Oriented Design by Sandi Metz https://www.youtube.com/watch?v=v-2yFMzxqwU Rocky Mountain Ruby 2012 - Go Ahead, Make a Mess by Sandi Metz https://www.youtube.com/watch?v=f5I1iyso29U   RailsConf 2014 - Step-by-Step Refactoring Toward Ember by Brandon Hays  https://www.youtube.com/watch?v=VMmaKj8hCR4 Katrina Owen - 467 tests, 0 failures, 0 confidence - Railsberry 2013 https://vimeo.com/68730418 BathRuby 2015 - Here Be Dragons, Katrina Owen https://www.youtube.com/watch?v=QAUHYzC9kFM Cascadia Ruby Conf 2012 Therapeutic Refactoring by Katrina Owen https://www.youtube.com/watch?v=J4dlF0kcThQ RailsConf 2015 - Getting a Handle on Legacy Code https://www.youtube.com/watch?v=lsFFjFp7mEE MountainWest RubyConf 2015 - Data-Driven Refactoring https://www.youtube.com/watch?v=AcdliNixNhs Baruco 2013: Design Patterns And The Proper Cultivation Thereof, Corey Haines https://www.youtube.com/watch?v=vqN3TQgsXzI Rocky Mountain Ruby 2013 SOLID and TDD, Sitting in a by Mike Nicholaides https://www.youtube.com/watch?v=FidRcixHQos RubyConf 2009 - SOLID Ruby by: Jim Weirich https://www.youtube.com/watch?v=dKRbsE061u4 content[:source]