Surviving a Legacy Codebase by Jeremy Thurgood

Few things strike more fear into the heart of a seasoned software developer than the words "legacy code". However, many of us spend a lot of time working on byzantine monstrosities inherited from contractors, third parties, or Bob who left the company three months ago. Over the past several years, I've sunk way more hours than I care to think about into making legacy codebases more malleable. I've picked up a few tricks and strategies along the way that make the process a little smoother and less painful, and I will be sharing them in this talk.

Transcript

1. SURVIVING A LEGACY CODEBASE IT’S (PROBABLY) NOT AS BAD AS YOU THINK Jeremy Thurgood PyconZA 2017
2. WHAT IS LEGACY CODE? PRELUDE
3. “CODE FOR AN OBSOLETE COMPUTER” THE VERY OLD DEFINITION
4. “CODE INHERITED FROM SOMEONE ELSE” THE MOST COMMON CURRENT DEFINITION
5. “CODE YOU’RE AFRAID TO MODIFY” A MORE VISCERAL DEFINITION
6. “CODE WITHOUT TESTS” —MICHAEL FEATHERS A PRACTICAL DEFINITION THAT POINTS TO A SOLUTION
7. GET IT TESTED ACT 1
8. WRITE SOME TESTS All done, let’s go home. IF ONLY IT WERE THAT EASY.
9. WHY IS IT NOT THAT EASY? It just wasn’t written to be testable. Spaghetti Giant functions Global mutable state Tightly coupled dependencies
10. IT GETS WORSE OVER TIME Changing legacy code is scary. It doesn’t get refactored. Things get hacked in.
11. ALL IS NOT LOST! We can make legacy code testable. Very carefully. With limited, controlled changes.
12. SMALL EXAMPLE: CREATE_VM I want to modify this celery task… …but it creates a remote XenServer API session. @app.task(time_limit=120) def create_vm(vm, xenserver, template, name, **others): session = getSession( xenserver.hostname, xenserver.username, xenserver.password) storage = session.xenapi.SR.get_all() # ... Another 180 lines of VM creation using the session ... def getSession(hostname, username, password): session = xenapi.Session('https://%s:443/' % (hostname)) session.xenapi.login_with_password(username, password) return session
13. SMALL EXAMPLE: CREATE_VM Factor out everything that uses the session… …build a test double for the session object… …and then write some tests. @app.task(time_limit=120) def create_vm(vm, xenserver, template, name, **others): session = getSession( xenserver.hostname, xenserver.username, xenserver.password) return _create_vm(session, vm, template, name, **others) def _create_vm(session, vm, template, name, **others): storage = session.xenapi.SR.get_all() # ... Another 180 lines of VM creation using the session ... class FakeXenServer(object): """Fake XenServer to use in tests.""" # ... 300+ lines of implementation ...
14. THINGS TO NOTE We made one small change to the legacy code. We did not change the behaviour of the legacy code. We did not change the signature of create_vm. We built a test double that will be useful elsewhere. We can now start making safe, tested changes.
15. TESTING LEGACY CODE FOCUS ON THE TASK AT HAND Only test the relevant code Hacks are okay WRITE INVESTIGATIVE TESTS “What does it do if I give it this input?” Copy “expected” values from actual output
16. TOOLS AND TECHNIQUES ACT 2
17. FIRST, SOME TOOLS STATIC ANALYSIS ake8 pylint AUTOMATED REFACTORING PyCharm Rope THE HUMAN BRAIN
18. BREAKING DEPENDENCIES Test doubles! …but how do we get them in there? Break up long functions Add parameters Encapsulate global references Use seams
19. ENCAPSULATING A GLOBAL DEPENDENCY Instead of using reactor directly, put it in an attribute… …then override that attribute in the test. class SmppTransceiverTransport(Transport): clock = reactor # ... Quite a lot of transport implementation code ... def check_stop_throttling(self, delay=None): # ... A few lines of throttle-stop-checking code ... self._unthrottle_delayedCall = self.clock.callLater( delay, self._check_stop_throttling) def test_smpp_transport(): clock = Clock() transport = SmppTransceiverTransport() transport.clock = clock # ... The rest of the test ...
21. SEAMS “A seam is a place where you can alter behavior in your program without editing in that place.” —Michael Feathers module imports function/method calls attribute lookups
22. EXPLOITING SEAMS Install a fake library Monkey-patch Subclass and override
23. BREAKING A DEPENDENCY WITH A SEAM We could extract getSession() as we did before… …by making nontrivial changes to untested code. :-( @app.task(time_limit=60) def updateServer(xenserver): session = getSession( xenserver.hostname, xenserver.username, xenserver.password) # ... Dozens of lines of code to update a server ... for vmref, vmobj in allvms.items(): updateVm.delay(xenserver, vmref, vmobj) # ... Dozens more lines of server-related code ... @app.task(time_limit=60) def updateVm(xenserver, vmref, vmobj): # ... A few lines of check and setup code ... session = getSession( xenserver.hostname, xenserver.username, xenserver.password) # ... Dozens of lines of code to update the VM ...
25. BREAKING A DEPENDENCY WITH A SEAM Instead, we monkey-patch in our own getSession(). It’s ugly, but we know we didn’t break anything. :-) def test_updateServer(monkeypatch): from xenserver import tasks from xenserver.tests.helpers import XenServerHelper xshelper = XenServerHelper() monkeypatch.setattr(tasks, 'getSession', xshelper.get_session) # ... Way too many lines of code to test updateServer() ...
26. CHANGING UNTESTED CODE Separate new code from old. PROS: New code can be tested in isolation Changes to old code are restricted CONS: Spaghetti with meatballs Old code doesn’t improve Use with care, clean up later.
27. THE LEGACY CODE ALGORITHM 1. Identify code that needs to change 2. Break dependencies 3. Write tests 4. Make changes 5. Refactor (where possible)
28. READ THIS BOOK
29. THE END OF THE SLIDES Now you get to ask me hard questions. (Or easy questions. I prefer those.) Image credits: Unicorn cat from (CC By 4.0) Book cover from product page animalsclipart.com amazon.ca