Total Rewrite

Presentation deck & speaker notes for my talk at LRUG on September 12th, 2016

Transcript

2. Who? • David Salgado • @digitalronin (twitter, github, stackoverflow) • CTO, Tech co-founder, lead dev. etc. • Technical Architect @ Ministry of Justice
3. I've done some rewrites Apologise in advance: presentation is a random walk rewrites touch on lots of different areas, & so does this talk
4. What is a ‘rewrite’?
5. Decide to replace
 a working system
 with one that is equivalent
 but different Let's unpack that
6. Decide to replace
 a working system
 with one that is equivalent
 but different Not like you ended up replacing every line of code during normal development
7. Decide to replace
 a working system
 with one that is equivalent
 but different If it's not working, you're fixing, not rewriting
8. Decide to replace
 a working system
 with one that is equivalent
 but different If it's not equivalent, you're developing, not rewriting
9. Decide to replace
 a working system
 with one that is equivalent
 but different If it's not different, you've just run "cp -r"
10. Why? • “This code sucks” • “We should use #{shiny_new_tech}” • “This design/architecture sucks” • (aka “because micro-services”) Usually (not always) it's the developers who are pushing the rewrite There is a big problem with all of these arguments
11. – Dan Milstein “Developers tend to spectacularly underestimate the effort involved...and spectacularly overestimate the value generated” Remember, we're talking about replacing a working system - so, the current code is generating value for the business Let's look at this idea a bit more closely
12. This is what your
 product owner hears
13. This is what your
 product owner hears “We want to spend a huge amount of effort
 (aka money), and deliver no benefit” If you've got a smart product owner, they might hear...
14. This is what your
 product owner hears “We want to spend a huge amount of effort
 (aka money), and deliver no immediate benefit” If you've got a really smart product owner...
15. This is what your
 product owner hears “We want to spend a huge amount of effort
 (aka money), and deliver no immediate benefit to the business” Digress, & talk about Incentives
16. Incentives Let's look at the incentives for doing a rewrite
17. Incentives • Faster to add new features • Enables feature XXXXX • Fewer critical failures • Less devops load • Happier devs • Recruitment Business
18. Incentives • Work on a better system • Write software, not maintain it • Play with new toys • New tech/language/patterns • + CV / career • Faster to add new features • Enables feature XXXXX • Fewer critical failures • Less devops load • Happier devs • Recruitment Developers Business write software - we'll come back to that
19. Not perfectly aligned * Rewrite could be good for devs, but bad for business, or vice versa (but usually devs are pushing for rewrite)
20. Not saying "pitch a rewrite you know is a bad idea for your organisation" - depends on your situation and your conscience Know your goals and the goals of those around you => more chance of a good outcome
21. “This code sucks” So common, it's a cliche (like builders/mechanics). OK, sometimes it's true, but there are other factors Why?
22. Writing vs. Reading Code • Easier • More fun • Reputation Comes back to writing versus reading "Here is my gift to the world" github stars, twitter followers - "Here is a thing" vs. "I made this lib suck less" Serious problem in our industry - could be a separate talk
23. Should you ever rewrite?
24. Probably not ...but sometimes, yes - the new tech/architecture really is necessary - the code really does suck So, let's look at how to do it
25. How (not) to rewrite Or rather, how not to. One, very common anti-pattern...
27. Big Bang • Feature freeze • Build new system (from scratch) • Migrate • Profit / Unicorns / Kittens / Ponies
28. Why not? • It will take (a lot) longer than you think • You've stopped delivering value to users • The feature freeze is a lie • You're locked in • => Massive Risk There is a better way
29. Instead of jumping off a cliff there is a better way to get where you're going
30. Incremental • Work in small chunks • Freedom to stop at any point • Minimise Risk (one sprint) one sprint is an ideal scenario, but you can always get close to that
31. Strangler Application The most common pattern for doing an incremental rewrite Martin Fowler, 2004 sinister name
32. Strangler vine, aka strangler fig Dark, tropical forests where sunlight doesn't reach the ground
33. Start with a tree. vine seed lands in crevice grows up and down, using the tree's structure, intercepting sunlight & nutrients Tree dies & only the vine remains
34. Start with a tree. vine seed lands in crevice grows up and down, using the tree's structure, intercepting sunlight & nutrients Tree dies & only the vine remains
35. Start with a tree. vine seed lands in crevice grows up and down, using the tree's structure, intercepting sunlight & nutrients Tree dies & only the vine remains
36. Strangler Application 1. Set up production instance of new, alongside old 2. Build feature in new, that looks like old 3. Delete feature in old 4. Repeat until old is gone In terms of a software rewrite, process looks like this More detail about how to implement this in practice
37. No. 1 rule - be prepared. Lots you can do before you start to make it easier to survive "Ash" - evil dead movies chainsaw where his hand used to be, & carries a shotgun Good mindset to have when embarking on a rewrite
38. Test Coverage • Prove that new == old • Use your favourite testing tool • Containerisation Feature tests (acceptance tests) Don't start until you can prove new == old Changing tech, could be unfamiliar. Old tech could be horrible. Test suite can be its own application Containers can help - new & old & test suite can all target the same backend data store
39. Walking Skeleton • Infrastructure • Deployment (CI/CD) • Web App. • Look & feel • Routing • Shared auth/session Writing the code is the easy part. Don't leave ops work to the end. Do it first look & feel - users shouldn't notice the difference. How to route some requests to new & some to old routing: sub-domain? path-based routing? sessions (to have a subset of users see new app)? shared auth. - port old auth to new app. can be quite a big task, move to standards? oauth or JWT
40. Spike one function • Simple • Genuine • Authenticated • a.k.a. “Tracer Bullet” genuine - not a static content page: something that really exercises your system e.g. "edit my account" "tracer bullet" in Dave Thomas' "The Pragmatic Programmer" - crappy name, but it's about seeing what you're aiming at
41. Back-end Systems • Choose component • Refactor (clarify boundaries) • Extract as a service • Replace with new version • Repeat Not just web apps - Can replace a backend system, one piece at a time Might be a bit more work One thing to beware of...
42. Scope-creep • “We can fix XXX during the rewrite” • This will kill you, and eat your brain • Fix then rewrite or rewrite then fix One of the few nice things about a rewrite is the clearly defined scope This is what the chainsaw and the shotgun are for; "while you're in that bit of the code..." Puts you in an indeterminate state; new != old. Can't roll back, b/c user experience is different rewrite & development are different activities - guard that separation
43. Summary • Incremental rewrite • Strangler Application • Prepare • Tests. Shared Authentication. DevOps • Fight scope-creep • (Mostly) Don’t rewrite, refactor From a dev. POV, I'd argue that launching a rewrite is the lazy option Unless you're changing tech. (for which you need a good reason), you can almost always refactor your way to clean, maintainable, performant code ...
44. but it's usually harder, and less fun than throwing away and starting from scratch
45. References • Things You Should Never Do, Part I
 http://www.joelonsoftware.com/articles/fog0000000069.html • How To Survive a Ground-Up Rewrite Without Losing Your Sanity
 http://onstartups.com/tabid/3339/bid/97052/How-To-Survive-a-Ground-Up- Rewrite-Without-Losing-Your-Sanity.aspx • Strangler Vine Pattern
 http://martinfowler.com/bliki/StranglerApplication.html • Walking Skeleton
 http://blog.codeclimate.com/blog/2014/03/20/kickstart-your-next- project-with-a-walking-skeleton/