Open Heart Surgery (In Production)

Symfony Live 2016

Transcript

1. Open Heart Surgery (In Production) Refactoring the region price search, the beating heart of our hotel search application
2. Who is this Jan Guy… • Background in Assembler / C / C++. • Then found out PHP devs party harder. • Joined trivago in 2012. • Worked Hotel Search product since then. • Feature development in most components. • Moved on to Release / Application Maintainer and core architecture • Overzealous, strenuously passionate loud-mouth @janvanthoor http://janvt.com
3. “…is the world’s largest online hotel search site, comparing rates from over 900,000 hotels on over 250 booking sites worldwide. trivago integrates over 200 million hotel ratings and 14 million photos in order to make finding the perfect hotel easier for users.” • 120 million visitors per month • 66 million requests per day • global internet traffic • average: 500 Mbit/s • peak: 850 Mbit/s • 4 million search queries per day • 8,7 million bookings in 2014 • 250+ booking sites • 55 live markets • 33 languages
4. The Region Price Whaaat? • \Trivago\BusinessCase\RegionPriceSearch\RegionPriceSearch • Serves all (region) prices (>80% of all prices served) • >60% of our total traffic • Main endpoint for “/“ and “search/region” routes • Snapshot shit (application state) • Java Main Service communication • Result decoration (hotel fields, filter data, etc….) • Everything and nothing….
5. Some Numbers: Files: 28 Unit Tests: 0 LOC: ~10,000 Avg. Lines per Function: 150 CRAP Index: 14,280 Avg. Dependencies: 15
7. No, but, like seriously, what’s, like, the real problem… • Development speed • Team growth (code readability and complexity) • Constant testing / high change rate (and not enough boy-scouts) • Jan (Basic design mistakes, logical and structural fallacies that existed from the beginning, mostly thought up by yours truly) • Stability (absolutely no testability) • Extensibility (a.k.a. ability to add and modify features a.k.a CTests) • Maintainability
8. WTFWWH “What The Fuck Went Wrong Here”
15. No, but, like seriously, what’s, like, the real problem… • No Separation of Concerns. • Basic principles of MVC went out the window with the first line of code. • Interfaces were shit. • Death to all Boy Scouts! Critical nature of WHAT the code did seemed to prevent everyone from asking WHY it was done that way. • Lack of documentation. • 0 testability.
17. I was drunk in Poland once….
19. So, how do you refactor a monster? • Dietmar (a.k.a. GET HELP) • Figure out where you’re headed • Sharpen your FLEX • Baby steps • Find a shitty “interface”. • Figure out what it should look like. • Shim the shit out of the implementation. • Adjust usages. • Now you have a clean “border” you can write tests for. • Write tests. • Start refactoring….
20. Where are we headed? • Effective architecture > dogmatic architecture. • Support a high change rate, it’s our business. • Separation of concerns! • Build frameworks for features, not classes full of features. • Find the right layer for your logic. Model, View, Controller. • “Strensible” decoupling: • Be just strict enough to stay sensible and extensible. • Code interacts best on a “need-to-know” basis. • Leads directly to re-usability (Don’t Fucking Repeat Yourself). • Testability. • Performance… (who gives a shit).
21. So, What is a Baby Step • Get an overview of functionality. For me, this requires: • Concentration. • Time. • Alcohol. • Spend lots of time thinking about and discussing existing and proposed interfaces: • How were they used. • How are they used currently. • How might they be used and how could they improve.
22. UNDERSTAND DEFINE SHIM ADJUST TEST REFACTOR REPEAT
23. Baby Step: Fix All the JavaScripts • Clean up interface between client and server. • Get rid of highly-inflexible “user/search” fields. • Backend was completely shimmed, awful performance. • Created the ground-work for further steps.
24. Baby Step: Clean up Java API Clean up interface between PHP and Java
27. Tagged Services for Result Decoration • Introduction of tagged service architecture for hotel decoration. • Highest change rate comes from features that “do something to an item” • Clickout link generation. • General item data (from DB). • HasNews flag (tHPM Pro feature). • etc…. • Each new “builder” defines it’s own dependencies. • Optional cache warming features. • Test coverage went from 0 to over 80%. • Features now are independent. • Hello “item feature framework”
33. Baby Step: Introduce Mappers • “Controller as a service”. • Move view logic out of controllers (often quite complex). • Keep complicated operations out of templating.
36. Snowball Effect • I still didn’t trust this “architecture”…. • But Dietmar was right. • Effective architecture and more test coverage create an amazing snowball effect. • Design and architecture decisions became more daring and radical. • Moving code to the proper place was suddenly easy. • The seemingly impossible became responsibly reachable.
37. #keepflexxin