Open Heart Surgery (In Production)

Symfony Live 2016

Transcript

1. Open Heart Surgery
   (In Production)
   Refactoring the region price search,
   the beating heart of our hotel search application
   

   View Slide

2. Who is this Jan Guy…
   • Background in Assembler / C / C++.
   • Then found out PHP devs party harder.
   • Joined trivago in 2012.
   • Worked Hotel Search product since then.
   • Feature development in most components.
   • Moved on to Release / Application Maintainer and core architecture
   • Overzealous, strenuously passionate loud-mouth
   @janvanthoor http://janvt.com
   

   View Slide

3. “…is the world’s largest online hotel search
   site, comparing rates from over 900,000
   hotels on over 250 booking sites worldwide.
   trivago integrates over 200 million hotel
   ratings and 14 million photos in order to make
   finding the perfect hotel easier for users.”
   • 120 million visitors per month
   • 66 million requests per day
   • global internet traffic
   • average: 500 Mbit/s
   • peak: 850 Mbit/s
   • 4 million search queries per day
   • 8,7 million bookings in 2014
   • 250+ booking sites
   • 55 live markets
   • 33 languages
   

   View Slide

4. The Region Price Whaaat?
   • \Trivago\BusinessCase\RegionPriceSearch\RegionPriceSearch
   • Serves all (region) prices (>80% of all prices served)
   • >60% of our total traffic
   • Main endpoint for “/“ and “search/region” routes
   • Snapshot shit (application state)
   • Java Main Service communication
   • Result decoration (hotel fields, filter data, etc….)
   • Everything and nothing….
   

   View Slide

5. Some Numbers:
   Files: 28
   Unit Tests: 0
   LOC: ~10,000
   Avg. Lines per Function: 150
   CRAP Index: 14,280
   Avg. Dependencies: 15
   

   View Slide

6. View Slide

7. No, but, like seriously,
   what’s, like, the real problem…
   • Development speed
   • Team growth (code readability and complexity)
   • Constant testing / high change rate (and not enough boy-scouts)
   • Jan (Basic design mistakes, logical and structural fallacies that existed
   from the beginning, mostly thought up by yours truly)
   • Stability (absolutely no testability)
   • Extensibility (a.k.a. ability to add and modify features a.k.a CTests)
   • Maintainability
   

   View Slide

8. WTFWWH
   “What The Fuck Went Wrong Here”
   

   View Slide

9. View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. View Slide

15. No, but, like seriously,
    what’s, like, the real problem…
    • No Separation of Concerns.
    • Basic principles of MVC went out the window with the first line of code.
    • Interfaces were shit.
    • Death to all Boy Scouts! Critical nature of WHAT the code did seemed to
    prevent everyone from asking WHY it was done that way.
    • Lack of documentation.
    • 0 testability.
    

    View Slide

16. View Slide

17. I was drunk in Poland once….
    

    View Slide

18. View Slide

19. So, how do you refactor a monster?
    • Dietmar (a.k.a. GET HELP)
    • Figure out where you’re headed
    • Sharpen your FLEX
    • Baby steps
    • Find a shitty “interface”.
    • Figure out what it should look like.
    • Shim the shit out of the implementation.
    • Adjust usages.
    • Now you have a clean “border” you can write tests for.
    • Write tests.
    • Start refactoring….
    

    View Slide

20. Where are we headed?
    • Effective architecture > dogmatic architecture.
    • Support a high change rate, it’s our business.
    • Separation of concerns!
    • Build frameworks for features, not classes full of features.
    • Find the right layer for your logic. Model, View, Controller.
    • “Strensible” decoupling:
    • Be just strict enough to stay sensible and extensible.
    • Code interacts best on a “need-to-know” basis.
    • Leads directly to re-usability (Don’t Fucking Repeat Yourself).
    • Testability.
    • Performance… (who gives a shit).
    

    View Slide

21. So, What is a Baby Step
    • Get an overview of functionality. For me, this requires:
    • Concentration.
    • Time.
    • Alcohol.
    • Spend lots of time thinking about and discussing existing and
    proposed interfaces:
    • How were they used.
    • How are they used currently.
    • How might they be used and how could they improve.
    

    View Slide

22. UNDERSTAND
    DEFINE
    SHIM
    ADJUST
    TEST
    REFACTOR
    REPEAT
    

    View Slide

23. Baby Step: Fix All the JavaScripts
    • Clean up interface between client and server.
    • Get rid of highly-inflexible “user/search” fields.
    • Backend was completely shimmed, awful performance.
    • Created the ground-work for further steps.
    

    View Slide

24. Baby Step: Clean up Java API
    Clean up interface between PHP and Java
    

    View Slide

25. View Slide

26. View Slide

27. Tagged Services for Result Decoration
    • Introduction of tagged service architecture for hotel decoration.
    • Highest change rate comes from features that “do something to an item”
    • Clickout link generation.
    • General item data (from DB).
    • HasNews flag (tHPM Pro feature).
    • etc….
    • Each new “builder” defines it’s own dependencies.
    • Optional cache warming features.
    • Test coverage went from 0 to over 80%.
    • Features now are independent.
    • Hello “item feature framework”
    

    View Slide

28. View Slide

29. View Slide

30. View Slide

31. View Slide

32. View Slide

33. Baby Step: Introduce Mappers
    • “Controller as a service”.
    • Move view logic out of controllers (often quite complex).
    • Keep complicated operations out of templating.
    

    View Slide

34. View Slide

35. View Slide

36. Snowball Effect
    • I still didn’t trust this “architecture”….
    • But Dietmar was right.
    • Effective architecture and more test coverage create an
    amazing snowball effect.
    • Design and architecture decisions became more daring
    and radical.
    • Moving code to the proper place was suddenly easy.
    • The seemingly impossible became responsibly reachable.
    

    View Slide

37. #keepflexxin
    

    View Slide