Upgrade to Pro — share decks privately, control downloads, hide ads and more …

25+ million members in 22 countries, how to sca...

25+ million members in 22 countries, how to scale with Symfony2

Talk given with Olivier Dolbeau (@odolbeau) at PHP Frameworks day 2016 in Kiev, Ukraine.

Avatar for Benjamin De Bernardi

Benjamin De Bernardi

September 03, 2016
Tweet

More Decks by Benjamin De Bernardi

Other Decks in Technology

Transcript

  1. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 25+ million members in 22 countries; How to scale with Symfony2? Benjamin De Bernardi - Olivier Dolbeau
  2. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r @genes0r Lead Engineer Benjamin De Bernardi
  3. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Olivier Dolbeau @odolbeau Web Architect
  4. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 100€ 25€ 25€ 25€ 100€ 25€
  5. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 15M 20M 10M 5M 1M 2015 2014 2013 2012 2011 2010 2009 25M members worldwide Over 25 million members and growing 1.5 million+ joining each month
  6. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  7. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 1. The past 2. Why symfony? 3. How? ◦ How to deal with 22 countries? ◦ Why do we love the EventDispatcher? ◦ Performance & asynchronous ◦ Doctrine 4. Traps 5. Here & now Talk overview:
  8. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r The past 1
  9. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r v1.0 2005-2006
  10. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  11. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r v2.0 2008-2011 (2014)
  12. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  13. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 2010 New countries 2011 2012
  14. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  15. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 3 770 lines of code 69 functions 926 lines for the longest one. lib.trip.php
  16. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Homemade framework
  17. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r NO TESTS
  18. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r v3.0 2011 - today
  19. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  20. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 2012 2013 2014 2015 New countries
  21. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Why Symfony? 2
  22. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Huge community.
  23. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  24. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Attract new people
  25. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r How? 3
  26. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries, how can we manage specificities?
  27. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ➔ One codebase for each country? ➔ Conditions everywhere in the code? ➔ Configuration and Context? 22 countries: how can we manage specificities?
  28. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ➔ One codebase for each country? ➔ Conditions everywhere in the code? ➔ Configuration and Context? 22 countries: how can we manage specificities?
  29. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Web request Listener Symfony parse & build a Request object Build Context from configuration and Request Response 22 countries: Configuration & Context
  30. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 1 file for each country 100+ configuration keys Inheritance Toggling of features Metadata 22 countries: Configuration & Context Configuration:
  31. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context
  32. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context Get config from a controller:
  33. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context Get config from a controller:
  34. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context Get config from the view:
  35. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context
  36. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean?
  37. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ➔ Symfony built-in EventDispatcher Lot of code, how to keep events clean?
  38. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r EventDispatcher how it works: Lot of code, how to keep events clean? Event Dispatcher Event Listener Listener
  39. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r User A rates User B: Lot of code, how to keep events clean? Event Dispatcher New Rating
  40. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r User A rates User B: Lot of code, how to keep events clean? Event Dispatcher New Rating Grade Listener Recalculate the grade of both users
  41. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r User A rates User B: Lot of code, how to keep events clean? Event Dispatcher New Rating Grade Listener Moderation Listener Ask for content verification
  42. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean? Sending the event: • Sent from anywhere
  43. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean? Registering a listener: • Update a configuration file
  44. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean? Listening the event: • Inside a registered listener
  45. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r + Perfect to decouple code inside the app. + Small listeners, easy to understand, to test. + Easy to dispatch events outside application. - Sometimes, hard to understand the flow. - Need tools/rules to keep a common logic. Dispatching events Pros/Cons
  46. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ▪ Avoid Entities ▪ Keep listener small and dedicated to a single task ▪ Limit dependencies Dispatching events one last thing:
  47. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r High traffic, how to improve performance?
  48. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Problem: The website is a bit slow
  49. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Solution: All side tasks can be done asynchronously
  50. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Solution: All side tasks can be done asynchronously Tasks: • Find a tool to transmit messages • Create workers • Use the event dispatcher
  51. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous jobs everywhere. • Performance • Job splitting
  52. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  53. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Dispatch Event with the EventDispatcher. • Sent from anywhere • Listened with RabbitMQ or internal Listeners
  54. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Check eventName & event interface Dispatch the event in the application Decorated EventDispatcher. Publish to RabbitMQ
  55. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Send an Email • Sent from anywhere • Built using the User context in a dedicated worker
  56. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r + Perfect to decouple code + Small workers, easy to understand + Easy to dispatch events outside application + Separate projects & deployment + Scalable! - Learning curve - Need tools/rules to keep a common logic - Hard to test a complete flow with mock/stubs - Mapping is time consuming Asynchronous event & process Pros/Cons
  57. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 129 workers Mail SMS Publication Push Indexation Image resizing Alerts Profile completion I love workers! Payment
  58. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous workers: Infrastructure: ▪ PHP ▪ Supervisord Code: ▪ Symfony command inside the main app ▪ Standalone app + Symfony components
  59. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous workers, Symfony: + Access to Entities + Every already defined services - Entire Symfony - Not dedicated - Deployed along the website - Not the main goal of Symfony - High memory usage
  60. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous workers, Standalone: + Dedicated to a single task/type + Maintainable small code + Deployed independently + Easily created & tested - Component splitting from the codebase - More monitoring
  61. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r About Doctrine...
  62. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r It’s not a troll.
  63. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Code fast!
  64. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Code well?
  65. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  66. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Queries from the view! Hard to control cache strategy No integrity check.
  67. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Doctrine Pros/Cons + Quick to bootstrap a new project + Quick to code a small project - Hard to respect MVC - Hard to deal with cache - Hard to deal with integrity problems in DB - Not made to decouple code (relations). - Not made to scale.
  68. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r We fell in traps... 4
  69. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Never pimp your kernel!
  70. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Before Environment Debug?
  71. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Before After Branding Media Environment Debug?
  72. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Put everything in the same place can be dangerous!
  73. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  74. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r This is our application. This is our business code.
  75. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Don’t mess with your tools!
  76. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  77. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  78. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r We fell in traps...
  79. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r We had parachutes!
  80. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Problems solved? + Use symfony only as a tool + Work a lot on onboarding + Try to decouple everything with components - We still have one big application - As everyone, we lack of documentation
  81. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 5
  82. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 40 269 Commits There are “only” 18 952 commits for symfony/symfony
  83. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 561 851 NCLOC There are “only” 278 909 NCLOC for symfony/symfony
  84. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ~ 20 minutes to run all the unit/functional tests
  85. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ~ 30 minutes to deploy the application
  86. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Too big!
  87. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r We’re working on it! ;)
  88. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r We’re hiring! In Paris & Warsaw! http://blbl.cr/dreamjobs