25+ million members in 22 countries, how to scale with Symfony2

25+ million members in 22 countries, how to scale with Symfony2

Talk given with Benjamin De Bernardi (@genes0r) at PHP Frameworks day in Kiev.

418997665c4a3368515ecf9c3d746b95?s=128

Olivier Dolbeau

September 03, 2016
Tweet

Transcript

  1. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 25+ million members in 22 countries; How to scale with Symfony2? Benjamin De Bernardi - Olivier Dolbeau
  2. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r @genes0r Lead Engineer Benjamin De Bernardi
  3. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Olivier Dolbeau @odolbeau Web Architect
  4. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 100€ 25€ 25€ 25€ 100€ 25€
  5. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 15M 20M 10M 5M 1M 2015 2014 2013 2012 2011 2010 2009 25M members worldwide Over 25 million members and growing 1.5 million+ joining each month
  6. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  7. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 1. The past 2. Why symfony? 3. How? ◦ How to deal with 22 countries? ◦ Why do we love the EventDispatcher? ◦ Performance & asynchronous ◦ Doctrine 4. Traps 5. Here & now Talk overview:
  8. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r The past 1
  9. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r v1.0 2005-2006
  10. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  11. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r v2.0 2008-2011 (2014)
  12. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  13. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 2010 New countries 2011 2012
  14. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  15. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 3 770 lines of code 69 functions 926 lines for the longest one. lib.trip.php
  16. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Homemade framework
  17. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r NO TESTS
  18. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r v3.0 2011 - today
  19. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  20. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 2012 2013 2014 2015 New countries
  21. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Why Symfony? 2
  22. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Huge community.
  23. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  24. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Attract new people
  25. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r How? 3
  26. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries, how can we manage specificities?
  27. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ➔ One codebase for each country? ➔ Conditions everywhere in the code? ➔ Configuration and Context? 22 countries: how can we manage specificities?
  28. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ➔ One codebase for each country? ➔ Conditions everywhere in the code? ➔ Configuration and Context? 22 countries: how can we manage specificities?
  29. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Web request Listener Symfony parse & build a Request object Build Context from configuration and Request Response 22 countries: Configuration & Context
  30. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 1 file for each country 100+ configuration keys Inheritance Toggling of features Metadata 22 countries: Configuration & Context Configuration:
  31. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context
  32. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context Get config from a controller:
  33. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context Get config from a controller:
  34. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context Get config from the view:
  35. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context
  36. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean?
  37. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ➔ Symfony built-in EventDispatcher Lot of code, how to keep events clean?
  38. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r EventDispatcher how it works: Lot of code, how to keep events clean? Event Dispatcher Event Listener Listener
  39. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r User A rates User B: Lot of code, how to keep events clean? Event Dispatcher New Rating
  40. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r User A rates User B: Lot of code, how to keep events clean? Event Dispatcher New Rating Grade Listener Recalculate the grade of both users
  41. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r User A rates User B: Lot of code, how to keep events clean? Event Dispatcher New Rating Grade Listener Moderation Listener Ask for content verification
  42. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean? Sending the event: • Sent from anywhere
  43. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean? Registering a listener: • Update a configuration file
  44. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean? Listening the event: • Inside a registered listener
  45. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r + Perfect to decouple code inside the app. + Small listeners, easy to understand, to test. + Easy to dispatch events outside application. - Sometimes, hard to understand the flow. - Need tools/rules to keep a common logic. Dispatching events Pros/Cons
  46. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ▪ Avoid Entities ▪ Keep listener small and dedicated to a single task ▪ Limit dependencies Dispatching events one last thing:
  47. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r High traffic, how to improve performance?
  48. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Problem: The website is a bit slow
  49. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Solution: All side tasks can be done asynchronously
  50. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Solution: All side tasks can be done asynchronously Tasks: • Find a tool to transmit messages • Create workers • Use the event dispatcher
  51. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous jobs everywhere. • Performance • Job splitting
  52. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  53. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Dispatch Event with the EventDispatcher. • Sent from anywhere • Listened with RabbitMQ or internal Listeners
  54. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Check eventName & event interface Dispatch the event in the application Decorated EventDispatcher. Publish to RabbitMQ
  55. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Send an Email • Sent from anywhere • Built using the User context in a dedicated worker
  56. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r + Perfect to decouple code + Small workers, easy to understand + Easy to dispatch events outside application + Separate projects & deployment + Scalable! - Learning curve - Need tools/rules to keep a common logic - Hard to test a complete flow with mock/stubs - Mapping is time consuming Asynchronous event & process Pros/Cons
  57. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 129 workers Mail SMS Publication Push Indexation Image resizing Alerts Profile completion I love workers! Payment
  58. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous workers: Infrastructure: ▪ PHP ▪ Supervisord Code: ▪ Symfony command inside the main app ▪ Standalone app + Symfony components
  59. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous workers, Symfony: + Access to Entities + Every already defined services - Entire Symfony - Not dedicated - Deployed along the website - Not the main goal of Symfony - High memory usage
  60. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous workers, Standalone: + Dedicated to a single task/type + Maintainable small code + Deployed independently + Easily created & tested - Component splitting from the codebase - More monitoring
  61. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r About Doctrine...
  62. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r It’s not a troll.
  63. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Code fast!
  64. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Code well?
  65. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  66. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Queries from the view! Hard to control cache strategy No integrity check.
  67. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Doctrine Pros/Cons + Quick to bootstrap a new project + Quick to code a small project - Hard to respect MVC - Hard to deal with cache - Hard to deal with integrity problems in DB - Not made to decouple code (relations). - Not made to scale.
  68. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r We fell in traps... 4
  69. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Never pimp your kernel!
  70. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Before Environment Debug?
  71. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Before After Branding Media Environment Debug?
  72. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Put everything in the same place can be dangerous!
  73. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  74. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r This is our application. This is our business code.
  75. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Don’t mess with your tools!
  76. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  77. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r
  78. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r We fell in traps...
  79. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r We had parachutes!
  80. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Problems solved? + Use symfony only as a tool + Work a lot on onboarding + Try to decouple everything with components - We still have one big application - As everyone, we lack of documentation
  81. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 5
  82. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 40 269 Commits There are “only” 18 952 commits for symfony/symfony
  83. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r 561 851 NCLOC There are “only” 278 909 NCLOC for symfony/symfony
  84. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ~ 20 minutes to run all the unit/functional tests
  85. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r ~ 30 minutes to deploy the application
  86. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r Too big!
  87. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r We’re working on it! ;)
  88. PHP Frameworks Day 2016 - 25+ million members in 22

    countries; How to scale with Symfony2 @odolbeau & @genes0r We’re hiring! In Paris & Warsaw! http://blbl.cr/dreamjobs