25+ million members in 22 countries, how to scale with Symfony2

PHP Frameworks Day 2016 - 25+ million members in 22
countries; How to scale with Symfony2 @odolbeau & @genes0r 25+ million members in 22 countries; How to scale with Symfony2? Benjamin De Bernardi - Olivier Dolbeau

countries; How to scale with Symfony2 @odolbeau & @genes0r @genes0r Lead Engineer Benjamin De Bernardi

countries; How to scale with Symfony2 @odolbeau & @genes0r Olivier Dolbeau @odolbeau Web Architect

countries; How to scale with Symfony2 @odolbeau & @genes0r 100€ 25€ 25€ 25€ 100€ 25€

countries; How to scale with Symfony2 @odolbeau & @genes0r 15M 20M 10M 5M 1M 2015 2014 2013 2012 2011 2010 2009 25M members worldwide Over 25 million members and growing 1.5 million+ joining each month

countries; How to scale with Symfony2 @odolbeau & @genes0r

countries; How to scale with Symfony2 @odolbeau & @genes0r 1. The past 2. Why symfony? 3. How? ◦ How to deal with 22 countries? ◦ Why do we love the EventDispatcher? ◦ Performance & asynchronous ◦ Doctrine 4. Traps 5. Here & now Talk overview:

countries; How to scale with Symfony2 @odolbeau & @genes0r The past 1

countries; How to scale with Symfony2 @odolbeau & @genes0r v1.0 2005-2006

countries; How to scale with Symfony2 @odolbeau & @genes0r v2.0 2008-2011 (2014)

countries; How to scale with Symfony2 @odolbeau & @genes0r 2010 New countries 2011 2012

countries; How to scale with Symfony2 @odolbeau & @genes0r 3 770 lines of code 69 functions 926 lines for the longest one. lib.trip.php

countries; How to scale with Symfony2 @odolbeau & @genes0r Homemade framework

countries; How to scale with Symfony2 @odolbeau & @genes0r NO TESTS

countries; How to scale with Symfony2 @odolbeau & @genes0r v3.0 2011 - today

countries; How to scale with Symfony2 @odolbeau & @genes0r 2012 2013 2014 2015 New countries

countries; How to scale with Symfony2 @odolbeau & @genes0r Why Symfony? 2

countries; How to scale with Symfony2 @odolbeau & @genes0r Huge community.

countries; How to scale with Symfony2 @odolbeau & @genes0r Attract new people

countries; How to scale with Symfony2 @odolbeau & @genes0r How? 3

countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries, how can we manage specificities?

countries; How to scale with Symfony2 @odolbeau & @genes0r ➔ One codebase for each country? ➔ Conditions everywhere in the code? ➔ Configuration and Context? 22 countries: how can we manage specificities?

countries; How to scale with Symfony2 @odolbeau & @genes0r Web request Listener Symfony parse & build a Request object Build Context from configuration and Request Response 22 countries: Configuration & Context

countries; How to scale with Symfony2 @odolbeau & @genes0r 1 file for each country 100+ configuration keys Inheritance Toggling of features Metadata 22 countries: Configuration & Context Configuration:

countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context

countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context Get config from a controller:

countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context Get config from the view:

countries; How to scale with Symfony2 @odolbeau & @genes0r 22 countries: Configuration & Context

countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean?

countries; How to scale with Symfony2 @odolbeau & @genes0r ➔ Symfony built-in EventDispatcher Lot of code, how to keep events clean?

countries; How to scale with Symfony2 @odolbeau & @genes0r EventDispatcher how it works: Lot of code, how to keep events clean? Event Dispatcher Event Listener Listener

countries; How to scale with Symfony2 @odolbeau & @genes0r User A rates User B: Lot of code, how to keep events clean? Event Dispatcher New Rating

countries; How to scale with Symfony2 @odolbeau & @genes0r User A rates User B: Lot of code, how to keep events clean? Event Dispatcher New Rating Grade Listener Recalculate the grade of both users

countries; How to scale with Symfony2 @odolbeau & @genes0r User A rates User B: Lot of code, how to keep events clean? Event Dispatcher New Rating Grade Listener Moderation Listener Ask for content verification

countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean? Sending the event: • Sent from anywhere

countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean? Registering a listener: • Update a configuration file

countries; How to scale with Symfony2 @odolbeau & @genes0r Lot of code, how to keep events clean? Listening the event: • Inside a registered listener

countries; How to scale with Symfony2 @odolbeau & @genes0r + Perfect to decouple code inside the app. + Small listeners, easy to understand, to test. + Easy to dispatch events outside application. - Sometimes, hard to understand the flow. - Need tools/rules to keep a common logic. Dispatching events Pros/Cons

countries; How to scale with Symfony2 @odolbeau & @genes0r ▪ Avoid Entities ▪ Keep listener small and dedicated to a single task ▪ Limit dependencies Dispatching events one last thing:

countries; How to scale with Symfony2 @odolbeau & @genes0r High traffic, how to improve performance?

countries; How to scale with Symfony2 @odolbeau & @genes0r Problem: The website is a bit slow

countries; How to scale with Symfony2 @odolbeau & @genes0r Solution: All side tasks can be done asynchronously

countries; How to scale with Symfony2 @odolbeau & @genes0r Solution: All side tasks can be done asynchronously Tasks: • Find a tool to transmit messages • Create workers • Use the event dispatcher

countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous jobs everywhere. • Performance • Job splitting

countries; How to scale with Symfony2 @odolbeau & @genes0r Dispatch Event with the EventDispatcher. • Sent from anywhere • Listened with RabbitMQ or internal Listeners

countries; How to scale with Symfony2 @odolbeau & @genes0r Check eventName & event interface Dispatch the event in the application Decorated EventDispatcher. Publish to RabbitMQ

countries; How to scale with Symfony2 @odolbeau & @genes0r Send an Email • Sent from anywhere • Built using the User context in a dedicated worker

countries; How to scale with Symfony2 @odolbeau & @genes0r + Perfect to decouple code + Small workers, easy to understand + Easy to dispatch events outside application + Separate projects & deployment + Scalable! - Learning curve - Need tools/rules to keep a common logic - Hard to test a complete flow with mock/stubs - Mapping is time consuming Asynchronous event & process Pros/Cons

countries; How to scale with Symfony2 @odolbeau & @genes0r 129 workers Mail SMS Publication Push Indexation Image resizing Alerts Profile completion I love workers! Payment

countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous workers: Infrastructure: ▪ PHP ▪ Supervisord Code: ▪ Symfony command inside the main app ▪ Standalone app + Symfony components

countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous workers, Symfony: + Access to Entities + Every already defined services - Entire Symfony - Not dedicated - Deployed along the website - Not the main goal of Symfony - High memory usage

countries; How to scale with Symfony2 @odolbeau & @genes0r Asynchronous workers, Standalone: + Dedicated to a single task/type + Maintainable small code + Deployed independently + Easily created & tested - Component splitting from the codebase - More monitoring

countries; How to scale with Symfony2 @odolbeau & @genes0r About Doctrine...

countries; How to scale with Symfony2 @odolbeau & @genes0r It’s not a troll.

countries; How to scale with Symfony2 @odolbeau & @genes0r Code fast!

countries; How to scale with Symfony2 @odolbeau & @genes0r Code well?

countries; How to scale with Symfony2 @odolbeau & @genes0r Queries from the view! Hard to control cache strategy No integrity check.

countries; How to scale with Symfony2 @odolbeau & @genes0r Doctrine Pros/Cons + Quick to bootstrap a new project + Quick to code a small project - Hard to respect MVC - Hard to deal with cache - Hard to deal with integrity problems in DB - Not made to decouple code (relations). - Not made to scale.

countries; How to scale with Symfony2 @odolbeau & @genes0r We fell in traps... 4

countries; How to scale with Symfony2 @odolbeau & @genes0r Never pimp your kernel!

countries; How to scale with Symfony2 @odolbeau & @genes0r Before Environment Debug?

countries; How to scale with Symfony2 @odolbeau & @genes0r Before After Branding Media Environment Debug?

countries; How to scale with Symfony2 @odolbeau & @genes0r Put everything in the same place can be dangerous!

countries; How to scale with Symfony2 @odolbeau & @genes0r This is our application. This is our business code.

countries; How to scale with Symfony2 @odolbeau & @genes0r Don’t mess with your tools!

countries; How to scale with Symfony2 @odolbeau & @genes0r We fell in traps...

countries; How to scale with Symfony2 @odolbeau & @genes0r We had parachutes!

countries; How to scale with Symfony2 @odolbeau & @genes0r Problems solved? + Use symfony only as a tool + Work a lot on onboarding + Try to decouple everything with components - We still have one big application - As everyone, we lack of documentation

countries; How to scale with Symfony2 @odolbeau & @genes0r 5

countries; How to scale with Symfony2 @odolbeau & @genes0r 40 269 Commits There are “only” 18 952 commits for symfony/symfony

countries; How to scale with Symfony2 @odolbeau & @genes0r 561 851 NCLOC There are “only” 278 909 NCLOC for symfony/symfony

countries; How to scale with Symfony2 @odolbeau & @genes0r ~ 20 minutes to run all the unit/functional tests

countries; How to scale with Symfony2 @odolbeau & @genes0r ~ 30 minutes to deploy the application

countries; How to scale with Symfony2 @odolbeau & @genes0r Too big!

countries; How to scale with Symfony2 @odolbeau & @genes0r We’re working on it! ;)

countries; How to scale with Symfony2 @odolbeau & @genes0r We’re hiring! In Paris & Warsaw! http://blbl.cr/dreamjobs

25+ million members in 22 countries, how to sca...

25+ million members in 22 countries, how to scale with Symfony2

Transcript

More Decks by Benjamin De Bernardi

Other Decks in Technology

Featured