Large-scale websites performance optimisation tricks. Airline case study.

Practical lessons learned while revamping a US airline website to resist huge Black Friday and Cyber Monday traffic values. Using HTTP status codes and PHP cleverly, we have made parallel requests possible, so that the user experience is greatly enhanced, and we pre-cache resource-consuming user searches. All on a solid Symfony2 foundation.

Transcript

1. November 2015 LARGE SCALE WEBSITES PERFORMANCE OPTIMISATION TRICKS AIRLINE CASE STUDY
2. Large scale websites performance optimisation tricks GEORGIANA GLIGOR ▸ Geek. Mother. Do-er. ▸ Crafting enterprise-level applications with PHP since 2003 ▸ architecture and devops consultant ▸ Symfony & Reactjs consultant ▸ PHP Cluj Meetup Organizer
3. Large scale websites performance optimisation tricks CUSTOMER ▸ low cost airline, similar to Ryan Air / Wizzair / EasyJet
4. Large scale websites performance optimisation tricks CUSTOMER ▸ low cost airline, similar to Ryan Air / Wizzair / EasyJet ▸ clever strategy of using airports, so that they don’t have a direct competition on 90% of their routes
5. Large scale websites performance optimisation tricks CUSTOMER ▸ low cost airline, similar to Ryan Air / Wizzair / EasyJet ▸ clever strategy of using airports, so that they don’t have a direct competition on 90% of their routes ▸ selling models: website, call-centre, travel agents
6. Large scale websites performance optimisation tricks CUSTOMER ▸ low cost airline, similar to Ryan Air / Wizzair / EasyJet ▸ clever strategy of using airports, so that they don’t have a direct competition on 90% of their routes ▸ selling models: website, call-centre, travel agents ▸ use their own reservation system
7. Large scale websites performance optimisation tricks LEGACY
8. Large scale websites performance optimisation tricks THE WANTS ‣ enables customer to purchase a full experience based on the destination, not just cheap plane tickets ‣ flexibility ‣ scalability ‣ be in front of the competition by withstanding the Black Friday + Cyber Monday test ‣ move at the speed of a greenfield project even if the business is up and running
9. Large scale websites performance optimisation tricks THE NEEDS ‣ introduce polyglot technology stack ‣ handle geographically distributed teams ‣ replace legacy monolithic app
10. Large scale websites performance optimisation tricks TECHNOLOGY STACK
11. Large scale websites performance optimisation tricks LEGACY AS DATA
12. Large scale websites performance optimisation tricks RESTFUL SERVICES ACCESS DATA
13. Large scale websites performance optimisation tricks ISOLATE HIGH-TRAFFIC
14. Large scale websites performance optimisation tricks OVERVIEW OF WEB-FACING COMPONENTS
15. Large scale websites performance optimisation tricks WELCOME SILOS
16. Large scale websites performance optimisation tricks WELCOME SILOS 1 silo = full setup of servers that run each application component ‣ frontend (Drupal, Javascript SPAs) ‣ backend (Symfony services) ‣ cache
17. Large scale websites performance optimisation tricks
18. Large scale websites performance optimisation tricks STATIC DATA
19. Large scale websites performance optimisation tricks STATIC DATA
20. Large scale websites performance optimisation tricks OPTIMISE ACCESS
21. Large scale websites performance optimisation tricks ANATOMY OF A SILO
22. Large scale websites performance optimisation tricks THE PIECES
23. Large scale websites performance optimisation tricks DATA ACCESS
24. Large scale websites performance optimisation tricks DATA CACHING
25. Large scale websites performance optimisation tricks FIXTURES
26. Large scale websites performance optimisation tricks
27. Large scale websites performance optimisation tricks FIXTURE-BASED DEVELOPMENT ‣ enforces discussing contracts before doing actual coding ‣ helps eliminate certain problems by having them discussed in advance ‣ allows developers to work independently and run all the stack on their machine ‣ each component is independently tested
28. Large scale websites performance optimisation tricks THE 204 TRICK
29. Large scale websites performance optimisation tricks ‣ HTTP status code 204 ‣ success code ‣ Content-Length: 0 ‣ the backend acknowledges the request and will carry on ‣ the client can continue processing
30. Large scale websites performance optimisation tricks
31. Large scale websites performance optimisation tricks BATCH PROCESSING EXAMPLE POST /batch-process public function batchProcessAction() { }
32. Large scale websites performance optimisation tricks BATCH PROCESSING EXAMPLE POST /batch-process public function batchProcessAction() { // 1 - check permissions }
33. Large scale websites performance optimisation tricks BATCH PROCESSING EXAMPLE POST /batch-process public function batchProcessAction() { // 1 - check permissions // 2 - validate user data }
34. Large scale websites performance optimisation tricks BATCH PROCESSING EXAMPLE POST /batch-process public function batchProcessAction() { // 1 - check permissions // 2 - validate user data // 3 - return 204 No Content }
35. Large scale websites performance optimisation tricks BATCH PROCESSING EXAMPLE POST /batch-process public function batchProcessAction() { // 1 - check permissions // 2 - validate user data // 3 - return 204 No Content // 4 - process batch }
36. Large scale websites performance optimisation tricks CLOSE CONNECTION: RAW PHP public function closeConnection() { header('HTTP/1.0 204 No Content'); header('Content-Length: 0', true); header('Content-Type: application/json', true); flush(); // or ob_flush() }
37. Large scale websites performance optimisation tricks CLOSE CONNECTION: SYMFONY-SPECIFIC public function closeConnection(Request $request) { $response = new StreamedResponse(); $response->setStatusCode(204); $response->headers->add(array('Connection' => 'close')); $c = function () { /* do nothing */ }; $response->setCallback($c); // needs to be a valid PHP callable $response->prepare($request); // ensure compliancy with HTTP specification $response->send(); return $response; }
38. Large scale websites performance optimisation tricks BLACKBOX: EVENT LOGGING SYSTEM ‣ cheap ‣ does not impact application performance ‣ logs are searchable ‣ logs tell the story of a user’s journey ‣ greatly help debugging
39. Large scale websites performance optimisation tricks BLACKBOX: EVENT LOGGING SYSTEM
40. Large scale websites performance optimisation tricks BLACKBOX: LOGGING SYSTEM
41. Large scale websites performance optimisation tricks BLACKBOX: LOGGING SYSTEM
42. Large scale websites performance optimisation tricks ERROR HANDLING
43. Large scale websites performance optimisation tricks ERROR TYPES ‣ recoverable ‣ user can continue inside the booking path ‣ non-recoverable ‣ user’s checkin process can’t proceed
44. Large scale websites performance optimisation tricks RECOVERABLE ERRORS ‣ 4xx status codes ‣ 401 unauthorised ‣ 403 forbidden ‣ 409 conflict ‣ 413 payload too large ‣ instruct the UI where to go back
45. Large scale websites performance optimisation tricks NON-RECOVERABLE ERRORS ‣ 5xx status codes ‣ 500 internal server error ‣ 503 service unavailable
46. Large scale websites performance optimisation tricks PARALLELISM
47. Large scale websites performance optimisation tricks ‣ asynchronously process multiple cURL handles ‣ same good old cURL, but on steroids ‣ http://php.net/manual/en/function.curl-multi- init.php#refsect1-function.curl-multi-init-examples
48. Large scale websites performance optimisation tricks PARALLELISM IN PRACTICE
49. Large scale websites performance optimisation tricks ANATOMY OF A SYMFONY SERVICE
50. Large scale websites performance optimisation tricks LESSONS LEARNED
51. Large scale websites performance optimisation tricks ‣ silos
52. Large scale websites performance optimisation tricks ‣ silos ‣ static data
53. Large scale websites performance optimisation tricks ‣ silos ‣ static data ‣ fixtures
54. Large scale websites performance optimisation tricks ‣ silos ‣ static data ‣ fixtures ‣ delegate responsibility
55. Large scale websites performance optimisation tricks ‣ silos ‣ static data ‣ fixtures ‣ delegate responsibility ‣ parallelism
56. Large scale websites performance optimisation tricks Q & A
57. THANK YOU! Georgiana Gligor @gbtekkie http://tekkie.ro/