wetter.com case study on Symfony_live 2012 Paris

After a lot of hard work, in march 2012 a complete rebuild of wetter.com went online. The site is #1 in germany for weather forecasts since years with millions of visits/day. It was built completely new on PHP 5.4, Symfony 2, MySQL, SolR, Varnish and NginX. 100 DAYS implemented the symfony2 parts of this project. This session describes the experiences we made when developing and launching this large scale application.

Transcript

3. wetter.com • 656 million visits in 2011 • 13,22 million unique users per month (AGOF 08/11) • Largest German weather portal
4. The Challenge…
5. The Challenge • Complete rewrite • New CMS, New Search, SF2-Frontend • Prime Contract: TFT (Tomorrow Focus Technologies) – CMS (Chameleon) – Search (SolR) • Integration of CMS, Search and Weather Data (Web30 services) with SF2 based system
6. Non-Functionals • up to ~ 5 Mio Visits per hour peak load • CMS loosely coupled • Maximum possible Cache Hit Rate • No PHP Sessions (if possible) • Seamless Switch-Over from old system
7. Functional Req‘s • Search for location • Display Maps – With overlays – With animation player • Display weather data • Inject Advertising ( -> Business model) • Page Structure editable without coding • Extensive own content (CMS) • Self Registration / Different User Levels
9. How it was done…
11. Architecture SF2 Frontend CMS CMS-DB Map Server Map DB Location-DB TWIG SolR Search Maps Data Varnish Location
12. Building a page
13. Caching • Browser Cache • Varnish Page Cache • Varnish ESI Cache • TWIG Template Cache • CMS Cache • MySQL Query Cache
14. And now: how it was REALLY done…
15. Varnish Config 670 Lines of VCL Backend Configuration, Probing, Balancing (40%) Caching Logic (50%) Dirty Hacking (Redirects, URL-Filtering etc.) (10%) 270 Lines of C code (DL, that was the fun part) Signed Cookie validation Lots of fun with saint mode (backend blacklisting)
16. Components • Custom Router URLs -> CMS URLs • Assetic • Asset handling via static servers • Twig Template Engine • User profiles • Symfony caching components • Symfony event handling
17. Symfony core • HttpKernelInterface standardizes request handling, subrequests ({% render %}) work the same way as main actions • ESI support out of the box • HttpCache if ESI is not available
18. Twig {% extends "cms://Home/weather/" ~ dayCount ~ "dayforecast/" %} {% block content %} <img src="{{ asset('img/shiny_weather.jpg') }}"/> {% endblock %}
19. Assetic • Awesome idea • Used heavily including a new smartsprites filter (Thanks, Alexandre) • However Symfony integration (AsseticBundle) killed performance when many assets are in use (design bug) • Hot-Fix: combine=true
23. SecurityBundle • Inner structure hard to understand • Lots of event listeners • Will only work smoothly if session is enabled • Had to do a few hacks to make it work without session
24. SensioFrameworkExtraBundle • Will set standard HTTP Cache headers understood and handled by varnish • Makes it really easy to configure caching /** * testAction * * @Cache(expires="+5 days") */
25. Symfony DI • Every SymfonyBundle can extend Symfonys DI by default • Provides everything you need for DI • Container gets compiled to raw PHP – Performance has never been a problem – If something goes really wrong you can debug in the compiled class
26. Debug Toolbar
27. Results and Learnings
28. Additional learnings • PHP 5.4 increases performance by 10%-30% • PHP-FPM allows graceful restarts • Ad-Server integration with own Javascript can be pain
29. Additional learnings • Make sure to cache 404 responses • Check Varnish with Backend Errors (500) • Test your caching (reserve some time for this) • Cache as close to the browser as possible • Avoid own caching logic, cache only if needed • Combination of varnish_log and varnish_replay is great for preparing a relaunch
30. Conclusion • Projects with Symfony2 are fun • Symfony2 contains great components and concepts to realize high traffic projects • Twig is an awesome template language • Varnish is THE way to cache • Debugging Varnish Config can be pain
31. Questions? software projects