Play by Play - Planning and dealing with load

Based on the experiences from two of the biggest Magento projects we have delivered to date we will explore how to plan and deliver a Magento site that can support even the biggest traffic requirements.

What do you need to know about your users before you begin planning. How can you make accurate predictions about how your site will perform. Where are your efforts best invested to ensure that your site stays up and that your users spend money. What technology can we use to increase site performance and capacity. What services can help us support peaks in traffic and how do we know it is working.

What lessons have we learned and how did our plans perform over the duration of a business critical promotion period.

Transcript

1. PLAY BY
   PLAY
   Planning for and dealing with high traffic volumes
   on your Magento store
   

   View Slide

2. WHO AM I
   • Alistair Stead ~ @alistairstead
   • Technical Assurance Manager @ Ibuildings UK / Session Digital
   • Lead Magento projects for:
   • Dreams
   • Warner Music International
   • Kookai
   

   View Slide

3. REAL SCENARIO
   • Your client has a sale every year on the same day.
   • They have a loyal customer base who know about the sale.
   • They have sent over 5 million email advertising large savings.
   • The sale prices and products must all go live at 08:00 on sale
   day
   

   View Slide

4. Thats a lot of people waiting at the door!
   

   View Slide

5. WHAT ARE THE EFFECTS OF
   HIGH TRAFFIC VOLUMES
   • Initial site slowdown as requests are queued.
   • Eventual failure to respond to requests.
   • Inaccessible servers for any service
   • No effect just business as usual.
   

   View Slide

6. WHERE SHOULD WE FOCUS
   OUR ATTENTIONS
   • Optimising code factors such as DB queries or poorly thought
   out logic.
   • Reduce the page weight by optimising or removing elements.
   • Reduce the number of HTTP requests.
   

   View Slide

7. BOTTLENECK
   

   View Slide

8. HOW DO WE FIND
   BOTTLENECKS IN THE CODE
   • Cache Grind (WebGrind, CallGrind) http://bit.ly/Hwfl42
   • XHPROF http://bit.ly/HweQXM
   

   View Slide

9. THEN WE NEED TO START TO
   EXPAND THE TECHNOLOGY
   • Lightweight faster servers (Apache, Nginx, Lighttpd)
   • Gateway caches (Varnish, Squid, Hardware cahing)
   •
   

   View Slide

10. OFF LOAD REQUEST TO
    OTHER SERVICES
    • CDN (Content Delivery Network)
    • Split the load using sub-domains to other servers
    

    View Slide

11. MONITORING UNDER HIGH
    LOAD
    • Munin
    • Cacti
    • Nagios
    • Many monitors and top
    

    View Slide

12. Simple solution
    

    View Slide

13. HOW TO ESTIMATE TRAFFIC
    AND CAPACITY
    • Acceptable response time.
    • Number of requests.
    • Number of concurrent requests.
    

    View Slide

14. C = (N*L) * (R / T) * D
    • C = Number of concurrent connections to handle based on the estimated traffic volume.
    • N = Total number of unique users.
    • L = Average user journey length in requests.
    • T = Total time in which the traffic will visit the site.
    • R = Average time of one request to be served by the system.
    • D = Deviation in traffic pattern.
    

    View Slide

15. EXAMPLE CONCURRENCY
    266 = (40,000 * 10) * (2 / 3600) * 1.2
    

    View Slide

16. PROFILING UNDER LOAD
    • Jmeter
    • Siege
    • Apache Bench
    

    View Slide

17. PROFILING OUTPUT
    

    View Slide

18. EXAMPLE TOPOLOGY
    Web
    DB
    Master
    DB
    Slave
    DB
    Slave
    Web Web Web Web Web Web Web
    

    View Slide

19. LESSONS LEARNER
    Planning for your next big promotion
    

    View Slide

20. HOW TO DEAL WITH THE
    UNEXPECTED
    • Have a plan for even the most unexpected events.
    • Even if you have agreed not to change content plan a solution.
    • Consider temporarily revoking the clients access to the admin.
    • Delay import, export and indexing until later.
    

    View Slide

21. PLAN FOR CACHE
    EXPIRATION
    • The default magento cache will expire approx. every 6 hours
    • If this happens during a traffic peak your CPU will melt.
    • Temporarily set a far future cache expiration date.
    

    View Slide

22. Trouble is on the way!
    

    View Slide

23. CACHE STORM
    • Don’t assume this will not happen, it will!
    • Have a plan of how to deal with it.
    • Better yet create an architecture that can support the traffic
    with a completely cold cache.
    

    View Slide

24. NETWORK SATURATION
    • Under normal events your very unlikely to consider this.
    • Adding many more servers can start to expose network
    limitations.
    • Read the fine print from your hosting company.
    • Do not assume the capacity of switches confirm it!
    

    View Slide

25. ALTERNATE CACHE BACKEND
    • Zend_Cache_Backend_BlackHole
    • Zend_Cache_Backend_Redis
    

    View Slide

26. NO CACHE BACKEND
    
    
    
    
    
    /...
    
    

    View Slide

27. NO CACHE BACKEND
    
    
    
    
    
    /...
    
    

    View Slide

28. REDIS CACHE
    
    
    
    /...
    
    

    View Slide

29. REDIS CACHE
    
    
    
    /...
    
    

    View Slide

30. LOG TO AN ALTERNATE
    INTERFACE
    • Logging in Magento is key to some admin functions.
    • But this results in many DB queries per page request.
    • Exception logging is file based and not suitable to be shared
    over NFS
    • global/log/core/writer_model Specify a more appropriate log
    writter
    

    View Slide

31. SCALING PROBLEMS
    • Merged JavaScript and CSS
    • Image cache and generation
    

    View Slide

32. ARCHITECTURAL CHANGES
    • SOA (Service Orientated Architecture)
    • HMVC (Hierarchical MVC) http://bit.ly/HwpqOj
    

    View Slide

33. Don’t wait for any of these issues on promotion day!
    

    View Slide

34. THANK YOU!
    • Email: astead@ibuildings.com
    • Skype: astead-ibuildings
    • Twitter: @alistairstead
    

    View Slide

35. QUESTIONS?
    

    View Slide

36. WE ARE HIRING!
    http://www.ibuildings.co.uk/about/careers/
    

    View Slide