Scaling Craft CMS Sites for Large Launches

My presentation at the 2018 Craft CMS conference, Dot All.

This talk provides a clear set of steps, guides, and testing/analysis methods for launching Craft sites that will be highly visible, need to have close to perfect uptime, have legal requirements about being available in multiple geographic regions, and/or just get a whole lot of traffic. It will address Craft setup and caching choices, infrastructure choices, performance optimization, load testing, and more. It will also focus on some of the great performance changes in Craft 3.

Transcript

1. Scaling Craft Sites for Large Launches
   Matt Weinberg
   2018/09/28
   Vector Media Group
   

   View Slide

2. Intro
   

   View Slide

3. Goals for the talk
   ● Address “things to think about” when launching
   mission-critical Craft sites
   ● Explain various approaches to scaling, high uptime,
   and redundancy
   ● Provide a pre-launch checklist
   ● Help everyone launch faster Craft sites with more
   uptime
   

   View Slide

4. Who am I?
   ● Co-founder and President of
   Development at Vector Media
   Group
   ● 50 people, based in NYC
   ● Craft service partners
   ● Significant amount of Craft
   work, including very large sites
   

   View Slide

5. Who am I?
   VectorMediaGroup.com
   matt@vectormediagroup.com
   @mrw on twitter
   

   View Slide

6. 1. Performant: quickly answer real-world traffic
   2. Redundant + Horizontally scalable: handle traffic
   surges and survive partial service degradations
   3. Visible: easily predict and monitor performance
   under various scenarios
   4. Secure: follows best practices
   What should “scaling” mean to you?
   

   View Slide

7. Performance
   Quickly answer real-world traffic
   

   View Slide

8. Performance
   ● Many factors involved in a user’s total load speed.
   Asset loading, CSS structure, etc.
   ○ Andrew Welch’s “Making a Craft CMS Website That FLIES”
   presentation yesterday
   ● Next few slides concentrate only on TTFB - time to
   first byte.
   ○ Other factors are very important for users but a little
   different from the scaling questions here
   ● Reducing web server load makes everything else
   easier
   

   View Slide

9. TTFB (Time To First Byte)
   ● Time between initial request
   and getting any response back
   ● Craft/PHP compiles, parses,
   and renders the templates
   ● Lower TTFB means:
   ○ Faster site for users
   ○ Server freeing up resources
   more quickly
   ● Measure in Chrome or
   webpagetest.org
   

   View Slide

10. ● Static assets (CSS, JS, images,
    etc…) will have much faster
    load than Craft templates
    ● Template Target:
    ○ Good: <200ms
    ○ Better: <100ms
    ○ Best: <75ms
    ● Fixing low hanging fruit is
    good but don’t obsess about a
    10ms difference
    TTFB (Time To First Byte)
    

    View Slide

11. ● Cache aggressively. Anything
    that can be cached, should be.
    ● Cache globally where possible.
    ● Use custom cache keys for geo
    and other differences
    ● Add config constants if needed
    Lowering TTFB in Craft
    

    View Slide

12. Lowering TTFB in Craft
    

    View Slide

13. Lowering TTFB in Craft
    ● Make sure OPCache (PHP’s built-in bytecode cache)
    is turned on in php.ini.
    ● Offload as much as you can from the frontend servers
    ○ Use CDNs such as Cloudflare, CloudFront, Google Cloud
    CDN, Akamai, etc to serve static content (JS/CSS/images)
    ○ Use hosted cloud services for DB/cache/queue. Trust the
    experts.
    ■ A managed host like Arcustech or a cloud DB service
    from Amazon (RDS/Aurora) or Google (Cloud SQL) is
    best for most sites without ops engineers.
    

    View Slide

14. Redundancy + Horizontal scalability
    Handle traffic surges and survive partial service
    degradations
    

    View Slide

15. Assume multiple tiers
    Load
    Balancer
    Craft
    Craft
    Craft
    DB
    Memory
    Cache
    DB
    Memory
    Cache
    

    View Slide

16. Assume multiple tiers
    ● From day 1: assume different server tiers, and Craft
    living on multiple front-end servers
    ● Load balancer means the failure of one frontend
    server doesn’t bring the site down
    ● Staging site (and local!) should implement this
    ● Multiple FE servers has a few implications for build
    process
    

    View Slide

17. ● Earlier: “Offload as much as you can from the
    frontend servers”
    ● Architecting this from the beginning will make it a
    lot easier later
    ● Everyone: use cloud or NAS object storage for assets
    ○ Recommend: Amazon S3, Google Cloud Storage
    ■ Name your bucket as a domain (e.g. assets.domain.com)
    ○ These are slow; front them with a CDN
    ● Using good managed hosting? Much of the rest will
    be handled
    Assume multiple tiers
    

    View Slide

18. Assume multiple tiers
    ● Using unmanaged hosting?
    ○ Load Balancer
    ■ Recommend: Amazon ALB, Google CLB, HAProxy
    ○ DB
    ■ Recommend: Amazon RDS, Google Cloud SQL
    ■ Consider PostgreSQL
    ● DataGrip by JetBrains
    ■ Consider Aurora/Aurora Serverless
    ■ Consider MariaDB or Percona Server
    ○ Memory cache
    ■ Recommend: Redis
    

    View Slide

19. Assume multiple frontends
    Shared memory cache
    

    View Slide

20. Assume multiple frontends
    Shared session storage
    

    View Slide

21. Assume multiple frontends
    Shared external queue
    

    View Slide

22. Assume multiple frontends
    If you’re using the same in-memory cache for more
    than one of caching, sessions, and queuing, you can
    make the config less repetitive using Yii components.
    

    View Slide

23. Assume multiple frontends
    ● NAS/shared file storage is another option
    ● We’ve used this successfully with Craft before
    ● Amazon EFS, Google Filestore, or any NAS/NFS
    mount
    ● Carefully check performance. Can be extremely slow
    if not configured correctly.
    ○ Consider provisioned or premium throughput options
    

    View Slide

24. Reverse proxy caching
    Load
    Balancer
    Craft
    Craft
    Craft
    DB
    Memory
    Cache
    DB
    Memory
    Cache
    Reverse
    proxy
    cache
    

    View Slide

25. Reverse proxy caching
    ● At some point you’ll want to start thinking about this
    ● Helps take as much load as possible off your
    application servers
    ● Cloudflare is very popular with Craft sites and has
    built-in caching
    ○ Using Arcustech? You’ll get a faster connection even on the
    free plan because of Railgun
    ● Varnish is very popular as well, but complex.
    ○ Fastly is a much easier to use hosted Varnish provider
    

    View Slide

26. Down the road...
    ● Integrate more dedicated services like Elasticsearch
    or Algolia
    ● Create an off-thread background caching and
    cache-breaking system
    ○ Including stale serving
    

    View Slide

27. Visibility
    Easily predict and monitor performance under
    various scenarios
    

    View Slide

28. Launch checklist
    ● Launch checklist with responsible people identified
    ● Each item should be assigned to 1 person, not a team
    ● Capture everything: Analytics tags, backups, SSL
    certificates, domain redirects, debugging turned off,
    load testing complete, etc...
    

    View Slide

29. Load testing
    ● Open Source: we like Siege and Bees With Machine
    Guns
    ● Commercial: we like Load Impact
    ○ Can subscribe for just a month
    ○ Chrome extension to record typical user paths and all
    requests
    

    View Slide

30. Load testing
    ● Load test before all launches
    ● More than just hitting a single URL a few times
    ● Simulate hitting all a page’s assets concurrently
    ● Test from different geo regions
    ● Increase traffic until it “breaks”
    ○ Knowing your site’s limit helps guide business decisions
    ○ First fix by optimizing low-hanging fruit (performance),
    then fix by adding hardware (horizontal scalability)
    

    View Slide

31. Load testing
    Load Impact: site limit reached
    

    View Slide

32. Identifying bottlenecks
    ● Craft debugging toolbar is a big help
    ● Slow query logging on your database is important
    ○ Tip: use debugging profiler to see template generating a
    query. Add unique limits to your entries queries
    (“.entries().limit(23)”) to help identify them in the
    debugger and DB logs
    ● New Relic is very helpful here
    ● Stackdriver/CloudWatch
    ● Create a Craft plugin as a performance dashboard
    

    View Slide

33. Load testing
    Load Impact: site limit reached
    

    View Slide

34. Security
    Follow best practices
    

    View Slide

35. Security
    ● Craft has great protections built-in
    ○ Twig escaping
    ○ One-click updates
    ○ Proper password hashing
    ○ Sane minimum server requirements
    ○ DB query parameterization
    ○ etc...
    

    View Slide

36. ● If you can, IP lock the control panel
    ○ Cloudflare calls this “Zone Lockdown”
    ○ Can also be done in .htaccess/nginx config, or a plugin
    ● If you can’t IP lock the control panel:
    ○ Consider turning ‘preventUserEnumeration’ to true
    ■ Possibly more confusing for some users but more secure
    ○ Change cpTrigger from “admin”, which is often scanned
    ● Consider tying Craft login to SSO - Google, Okta, etc…
    ○ Via plugin, Cloudflare Access, or similar
    ● Weak user passwords are your biggest threat
    Security
    

    View Slide

37. Thank You + Q&A
    ● Contact me any time with any questions
    ● matt@vectormediagroup.com
    ● @mrw
    ● https://www.VectorMediaGroup.com
    

    View Slide