Using Varnish Rules to protect your website from falling over because of too much traffic

Varnish Cache is an effective tool for protecting websites from traffic load. When developing websites with Caching in mind, developers can increase the headroom on the site. However there can be instances when you may want to hold traffic at the door and stop too much traffic from being on your website.
This presentation covers an effective way of doing that with www.section.io

Transcript

1. At last, a flexible Content Delivery Solution which software engineers can control Code Local > Deliver Global Traffic Throttling Mechanisms and Keeping Customers Happy @jstangroome
2. Performance Vs Availability Website Performance Optimisation: focus on improving the speed Speed and Availability: dealing with occasional spikes of very high traffic Throttling: limiting traffic coming from legitimate visitors
3. Why Throttle? • Provide the best experience to a portion of the visitors • Don’t degrade the experience of all visitors as the number of concurrent visitors crashes your site. • Provide a quality service to the maximum number of visitors that your current infrastructure can handle
4. What Happens to the Rest of the Customers? Visitors that can’t enter the site are placed in holding pattern and are allowed to enter when other customers leave. Much like a physical store at sale time!
5. Choosing What to Throttle The threshold for throttling is directly connected to the number of concurrent HTTP requests that your website infrastructure can handle at a good level of performance… …therefore…. ....limit the number of incoming HTTP requests
6. That Sounds Hard • Don’t interrupt the users session • I would need to change config to load balancer • Do I require a reverse-proxy positioned between the user-agents and the origin web servers • How do I throttle by connection? • What happens if browser connection times-out • I can’t use throttle by IP address as this may throttle a large portion of visitors connecting through an enterprise environment or using a cellular network
7. Throttling by Cookie • Give your visitors the most consistent experience and throttle by cookie. • Using this method you can: • Choose which cookies, hence users, will be throttled • Ensure that visitors are not be subjected to broken resources on a page load, • Ensure customers aren’t blocked midway through a transaction.
8. How to Throttle by Cookie • You require more capable infrastructure that is able to parse and manipulate the HTTP data - something that many simple load balancers cannot do. • A good reverse-proxy, like Varnish Cache is a good tool for this job.
9. Server State Versus Client State • When using cookies for throttling you need to decide: • where to keep track of which cookies are throttled and which are allowed. A simple mechanism is to specify in the cookie value whether to throttle or allow Positives: 1. Very low resource requirements on your infrastructure Negatives: 1. An intelligent visitor may manipulate the cookie value 2. How do you know the right number to allow 3. How can you tell accurately how many are on site already Note: you will probably end up using a percentage threshold to throttle visitors instead of an absolute number
10. Alternatives Track the state of a given visitor cookie on your infrastructure. To do this well will typically require a shared, fast, and highly-available data store accessible by all your servers responsible for implementing the throttling…..you may as well spend the money on the primary site!!! If you can do server-side state tracking for throttling this can enable features like: Throttling to an absolute number of visitors instead of a ratio. Providing a guide to throttled visitors about how long the wait may be. Allowing the throttled visitors waiting longest to access the site first when traffic subsides.