Varnish for PHP developers (LaraconEU 2016)

Varnish has the ability to reduce server load, improve page speeds and help scale an application from a couple of hundred users to tens of thousands. But Varnish isn't easy.

This talk guides you through what Varnish is, how it works and leaves you with practical pointers to implement it on your own infrastructure.

Transcript

1. VARNISH
   FOR PHP DEVELOPERS
   

   View Slide

2. WHAT'S THIS TALK ABOUT?
   

   View Slide

3. HI, I'M MATTIAS GENIAR!
   

   View Slide

4. View Slide

5. WHAT IS VARNISH?
   

   View Slide

6. IMPORTANT VARNISH VERSIONS
   

   View Slide

7. WHY USE VARNISH?
   

   View Slide

8. View Slide

9. REMINDER: HTTP HEADERS
   

   View Slide

10. REQUEST HEADERS
    

    View Slide

11. GET / HTTP/1.1
    Accept: */*
    Accept-Encoding: gzip, deflate
    Host: ma.ttias.be
    User-Agent: Chrome/52
    Cookie: laravel_session=eyJpdiI6...
    

    View Slide

12. RESPONSE HEADERS
    HTTP/1.1 200 OK
    Cache-Control: private, max-age=0
    Content-Encoding: gzip
    Content-Length: 9944
    Content-Type: text/html;
    charset=UTF-8
    Server: Apache
    Date: Tue, 23 Aug 2016 14:15:50 GMT
    

    View Slide

13. View Slide

14. HOW VARNISH WORKS
    

    View Slide

15. View Slide

16. View Slide

17. View Slide

18. View Slide

19. View Slide

20. View Slide

21. View Slide

22. View Slide

23. View Slide

24. View Slide

25. View Slide

26. View Slide

27. View Slide

28. VARNISH CONFIGURATION
    LANGUAGE
    $ cat /etc/varnish/default.vcl
    vcl 4.0;
    sub vcl_recv { }
    sub vcl_backend_response { }
    sub vcl_backend_fetch { }
    sub vcl_deliver { }
    

    View Slide

29. MANIPULATING THE CLIENT
    REQUEST
    sub vcl_recv {
    # Only cache GET or HEAD requests.
    if (req.method != "GET" && req.method != "HEAD") {
    return (pass);
    }
    # Remove any Google Analytics based cookies
    set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
    ...
    }
    

    View Slide

30. MANIPULATING THE CLIENT
    REQUEST
    sub vcl_recv {
    if (req.http.host ~ "^(www\.)?domain\.tld$" || req.http.host ~ "test"){
    set req.backend = specific_backend;
    return (pass);
    }
    }
    

    View Slide

31. MANIPULATING THE CLIENT
    REQUEST
    sub vcl_recv {
    if (req.url ~ "^[^?]*\.(css|gif|gz|ico|jpeg|jpg|js|png|xml)(\?.*)?$") {
    unset req.http.Cookie;
    return (lookup);
    }
    }
    

    View Slide

32. MANIPULATING THE RESPONSE
    sub vcl_backend_response {
    if (bereq.url ~ "^[^?]*\.(css|gif|gz|ico|jpeg|jpg|js|png|xml)(\?.*)?$") {
    unset bereq.http.Set-Cookie; # Case insensitive headers
    return (deliver);
    }
    }
    

    View Slide

33. MANIPULATING THE RESPONSE
    sub vcl_backend_response {
    # Don't cache 50x responses
    if (beresp.status == 500 || beresp.status == 502 || beresp.status == 503) {
    return (abandon);
    }
    # Allow stale content, in case the backend goes down.
    # make Varnish keep all objects for 2 hours beyond their TTL
    set beresp.grace = 2h;
    }
    

    View Slide

34. MANIPULATING THE DELIVERY
    sub vcl_deliver {
    if (obj.hits > 0) {
    set resp.http.X-Cache = "HIT";
    } else {
    set resp.http.X-Cache = "MISS";
    }
    set resp.http.X-Cache-Hits = obj.hits;
    unset resp.http.X-Powered-By;
    unset resp.http.Server;
    }
    

    View Slide

35. CHALLENGE: VISUALISING CHANGES
    

    View Slide

36. EXAMPLE
    +
    

    View Slide

37. View Slide

38. View Slide

39. View Slide

40. View Slide

41. View Slide

42. View Slide

43. DETERMINING HASH KEYS
    sub vcl_hash {
    hash_data(req.url); // ie: /blog/your-clickbait-title
    hash_data(req.http.host); // ie: ma.ttias.be
    hash_data(req.http.Cookie); // ie: language=nl
    }
    

    View Slide

44. CHALLENGE: URLS
    sub vcl_recv {
    # Sort all query arguments
    set req.url = std.querysort(req.url); // ie: "?lang=nl&p=5" == "?p=5&lang=nl"
    # Remove GA tags: domain.tld?utm_source=email&utm_medium=buffer&...
    set req.url = regsuball(req.url, "(utm_source|utm_medium|...)=(...)", "");
    }
    

    View Slide

45. CHALLENGE: COOKIES
    sub vcl_hash {
    // ...
    hash_data(req.http.Cookie); // ie: language=nl
    }
    

    View Slide

46. CHALLENGE: COOKIES
    import cookie;
    sub vcl_recv {
    # Use libvmod-cookie, parse the "Cookie:" header from the client
    cookie.parse(req.http.cookie);
    # Filter all except these cookies from it
    cookie.filter_except("laravel_session,custom_cookie");
    # Set the "Cookie:" header back to the parsed/filtered value
    set req.http.cookie = cookie.get_string();
    }
    

    View Slide

47. CHALLENGE: HOST
    sub vcl_recv {
    # domain.tld vs www.domain.tld, force www subdomain
    if (req.http.host == "domain.tld") {
    return (synth(720, "http://www.domain.tld"));
    }
    }
    sub vcl_synth {
    if (resp.status == 720) {
    set resp.http.Location = resp.reason;
    set resp.status = 301;
    return (deliver);
    }
    }
    

    View Slide

48. CACHING API REQUESTS
    sub vcl_hash {
    hash_data(req.url); // ie: /api/companies/5/products
    hash_data(req.http.host); // ie: ma.ttias.be
    if (req.http.Authorization) {
    hash_data(req.http.Authorization); // ie: Bearer Access-Token
    }
    if (req.http.X-Custom-Header) {
    hash_data(req.http.X-Custom-Header); // ie: app specific
    }
    }
    

    View Slide

49. View Slide

50. FLUSHING THE CACHE
    

    View Slide

51. PURGE REQUEST
    acl purge {
    "localhost";
    "192.168.55.0"/24;
    }
    sub vcl_recv {
    # allow PURGE from localhost and 192.168.55...
    if (req.method == "PURGE") {
    if (!client.ip ~ purge) {
    return(synth(405,"Not allowed."));
    }
    return (purge);
    }
    }
    $ curl -X PURGE -H "Host: domain.tld" http://31.193.180.217/blog/page1
    

    View Slide

52. BAN REQUEST
    $ varnishadm \
    -S /etc/varnish/secret \
    "ban req.http.host ~ (www.)?domain.tld"
    $ varnishadm \
    -S /etc/varnish/secret \
    -T 31.193.180.217:6082 \
    "ban req.http.host ~ www.domain.tld && req.url ~ .css"
    $ cat /etc/varnish/secret
    yb9exW6O57c77b9T9lHxkLAShsJ-QbXAdB7CMlGegGK1Q5ViJ5NTwRW
    

    View Slide

53. View Slide

54. EDGE SIDE INCLUDES
    

    View Slide

55. View Slide

56. View Slide

57. 
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    

    View Slide

58. View Slide

59. TTL: HOW LONG TO CACHE?
    header("Cache-Control: public, max-age=900");
    sub vcl_backend_response {
    if (bereq.url ~ "/blog/post-on-hackernews") {
    set beresp.ttl = 600s;
    return (deliver);
    }
    }
    

    View Slide

60. GRACE MODE
    sub vcl_backend_response {
    # All objects get a 10 minute cache lifetime
    set beresp.ttl = 10m;
    # But just in case, keep objects in cache for an extra 20 min
    # Can be served to clients while fetching new content
    # Total time object can be in the cache: 10min + 20min = 30min
    set beresp.grace = 20m;
    }
    

    View Slide

61. View Slide

62. varnishhist
    

    View Slide

63. varnishlog
    

    View Slide

64. varnishncsa
    

    View Slide

65. View Slide

66. GETTING STARTED
    

    View Slide

67. THANK YOU!
    

    View Slide