Varnish in-depth training - Symfony Live Berlin 2017

More info at https://feryn.eu/speaking/varnish-in-depth-symfony-live-berlin-2017/

Transcript

1. By Thijs Feryn
   Varnish in -depth
   training
   

   View Slide

2. Slow websites suck
   

   View Slide

3. Web performance is
   an essential part of
   the user experience
   

   View Slide

4. Slow ~ Down
   

   View Slide

5. Saturated
   market
   

   View Slide

6. View Slide

7. View Slide

8. View Slide

9. View Slide

10. Heavy load
    

    View Slide

11. Mo money
    Mo servers
    

    View Slide

12. Identify slowest parts
    

    View Slide

13. View Slide

14. View Slide

15. Reduce the
    impact of the
    code on the
    server
    

    View Slide

16. Not just raw speed
    

    View Slide

17. Scale
    

    View Slide

18. Refactor slow code
    Write fast code
    

    View Slide

19. Optimize database
    

    View Slide

20. Improve the API call
    

    View Slide

21. Optimize runtime
    

    View Slide

22. After a while you
    hit the limits
    

    View Slide

23. Cache
    

    View Slide

24. Don’t
    recompute
    if the data
    hasn’t
    changed
    

    View Slide

25. 3 x 2 = ?
    

    View Slide

26. What can you cache?
    Byte code
    Database output External services
    Files from disk
    Pages
    

    View Slide

27. Caching is not a
    compensation for
    poor code
    

    View Slide

28. Caching is an
    essential
    architectural
    strategy
    

    View Slide

29. Pages
    

    View Slide

30. Normally
    User Server
    

    View Slide

31. With Varnish
    User Varnish Server
    

    View Slide

32. View Slide

33. Hi, I’m Thijs
    

    View Slide

34. I’m
    @ThijsFeryn
    on Twitter
    

    View Slide

35. I’m an
    Evangelist
    At
    

    View Slide

36. I’m an
    Evangelist
    At
    

    View Slide

37. I’m a
    at
    board member
    

    View Slide

38. View Slide

39. Let's dive right in!
    

    View Slide

40. Varnish
    

    View Slide

41. Cache?
    

    View Slide

42. Loadbalancer?
    

    View Slide

43. Proxy?
    

    View Slide

44. Web
    application
    firewall?
    

    View Slide

45. HTTP
    accelerator
    

    View Slide

46. Regular proxy
    User Proxy Server
    Office
    

    View Slide

47. With reverse
    proxy
    User Proxy Server
    Datacenter
    

    View Slide

48. Bit of history
    

    View Slide

49. View Slide

50. View Slide

51. View Slide

52. View Slide

53. View Slide

54. View Slide

55. Varnish Moral
    License
    http://phk.freebsd.dk/VML/
    

    View Slide

56. View Slide

57. View Slide

58. Main project
    “sponsors”
    

    View Slide

59. How does it work?
    

    View Slide

60. varnishd
    VCL file
    Compiled & linked as
    shared object
    Client
    HTTP
    Backend
    HTTP
    Memory
    Logs
    Writes verbose
    “transaction” logs to
    memory
    varnishlog
    varnishtop
    Read memory logs,
    displays information
    varnishncsa
    varnishadm
    Allow basic
    management tasks
    Custom
    protocol
    

    View Slide

61. Install & configure
    

    View Slide

62. https://packagecloud.io/varnishcache/varnish5
    https://packagecloud.io/varnishcache/varnish41
    

    View Slide

63. $ curl -s https://packagecloud.io/install/repositories/
    varnishcache/varnish5/script.deb.sh | sudo bash
    $ apt-get install varnish
    Debian & Ubuntu
    

    View Slide

64. $ apt-cache policy varnish
    varnish:
    Installed: 5.2.0-1~stretch
    Candidate: 5.2.0-1~stretch
    Version table:
    *** 5.2.0-1~stretch 500
    500 https://packagecloud.io/varnishcache/varnish5/debian stretch/main
    amd64 Packages
    100 /var/lib/dpkg/status
    5.1.3-1~stretch 500
    500 https://packagecloud.io/varnishcache/varnish5/debian stretch/main
    amd64 Packages
    5.1.2-1~stretch 500
    500 https://packagecloud.io/varnishcache/varnish5/debian stretch/main
    amd64 Packages
    5.0.0-7+deb9u1 500
    500 http://deb.debian.org/debian stretch/main amd64 Packages
    500 http://security.debian.org stretch/updates/main amd64 Packages
    

    View Slide

65. $ apt-get install varnish=5.2.0-1~stretch
    

    View Slide

66. $ curl -s https://packagecloud.io/install/repositories/
    varnishcache/varnish5/script.rpm.sh | sudo bash
    $ yum install varnish
    RHEL & CentOS
    

    View Slide

67. Config
    file
    

    View Slide

68. View Slide

69. SysV Systemd
    Ubuntu/Debian /etc/default/varnish /etc/systemd/system/varnish.service
    RHEL/CentOS /etc/sysconfig/varnish /etc/varnish/varnish.params
    

    View Slide

70. cp /lib/systemd/system/varnish.service \
    /etc/systemd/system/
    Copy defaults file
    Ubuntu &
    Debian with
    systemd
    

    View Slide

71. Startup options
    

    View Slide

72. [Unit]
    Description=Varnish Cache, a high-performance HTTP accelerator
    [Service]
    Type=forking
    # Maximum number of open files (for ulimit -n)
    LimitNOFILE=131072
    # Locked shared memory - should suffice to lock the shared memory log
    # (varnishd -l argument)
    # Default log size is 80MB vsl + 1M vsm + header -> 82MB
    # unit is bytes
    LimitMEMLOCK=85983232
    # On systemd >= 228 enable this to avoid "fork failed" on reload.
    #TasksMax=infinity
    # Maximum size of the corefile.
    LimitCORE=infinity
    # Set WARMUP_TIME to force a delay in reload-vcl between vcl.load and vcl.use
    # This is useful when backend probe definitions need some time before declaring
    # configured backends healthy, to avoid routing traffic to a non-healthy backend.
    #WARMUP_TIME=0
    ExecStart=/usr/sbin/varnishd -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/
    secret -s malloc,256m
    ExecReload=/usr/share/varnish/reload-vcl
    [Install]
    WantedBy=multi-user.target
    Debian
    Stretch with
    systemd
    

    View Slide

73. $ vim /etc/systemd/system/varnish.service
    $ systemctl daemon-reload
    $ service varnish reload
    Reload with systemd
    

    View Slide

74. DAEMON_OPTS="-a :80 \
    -T localhost:6082 \
    -f /etc/varnish/default.vcl \
    -S /etc/varnish/secret \
    -s malloc,256m"
    Ubuntu
    Trusty without
    systemd
    

    View Slide

75. vim /etc/default/varnish
    service varnish reload
    Reload with sysv
    

    View Slide

76. -a: binding address & port
    -T: admin binding
    -f: VCL file
    -S: secret file
    -s: storage
    -j: jailing
    -l: shared memory log size
    -t: default TTL
    -p: runtime parameters
    Options
    

    View Slide

77. DAEMON_OPTS="-j unix,user=www-data \
    -a main=0000:80 \
    -a proxy=0.0.0.0:81,PROXY \
    -T localhost:6082 \
    -f /etc/varnish/default.vcl \
    -S /etc/varnish/secret \
    -l 100m,10m \
    -t 60 \
    -p feature=+esi_disable_xml_check \
    -p connect_timeout=20s \
    -p first_byte_timeout=100s \
    -p between_bytes_timeout=5s \
    -s malloc,3g"
    

    View Slide

78. The backend
    

    View Slide

79. View Slide

80. Point of
    entry? Bind
    to 80
    Otherwise
    6081
    

    View Slide

81. 8080
    HTTP-ALT
    

    View Slide

82. Listen 8080
    Apache
    
    ports.conf
    vhost
    

    View Slide

83. listen 8080;
    Nginx
    vhost
    

    View Slide

84. ✓In VCL file
    ✓In config file using "-b"
    Link backend to Varnish
    

    View Slide

85. DAEMON_OPTS="-j unix,user=www-data \
    -a main=0.0.0.0:80 \
    -a proxy=0.0.0.0:81,PROXY \
    -T localhost:6082 \
    -b 127.0.0.1:8080 \
    -S /etc/varnish/secret \
    -s malloc,3g"
    Can't
    have -f
    In config file
    

    View Slide

86. vcl 4.0;

    

    backend default {

    .host = "127.0.0.1";

    .port = "8080";

    }
    In VCL file
    Minimal
    VCL
    

    View Slide

87. Typical setups
    

    View Slide

88. All-in-one
    User
    Varnish (80)
    Nginx (8080)
    

    View Slide

89. Dedicated Varnish
    User Varnish (80)
    Nginx (80)
    

    View Slide

90. Multiple webservers
    User
    Varnish (80)
    Nginx (8080)
    Varnish (80)
    Nginx (8080)
    KeepAliveD
    

    View Slide

91. Multiple webservers & dedicated Varnish
    User
    Varnish
    (80)
    KeepAliveD
    Varnish
    (80)
    Nginx
    (80)
    Nginx
    (80)
    

    View Slide

92. Loadbalancing
    User
    Varnish
    (80)
    KeepAliveD
    Varnish
    (80)
    Nginx
    (80)
    Nginx
    (80)
    

    View Slide

93. HAProxy loadbalancers
    User
    HAProxy (80)
    Varnish (6081)
    KeepAliveD
    HAProxy (80)
    Varnish (6081)
    Nginx (80)
    Nginx (80)
    

    View Slide

94. Share the cache
    

    View Slide

95. Share the cache
    User
    Nginx (80)
    Nginx (80)
    1
    2
    3
    4
    5
    6
    HAProxy (80)
    Varnish ( 6080 & 6081)
    HAProxy (80)
    Varnish ( 6080 & 6081)
    

    View Slide

96. Share the cache 2
    User
    HAProxy (80)
    Varnish ( 6080 & 6081)
    Nginx (80)
    Nginx (80)
    1
    5
    3
    4
    2
    6
    HAProxy (80)
    Varnish ( 6080 & 6081)
    

    View Slide

97. vcl 4.0;
    import std;
    backend nginx {
    .host = "nginx";
    .port = "80";
    }
    backend nginx2 {
    .host = "nginx2";
    .port = "80";
    }
    backend varnish {
    .host = "varnish";
    .port = "6080";
    }
    backend varnish2 {
    .host = "varnish2";
    .port = "6080";
    }
    sub vcl_recv {
    if(server.hostname == "varnish") {
    if(std.port(server.ip) == 6080) {
    set req.backend_hint = nginx;
    set req.http.x-esi = "yes";
    } else {
    set req.backend_hint = varnish2;
    set req.http.x-esi = "no";
    }
    } else {
    if(std.port(server.ip) == 6080) {
    set req.backend_hint = nginx2;
    set req.http.x-esi = "yes";
    } else {
    set req.backend_hint = varnish;
    set req.http.x-esi = "no";
    }
    }
    }
    sub vcl_backend_response {
    if (bereq.http.x-esi == "yes" && beresp.http.Surrogate-Control ~ "ESI/1.0") {
    unset beresp.http.Surrogate-Control;
    set beresp.do_esi = true;
    }
    }
    

    View Slide

98. sub vcl_recv {
    if(server.hostname == "varnish") {
    if(std.port(server.ip) == 6080) {
    set req.backend_hint = nginx;
    set req.http.x-esi = "yes";
    } else {
    set req.backend_hint = varnish2;
    set req.http.x-esi = "no";
    }
    } else {
    if(std.port(server.ip) == 6080) {
    set req.backend_hint = nginx2;
    set req.http.x-esi = "yes";
    } else {
    set req.backend_hint = varnish;
    set req.http.x-esi = "no";
    }
    }
    }
    

    View Slide

99. What about TLS/SSL?
    

    View Slide

100. ✓HAProxy
     ✓Nginx
     ✓Pound
     ✓Hitch
     Terminate TLS/SSL
     

     View Slide

101. HAProxy
     172.18.0.3
     Varnish
     172.18.0.8
     Nginx
     Public
     HTTP & HTTPS
     HTTP
     HTTP
     Certificates go here
     

     View Slide

102. How do you know if it
     was HTTP or HTTPS?
     

     View Slide

103. X-Forwarded-Proto: https
     X-Forwarded-Proto: http
     

     View Slide

104. if (strpos($_SERVER['HTTP_X_FORWARDED_PROTO'], 'https') !== false)
     $_SERVER['HTTPS']='on';
     

     View Slide

105. View Slide

106. SetEnvIf X-Forwarded-Proto "https" HTTPS=on
     Header append Vary: X-Forwarded-Proto
     
     RewriteEngine on
     RewriteCond %{HTTPS} !=on
     RewriteCond %{HTTP:X-Forwarded-Proto} !https [NC]
     RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI}
     [L,R=301]
     
     

     View Slide

107. Demo setup
     

     View Slide

108. https://github.com/ThijsFeryn/
     varnishtraining
     

     View Slide

109. $ docker-compose up
     $ docker-compose down
     $ docker-compose up -d
     $ docker-compose logs
     $ docker-compose logs -f
     $ docker-compose logs varnish
     $ docker-compose logs -f varnish
     

     View Slide

110. HAProxy (80 & 81)
     Varnish
     (6080, 6081, 6082, 6083)
     Nginx (80)
     PHP-FPM (9000)
     Varnish 2
     (6080, 6081, 6082, 6083)
     Nginx 2 (80)
     PHP-FPM 2 (9000)
     MySQL (3306)
     Redis (6379)
     Demo setup
     

     View Slide

111. ✓sflive-varnish2: 172.18.0.10
     ✓sflive-varnish: 172.18.0.8
     ✓sflive-nginx2: 172.18.0.6
     ✓sflive-nginx: 172.18.0.2
     ✓sflive-php2: 172.18.0.5
     ✓sflive-php: 172.18.0.4
     ✓sflive-haproxy: 172.18.0.3
     ✓sflive-redis: 172.18.0.7
     ✓sflive-mysql: 172.18.0.9
     Demo setup IPs
     

     View Slide

112. View Slide

113. FROM debian:stretch as builder
     WORKDIR /usr/lib/varnish/vmods/
     RUN apt-get update \
     && apt-get install -y curl \
     && curl -s https://packagecloud.io/install/repositories/varnishcache/varnish5/
     script.deb.sh | bash \
     && apt-get install -y unzip automake varnish=5.2.0-1~stretch varnish-
     dev=5.2.0-1~stretch build-essential libtool docutils-common libmhash-dev --allow-
     unauthenticated \
     && curl -O https://download.varnish-software.com/varnish-modules/varnish-
     modules-0.12.1.tar.gz \
     && tar xvzf varnish-modules-0.12.1.tar.gz \
     && (cd varnish-modules-0.12.1 && ./configure && make && make install) \
     && curl -O -L https://github.com/gquintard/libvmod-accept/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-accept-master && ./autogen.sh && ./configure && make && make
     install) \
     && curl -O -L https://github.com/varnish/libvmod-digest/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-digest-master && ./autogen.sh && ./configure && make && make
     install)
     FROM debian:stretch
     WORKDIR /etc/varnish
     EXPOSE 6080
     EXPOSE 6081
     EXPOSE 6082
     EXPOSE 6083
     ENV SIZE 100m
     ENV SECRET E29F6DE6-3803-449E-8A01-AA537A8715B1
     ARG BACKEND=nginx
     

     View Slide

114. FROM debian:stretch as builder
     WORKDIR /usr/lib/varnish/vmods/
     RUN apt-get update \
     && apt-get install -y curl \
     && curl -s https://packagecloud.io/install/repositories/varnishcache/varnish5/
     script.deb.sh | bash \
     && apt-get install -y unzip automake varnish=5.2.0-1~stretch varnish-
     dev=5.2.0-1~stretch build-essential libtool docutils-common libmhash-dev --allow-
     unauthenticated \
     && curl -O https://download.varnish-software.com/varnish-modules/varnish-
     modules-0.12.1.tar.gz \
     && tar xvzf varnish-modules-0.12.1.tar.gz \
     && (cd varnish-modules-0.12.1 && ./configure && make && make install) \
     && curl -O -L https://github.com/gquintard/libvmod-accept/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-accept-master && ./autogen.sh && ./configure && make && make
     install) \
     && curl -O -L https://github.com/varnish/libvmod-digest/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-digest-master && ./autogen.sh && ./configure && make && make
     install)
     FROM debian:stretch
     WORKDIR /etc/varnish
     EXPOSE 6080
     EXPOSE 6081
     EXPOSE 6082
     EXPOSE 6083
     ENV SIZE 100m
     ENV SECRET E29F6DE6-3803-449E-8A01-AA537A8715B1
     ARG BACKEND=nginx
     

     View Slide

115. FROM debian:stretch as builder
     WORKDIR /usr/lib/varnish/vmods/
     RUN apt-get update \
     && apt-get install -y curl \
     && curl -s https://packagecloud.io/install/repositories/varnishcache/varnish5/
     script.deb.sh | bash \
     && apt-get install -y unzip automake varnish=5.2.0-1~stretch varnish-
     dev=5.2.0-1~stretch build-essential libtool docutils-common libmhash-dev --allow-
     unauthenticated \
     && curl -O https://download.varnish-software.com/varnish-modules/varnish-
     modules-0.12.1.tar.gz \
     && tar xvzf varnish-modules-0.12.1.tar.gz \
     && (cd varnish-modules-0.12.1 && ./configure && make && make install) \
     && curl -O -L https://github.com/gquintard/libvmod-accept/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-accept-master && ./autogen.sh && ./configure && make && make
     install) \
     && curl -O -L https://github.com/varnish/libvmod-digest/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-digest-master && ./autogen.sh && ./configure && make && make
     install)
     FROM debian:stretch
     WORKDIR /etc/varnish
     EXPOSE 6080
     EXPOSE 6081
     EXPOSE 6082
     EXPOSE 6083
     ENV SIZE 100m
     ENV SECRET E29F6DE6-3803-449E-8A01-AA537A8715B1
     ARG BACKEND=nginx
     

     View Slide

116. FROM debian:stretch as builder
     WORKDIR /usr/lib/varnish/vmods/
     RUN apt-get update \
     && apt-get install -y curl \
     && curl -s https://packagecloud.io/install/repositories/varnishcache/varnish5/
     script.deb.sh | bash \
     && apt-get install -y unzip automake varnish=5.2.0-1~stretch varnish-
     dev=5.2.0-1~stretch build-essential libtool docutils-common libmhash-dev --allow-
     unauthenticated \
     && curl -O https://download.varnish-software.com/varnish-modules/varnish-
     modules-0.12.1.tar.gz \
     && tar xvzf varnish-modules-0.12.1.tar.gz \
     && (cd varnish-modules-0.12.1 && ./configure && make && make install) \
     && curl -O -L https://github.com/gquintard/libvmod-accept/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-accept-master && ./autogen.sh && ./configure && make && make
     install) \
     && curl -O -L https://github.com/varnish/libvmod-digest/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-digest-master && ./autogen.sh && ./configure && make && make
     install)
     FROM debian:stretch
     WORKDIR /etc/varnish
     EXPOSE 6080
     EXPOSE 6081
     EXPOSE 6082
     EXPOSE 6083
     ENV SIZE 100m
     ENV SECRET E29F6DE6-3803-449E-8A01-AA537A8715B1
     ARG BACKEND=nginx
     

     View Slide

117. FROM debian:stretch as builder
     WORKDIR /usr/lib/varnish/vmods/
     RUN apt-get update \
     && apt-get install -y curl \
     && curl -s https://packagecloud.io/install/repositories/varnishcache/varnish5/
     script.deb.sh | bash \
     && apt-get install -y unzip automake varnish=5.2.0-1~stretch varnish-
     dev=5.2.0-1~stretch build-essential libtool docutils-common libmhash-dev --allow-
     unauthenticated \
     && curl -O https://download.varnish-software.com/varnish-modules/varnish-
     modules-0.12.1.tar.gz \
     && tar xvzf varnish-modules-0.12.1.tar.gz \
     && (cd varnish-modules-0.12.1 && ./configure && make && make install) \
     && curl -O -L https://github.com/gquintard/libvmod-accept/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-accept-master && ./autogen.sh && ./configure && make && make
     install) \
     && curl -O -L https://github.com/varnish/libvmod-digest/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-digest-master && ./autogen.sh && ./configure && make && make
     install)
     FROM debian:stretch
     WORKDIR /etc/varnish
     EXPOSE 6080
     EXPOSE 6081
     EXPOSE 6082
     EXPOSE 6083
     ENV SIZE 100m
     ENV SECRET E29F6DE6-3803-449E-8A01-AA537A8715B1
     ARG BACKEND=nginx
     

     View Slide

118. install) \
     && curl -O -L https://github.com/varnish/libvmod-digest/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-digest-master && ./autogen.sh && ./configure && make && make
     install)
     FROM debian:stretch
     WORKDIR /etc/varnish
     EXPOSE 6080
     EXPOSE 6081
     EXPOSE 6082
     EXPOSE 6083
     ENV SIZE 100m
     ENV SECRET E29F6DE6-3803-449E-8A01-AA537A8715B1
     ARG BACKEND=nginx
     RUN apt-get update \
     && apt-get install -y curl \
     && curl -s https://packagecloud.io/install/repositories/varnishcache/varnish5/
     script.deb.sh | bash \
     && apt-get install -y varnish=5.2.0-1~stretch libmhash-dev --no-install-recommends
     --allow-unauthenticated \
     && rm -rf /var/lib/apt/lists/* \
     && /bin/bash -c ' echo "$SECRET" > /etc/varnish/secret' \
     && rm -rf /usr/lib/varnish/vmods/libvmod_*.so
     COPY --from=builder /usr/lib/varnish/vmods/libvmod_*.so /usr/lib/varnish/vmods/
     CMD ["/bin/bash","-c","/usr/sbin/varnishd -j unix,user=varnish -s malloc,$SIZE -F -a
     internal=0.0.0.0:6080 -a main=0.0.0.0:6081 -a proxy=0.0.0.0:6083,PROXY -T 0.0.0.0:6082
     -f /etc/varnish/default.vcl -S /etc/varnish/secret -p feature=+esi_disable_xml_check -
     p vsl_mask=+Hash"]
     

     View Slide

119. install) \
     && curl -O -L https://github.com/varnish/libvmod-digest/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-digest-master && ./autogen.sh && ./configure && make && make
     install)
     FROM debian:stretch
     WORKDIR /etc/varnish
     EXPOSE 6080
     EXPOSE 6081
     EXPOSE 6082
     EXPOSE 6083
     ENV SIZE 100m
     ENV SECRET E29F6DE6-3803-449E-8A01-AA537A8715B1
     ARG BACKEND=nginx
     RUN apt-get update \
     && apt-get install -y curl \
     && curl -s https://packagecloud.io/install/repositories/varnishcache/varnish5/
     script.deb.sh | bash \
     && apt-get install -y varnish=5.2.0-1~stretch libmhash-dev --no-install-recommends
     --allow-unauthenticated \
     && rm -rf /var/lib/apt/lists/* \
     && /bin/bash -c ' echo "$SECRET" > /etc/varnish/secret' \
     && rm -rf /usr/lib/varnish/vmods/libvmod_*.so
     COPY --from=builder /usr/lib/varnish/vmods/libvmod_*.so /usr/lib/varnish/vmods/
     CMD ["/bin/bash","-c","/usr/sbin/varnishd -j unix,user=varnish -s malloc,$SIZE -F -a
     internal=0.0.0.0:6080 -a main=0.0.0.0:6081 -a proxy=0.0.0.0:6083,PROXY -T 0.0.0.0:6082
     -f /etc/varnish/default.vcl -S /etc/varnish/secret -p feature=+esi_disable_xml_check -
     p vsl_mask=+Hash"]
     

     View Slide

120. install) \
     && curl -O -L https://github.com/varnish/libvmod-digest/archive/master.zip \
     && unzip master.zip \
     && (cd libvmod-digest-master && ./autogen.sh && ./configure && make && make
     install)
     FROM debian:stretch
     WORKDIR /etc/varnish
     EXPOSE 6080
     EXPOSE 6081
     EXPOSE 6082
     EXPOSE 6083
     ENV SIZE 100m
     ENV SECRET E29F6DE6-3803-449E-8A01-AA537A8715B1
     ARG BACKEND=nginx
     RUN apt-get update \
     && apt-get install -y curl \
     && curl -s https://packagecloud.io/install/repositories/varnishcache/varnish5/
     script.deb.sh | bash \
     && apt-get install -y varnish=5.2.0-1~stretch libmhash-dev --no-install-recommends
     --allow-unauthenticated \
     && rm -rf /var/lib/apt/lists/* \
     && /bin/bash -c ' echo "$SECRET" > /etc/varnish/secret' \
     && rm -rf /usr/lib/varnish/vmods/libvmod_*.so
     COPY --from=builder /usr/lib/varnish/vmods/libvmod_*.so /usr/lib/varnish/vmods/
     CMD ["/bin/bash","-c","/usr/sbin/varnishd -j unix,user=varnish -s malloc,$SIZE -F -a
     internal=0.0.0.0:6080 -a main=0.0.0.0:6081 -a proxy=0.0.0.0:6083,PROXY -T 0.0.0.0:6082
     -f /etc/varnish/default.vcl -S /etc/varnish/secret -p feature=+esi_disable_xml_check -
     p vsl_mask=+Hash"]
     

     View Slide

121. What’s the real IP?
     172.18.0.1 ?
     

     View Slide

122. HAProxy
     172.18.0.3
     Varnish
     172.18.0.8
     Nginx
     Public
     172.18.0.1
     

     View Slide

123. X-Forwarded-For
     

     View Slide

124. Nginx
     Public
     172.18.0.1
     Client IP: 172.18.0.1
     X-Forwarded-For:
     

     View Slide

125. Varnish
     172.18.0.8
     Nginx
     Public
     172.18.0.1
     Client IP: 172.18.0.8
     X-Forwarded-For: 172.18.0.1
     

     View Slide

126. HAProxy
     172.18.0.3
     Varnish
     172.18.0.8
     Nginx
     Public
     172.18.0.1
     Client IP: 172.18.0.8
     X-Forwarded-For: 172.18.0.3
     

     View Slide

127. PROXY Protocol
     

     View Slide

128. Add original IP as
     TCP preamble
     

     View Slide

129. DAEMON_OPTS="-j unix,user=www-data \
     -a :80 \
     -a :81,PROXY \
     -T localhost:6082 \
     -f /etc/varnish/default.vcl \
     -S /etc/varnish/secret \
     -l 100m,10m \
     -t 60 \
     -p feature=+esi_disable_xml_check \
     -p connect_timeout=20s \
     -p first_byte_timeout=100s \
     -p between_bytes_timeout=5s \
     -s malloc,3g"
     

     View Slide

130. Varnish automatically sets
     X-Forwarded-For to the
     original IP using PROXY
     protocol
     

     View Slide

131. HAProxy
     172.18.0.3
     Varnish
     172.18.0.8
     Nginx
     Public
     172.18.0.1
     Client IP: 172.18.0.8
     X-Forwarded-For: 172.18.0.1
     

     View Slide

132. View Slide

133. There are rules
     

     View Slide

134. ✓Idempotence
     ✓State
     ✓Expiration
     ✓Conditional requests
     ✓Cache variations
     Varnish speaks HTTP
     

     View Slide

135. Idempotence
     Execute
     multiple
     times
     Result
     doesn't
     change
     

     View Slide

136. ✓ GET
     ✓ HEAD
     - POST
     - PUT
     - DELETE
     - PATCH
     Idempotence
     Result
     changes
     Result
     doesn't
     change
     Don't
     cache
     

     View Slide

137. State
     

     View Slide

138. State
     ~
     user specific
     data
     Cookies
     Auth
     headers
     

     View Slide

139. About cookies
     

     View Slide

140. ✓ Request header
     ✓ Sent by client (~browser)
     ✓ Sent on every request
     ✓ Describes state
     ✓ Is not cached
     ✓ 3rd party tracking cookies vs own
     cookies
     About cookies
     Cookie: key=value;key2=value2
     

     View Slide

141. ✓ Response header
     ✓ Sent by backend server
     ✓ Only when backend is called
     ✓ Changes state
     ✓ Is not cached
     ✓ Blacklisted for 120s (by default)
     About cookies
     Set-Cookie: key=another_value
     

     View Slide

142. Time
     To
     Live
     

     View Slide

143. ✓ 120s by default
     ✓ Respects HTTP cache-control header
     ✓ Respects expires header
     ✓ Override in VCL file
     Time to live
     

     View Slide

144. 1.VCL
     2.s-maxage
     3.max-age
     4.expires
     5.Default value 120s
     TTL Order
     

     View Slide

145. Use TTL not to cache
     Cache-Control: max-age=0
     Cache-Control: s-maxage=0
     Cache-Control: private
     Cache-Control: no-cache
     Cache-Control: no-store
     Expires: Fri, 1 Jan 1971 00:00:00 GMT
     

     View Slide

146. Combine values
     Cache-Control: public, max-
     age=1000, s-maxage=3600
     

     View Slide

147. Age
     Age: 10
     How old is the
     cached object?
     

     View Slide

148. Real cache duration
     max-age - Age
     s-maxage - Age
     

     View Slide

149. Don’t set the Age
     header yourself!
     

     View Slide

150. Conditional
     requests
     

     View Slide

151. Only fetch
     payload that has
     changed
     

     View Slide

152. Otherwise:
     HTTP/1.1 304 Not Modified
     

     View Slide

153. In both directions
     User Varnish Server
     304 Not Modified 304 Not Modified
     

     View Slide

154. ✓ Conserve bandwidth
     ✓ Reduce load on the backend when
     revalidating
     Conditional requests
     

     View Slide

155. Conditional requests
     HTTP/1.1 200 OK
     Host: localhost
     Etag: 7c9d70604c6061da9bb9377d3f00eb27
     Content-type: text/html; charset=UTF-8
     Hello world output
     GET /if_none_match.php HTTP/1.1
     Host: localhost
     User-Agent: curl/7.48.0
     

     View Slide

156. Conditional requests
     HTTP/1.0 304 Not Modified
     Host: localhost
     Etag: 7c9d70604c6061da9bb9377d3f00eb27
     GET /if_none_match.php HTTP/1.1
     Host: localhost
     User-Agent: curl/7.48.0
     If-None-Match:
     7c9d70604c6061da9bb9377d3f00eb27
     

     View Slide

157. Conditional requests
     HTTP/1.1 200 OK
     Host: localhost
     Last-Modified: Fri, 22 Jul 2016 10:11:16 GMT
     Content-type: text/html; charset=UTF-8
     Hello world output
     GET /if_none_match.php HTTP/1.1
     Host: localhost
     User-Agent: curl/7.48.0
     

     View Slide

158. Conditional requests
     HTTP/1.0 304 Not Modified
     Host: localhost
     Last-Modified: Fri, 22 Jul 2016 10:11:16 GMT
     GET /if_none_match.php HTTP/1.1
     Host: localhost
     User-Agent: curl/7.48.0
     If-Last-Modified: Fri, 22 Jul 2016 10:11:16
     GMT
     

     View Slide

159. Store etag or
     modification date
     

     View Slide

160. Validate early on
     

     View Slide

161. Exit quickly
     

     View Slide

162. Varnish can revalidate
     aynchronously
     

     View Slide

163. And serve stale data while
     that happens
     

     View Slide

164. Grace mode
     

     View Slide

165. Effective TTL = TTL + grace
     

     View Slide

166. Cache-Control: public, max-age=100, s-maxage=3600,
     stale-while-revalidate=7200
     Stale-While-Revalidate
     Set grace mode
     

     View Slide

167. Variations
     

     View Slide

168. Cache is
     not the
     same for
     everyone
     

     View Slide

169. ✓ URL
     ✓ Hostname
     ✓ IP if hostname is not set
     ✓ Vary header
     Basic variations
     

     View Slide

170. Vary header
     Vary: Accept-Encoding
     Vary: Accept-Language
     Vary: Cookie
     Watch out!
     

     View Slide

171. The flow
     

     View Slide

172. View Slide

173. View Slide

174. Varnish
     Configuration
     Language
     

     View Slide

175. DSL compiled
     and linked as
     shared object
     

     View Slide

176. /etc/varnish/default.vcl
     

     View Slide

177. Hooks
     &
     subroutines
     

     View Slide

178. VCL
     Recv
     VCL
     Hash
     VCL
     Miss
     VCL
     Hit
     VCL
     Backend
     Response
     VCL
     Deliver
     VCL
     Purge
     VCL
     Error
     VCL
     Pipe
     VCL
     Pass
     VCL
     Synth
     VCL
     Backend
     Error
     

     View Slide

179. ✓vcl_recv: receive request
     ✓vcl_hash: compose cache key
     ✓vcl_miss: not found in cache
     ✓vcl_hit: found in cache
     ✓vcl_pass: don’t store in cache
     ✓vcl_pipe: bypass cache
     ✓vcl_backend_fetch: connect to
     backend
     ✓vcl_backend_response: response from
     backend
     ✓vcl_backend_error: backend fetch
     failed
     

     View Slide

180. ✓vcl_purge: after successful purge
     ✓vcl_synth: send synthetic output
     ✓vcl_deliver: return data to client
     ✓vcl_init: initialize VMODs
     ✓vcl_fini: discard VMODs
     ✓vcl_fail: stop execution
     

     View Slide

181. Actions
     

     View Slide

182. ✓ hash: lookup in cache
     ✓ pass: don't cache
     ✓ synth: synthetic HTML output
     ✓ pipe: bypass cache
     ✓ purge: remove from cache
     VCL_RECV
     

     View Slide

183. ✓ synth: synthetic HTML output
     ✓ pipe: bypass cache
     VCL_PIPE
     

     View Slide

184. ✓ fetch: fetch data from
     backend, don't cache
     ✓ restart: restart transaction
     ✓ synth: synthetic HTML output
     VCL_PASS
     

     View Slide

185. ✓ deliver: send cached object
     ✓ miss: synchronous refresh
     despite hit
     ✓ pass: fetch data from
     backend despite hit, don't
     cache
     ✓ restart: restart transaction
     ✓ synth: synthetic HTML output
     VCL_HIT
     

     View Slide

186. ✓ fetch: fetch data from
     backend
     ✓ pass: fetch data from
     backend, don't cache
     ✓ restart: restart transaction
     ✓ synth: synthetic HTML output
     VCL_MISS
     

     View Slide

187. ✓ lookup: look for cached
     object by using hash key
     VCL_HASH
     

     View Slide

188. ✓ restart: restart transaction
     ✓ synth: synthetic HTML output
     VCL_PURGE
     

     View Slide

189. ✓ deliver: deliver object to
     client
     ✓ restart: restart transaction
     ✓ synth: synthetic HTML output
     VCL_DELIVER
     

     View Slide

190. ✓ deliver: deliver synthetic
     output to client
     ✓ restart: restart transaction
     VCL_SYNTH
     

     View Slide

191. ✓ fetch: fetch object from
     backend
     ✓ abandon: abandon request
     and send HTTP 503 error
     VCL_BACKEND_FETCH
     

     View Slide

192. ✓ deliver: send fetched data to
     client
     ✓ abandon: abandon request
     and send HTTP 503 error
     ✓ retry: retry backend request
     VCL_BACKEND_RESPONSE
     

     View Slide

193. ✓ deliver: send fetched data to
     client
     ✓ retry: retry backend request
     VCL_BACKEND_ERROR
     

     View Slide

194. Typical flows
     

     View Slide

195. ✓ vcl_recv: hash
     ✓ vcl_hash: lookup
     ✓ vcl_miss: fetch
     ✓ vcl_backend_request: fetch
     ✓ vcl_backend_response: deliver
     ✓ vcl_deliver: deliver
     MISS
     

     View Slide

196. ✓ vcl_recv: hash
     ✓ vcl_hash: lookup
     ✓ vcl_hit: deliver
     ✓ vcl_deliver: deliver
     HIT
     

     View Slide

197. ✓ vcl_recv: pass
     ✓ vcl_pass: fetch
     ✓ vcl_backend_request: fetch
     ✓ vcl_backend_response: deliver
     ✓ vcl_deliver: deliver
     PASS
     

     View Slide

198. Objects
     

     View Slide

199. ✓ req: incoming request object
     ✓ req_top: top level esi request
     ✓ bereq: request object to send to
     backend
     ✓ beresp: backend response
     ✓ resp: response to send back to
     client
     ✓ obj: cached object
     ✓ client: client information
     ✓ server: server information
     ✓ local: local TCP information
     ✓ remote: remote TCP information
     ✓ storage: storage information
     VCL Objects
     

     View Slide

200. Variables
     

     View Slide

201. ✓ req.url
     ✓ req.http.host
     ✓ req.http.user-agent
     ✓ req.backend_hint
     ✓ req.method
     ✓ …
     req variables
     

     View Slide

202. ✓ bereq.backend
     ✓ bereq.http.user-agent
     ✓ bereq.method
     ✓ bereq.url
     ✓ …
     bereq variables
     

     View Slide

203. ✓ beresp.age
     ✓ beresp.backend.ip
     ✓ beresp.backend.name
     ✓ beresp.do_esi
     ✓ beresp.grace
     ✓ beresp.keep
     ✓ beresp.http.set-cookie
     ✓ beresp.ttl
     ✓ beresp.status
     ✓ beresp.uncacheable
     beresp variables
     

     View Slide

204. ✓ client.ip
     ✓ client.identity
     client variables
     local variables
     ✓ local.ip
     remote variables
     ✓ remote.ip
     

     View Slide

205. ✓ obj.age
     ✓ obj.grace
     ✓ obj.hits
     ✓ obj.http.cache-control
     ✓ obj.reason
     ✓ obj.status
     ✓ obj.ttl
     object variables
     

     View Slide

206. ✓ resp.http.user-agent
     ✓ resp.is_streaming
     ✓ resp.reason
     ✓ resp.status
     resp variables
     

     View Slide

207. ✓ storage..free_space
     ✓ storage..used_space
     ✓ storage..happy
     storage variables
     

     View Slide

208. Default
     behaviour
     

     View Slide

209. vcl 4.0;
     sub vcl_recv {
     if (req.method == "PRI") {
     return (synth(405));
     }
     if (req.method != "GET" &&
     req.method != "HEAD" &&
     req.method != "PUT" &&
     req.method != "POST" &&
     req.method != "TRACE" &&
     req.method != "OPTIONS" &&
     req.method != “PATCH" &&
     req.method != "DELETE") {
     return (pipe);
     }
     if (req.method != "GET" && req.method != "HEAD") {
     return (pass);
     }
     if (req.http.Authorization || req.http.Cookie) {
     return (pass);
     }
     return (hash);
     }
     Idempotence
     State
     Action
     Receive
     request
     

     View Slide

210. sub vcl_hash {
     hash_data(req.url);
     if (req.http.host) {
     hash_data(req.http.host);
     } else {
     hash_data(server.ip);
     }
     return (lookup);
     }
     Lookup in
     cache
     Variations
     Action
     

     View Slide

211. sub vcl_purge {
     return (synth(200, "Purged"));
     }
     sub vcl_hit {
     if (obj.ttl >= 0s) {
     return (deliver);
     }
     if (obj.ttl + obj.grace > 0s) {
     return (deliver);
     }
     return (miss);
     }
     sub vcl_miss {
     return (fetch);
     }
     sub vcl_deliver {
     return (deliver);
     }
     Remove
     from cache
     Found in
     cache
     Not found in
     cache
     Return
     HTTP response to
     client
     

     View Slide

212. sub vcl_synth {
     set resp.http.Content-Type = "text/html; charset=utf-8";
     set resp.http.Retry-After = "5";
     synthetic( {"
     
     
     "} + resp.status + " " + resp.reason + {"
     
     
     Error "} + resp.status + " " + resp.reason + {"
     "} + resp.reason + {"
     Guru Meditation:
     XID: "} + req.xid + {"
     
     Varnish cache server
     
     
     "} );
     return (deliver);
     }
     Send
     custom HTML
     

     View Slide

213. sub vcl_backend_fetch {
     return (fetch);
     }
     sub vcl_backend_response {
     if (bereq.uncacheable) {
     return (deliver);
     } else if (beresp.ttl <= 0s ||
     beresp.http.Set-Cookie ||
     beresp.http.Surrogate-control ~ "no-store" ||
     (!beresp.http.Surrogate-Control &&
     beresp.http.Cache-Control ~ "no-cache|no-store|private") ||
     beresp.http.Vary == "*") {
     set beresp.ttl = 120s;
     set beresp.uncacheable = true;
     }
     return (deliver);
     }
     Send
     backend
     request
     Receive
     backend
     response
     TTL
     State
     Receive
     backend
     response
     

     View Slide

214. sub vcl_backend_error {
     set beresp.http.Content-Type = "text/html; charset=utf-8";
     set beresp.http.Retry-After = "5";
     synthetic( {"
     
     
     "} + beresp.status + " " + beresp.reason + {"
     
     
     Error "} + beresp.status + " " + beresp.reason + {"
     "} + beresp.reason + {"
     Guru Meditation:
     XID: "} + bereq.xid + {"
     
     Varnish cache server
     
     
     "} );
     return (deliver);
     }
     Send
     custom HTML on
     backend error
     

     View Slide

215. Minimal VCL
     

     View Slide

216. vcl 4.0;
     backend default {
     .host = "127.0.0.1";
     .port = "8080";
     }
     Minimal VCL
     

     View Slide

217. vcl 4.0;
     backend default {
     .host = "127.0.0.1";
     .port = "80";
     .max_connections = 300;
     .first_byte_timeout = 300s;
     .connect_timeout = 5s;
     .between_bytes_timeout = 2s;
     }
     More backend work
     

     View Slide

218. vcl 4.0;
     backend default {
     .host = "127.0.0.1";
     .port = "80";
     .max_connections = 300;
     .first_byte_timeout = 300s;
     .connect_timeout = 5s;
     .between_bytes_timeout = 2s;
     .probe = {
     .url = "/";
     .interval = 5s;
     .timeout = 1s;
     .window = 5;
     .threshold = 3;
     }
     }
     Even more backend work
     

     View Slide

219. vcl 4.0;
     backend default {
     .host = "127.0.0.1";
     .port = "80";
     .max_connections = 300;
     .first_byte_timeout = 300s;
     .connect_timeout = 5s;
     .between_bytes_timeout = 2s;
     .probe = {
     .request =
     "HEAD / HTTP/1.1"
     "Host: localhost"
     "Connection: close"
     "User-Agent: Varnish Health Probe";
     .interval = 5s;
     .timeout = 1s;
     .window = 5;
     .threshold = 3;
     }
     }
     Even more backend work
     

     View Slide

220. VMOD
     

     View Slide

221. Varnish modules
     

     View Slide

222. Written in C
     

     View Slide

223. Exposes new VCL
     objects
     

     View Slide

224. Added functionality
     

     View Slide

225. vcl 4.0;
     import directors;
     import std;
     import cookie;
     sub vcl_init {
     new vdir = directors.round_robin();
     vdir.add_backend(…);
     vdir.add_backend(…);
     }
     VMOD
     

     View Slide

226. vmod_std
     

     View Slide

227. sub vcl_recv {
     if(std.file_exists("/etc/varnish/file.txt")) {
     return(synth(200,std.fileread("/etc/varnish/file.txt")));
     } else {
     return(synth(200,"File does not exist"));
     }
     }
     std.fileread
     

     View Slide

228. sub vcl_recv {
     return(synth(200,"FOO: " + std.getenv("FOO")));
     }
     std.getenv
     

     View Slide

229. vcl 4.0;
     import std;
     backend default {
     .host = "nginx";
     .port = "80";
     .probe = {
     .url = "/";
     .interval = 1s;
     .timeout = 1s;
     .window = 5;
     .threshold = 3;
     }
     }
     sub vcl_recv {
     if(std.healthy(req.backend_hint)) {
     return(synth(200,"Backend is healthy"));
     } else {
     return(synth(200,"Backend is unhealthy"));
     }
     }
     std.healthy
     

     View Slide

230. sub vcl_recv {
     std.timestamp("Before std.log");
     std.log("This should appear in the Shared Memory Log");
     std.timestamp("After std.log");
     std.syslog(9,"This should appear in the Syslog");
     std.timestamp("After std.syslog");
     return(synth(200,"OK"));
     }
     Logging
     

     View Slide

231. sub vcl_recv {
     return(synth(200,"Client port: " + std.port(client.ip) + ", server
     port: " + std.port(server.ip)));
     }
     std.port
     

     View Slide

232. sub vcl_recv {
     return(synth(200,"Unsorted: " + regsub(req.url,"\?(.+)$","\1") + ",
     sorted: " + regsub(std.querysort(req.url),"\?(.+)$","\1")));
     }
     std.querysort
     

     View Slide

233. sub vcl_recv {
     return(synth(200,std.real2integer(std.random(1,10),0)));
     }
     std.random & std.real2integer
     

     View Slide

234. sub vcl_recv {
     return(synth(200,std.toupper("yes") + " " + std.tolower("YES")));
     }
     std.toupper & std.tolower
     

     View Slide

235. https://varnish-cache.org/docs/5.2/
     reference/vmod_std.generated.html
     

     View Slide

236. Load balancing
     

     View Slide

237. vcl 4.0;
     import directors;
     backend server1 {
     .host = “1.2.3.4”;
     .port = "80";
     .max_connections = 300;
     .first_byte_timeout = 300s;
     .connect_timeout = 5s;
     .between_bytes_timeout = 2s;
     .probe = {
     .request =
     "HEAD / HTTP/1.1"
     "Host: localhost"
     "Connection: close"
     "User-Agent: Varnish Health Probe";
     .interval = 5s;
     .timeout = 1s;
     .window = 5;
     .threshold = 3;
     }
     }
     Loadbalancing: server 1
     

     View Slide

238. backend server2 {
     .host = “1.2.3.5”;
     .port = "80";
     .max_connections = 300;
     .first_byte_timeout = 300s;
     .connect_timeout = 5s;
     .between_bytes_timeout = 2s;
     .probe = {
     .request =
     "HEAD / HTTP/1.1"
     "Host: localhost"
     "Connection: close"
     "User-Agent: Varnish Health Probe";
     .interval = 5s;
     .timeout = 1s;
     .window = 5;
     .threshold = 3;
     }
     }
     Loadbalancing: server 2
     

     View Slide

239. sub vcl_init {
     new vdir = directors.round_robin();
     vdir.add_backend(server1);
     vdir.add_backend(server2);
     }
     sub vcl_recv {
     set req.backend_hint = vdir.backend();
     }
     Loadbalancing: round robin
     

     View Slide

240. sub vcl_init {
     new vdir = directors.random();
     vdir.add_backend(server1,2);
     vdir.add_backend(server2,3);
     }
     sub vcl_recv {
     set req.backend_hint = vdir.backend();
     }
     Loadbalancing: random
     

     View Slide

241. sub vcl_init {
     new vdir = directors.fallback();
     vdir.add_backend(server1);
     vdir.add_backend(server2);
     }
     sub vcl_recv {
     set req.backend_hint = vdir.backend();
     }
     Loadbalancing: fallback
     

     View Slide

242. sub vcl_init {
     new vdir = directors.hash();
     vdir.add_backend(server1);
     vdir.add_backend(server2);
     }
     sub vcl_recv {
     set req.backend_hint = vdir.backend(req.url);
     }
     Loadbalancing: URL hash
     

     View Slide

243. sub vcl_init {
     new vdir = directors.hash();
     vdir.add_backend(server1);
     vdir.add_backend(server2);
     }
     sub vcl_recv {
     set req.backend_hint = vdir.backend(client.identity);
     }
     Loadbalancing: IP hash
     

     View Slide

244. sub vcl_recv {
     if(req.url ~ “^/products”) {
     set req.backend_hint = server1;
     } else {
     set req.backend_hint = server2;
     }
     }
     Conditional loadbalancing
     

     View Slide

245. Install other
     VMODs
     

     View Slide

246. apt-get install -y varnish varnish-dev build-essential
     curl -O https://download.varnish-software.com/varnish-
     modules/varnish-modules-0.12.1.tar.gz
     tar xvzf varnish-modules-0.12.1.tar.gz
     cd varnish-modules-0.12.1
     ./configure
     make
     make install
     Install official VMODs
     

     View Slide

247. apt-get install -y varnish varnish-modules
     Install basic VMODs
     If you install Varnish
     5.0.0 via Debian repo
     

     View Slide

248. vmod_cookie
     

     View Slide

249. vcl 4.0;
     sub vcl_recv {
     if (req.http.Cookie) {
     set req.http.Cookie = ";" + req.http.Cookie;
     set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
     set req.http.Cookie = regsuball(req.http.Cookie, ";(PHPSESSID)=", "; \1=");
     set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
     if (req.http.cookie ~ "^\s*$") {
     unset req.http.cookie;
     }
     }
     }
     Without vmod_cookie
     

     View Slide

250. vcl 4.0;
     import cookie;
     sub vcl_recv {
     if (req.http.cookie) {
     cookie.parse(req.http.cookie);
     cookie.filter_except("PHPSESSID");
     set req.http.cookie = cookie.get_string();
     if (req.http.cookie ~ "^\s*$") {
     unset req.http.cookie;
     }
     }
     }
     With vmod_cookie
     

     View Slide

251. vcl 4.0;
     import cookie;
     sub vcl_recv {
     cookie.parse(req.http.cookie);
     if(cookie.isset("PHPSESSID")) {
     return(synth(200,"PHPSESSID is set and contains the following value: " +
     cookie.get("PHPSESSID") + ""));
     }
     }
     Other vmod_cookie examples
     

     View Slide

252. vcl 4.0;
     import cookie;
     sub vcl_recv {
     cookie.parse(req.http.cookie);
     if(cookie.isset(“PHPSESSID") && req.url ~ "^/products/[0-9]+$") {
     cookie.delete("PHPSESSID");
     }
     set req.http.cookie = cookie.get_string();
     }
     Other vmod_cookie examples
     

     View Slide

253. https://github.com/varnish/varnish-
     modules/blob/master/docs/
     vmod_cookie.rst
     

     View Slide

254. vmod_accept
     

     View Slide

255. https://github.com/gquintard/libvmod-accept
     

     View Slide

256. vcl 4.0;
     import accept;
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_init {
     new rule = accept.rule("en");
     rule.add("nl");
     }
     sub vcl_recv {
     set req.http.x-old-accept-language= req.http.Accept-Language;
     set req.http.Accept-Language = rule.filter(req.http.Accept-Language);
     return(synth(200,"Raw Accept-Language: "+req.http.x-old-accept-
     language+"Filtered Accept-Language: "+req.http.Accept-Language+"
     ul>"));
     }
     vmod_accept
     

     View Slide

257. • Raw Accept-Language: nl,en-US;q=0.8,en;q=0.6
     • Filtered Accept-Language: nl
     vmod_accept output
     

     View Slide

258. vcl 4.0;
     import accept;
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_init {
     new language = accept.rule("en");
     language.add("nl");
     new encoding = accept.rule("deflate");
     language.add("gzip");
     new contenttype = accept.rule("text/plain");
     contenttype.add("text/html");
     contenttype.add("application/json");
     }
     sub vcl_recv {
     set req.http.Accept-Language = language.filter(req.http.Accept-Language);
     set req.http.Accept-Encoding = encoding.filter(req.http.Accept-Encoding);
     set req.http.Accept = contenttype.filter(req.http.Accept);
     }
     vmod_accept
     

     View Slide

259. Why is this useful?
     

     View Slide

260. Accept-Language: nl,en-US;q=0.8,en;q=0.6
     Too many
     variations
     Impacts hit
     rate
     Why is vmod_accept useful?
     Accept-Language: nl
     Less variations
     

     View Slide

261. More VMODs
     coming up later
     

     View Slide

262. In an ideal world
     

     View Slide

263. HTTP best practices
     >
     custom VCL
     

     View Slide

264. Reality
     sucks
     

     View Slide

265. View Slide

266. View Slide

267. Don’t trust the end-user
     

     View Slide

268. Cache-control ?
     

     View Slide

269. Legacy
     

     View Slide

270. Write VCL
     

     View Slide

271. Normalize
     

     View Slide

272. vcl 4.0;
     import accept;
     import std;
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_init {
     new language = accept.rule("en");
     language.add("nl");
     new encoding = accept.rule("");
     language.add("deflate");
     language.add("gzip");
     new contenttype = accept.rule("text/plain");
     contenttype.add("text/html");
     contenttype.add("application/json");
     }
     sub vcl_recv {
     set req.http.Accept-Language = language.filter(req.http.Accept-Language);
     set req.http.Accept-Encoding = encoding.filter(req.http.Accept-Encoding);
     set req.http.Accept = contenttype.filter(req.http.Accept);
     if (req.http.User-Agent ~ "MSIE 6" || req.http.Accept-Encoding ~ "^\s*$") {
     unset req.http.Accept-Encoding;
     }
     set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
     set req.url = std.querysort(req.url);
     Normalize
     

     View Slide

273. if (req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=") {
     set req.url = regsuball(req.url, "&(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|
     siteurl)=([A-z0-9_\-\.%25]+)", "");
     set req.url = regsuball(req.url, "\?(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|
     siteurl)=([A-z0-9_\-\.%25]+)", "?");
     set req.url = regsub(req.url, "\?&", "?");
     set req.url = regsub(req.url, "\?$", "");
     }
     if (req.url ~ "\#") {
     set req.url = regsub(req.url, "\#.*$", "");
     }
     if (req.url ~ "\?$") {
     set req.url = regsub(req.url, "\?$", "");
     }
     }
     Normalize
     

     View Slide

274. Static assets
     

     View Slide

275. vcl 4.0;
     sub vcl_recv {
     if (req.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|
     gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|otf|ogg|
     ogm|opus|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|
     wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
     unset req.http.Cookie;
     return (hash);
     }
     }
     sub vcl_backend_response {
     if (bereq.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|
     flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|otf|
     ogg|ogm|opus|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|
     txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
     unset beresp.http.set-cookie;
     }
     if (bereq.url ~ "^[^?]*\.(7z|avi|bz2|flac|flv|gz|mka|mkv|mov|mp3|mp4|
     mpeg|mpg|ogg|ogm|opus|rar|tar|tgz|tbz|txz|wav|webm|xz|zip)(\?.*)?$") {
     unset beresp.http.set-cookie;
     set beresp.do_stream = true;
     set beresp.do_gzip = false;
     }
     }
     Cache static assets
     

     View Slide

276. Do you really
     want to cache
     static assets?
     

     View Slide

277. Nginx or
     Apache can
     be fast
     enough for
     that
     

     View Slide

278. vcl 4.0;
     import std;
     sub vcl_recv {
     if (req.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|
     gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|otf|ogg|
     ogm|opus|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|
     wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
     unset req.http.Cookie;
     return (pass);
     }
     }
     Don’t cache static assets
     

     View Slide

279. URL whitelist/blacklist
     

     View Slide

280. sub vcl_recv {
     if (req.url ~ "^/status\.php$" ||
     req.url ~ "^/update\.php$" ||
     req.url ~ "^/admin$" ||
     req.url ~ "^/admin/.*$" ||
     req.url ~ "^/user$" ||
     req.url ~ "^/user/.*$" ||
     req.url ~ "^/flag/.*$" ||
     req.url ~ "^.*/ajax/.*$" ||
     req.url ~ "^.*/ahah/.*$") {
     return (pass);
     }
     }
     URL blacklist
     

     View Slide

281. sub vcl_recv {
     if (req.url ~ "^/products/?"
     return (hash);
     }
     }
     URL whitelist
     

     View Slide

282. Those damn
     cookies again!
     

     View Slide

283. vcl 4.0;
     sub vcl_recv {
     set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
     if (req.http.cookie ~ "^\s*$") {
     unset req.http.cookie;
     }
     }
     Remove tracking cookies
     

     View Slide

284. vcl 4.0;
     sub vcl_recv {
     if (req.http.Cookie) {
     set req.http.Cookie = ";" + req.http.Cookie;
     set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
     set req.http.Cookie = regsuball(req.http.Cookie, ";(PHPSESSID)=", "; \1=");
     set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
     if (req.http.cookie ~ "^\s*$") {
     unset req.http.cookie;
     }
     }
     }
     Only keep session cookie
     

     View Slide

285. vcl 4.0;
     sub vcl_recv {
     if (req.http.cookie) {
     cookie.parse(req.http.cookie);
     cookie.filter_except("PHPSESSID");
     set req.http.cookie = cookie.get_string();
     if (req.http.cookie ~ "^\s*$") {
     unset req.http.cookie;
     }
     }
     }
     The VMOD way
     

     View Slide

286. sub vcl_recv {
     if (req.http.Cookie) {
     set req.http.Cookie = ";" + req.http.Cookie;
     set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
     set req.http.Cookie = regsuball(req.http.Cookie, ";(language)=", "; \1=");
     set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
     set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
     if (req.http.cookie ~ "^\s*$") {
     unset req.http.cookie;
     return(pass);
     }
     return(hash);
     }
     }
     sub vcl_hash {
     hash_data(regsub( req.http.Cookie, "^.*language=([^;]*);*.*$", "\1" ));
     }
     Language cookie cache variation
     

     View Slide

287. vcl 4.0;
     import cookie;
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_recv {
     if (req.http.cookie) {
     cookie.parse(req.http.cookie);
     cookie.filter_except("language");
     set req.http.cookie = cookie.get_string();
     if (req.http.cookie ~ "^\s*$") {
     unset req.http.cookie;
     }
     return(hash);
     }
     }
     sub vcl_hash {
     hash_data(cookie.get("language"));
     }
     The VMOD way
     

     View Slide

288. Alternative
     language
     cache
     variation
     

     View Slide

289. sub vcl_hash {
     hash_data(req.http.Accept-Language);
     }
     Language cookie cache variation
     Or just send a
     “Vary:Accept-Language”
     header
     

     View Slide

290. sub vcl_recv {
     return(hash);
     }
     Cache, even with cookies
     

     View Slide

291. sub vcl_hash {
     hash_data(req.http.Cookie);
     }
     Hash all cookies
     

     View Slide

292. Block caching
     

     View Slide

293. View Slide

294. Code
     renders
     single HTTP
     response
     

     View Slide

295. Lowest
     denominator:
     no cache
     

     View Slide

296. 
     Edge Side Includes
     ✓Placeholder
     ✓Parsed by Varnish
     ✓Output is a composition of blocks
     ✓State per block
     ✓TTL per block
     

     View Slide

297. sub vcl_recv {
     set req.http.Surrogate-Capability = "key=ESI/1.0";
     }
     sub vcl_backend_response {
     if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
     unset beresp.http.Surrogate-Control;
     set beresp.do_esi = true;
     }
     }
     Edge Side Includes
     

     View Slide

298. header("Cache-Control: public,must-revalidate,s-maxage=10");
     echo "Date in the ESI tag: ".date('Y-m-d H:i:s').'
     ';
     header("Cache-Control: no-store");
     header(“Surrogate-Control: content='ESI/1.0'");
     echo ''.PHP_EOL;
     echo "Date in the main page: ".date('Y-m-d H:i:s').'
     ';
     Main page
     ESI frame:
     esi.php
     Cached for
     10 seconds
     Not cached
     

     View Slide

299. ESI_xmlerror No ESI processing, first char not 'feature esi_disable_xml_check)
     Expects HTML/
     XML tags
     

     View Slide

300. -p feature=+esi_disable_xml_check
     Add as
     startup option
     

     View Slide

301. req_top
     Get
     information about
     parent request in an
     ESI call
     

     View Slide

302. vcl 4.0;
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_recv {
     if (req.esi_level > 0) {
     set req.http.x-parent-url = req_top.url;
     }
     }
     sub vcl_backend_response {
     if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
     unset beresp.http.Surrogate-Control;
     set beresp.do_esi = true;
     }
     }
     Get parent URL
     

     View Slide

303. ESI
     vs
     AJAX
     

     View Slide

304. ✓ Server-side
     ✓ Standardized
     ✓ Processed on the
     “edge”, no in the
     browser
     ✓ Generally faster
     Edge-Side Includes
     - Sequential
     - One fails, all fail
     - Limited
     implementation in
     Varnish
     

     View Slide

305. ✓ Client-side
     ✓ Common knowledge
     ✓ Parallel processing
     ✓ Graceful
     degradation
     AJAX
     - Processed by the
     browser
     - Extra roundtrips
     - Somewhat slower
     

     View Slide

306. Choose wisely
     

     View Slide

307. Assemble at the view layer
     

     View Slide

308. {{ render_esi(controller('AppBundle:News:latest', { 'maxPerPage': 5 })) }}
     {{ render_esi(url('latest_news', { 'maxPerPage': 5 })) }}
     In your Twig
     templates
     Silex or
     Symfony
     Falls back to
     internal subrequests
     on failure
     Does ESI
     

     View Slide

309. {{ render_hinclude(controller('AppBundle:News:latest', { 'maxPerPage': 5 })) }}
     {{ render_hinclude(url(‘latest_news', { 'maxPerPage': 5 })) }}
     In your Twig
     templates
     Silex or
     Symfony
     Requires hinclude.js
     Does AJAX
     

     View Slide

310. 
     ESI vs HInclude
     
     
     
     Extra
     parameters
     

     View Slide

311. Silex example
     

     View Slide

312. use Symfony\Component\HttpFoundation\Response;
     require_once dirname(__DIR__).'/vendor/autoload.php';
     $app = new Silex\Application();
     $app['debug'] = true;
     $app->register(new Silex\Provider\TwigServiceProvider(), [
     'twig.path' => dirname(__DIR__).'/views'
     ]);
     $app->register(new Silex\Provider\HttpCacheServiceProvider());
     $app->register(new Silex\Provider\HttpFragmentServiceProvider());
     $app->get('/', function() use($app) {
     $response = new Response($app['twig']->render('index.twig'));
     $response->setSharedMaxAge(10);
     return $response;
     });
     $app->get('/header', function() use($app) {
     $response = new Response($app['twig']->render('header.twig'));
     $response->setPrivate();
     return $response;
     })->bind('header');
     $app->get('/footer', function() use($app) {
     $response = new Response($app['twig']->render('footer.twig'));
     $response->setSharedMaxAge(3);
     return $response;
     })->bind('footer');
     $app->run();
     

     View Slide

313. 
     
     
     {{ render_esi(url('header')) }}
     Main content
     
     Date and time
     {{ "now"|date("m/d/Y H:i:s") }}
     
     {{ render_hinclude(url('footer')) }}
     <br/>script><br/></body><br/></html><br/>index.twig<br/>

     View Slide

314. Header
     
     Date and time
     {{ "now"|date("m/d/Y H:i:s") }}
     
     
     header.twig
     

     View Slide

315. 
     Footer
     
     Date and time
     {{ "now"|date("m/d/Y H:i:s") }}
     
     footer.twig
     

     View Slide

316. Control
     Time
     To
     Live
     

     View Slide

317. sub vcl_backend_response {
     set beresp.ttl = 3h;
     }
     Control Time To Live
     

     View Slide

318. sub vcl_backend_response {
     if (beresp.ttl <= 0s || beresp.http.Set-Cookie ||
     beresp.http.Vary == "*") {
     set beresp.ttl = 120s;
     set beresp.uncacheable = true;
     return (deliver);
     }
     }
     Control Time To Live
     

     View Slide

319. Debugging
     

     View Slide

320. sub vcl_deliver {
     if (obj.hits > 0) {
     set resp.http.X-Cache = "HIT";
     } else {
     set resp.http.X-Cache = "MISS";
     }
     }
     Debugging
     

     View Slide

321. Anonymize
     

     View Slide

322. sub vcl_deliver {
     unset resp.http.X-Powered-By;
     unset resp.http.Server;
     unset resp.http.X-Drupal-Cache;
     unset resp.http.X-Varnish;
     unset resp.http.Via;
     unset resp.http.Link;
     unset resp.http.X-Generator;
     }
     Anonymize
     

     View Slide

323. Breaking news isn't breaking
     

     View Slide

324. Purging
     

     View Slide

325. acl purgers {
     "localhost";
     "127.0.0.1";
     "::1";
     }
     sub vcl_recv {
     if (req.method == "PURGE") {
     if (client.ip !~ purgers) {
     return (synth(405, "Method not allowed"));
     }
     return (purge);
     }
     }
     Purging
     

     View Slide

326. Purging
     curl -XPURGE "http://example.com/products"
     Immediately
     remove from
     memory
     

     View Slide

327. header('Cache-control: no-cache');
     $curl = curl_init('http://localhost/');
     curl_setopt_array($curl,[
     CURLOPT_RETURNTRANSFER => true,
     CURLOPT_CUSTOMREQUEST => 'PURGE'
     ]);
     echo curl_exec($curl).PHP_EOL;
     Purging
     

     View Slide

328. header('Cache-control: no-cache');
     require 'guzzle.phar';
     $client = new GuzzleHttp\Client(['base_uri' => 'http://localhost:6081/']);
     $response = $client->request('PURGE', '/');
     if($response->getStatusCode() == 200) {
     echo "PURGED".PHP_EOL;
     } else {
     echo $response->getBody();
     }
     Purging
     

     View Slide

329. acl purge {
     "localhost";
     "127.0.0.1";
     "::1";
     }
     sub vcl_recv {
     if (req.method == "PURGE") {
     if (client.ip !~ purgers) {
     return (synth(405, "Method not allowed"));
     }
     if(req.http.x-purge-regex) {
     ban("req.http.host == " + req.http.host + " && req.url ~ " + req.http.x-purge-regex);
     } else {
     ban("req.http.host == " + req.http.host + " && req.url == " + req.url);
     }
     return (synth(200, "Purged"));
     }
     }
     Banning
     

     View Slide

330. Banning
     curl -XPURGE "http://example.com/products"
     curl -XPURGE -H "x-purge-regex:/products" "http://example.com"
     Add to ban list, remove
     on next request
     Remove patterns via
     “X-PURGE-REGEX”
     

     View Slide

331. Lurker-friendly bans
     

     View Slide

332. Ban lurker
     Object
     User
     Varnish
     Server
     Sends HTTP
     response
     Response
     stored in
     object
     Sends BAN to
     Varnish
     Ban lurker
     thread
     Ban list
     Reads
     ban list
     Removes
     object
     ban req.http.host == localhost && req.url ~ /products
     

     View Slide

333. Ban lurker
     ban req.http.host == localhost && req.url ~ /products
     Lurker can’t match
     request info
     Lurker can only
     match what’s in the
     object
     Next visitor
     triggers cache
     remove
     

     View Slide

334. acl purge {
     "localhost";
     "127.0.0.1";
     "::1";
     }
     sub vcl_recv {
     if (req.method == "PURGE") {
     if (client.ip !~ purgers) {
     return (synth(405, "Method not allowed"));
     }
     if(req.http.x-purge-regex) {
     ban("obj.http.x-host == " + req.http.host + " && obj.http.x-url ~ " + req.http.x-purge-regex);
     } else {
     ban("obj.http.host == " + req.http.host + " && obj.http.x-url == " + req.url);
     }
     return (synth(200, "Purged"));
     }
     }
     sub vcl_backend_response {
     set beresp.http.x-url = bereq.url;
     set beresp.http.x-host = bereq.http.host;
     }
     sub vcl_deliver {
     unset resp.http.x-url;
     unset resp.http.x-host;
     }
     Lurker-friendly bans
     Store request info
     in response object
     

     View Slide

335. Lurker-friendly bans
     ban obj.http.x-host == localhost && obj.http.x-url ~ /products
     Lurker can match
     response info
     Ban lurker
     removes item from
     cache async
     

     View Slide

336. varnish> param.show ban_lurker_age
     200
     ban_lurker_age
     Value is: 60.000 [seconds] (default)
     Minimum is: 0.000
     The ban lurker will ignore bans until they are this old. When
     a ban is added, the active traffic will be tested against it as
     part of object lookup. Because many applications issue bans in
     bursts, this parameter holds the ban-lurker off until the rush
     is over.
     This should be set to the approximate time which a ban-burst
     takes.
     

     View Slide

337. Banning
     through
     Varnishadm
     

     View Slide

338. varnishadm> ban obj.http.x-host == localhost && obj.http.x-url ~ /products
     Varnishadm banning
     

     View Slide

339. Banning
     through
     socket
     

     View Slide

340. Varnishadm is a
     binary on top of
     the socket
     

     View Slide

341. Varnishadm uses
     the secret file to
     authenticate
     automatically
     

     View Slide

342. $ telnet varnish 6082
     Connected to localhost.
     Escape character is '^]'.
     107 59
     cohvooigdtqvkpwewhdxkqiwkfkpwsly
     Authentication required.
     auth 5a9c5722f31cc3c92f0e4616571624df7bddde2f8e42aaffe795dc80fb8c91dd
     200 240
     -----------------------------
     Varnish Cache CLI 1.0
     -----------------------------
     Linux,4.9.49-moby,x86_64,-junix,-smalloc,-smalloc,-hcritbit
     varnish-5.2.0 revision 4c4875cbf
     Type 'help' for command list.
     Type 'quit' to close CLI session.
     ban obj.http.x-host == localhost && obj.http.x-url ~ /products
     200 0
     

     View Slide

343. $challenge = $argv[1];
     $secret = trim(fgets(STDIN));
     $pack = $challenge . "\x0A" . $secret . "\x0A" . $challenge . "\x0A";
     $key = hash('sha256', $pack);
     echo $key.PHP_EOL;
     Use secret & process challenge
     cohvooigdtqvkpwewhdxkqiwkfkpwsly
     5a9c5722f31cc3c92f0e4616571624df7bddde2f8e42aaffe795dc80fb8c91dd
     E29F6DE6-3803-449E-8A01-AA537A8715B1
     

     View Slide

344. $ telnet varnish 6082
     Connected to localhost.
     Escape character is '^]'.
     107 59
     cohvooigdtqvkpwewhdxkqiwkfkpwsly
     Authentication required.
     auth 5a9c5722f31cc3c92f0e4616571624df7bddde2f8e42aaffe795dc80fb8c91dd
     200 240
     -----------------------------
     Varnish Cache CLI 1.0
     -----------------------------
     Linux,4.9.49-moby,x86_64,-junix,-smalloc,-smalloc,-hcritbit
     varnish-5.2.0 revision 4c4875cbf
     Type 'help' for command list.
     Type 'quit' to close CLI session.
     ban obj.http.x-host == localhost && obj.http.x-url ~ /products
     200 0
     

     View Slide

345. Secure your access the the admin socket
     

     View Slide

346. DAEMON_OPTS="-j unix,user=www-data \
     -a :80 \
     -a :81,PROXY \
     -T localhost:6082 \
     -f /etc/varnish/default.vcl \
     -S /etc/varnish/secret \
     -l 100m,10m \
     -t 60 \
     -p feature=+esi_disable_xml_check \
     -p connect_timeout=20s \
     -p first_byte_timeout=100s \
     -p between_bytes_timeout=5s \
     -s malloc,3g"
     

     View Slide

347. Admin socket does
     more than banning
     

     View Slide

348. -----------------------------
     Varnish Cache CLI 1.0
     -----------------------------
     Linux,4.9.49-moby,x86_64,-junix,-smalloc,-smalloc,-hcritbit
     varnish-5.2.0 revision 4c4875cbf
     Type 'help' for command list.
     Type 'quit' to close CLI session.
     help
     200 613
     auth
     backend.list [-p] []
     backend.set_health [auto|healthy|sick]
     ban [&& ...]
     ban.list
     banner
     help []
     panic.clear [-z]
     panic.show
     param.set
     param.show [-l] []
     ping []
     quit
     start
     status
     stop
     storage.list
     vcl.discard
     vcl.inline [auto|cold|warm]
     vcl.label
     vcl.list
     vcl.load [auto|cold|warm]
     vcl.show [-v]
     vcl.state [auto|cold|warm]
     vcl.use
     

     View Slide

349. Refresh content
     

     View Slide

350. vcl 4.0;
     acl refreshers {
     "172.18.0.0"/24;
     "localhost";
     }
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_recv {
     if (req.method == "REFRESH") {
     if (client.ip !~ refreshers) {
     return (synth(405, "Method not allowed"));
     }
     set req.method = "GET";
     set req.hash_always_miss = true;
     }
     }
     Refresh content
     Perform miss,
     fetch content,
     overwrite object
     

     View Slide

351. Refresh content
     curl -XREFRESH "http://example.com/products"
     

     View Slide

352. Tag-base invalidation
     

     View Slide

353. vmod_xkey
     

     View Slide

354. vcl 4.0;
     import xkey;
     acl purgers {
     "172.18.0.0"/24;
     }
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_recv {
     if (req.method == "PURGE") {
     if (client.ip !~ purgers) {
     return (synth(405, "Method now allowed"));
     }
     set req.http.n-gone = xkey.purge(req.http.key);
     return (synth(200, "Invalidated "+req.http.n-gone+" objects for key "+ req.http.key));
     }
     if (req.method == "SOFTPURGE") {
     if (client.ip !~ purgers) {
     return (synth(405, "Method not allowed"));
     }
     set req.http.n-gone = xkey.softpurge(req.http.key);
     return (synth(200, "Invalidated "+req.http.n-gone+" objects via softpurge for key "+
     req.http.key));
     }
     }
     

     View Slide

355. header('Cache-control: public, s-maxage=100');
     header("xkey: a");
     header("xkey: b",false);
     echo date('Y-m-d H:i:s').'
     '.PHP_EOL;
     Add “xkey” headers
     Register tags
     

     View Slide

356. Invalidate tags
     curl -XPURGE -H"key: b" "http://example.com/"
     PURGE / HTTP/1.1
     Host: localhost
     User-Agent: curl/7.55.1
     Accept: */*
     key:b
     

     View Slide

357. Header-based
     invalidation
     

     View Slide

358. Header-based
     invalidation
     If-Modified-Since: Tue, 14 Jun 2016 11:49:18 GMT
     Last-Modified: Tue, 14 Jun 2016 11:49:32 GMT
     Cache-Control: max-age=10
     HTTP 200: OK HTTP 304: Not modified
     

     View Slide

359. Header-based
     invalidation
     If-None-Match: 57601698ae1e2
     Etag: 57601698ae1e2
     Cache-Control: max-age=10
     HTTP 200: OK HTTP 304: Not modified
     

     View Slide

360. Grace mode
     

     View Slide

361. beresp.grace
     Keep servering
     stale object while
     fetching data from
     backend
     Avoid
     excessive request
     queueing
     

     View Slide

362. sub vcl_backend_response {
     set beresp.grace = 2h;
     }
     Grace
     

     View Slide

363. Cache-Control: public, max-age=100, s-maxage=3600,
     stale-while-revalidate=7200
     Stale-While-Revalidate
     

     View Slide

364. sub vcl_backend_response {
     set beresp.grace = 6h;
     }
     Grace mode
     Monitoring & logging
     

     View Slide

365. Varnishstat
     Realtime
     statistics
     

     View Slide

366. View Slide

367. usage: varnishstat [-1lV] [-f field] [-t seconds|] [-n varnish_name] [-N filename]
     -1 # Print the statistics to stdout.
     -f field # Field inclusion glob
     # If it starts with '^' it is used as an exclusion list.
     -l # Lists the available fields to use with the -f option.
     -n varnish_name # The varnishd instance to get logs from.
     -N filename # Filename of a stale VSM instance.
     -t seconds| # Timeout before returning error on initial VSM connection.
     -V # Display the version number and exit.
     -x # Print statistics to stdout as XML.
     -j # Print statistics to stdout as JSON.
     Varnishstat usage
     

     View Slide

368. ~# varnishstat -f MAIN.cache_hit -1
     MAIN.cache_hit 13049135 5.39 Cache hits
     Varnishstat usage
     ~# varnishstat -f MAIN.cache_hit -j -1
     {
     "timestamp": "2016-06-14T16:10:32",
     "MAIN.cache_hit": {
     "description": "Cache hits",
     "type": "MAIN", "flag": "c", "format": "i",
     "value": 13050992
     }
     }
     

     View Slide

369. Varnishstat usage
     ~# varnishstat -f MAIN.n_object -f MAIN.n_lru_nuked -j
     {
     "timestamp": "2016-06-14T16:14:49",
     "MAIN.n_object": {
     "description": "object structs made",
     "type": "MAIN", "flag": "g", "format": "i",
     "value": 46295
     },
     "MAIN.n_lru_nuked": {
     "description": "Number of LRU nuked objects",
     "type": "MAIN", "flag": "g", "format": "i",
     "value": 0
     }
     }
     

     View Slide

370. varnishstat -f MAIN.cache*
     

     View Slide

371. ✓ Session
     ✓ Client
     ✓ Uptime
     ✓ Hit/miss
     ✓ Backend
     ✓ Fetch
     ✓ Threading
     ✓ Cache
     objects
     ✓ Memory
     ✓ Invalidation
     Varnishstat counters
     

     View Slide

372. VSL
     

     View Slide

373. Varnish
     Shared memory
     Logging
     

     View Slide

374. ✓ In-memory logs
     ✓ Generated by varnishd
     ✓ 81 MB by default
     ✓ Customize with “-l” setting
     ✓ Varnishlog command
     ✓ Varnishtop command
     VSL
     

     View Slide

375. * << Request >> 10973258
     - Begin req 10973257 rxreq
     - Timestamp Start: 1501507281.942533 0.000000 0.000000
     - Timestamp Req: 1501507281.942533 0.000000 0.000000
     - ReqStart 127.0.0.1 59753
     - ReqMethod GET
     - ReqURL /
     - ReqProtocol HTTP/1.1
     - ReqHeader Host: feryn.eu
     

     View Slide

376. Transactions
     

     View Slide

377. ✓ Items of work
     ✓ Identified by VXID
     ✓ 2 kinds:
     ✓ Sessions
     ✓ Requests
     Transactions
     

     View Slide

378. ✓ Identifies TCP
     connection
     ✓ Contains multiple
     requests
     Transactions
     ✓ Client request
     ✓ Backend request
     ✓ ESI subrequest
     Session Request
     

     View Slide

379. View Slide

380. ✓ VXID (default)
     ✓ Session
     ✓ Request
     ✓ Raw
     Transactions grouping
     

     View Slide

381. Example
     composition that will
     be monitored
     

     View Slide

382. varnishlog -i Begin,ReqUrl,Link,BereqURL
     

     View Slide

383. * << BeReq >> 98318
     - Begin bereq 98317 fetch
     - BereqURL /
     * << BeReq >> 98320
     - Begin bereq 98319 fetch
     - BereqURL /header
     * << Request >> 98319
     - Begin req 98317 esi
     - ReqURL /header
     - Link bereq 98320 fetch
     * << BeReq >> 98322
     - Begin bereq 98321 fetch
     - BereqURL /nav
     * << Request >> 98321
     - Begin req 98317 esi
     - ReqURL /nav
     - Link bereq 98322 fetch
     * << Request >> 98317
     - Begin req 98316 rxreq
     - ReqURL /
     - Link bereq 98318 fetch
     - Link req 98319 esi
     - Link req 98321 esi
     * << BeReq >> 98324
     - Begin bereq 98323 fetch
     - BereqURL /footer
     * << Request >> 98323
     - Begin req 98316 rxreq
     - ReqURL /footer
     - Link bereq 98324 fetch
     * << Session >> 98316
     - Begin sess 0 HTTP/1
     - Link req 98317 rxreq
     - Link req 98323 rxreq
     

     View Slide

384. varnishlog -i Begin,ReqUrl,Link,BereqURL -g session
     

     View Slide

385. * << Session >> 14
     - Begin sess 0 HTTP/1
     - Link req 65539 rxreq
     - Link req 65545 rxreq
     ** << Request >> 65539
     -- Begin req 14 rxreq
     -- ReqURL /
     -- Link bereq 65540 fetch
     -- Link req 65541 esi
     -- Link req 65543 esi
     ** << Request >> 65545
     -- Begin req 14 rxreq
     -- ReqURL /footer
     -- Link bereq 65546 fetch
     *** << BeReq >> 65540
     --- Begin bereq 65539 fetch
     --- BereqURL /
     *** << Request >> 65541
     --- Begin req 65539 esi
     --- ReqURL /header
     --- Link bereq 65542 fetch
     *** << Request >> 65543
     --- Begin req 65539 esi
     --- ReqURL /nav
     --- Link bereq 65544 fetch
     *** << BeReq >> 65546
     --- Begin bereq 65545 fetch
     --- BereqURL /footer
     **** << BeReq >> 65542
     ---- Begin bereq 65541 fetch
     ---- BereqURL /header
     **** << BeReq >> 65544
     ---- Begin bereq 65543 fetch
     ---- BereqURL /nav
     

     View Slide

386. View Slide

387. Tags
     

     View Slide

388. Request tags
     

     View Slide

389. ✓ ReqMethod
     ✓ ReqUrl
     ✓ ReqProtocol
     ✓ ReqHeader
     Request tags
     

     View Slide

390. - ReqStart 127.0.0.1 56312
     - ReqMethod GET
     - ReqURL /
     - ReqProtocol HTTP/1.1
     - ReqHeader Host: localhost
     - ReqHeader Connection: keep-alive
     - ReqHeader Upgrade-Insecure-Requests: 1
     - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X
     10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/
     59.0.3071.115 Safari/537.36
     - ReqHeader Accept: text/html,application/xhtml+xml,application/
     xml;q=0.9,image/webp,image/apng,*/*;q=0.8
     - ReqHeader Accept-Encoding: gzip, deflate, br
     - ReqHeader Accept-Language: nl,en-US;q=0.8,en;q=0.6
     - ReqHeader X-Forwarded-For: 127.0.0.1
     

     View Slide

391. Response tags
     

     View Slide

392. ✓ RespProtocol
     ✓ RespStatus
     ✓ RespReason
     ✓ RespHeader
     Response tags
     

     View Slide

393. - RespProtocol HTTP/1.1
     - RespStatus 200
     - RespReason OK
     - RespHeader Host: localhost
     - RespHeader Cache-Control: public, s-maxage=500
     - RespHeader Date: Tue, 01 Aug 2017 08:56:44 GMT
     - RespHeader ETag: "c5afddc587599a72d467caca23e980bf"
     - RespHeader Vary: Accept-Language
     - RespHeader Content-Length: 3098
     - RespHeader Content-Type: text/html; charset=UTF-8
     - RespHeader X-Varnish: 32770
     - RespHeader Age: 10
     

     View Slide

394. Backend tags
     

     View Slide

395. - BackendOpen 19 boot.default 127.0.0.1 8080 127.0.0.1 62552
     - BackendClose 19 boot.default
     

     View Slide

396. Backend request tags
     

     View Slide

397. ✓ BereqMethod
     ✓ BereqUrl
     ✓ BereqProtocol
     ✓ BereqHeader
     Backend request tags
     

     View Slide

398. Backend response
     tags
     

     View Slide

399. ✓ BerespProtocol
     ✓ BerespStatus
     ✓ BerespReason
     ✓ BerespHeader
     Backens response tags
     

     View Slide

400. Object tags
     

     View Slide

401. ✓ ObjProtocol
     ✓ ObjStatus
     ✓ ObjReason
     ✓ ObjHeader
     Object tags
     

     View Slide

402. VCL tags
     

     View Slide

403. ✓ VCL_Call
     ✓ VCL_Return
     ✓ VCL_Error
     ✓ VCL_Log
     ✓ VCL_Acl
     VCL tags
     

     View Slide

404. * << Request >> 5
     - ReqURL /
     - VCL_call RECV
     - VCL_return hash
     - VCL_call HASH
     - VCL_return lookup
     - VCL_call MISS
     - VCL_return fetch
     - VCL_call DELIVER
     - VCL_return deliver
     ** << BeReq >> 6
     -- BereqURL /
     -- VCL_call BACKEND_FETCH
     -- VCL_return fetch
     -- VCL_call BACKEND_RESPONSE
     -- VCL_return deliver
     

     View Slide

405. * << Request >> 6
     - ReqURL /
     - VCL_call RECV
     - ReqURL /
     - VCL_return hash
     - VCL_call HASH
     - VCL_return lookup
     - VCL_call HIT
     - VCL_return deliver
     - VCL_call DELIVER
     - VCL_return deliver
     

     View Slide

406. * << Request >> 32789
     - ReqURL /header
     - ExpBan 98355 banned lookup
     ExpBan tag
     

     View Slide

407. * << Request >> 98369
     - ReqURL /footer
     - Hit 65597
     Hit tag
     varnishlog -i "ReqUrl,Hit"
     

     View Slide

408. ** << Request >> 6
     -- ReqURL /footer
     -- HitPass 3
     HitPass tag
     

     View Slide

409. TTL tag
     

     View Slide

410. %s %d %d %d %d [ %d %d %u %u ]
     | | | | | | | | |
     | | | | | | | | +- Max-Age from Cache-Control header
     | | | | | | | +---- Expires header
     | | | | | | +------- Date header
     | | | | | +---------- Age (incl Age: header value)
     | | | | +--------------- Reference time for TTL
     | | | +------------------ Keep
     | | +--------------------- Grace
     | +------------------------ TTL
     +--------------------------- "RFC" or "VCL"
     TTL tag
     

     View Slide

411. -- TTL VCL 120 10 0 1501597242
     ✓ TTL decided by the VCL
     ✓ 120 seconds cached
     ✓ 10 seconds grace time
     ✓ 0 seconds keep time
     ✓ Reference time: 1501597242

     (2017-08-01 14:20:42)
     

     View Slide

412. -- RFC 500 10 -1 1501598872 1501598872 1501598872 0 500
     ✓ 500 seconds TTL (via headers)
     ✓ 10 seconds grace
     ✓ No keep value
     ✓ 2017-08-01 14:47:52 date, age & reference time
     ✓ Cache-control headers sets 500 second TTL
     

     View Slide

413. -- RFC 500 10 -1 1501598872 1501598869 1501598872 0 500
     ✓ Don’t screw with the age header
     ✓ Only 497 second effective TTL
     ✓ Custom Age header (3 seconds off)
     

     View Slide

414. Begin tag
     

     View Slide

415. - Begin bereq 98317 fetch
     - Begin req 98317 esi
     - Begin req 98316 rxreq
     - Begin sess 0 HTTP/1
     

     View Slide

416. Link tag
     

     View Slide

417. - Link bereq 98312 fetch
     - Link req 98311 rxreq
     - Link req 98318 esi
     

     View Slide

418. Timestamp tag
     

     View Slide

419. %s: %f %f %f
     | | | |
     | | | +- Time since last timestamp
     | | +---- Time since start of work unit
     | +------- Absolute time of event
     +----------- Event label
     Timestamp tag
     

     View Slide

420. * << Request >> 65539
     - Timestamp Start: 1501601912.758662 0.000000 0.000000
     - Timestamp Req: 1501601912.758662 0.000000 0.000000
     - Timestamp Fetch: 1501601912.806733 0.048071 0.048071
     - Timestamp Process: 1501601912.806750 0.048088 0.000017
     - Timestamp Resp: 1501601912.806787 0.048125 0.000037
     ** << BeReq >> 65540
     -- Timestamp Start: 1501601912.758753 0.000000 0.000000
     -- Timestamp Bereq: 1501601912.758952 0.000199 0.000199
     -- Timestamp Beresp: 1501601912.806677 0.047924 0.047725
     -- Timestamp BerespBody: 1501601912.806749 0.047996 0.000072
     

     View Slide

421. Filtering output
     

     View Slide

422. ✓ -i: include tags
     ✓ -I: include tags by regex
     ✓ -x: exclude tags
     ✓ -X: exclude by regex
     Filtering output
     

     View Slide

423. Include tags
     

     View Slide

424. varnishlog -i ReqUrl,VCL_call,VCL_return -g session
     varnishlog -i "ReqUrl,VCL_*" -g session
     

     View Slide

425. * << Session >> 252394
     ** << Request >> 252395
     -- ReqURL /
     -- VCL_call RECV
     -- VCL_return hash
     -- VCL_call HASH
     -- VCL_return lookup
     -- VCL_call HIT
     -- VCL_return deliver
     -- VCL_call DELIVER
     -- VCL_return deliver
     *** << Request >> 252397
     --- ReqURL /header
     --- VCL_call RECV
     --- VCL_return hash
     --- VCL_call HASH
     --- VCL_return lookup
     --- VCL_call HIT
     --- VCL_return deliver
     --- VCL_call DELIVER
     --- VCL_return deliver
     *** << Request >> 252399
     --- ReqURL /nav
     --- VCL_call RECV
     --- VCL_return hash
     --- VCL_call HASH
     --- VCL_return lookup
     --- VCL_call HIT
     --- VCL_return deliver
     --- VCL_call DELIVER
     --- VCL_return deliver
     *** << BeReq >> 252396
     --- VCL_call BACKEND_FETCH
     --- VCL_return fetch
     --- VCL_call BACKEND_RESPONSE
     --- VCL_return deliver
     **** << BeReq >> 252398
     ---- VCL_call BACKEND_FETCH
     ---- VCL_return fetch
     ---- VCL_call BACKEND_RESPONSE
     ---- VCL_return deliver
     **** << BeReq >> 252400
     ---- VCL_call BACKEND_FETCH
     ---- VCL_return fetch
     ---- VCL_call BACKEND_RESPONSE
     ---- VCL_return deliver
     

     View Slide

426. Exclude tags
     

     View Slide

427. varnishlog -i "Req*" -x ReqHeader,ReqUnset
     

     View Slide

428. * << Request >> 314125
     - ReqStart 127.0.0.1 64585
     - ReqMethod GET
     - ReqURL /
     - ReqProtocol HTTP/1.1
     - ReqAcct 476 0 476 311 0 311
     * << Request >> 314126
     - ReqStart 127.0.0.1 64585
     - ReqMethod GET
     - ReqURL /footer
     - ReqProtocol HTTP/1.1
     - ReqAcct 370 0 370 309 0 309
     

     View Slide

429. Include tags by regex
     

     View Slide

430. varnishlog -I "reqheader:Accept-Language" -i requrl
     

     View Slide

431. * << Request >> 374378
     - ReqURL /
     - ReqHeader Accept-Language: nl,en-US;q=0.8,en;q=0.6
     * << Request >> 374379
     - ReqURL /footer
     - ReqHeader Accept-Language: nl,en-US;q=0.8,en;q=0.6
     

     View Slide

432. Exclude tags by regex
     

     View Slide

433. * << Request >> 374384
     - ReqURL /footer
     - RespHeader Host: localhost
     - RespHeader X-Powered-By: PHP/7.0.15
     - RespHeader Cache-Control: public, s-maxage=500
     - RespHeader Date: Wed, 02 Aug 2017 11:27:21 GMT
     - RespHeader ETag: "d47ac09f5351f8f4c97c99ef5b3d2ecd"
     - RespHeader Vary: Accept-Language
     - RespHeader Content-Length: 80
     - RespHeader Content-Type: text/html; charset=UTF-8
     - RespHeader X-Varnish: 374384 15367
     - RespHeader Age: 334
     - RespHeader Via: 1.1 varnish-v4
     - RespHeader Connection: keep-alive
     

     View Slide

434. varnishlog -i RespHeader,ReqUrl -X "RespHeader:(x|X)-"
     

     View Slide

435. * << Request >> 374384
     - ReqURL /footer
     - RespHeader Host: localhost
     - RespHeader Cache-Control: public, s-maxage=500
     - RespHeader Date: Wed, 02 Aug 2017 11:27:21 GMT
     - RespHeader ETag: "d47ac09f5351f8f4c97c99ef5b3d2ecd"
     - RespHeader Vary: Accept-Language
     - RespHeader Content-Length: 80
     - RespHeader Content-Type: text/html; charset=UTF-8
     - RespHeader Age: 334
     - RespHeader Via: 1.1 varnish-v4
     - RespHeader Connection: keep-alive
     

     View Slide

436. All-in-one
     

     View Slide

437. varnishlog -i "RespHeader,Req*" -X "RespHeader:(x|X)-" -I
     "timestamp:Resp" -x reqprotocol,reqacct -g request
     

     View Slide

438. * << Request >> 59383
     - ReqStart 127.0.0.1 53195
     - ReqMethod GET
     - ReqURL /
     - ReqHeader Host: localhost
     - ReqHeader Connection: keep-alive
     - ReqHeader Cache-Control: max-age=0
     - ReqHeader Upgrade-Insecure-Requests:
     1
     - ReqHeader User-Agent: Mozilla/5.0
     (Macintosh; Intel Mac OS X 10_12_5)
     AppleWebKit/537.36 (KHTML, like Gecko)
     Chrome/59.0.3071.115 Safari/537.36
     - ReqHeader Accept: text/
     html,application/
     xhtml+xml,application/xml;q=0.9,image/
     webp,image/apng,*/*;q=0.8
     - ReqHeader Accept-Encoding: gzip,
     deflate, br
     - ReqHeader Accept-Language: nl,en-
     US;q=0.8,en;q=0.6
     - ReqHeader If-None-Match:
     W/"27f341f8e459dd35f1087c55351cacda"
     - ReqHeader X-Forwarded-For: 127.0.0.1
     - ReqUnset Accept-Language: nl,en-
     US;q=0.8,en;q=0.6
     - ReqHeader accept-language: nl
     - ReqHeader Surrogate-Capability:
     key=ESI/1.0
     - ReqUnset Accept-Encoding: gzip,
     deflate, br
     - ReqHeader Accept-Encoding: gzip
     - RespHeader Host: localhost
     - RespHeader Cache-Control: public, s-
     maxage=500
     - RespHeader Date: Wed, 02 Aug 2017
     11:46:49 GMT
     - RespHeader ETag:
     "27f341f8e459dd35f1087c55351cacda"
     - RespHeader Vary: Accept-Language
     - RespHeader Content-Length: 3098
     - RespHeader Content-Type: text/html;
     charset=UTF-8
     - RespHeader Age: 152
     - RespHeader Via: 1.1 varnish-v4
     - RespHeader ETag:
     W/"27f341f8e459dd35f1087c55351cacda"
     - RespHeader Connection: keep-alive
     - Timestamp Resp: 1501674561.472358
     0.000068 0.000020
     

     View Slide

439. ✓ All response headers
     ✓ All tags that start with “Req”
     ✓ Exclude “x-“ response headers
     ✓ Include “response time” timestamp
     ✓ Exclude request protocol log lines,
     and the request accountancy log
     lines
     All-in-one
     

     View Slide

440. VSL queries
     

     View Slide

441. Filtering fields from
     all transactions
     Filtering transactions
     

     View Slide

442. 
     {level}taglist:record-prefix[field]
     

     View Slide

443. ReqUrl eq ‘/‘
     {level}taglist:record-prefix[field]
     

     View Slide

444. Timestamp:Resp[2] > 1.0
     {level}taglist:record-prefix[field]
     

     View Slide

445. {2}Timestamp:Resp[2] > 1.0
     {level}taglist:record-prefix[field]
     

     View Slide

446. varnishlog -i VCL_call,VCL_return -g request -q "ReqURL eq '/'"
     

     View Slide

447. * << Request >> 374400
     - VCL_call RECV
     - VCL_return hash
     - VCL_call HASH
     - VCL_return lookup
     - VCL_call HIT
     - VCL_return deliver
     - VCL_call DELIVER
     - VCL_return deliver
     

     View Slide

448. varnishlog -i ReqUrl -q "VCL_call eq 'MISS' or VCL_call eq 'PASS'"
     varnishlog -i ReqUrl -I "Timestamp:Resp" -q "Timestamp:Resp[2] > 1.0"
     

     View Slide

449. Other options
     

     View Slide

450. varnishlog -n myVarnishInstance
     varnishlog -d
     varnishlog -w file
     varnishlog -r file
     varnishlog -A -a -w file
     varnishlog -i "ReqUrl,VCL_*" -D -a -A -w /var/log/varnish/custom.log -
     P /var/run/custom_varnishlog.pid
     

     View Slide

451. Varnishtop
     

     View Slide

452. Same syntax as varnishlog.
     Output is incremental
     

     View Slide

453. $ varnishtop -I reqheader:Accept-Language -q "ReqUrl eq '/'"
     list length 4
     0.86 ReqHeader Accept-Language: en-US
     0.80 ReqHeader Accept-Language: nl-NL,nl;q=0.8,en-
     US;q=0.6,en;q=0.4
     0.54 ReqHeader Accept-Language: nl,en-US;q=0.8,en;q=0.6
     0.39 ReqHeader Accept-Language: nl-BE
     

     View Slide

454. Common Varnishlog
     scenarios
     

     View Slide

455. Why didn’t Varnish serve this request from cache?
     

     View Slide

456. Request not
     cacheable
     

     View Slide

457. varnishlog -i "Req*,VCL*" -x ReqAcct,ReqStart -q "VCL_call
     eq 'PASS'"
     

     View Slide

458. * << Request >> 12
     - ReqMethod GET
     - ReqURL /
     - ReqProtocol HTTP/1.1
     - ReqHeader Host: localhost
     - ReqHeader User-Agent: curl/7.48.0
     - ReqHeader Accept: */*
     - ReqHeader Cookie: myCookie=bla
     - ReqHeader X-Forwarded-For: 127.0.0.1
     - VCL_call RECV
     - VCL_return pass
     - VCL_call HASH
     - VCL_return lookup
     - VCL_call PASS
     - VCL_return fetch
     - VCL_call DELIVER
     - VCL_return deliver
     

     View Slide

459. << Request >> 32779
     - ReqMethod POST
     - ReqURL /
     - ReqProtocol HTTP/1.1
     - ReqHeader Host: localhost
     - ReqHeader User-Agent: curl/7.48.0
     - ReqHeader Accept: */*
     - ReqHeader X-Forwarded-For: 127.0.0.1
     - VCL_call RECV
     - VCL_return pass
     - VCL_call HASH
     - VCL_return lookup
     - VCL_call PASS
     - VCL_return fetch
     - VCL_call DELIVER
     - VCL_return deliver
     

     View Slide

460. * << Request >> 15
     - ReqMethod GET
     - ReqURL /
     - ReqProtocol HTTP/1.1
     - ReqHeader Host: localhost
     - ReqHeader Authorization: Basic dGhpanM6ZmVyeW4=
     - ReqHeader User-Agent: curl/7.48.0
     - ReqHeader Accept: */*
     - ReqHeader X-Forwarded-For: 127.0.0.1
     - VCL_call RECV
     - VCL_return pass
     - VCL_call HASH
     - VCL_return lookup
     - VCL_call PASS
     - VCL_return fetch
     - VCL_call DELIVER
     - VCL_return deliver
     

     View Slide

461. Response not
     cacheable
     

     View Slide

462. varnishlog -i ReqUrl,VCL_*,Beresp*,TTL -q "HitPass" -g
     request
     

     View Slide

463. * << Request >> 19010384
     - ReqURL /my-url
     - VCL_call RECV
     - VCL_return hash
     - VCL_call HASH
     - VCL_return lookup
     - VCL_call PASS
     - VCL_return fetch
     - VCL_call DELIVER
     - VCL_return deliver
     ** << BeReq >> 19010385
     -- VCL_call BACKEND_FETCH
     -- VCL_return fetch
     -- BerespProtocol HTTP/1.1
     -- BerespStatus 200
     -- BerespReason OK
     -- BerespHeader Date: Thu, 03 Aug
     2017 08:15:22 GMT
     -- BerespHeader Server: Apache/
     2.4.10 (Debian)
     -- BerespHeader Last-Modified: Tue,
     01 Aug 2017 07:21:00 GMT
     -- BerespHeader ETag:
     "5c0d-555abfd3f422f-gzip"
     -- BerespHeader Vary: Accept-
     Encoding
     -- BerespHeader Content-Encoding:
     gzip
     -- BerespHeader Cache-Control: max-
     age=0
     -- BerespHeader Expires: Thu, 03 Aug
     2017 08:15:22 GMT
     -- BerespHeader Content-Length: 7686
     -- BerespHeader Content-Type:
     application/json
     -- TTL RFC 0 10 -1 1501748123
     1501748123 1501748122 1501748122 0
     -- VCL_call BACKEND_RESPONSE
     -- TTL VCL 120 10 0 1501748123
     -- VCL_return deliver
     

     View Slide

464. * << Request >> 65551
     - ReqURL /set-cookie
     - VCL_call RECV
     - VCL_return hash
     - VCL_call HASH
     - VCL_return lookup
     - VCL_call PASS
     - VCL_return fetch
     - VCL_call DELIVER
     - VCL_return deliver
     ** << BeReq >> 65552
     -- VCL_call BACKEND_FETCH
     -- VCL_return fetch
     -- BerespProtocol HTTP/1.1
     -- BerespStatus 200
     -- BerespReason OK
     -- BerespHeader Cache-control: s-maxage=10
     -- BerespHeader Set-Cookie: myCookie=bla
     -- BerespHeader Content-type: text/html; charset=UTF-8
     -- BerespHeader Date: Thu, 03 Aug 2017 08:39:04 GMT
     -- TTL RFC 10 10 -1 1501749545 1501749545 1501749544 0 10
     -- VCL_call BACKEND_RESPONSE
     -- TTL VCL 120 10 0 1501749545
     -- VCL_return deliver
     

     View Slide

465. Authentication
     

     View Slide

466. Authorization: Basic QWxhZGRpbjpPcGVuU2VzYW1l
     

     View Slide

467. PHPSESSID: laken37oumlkrndoo91k3trlv5
     

     View Slide

468. vcl 4.0;
     sub vcl_recv {
     if (req.method == "PRI") {
     return (synth(405));
     }
     if (req.method != "GET" &&
     req.method != "HEAD" &&
     req.method != "PUT" &&
     req.method != "POST" &&
     req.method != "TRACE" &&
     req.method != "OPTIONS" &&
     req.method != “PATCH" &&
     req.method != "DELETE") {
     return (pipe);
     }
     if (req.method != "GET" && req.method != "HEAD") {
     return (pass);
     }
     if (req.http.Authorization || req.http.Cookie) {
     return (pass);
     }
     return (hash);
     }
     

     View Slide

469. View Slide

470. What if we
     could create
     cache
     variations for
     logged-in
     users?
     

     View Slide

471. How do you identify a
     logged-in user without
     accessing the backend
     every time?
     

     View Slide

472. Push session
     information from
     the server
     to the client
     

     View Slide

473. JSON Web Tokens
     

     View Slide

474. Marco Pivetta
     introduced
     me to JWT
     https://twitter.com/ocramius
     

     View Slide

475. JWT
     eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOi
     JodHRwOlwvXC9sb2NhbGhvc3RcLyIsImlhdCI6MTUwODc0N
     zM2MywiZXhwIjoxNTA4NzUwOTYzLCJ1aWQiOjEsInVzZXJu
     YW1lIjoiVGhpanMifQ.EPemqBaH74Sbs0bZqAaR8i7uVYO3
     89VOlJvWC3ocL7g
     ✓3 parts
     ✓Dot separated
     ✓Base64 encoded JSON
     ✓Header
     ✓Payload
     ✓Signature (HMAC with secret)
     

     View Slide

476. eyJpc3MiOiJodHRwOlwvXC
     9sb2NhbGhvc3RcLyIsImlh
     dCI6MTUwODc0NzM2MywiZX
     hwIjoxNTA4NzUwOTYzLCJ1
     aWQiOjEsInVzZXJuYW1lIj
     oiVGhpanMifQ
     {
     "alg": "HS256",
     "typ": "JWT"
     }
     {
     "iss": "http://localhost/",
     "iat": 1508746861,
     "exp": 1508750461,
     "uid": 1,
     "username": "Thijs"
     }
     HMACSHA256(
     base64UrlEncode(header) + "." +
     base64UrlEncode(payload),
     secret
     )
     eyJ0eXAiOiJKV1QiLCJhb
     GciOiJIUzI1NiJ9
     EPemqBaH74Sbs0bZqAaR8i
     7uVYO389VOlJvWC3ocL7g
     

     View Slide

477. https://jwt.io
     

     View Slide

478. View Slide

479. JWT
     Cookie:jwt_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.
     eyJpc3MiOiJodHRwOlwvXC9sb2NhbGhvc3RcLyIsImlhdCI6MTUwOD
     c0NzM2MywiZXhwIjoxNTA4NzUwOTYzLCJ1aWQiOjEsInVzZXJuYW1l
     IjoiVGhpanMifQ.EPemqBaH74Sbs0bZqAaR8i7uVYO389VOlJvWC3o
     cL7g
     ✓Stored in a cookie
     ✓Can be validated by Varnish
     ✓Payload can be processed by any
     language (e.g. Javascript)
     

     View Slide

480. Application creates JWT
     alongside the regular
     session
     

     View Slide

481. Template loads stateful
     data from JWT
     in Javascript
     

     View Slide

482. {
     "iss": "http://localhost/",
     "iat": 1508746861,
     "exp": 1508750461,
     "uid": 1,
     "username": "Thijs"
     }
     Issuer
     

     View Slide

483. {
     "iss": "http://localhost/",
     "iat": 1508746861,
     "exp": 1508750461,
     "uid": 1,
     "username": "Thijs"
     }
     Issued at
     

     View Slide

484. {
     "iss": "http://localhost/",
     "iat": 1508746861,
     "exp": 1508750461,
     "uid": 1,
     "username": "Thijs"
     }
     Expires
     at
     

     View Slide

485. {
     "iss": "http://localhost/",
     "iat": 1508746861,
     "exp": 1508750461,
     "uid": 1,
     "username": "Thijs"
     }
     User ID
     

     View Slide

486. {
     "iss": "http://localhost/",
     "iat": 1508746861,
     "exp": 1508750461,
     "uid": 1,
     "username": "Thijs"
     }
     Username
     

     View Slide

487. {
     "iss": "http://localhost/",
     "iat": 1508746861,
     "exp": 1508750461,
     "uid": 1,
     "username": "Thijs"
     }
     Let Javascript
     parse this one
     

     View Slide

488. User Varnish Server
     Issues JWT
     Validates
     JWT
     Reads JWT
     Knows
     secret key
     Knows
     secret key
     Does
     not know secret
     key
     

     View Slide

489. Using vmod_digest for
     HMAC & base64
     

     View Slide

490. vcl 4.0;
     import std;
     import var;
     import cookie;
     import digest;
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_recv {
     if (req.http.Cookie) {
     cookie.parse(req.http.cookie);
     cookie.filter_except("jwt_cookie");
     set req.http.cookie = cookie.get_string();
     if (req.http.Cookie ~ "^\s*$") {
     unset req.http.Cookie;
     }
     }
     if ((req.method != "GET" && req.method != "HEAD") || req.http.Authorization) {
     return (pass);
     }
     call jwt;
     return(hash);
     }
     

     View Slide

491. vcl 4.0;
     import std;
     import var;
     import cookie;
     import digest;
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_recv {
     if (req.http.Cookie) {
     cookie.parse(req.http.cookie);
     cookie.filter_except("jwt_cookie");
     set req.http.cookie = cookie.get_string();
     if (req.http.Cookie ~ "^\s*$") {
     unset req.http.Cookie;
     }
     }
     if ((req.method != "GET" && req.method != "HEAD") || req.http.Authorization) {
     return (pass);
     }
     call jwt;
     return(hash);
     }
     

     View Slide

492. vcl 4.0;
     import std;
     import var;
     import cookie;
     import digest;
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_recv {
     if (req.http.Cookie) {
     cookie.parse(req.http.cookie);
     cookie.filter_except("jwt_cookie");
     set req.http.cookie = cookie.get_string();
     if (req.http.Cookie ~ "^\s*$") {
     unset req.http.Cookie;
     }
     }
     if ((req.method != "GET" && req.method != "HEAD") || req.http.Authorization) {
     return (pass);
     }
     call jwt;
     return(hash);
     }
     

     View Slide

493. vcl 4.0;
     import std;
     import var;
     import cookie;
     import digest;
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_recv {
     if (req.http.Cookie) {
     cookie.parse(req.http.cookie);
     cookie.filter_except("jwt_cookie");
     set req.http.cookie = cookie.get_string();
     if (req.http.Cookie ~ "^\s*$") {
     unset req.http.Cookie;
     }
     }
     if ((req.method != "GET" && req.method != "HEAD") || req.http.Authorization) {
     return (pass);
     }
     call jwt;
     return(hash);
     }
     

     View Slide

494. vcl 4.0;
     import std;
     import var;
     import cookie;
     import digest;
     backend default {
     .host = "nginx";
     .port = "80";
     }
     sub vcl_recv {
     if (req.http.Cookie) {
     cookie.parse(req.http.cookie);
     cookie.filter_except("jwt_cookie");
     set req.http.cookie = cookie.get_string();
     if (req.http.Cookie ~ "^\s*$") {
     unset req.http.Cookie;
     }
     }
     if ((req.method != "GET" && req.method != "HEAD") || req.http.Authorization) {
     return (pass);
     }
     call jwt;
     return(hash);
     }
     

     View Slide

495. sub vcl_synth {
     #301 & 302 synths should be actual redirects
     if (resp.status == 301 || resp.status == 302) {
     set resp.http.location = resp.reason;
     set resp.reason = "Moved";
     return (deliver);
     }
     }
     

     View Slide

496. sub jwt {
     var.set("key",std.fileread("/etc/varnish/jwt.key"));
     std.log("Ready to perform some JWT magic");
     if(cookie.isset("jwt_cookie")) {
     var.set("token", cookie.get("jwt_cookie"));
     var.set("header", regsub(var.get("token"),"([^\.]+)\.[^\.]+\.[^\.]+","\1"));
     var.set("type", regsub(digest.base64url_decode(var.get("header")),{"^.*?"typ"\s*:
     \s*"(\w+)".*?$"},"\1"));
     var.set("algorithm", regsub(digest.base64url_decode(var.get("header")),
     {"^.*?"alg"\s*:\s*"(\w+)".*?$"},"\1"));
     if(var.get("type") == "JWT" && var.get("algorithm") == "HS256") {
     var.set("rawPayload",regsub(var.get("token"),"[^\.]+\.([^\.]+)\.[^\.]+$","\1"));
     var.set("signature",regsub(var.get("token"),"^[^\.]+\.[^\.]+\.([^\.]+)$","\1"));
     var.set("currentSignature",digest.base64url_nopad_hex(digest.hmac_sha256(var.get("key"),
     var.get("header") + "." + var.get("rawPayload"))));
     var.set("payload", digest.base64url_decode(var.get("rawPayload")));
     var.set("exp",regsub(var.get("payload"),{"^.*?"exp"\s*:\s*([0-9]+).*?$"},"\1"));
     var.set("userId",regsub(var.get("payload"),{"^.*?"uid"\s*:\s*([0-9]+).*?
     $"},"\1"));
     if(var.get("userId") ~ "^\d+$") {
     if(std.time(var.get("exp"),now) >= now) {
     if(var.get("signature") == var.get("currentSignature")) {
     set req.http.X-Login="true";
     } else {
     std.log("JWT: signature doesn't match. Received: " +
     var.get("signature") + ", expected: " + var.get("currentSignature"));
     

     View Slide

497. sub jwt {
     var.set("key",std.fileread("/etc/varnish/jwt.key"));
     std.log("Ready to perform some JWT magic");
     if(cookie.isset("jwt_cookie")) {
     var.set("token", cookie.get("jwt_cookie"));
     var.set("header", regsub(var.get("token"),"([^\.]+)\.[^\.]+\.[^\.]+","\1"));
     var.set("type", regsub(digest.base64url_decode(var.get("header")),{"^.*?"typ"\s*:
     \s*"(\w+)".*?$"},"\1"));
     var.set("algorithm", regsub(digest.base64url_decode(var.get("header")),
     {"^.*?"alg"\s*:\s*"(\w+)".*?$"},"\1"));
     if(var.get("type") == "JWT" && var.get("algorithm") == "HS256") {
     var.set("rawPayload",regsub(var.get("token"),"[^\.]+\.([^\.]+)\.[^\.]+$","\1"));
     var.set("signature",regsub(var.get("token"),"^[^\.]+\.[^\.]+\.([^\.]+)$","\1"));
     var.set("currentSignature",digest.base64url_nopad_hex(digest.hmac_sha256(var.get("key"),
     var.get("header") + "." + var.get("rawPayload"))));
     var.set("payload", digest.base64url_decode(var.get("rawPayload")));
     var.set("exp",regsub(var.get("payload"),{"^.*?"exp"\s*:\s*([0-9]+).*?$"},"\1"));
     var.set("userId",regsub(var.get("payload"),{"^.*?"uid"\s*:\s*([0-9]+).*?
     $"},"\1"));
     if(var.get("userId") ~ "^\d+$") {
     if(std.time(var.get("exp"),now) >= now) {
     if(var.get("signature") == var.get("currentSignature")) {
     set req.http.X-Login="true";
     } else {
     std.log("JWT: signature doesn't match. Received: " +
     var.get("signature") + ", expected: " + var.get("currentSignature"));
     

     View Slide

498. var.set("header", regsub(var.get("token"),"([^\.]+)\.[^\.]+\.[^\.]+","\1"));
     First chunk
     contains the
     header
     

     View Slide

499. var.set("type", regsub(digest.base64url_decode(var.get("header")),
     {"^.*?"typ"\s*:\s*"(\w+)".*?$"},"\1"));
     Get type field
     from header
     

     View Slide

500. var.set("algorithm", regsub(digest.base64url_decode(var.get("header")),
     {"^.*?"alg"\s*:\s*"(\w+)".*?$"},"\1"));
     Get algorithm
     field from header
     

     View Slide

501. sub jwt {
     var.set("key",std.fileread("/etc/varnish/jwt.key"));
     std.log("Ready to perform some JWT magic");
     if(cookie.isset("jwt_cookie")) {
     var.set("token", cookie.get("jwt_cookie"));
     var.set("header", regsub(var.get("token"),"([^\.]+)\.[^\.]+\.[^\.]+","\1"));
     var.set("type", regsub(digest.base64url_decode(var.get("header")),{"^.*?"typ"\s*:
     \s*"(\w+)".*?$"},"\1"));
     var.set("algorithm", regsub(digest.base64url_decode(var.get("header")),
     {"^.*?"alg"\s*:\s*"(\w+)".*?$"},"\1"));
     if(var.get("type") == "JWT" && var.get("algorithm") == "HS256") {
     var.set("rawPayload",regsub(var.get("token"),"[^\.]+\.([^\.]+)\.[^\.]+$","\1"));
     var.set("signature",regsub(var.get("token"),"^[^\.]+\.[^\.]+\.([^\.]+)$","\1"));
     var.set("currentSignature",digest.base64url_nopad_hex(digest.hmac_sha256(var.get("key"),
     var.get("header") + "." + var.get("rawPayload"))));
     var.set("payload", digest.base64url_decode(var.get("rawPayload")));
     var.set("exp",regsub(var.get("payload"),{"^.*?"exp"\s*:\s*([0-9]+).*?$"},"\1"));
     var.set("userId",regsub(var.get("payload"),{"^.*?"uid"\s*:\s*([0-9]+).*?
     $"},"\1"));
     if(var.get("userId") ~ "^\d+$") {
     if(std.time(var.get("exp"),now) >= now) {
     if(var.get("signature") == var.get("currentSignature")) {
     set req.http.X-Login="true";
     } else {
     std.log("JWT: signature doesn't match. Received: " +
     var.get("signature") + ", expected: " + var.get("currentSignature"));
     

     View Slide

502. var.set(“rawPayload",regsub(var.get("token"),"[^\.]+\.([^\.]+)\.[^\.]+
     $","\1"));
     var.set("payload", digest.base64url_decode(var.get("rawPayload")));
     Second chunk
     contains the raw
     payload
     Base64
     decode to get
     JSON payload
     

     View Slide

503. var.set("signature",regsub(var.get("token"),"^[^\.]+\.[^\.]+\.([^\.]+)
     $","\1"));
     var.set("currentSignature",digest.base64url_nopad_hex(digest.hmac_sha256(var
     .get("key"),var.get("header") + "." + var.get("rawPayload"))));
     Third chunk
     contains the raw
     signature
     Build
     signature using
     key, header &
     payload
     

     View Slide

504. var.set("exp",regsub(var.get("payload"),{"^.*?"exp"\s*:\s*([0-9]+).*?
     $"},"\1"));
     var.set("userId",regsub(var.get("payload"),{"^.*?"uid"\s*:\s*([0-9]+).*?
     $"},"\1"));
     Extract
     expiration time
     from JSON
     payload
     Extract user ID
     from JSON
     payload
     

     View Slide

505. var.get("header") + "." + var.get("rawPayload"))));
     var.set("payload", digest.base64url_decode(var.get("rawPayload")));
     var.set("exp",regsub(var.get("payload"),{"^.*?"exp"\s*:\s*([0-9]+).*?$"},"\1"));
     var.set("userId",regsub(var.get("payload"),{"^.*?"uid"\s*:\s*([0-9]+).*?
     $"},"\1"));
     if(var.get("userId") ~ "^\d+$") {
     if(std.time(var.get("exp"),now) >= now) {
     if(var.get("signature") == var.get("currentSignature")) {
     set req.http.X-Login="true";
     } else {
     std.log("JWT: signature doesn't match. Received: " +
     var.get("signature") + ", expected: " + var.get("currentSignature"));
     }
     } else {
     std.log("JWT: token has expired");
     }
     } else {
     std.log("UserId '"+ var.get("userId") +"', is not numeric");
     }
     } else {
     std.log("JWT: type is not JWT or algorithm is not HS256");
     }
     std.log("JWT processing finished. UserId: " + var.get("userId") + ". X-Login: " +
     req.http.X-login);
     }
     if(req.url ~ "^/(\?[^\?]+)?$" && req.http.X-Login != "true") {
     return(synth(302,"/post.php"));
     }
     }
     

     View Slide

506. var.get("header") + "." + var.get("rawPayload"))));
     var.set("payload", digest.base64url_decode(var.get("rawPayload")));
     var.set("exp",regsub(var.get("payload"),{"^.*?"exp"\s*:\s*([0-9]+).*?$"},"\1"));
     var.set("userId",regsub(var.get("payload"),{"^.*?"uid"\s*:\s*([0-9]+).*?
     $"},"\1"));
     if(var.get("userId") ~ "^\d+$") {
     if(std.time(var.get("exp"),now) >= now) {
     if(var.get("signature") == var.get("currentSignature")) {
     set req.http.X-Login="true";
     } else {
     std.log("JWT: signature doesn't match. Received: " +
     var.get("signature") + ", expected: " + var.get("currentSignature"));
     }
     } else {
     std.log("JWT: token has expired");
     }
     } else {
     std.log("UserId '"+ var.get("userId") +"', is not numeric");
     }
     } else {
     std.log("JWT: type is not JWT or algorithm is not HS256");
     }
     std.log("JWT processing finished. UserId: " + var.get("userId") + ". X-Login: " +
     req.http.X-login);
     }
     if(req.url ~ "^/(\?[^\?]+)?$" && req.http.X-Login != "true") {
     return(synth(302,"/post.php"));
     }
     }
     

     View Slide

507. Cache variations
     

     View Slide

508. Vary: X-Login
     Vary
     header sent by
     application
     X-Login
     header set by
     Varnish
     Creates
     cache variations
     in Varnish
     

     View Slide

509. Write
     cacheable
     applications
     

     View Slide

510. ✓ Cache-control
     ✓ Content negotiation
     ✓ Cache variations
     ✓ Conditional requests
     ✓ Content composition
     ✓ JWT authentication
     Checklist
     

     View Slide

511. Let’s build a cacheable Symfony app
     

     View Slide

512. View Slide

513. https://feryn.eu
     https://twitter.com/ThijsFeryn
     https://instagram.com/ThijsFeryn
     

     View Slide

514. View Slide