Share
Embed
Leverage HTTP to deliver cacheable websites - Gears 2017
More info about this presentation: https://talks.feryn.eu/talks/169/leverage-http-to-deliver-cacheable-websites-gears-gent
Thijs Feryn PRO
May 24, 2017
Transcript
88 slides
-
HTTP caching mechanisms Expires: Sat, 09 Sep 2017 14:30:00 GMT
Cache-control: public, max-age=3600, s-maxage=86400 Cache-control: private, no-cache, no-store -
✓Multi-lingual (Accept-Language) ✓Nav ✓Header ✓Footer ✓Main ✓Login page & private
content Introducing the app -
<?php require_once __DIR__ . '/../vendor/autoload.php'; use Symfony\Component\HttpFoundation\Response; use Symfony\Component\HttpFoundation\RedirectResponse; use
Symfony\Component\HttpFoundation\Request; use Symfony\Component\Translation\Loader\YamlFileLoader; $app = new Silex\Application(); $app['locale'] = 'en'; $app->register(new Silex\Provider\TwigServiceProvider(), ['twig.path' => __DIR__.'/../views']); $app->register(new Silex\Provider\TranslationServiceProvider(), ['locale_fallbacks' => ['en','nl']]); $app->register(new Silex\Provider\SessionServiceProvider()); $app->extend('translator', function($translator, $app) { $translator->addLoader('yaml', new YamlFileLoader()); $translator->addResource('yaml', dirname(__DIR__).'/locales/en.yml', 'en'); $translator->addResource('yaml',dirname( __DIR__).'/locales/nl.yml', 'nl'); return $translator; }); $app['credentials'] = [ 'admin' => '$2y$10$431rvq1qS9ewNFP0Gti/o.kBbuMK4zs8IDTLlxm5uzV7cbv8wKt0K' ]; $app->before(function (Request $request) use ($app){ $request->setLocale($request->getPreferredLanguage()); $app['translator']->setLocale($request->getPreferredLanguage()); }); $app->after(function(Request $request, Response $response) use ($app){ $response->headers->set('Content-Length',strlen($response->getContent())); }); -
$app->get('/', function () use($app) { if($app['session']->has('username')) { $loginLogoutUrl = $app['url_generator']->generate('logout');
$loginLogoutLabel = 'log_out'; } else { $loginLogoutUrl = $app['url_generator']->generate('login'); $loginLogoutLabel = 'log_in'; } $response = new Response($app['twig']->render('index.twig', ['loginLogoutUrl'=>$loginLogoutUrl,'loginLogoutLabel'=>$loginLogoutLabel]),200); return $response; })->bind('home'); $app->get('/login', function (Request $request) use($app) { if($app['session']->has('username')) { return new RedirectResponse($app['url_generator']->generate('home')); } $response = new Response($app['twig']->render('login.twig'),200); return $response; })->bind('login'); $app->get('/logout', function () use($app) { $response = new RedirectResponse($app['url_generator']->generate('login')); $app['session']->invalidate(); return $response; })->bind('logout'); Routes -
<!DOCTYPE html> <html xmlns="http://www.w3.org/1999/html" xmlns:hx="http://purl.org/NET/hinclude"> <head> <title>{% block title %}{%
endblock %} - Developing cacheable websites</title> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/ bootstrap.min.css" integrity="sha384- BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous"> <script src="//rawgit.com/mnot/hinclude/master/hinclude.js"></script> <script src="https://code.jquery.com/jquery-2.2.4.min.js" integrity="sha256-BbhdlvQf/xTY9gja0Dq3HiwQF8LaCRTXxZKRutelT44=" crossorigin="anonymous"></script> <script src="//rawgit.com/carhartl/jquery-cookie/master/src/jquery.cookie.js"></ script> </head> <body> <div class="container-fluid"> {{ include('header.twig') }} <div class="row"> <div class="col-sm-3 col-lg-2"> {{ include('nav.twig') }} </div> <div class="col-sm-9 col-lg-10"> {% block content %}{% endblock %} </div> </div> {{ include('footer.twig') }} </div> </body> </html> Base template -
{% extends "base.twig" %} {% block title %}Home{% endblock %}
{% block content %} <div class="page-header"> <h1>{{ 'example' | trans }} <small>{{ 'rendered' | trans({'%date%':"now"|date("Y- m-d H:i:s")}) }}</small></h1> </div> <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris consequat orci eget libero sollicitudin, non ultrices turpis mollis. Aliquam sit amet tempus elit. Ut viverra risus enim, ut venenatis justo accumsan nec. Praesent a dolor tellus. Maecenas non mauris leo. Pellentesque lobortis turpis at dapibus laoreet. Mauris rhoncus nulla et urna mollis, et lobortis magna ornare. Etiam et sapien consequat, egestas felis sit amet, dignissim enim.</p> <p>Quisque quis mollis justo, imperdiet fermentum velit. Aliquam nulla justo, consectetur et diam non, luctus commodo metus. Vestibulum fermentum efficitur nulla non luctus. Nunc lorem nunc, mollis id efficitur et, aliquet sit amet ante. Sed ipsum turpis, vehicula eu semper eu, malesuada eget leo. Vestibulum venenatis dui id pulvinar suscipit. Etiam nec massa pharetra justo pharetra dignissim quis non magna. Integer id convallis lectus. Nam non ullamcorper metus. Ut vestibulum ex ut massa posuere tincidunt. Vestibulum hendrerit neque id lorem rhoncus aliquam. Duis a facilisis metus, a faucibus nulla.</p> {% endblock %} Home page template -
<div class="jumbotron"> <div class="page-header"> <h1>{{ 'welcome'|trans }} <em id="usernameLabel"></em></h1> <small>{{
'rendered' | trans({'%date%':"now"|date("Y-m-d H:i:s")}) }}</ small> </div> </div> Header template -
<nav class="navbar navbar-default navbar-fixed-side"> <ul class="nav"> <li><a href="{{ url('home') }}">Home</a></li>
<li><a href="{{ loginLogoutUrl| default(url('login')) }}">{{ loginLogoutLabel|default('log_in') | trans }}</a></ li> <li><a href="{{ url('private') }}">Private</a></li> </ul> <small>{{ 'rendered' | trans({'%date%':"now"|date("Y-m-d H:i:s")}) }}</small> </nav> Navigation template -
home: Home welcome : Welcome to the site rendered :
Rendered at %date% example : An example page log_in : Log in login : Login log_out : Log out username : Username password : Password private : Private privatetext : Looks like some very private data English locale file -
home: Start welcome : Welkom op de site rendered :
Samengesteld op %date% example : Een voorbeeldpagina log_in : Inloggen login : Login log_out : Uitloggen username : Gebruikersnaam password : Wachtwoord private : Privé privatetext : Deze tekst ziet er vrij privé uit Dutch locale file -
$app->get('/', function () use($app) { $response = new Response($app['twig']->render('index.twig'),200); $response
->setSharedMaxAge(500) ->setPublic(); return $response; })->bind('home'); Cache-Control: public, s-maxage=500 -
$app->get('/private', function () use($app) { if(!$app['session']->has('username')) { return new RedirectResponse($app['url_generator']-
>generate('login')); } $response = new Response($app['twig']->render('private.twig'),200); $response->headers->addCacheControlDirective('no-store', true); $response->headers->addCacheControlDirective('no-cache', true); $response ->setSharedMaxAge(0) ->setPrivate(); return $response; })->bind('private'); Cache-Control: private, no-cache, no-store, s-maxage=0 -
<esi:include src="/header" /> Edge Side Includes ✓Placeholder ✓Parsed by Varnish
✓Output is a composition of blocks ✓State per block ✓TTL per block -
sub vcl_recv { set req.http.Surrogate-Capability = "key=ESI/1.0"; } sub vcl_backend_response
{ if (beresp.http.Surrogate-Control ~ "ESI/1.0") { unset beresp.http.Surrogate-Control; set beresp.do_esi = true; } } Edge Side Includes -
<esi:include src="/header" /> Choose wisely <hx:include src="/header"></hx:include> ESI, parsed by
Varnish HInclude, parsed by Javascript -
<div class="container-fluid"> {{ include('header.twig') }} <div class="row"> <div class="col-sm-3 col-lg-2">
{{ include('nav.twig') }} </div> <div class="col-sm-9 col-lg-10"> {% block content %}{% endblock %} </div> </div> {{ include('footer.twig') }} </div> <div class="container-fluid"> {{ render_esi(url('header')) }} <div class="row"> <div class="col-sm-3 col-lg-2"> {{ render_esi(url('nav')) }} </div> <div class="col-sm-9 col-lg-10"> {% block content %}{% endblock %} </div> </div> {{ render_hinclude(url('footer')) }} </div> -
<div class="container-fluid"> <esi:include src="/header" /> <div class="row"> <div class="col-sm-3 col-lg-2">
<esi:include src="/nav" /> </div> <div class="col-sm-9 col-lg-10"> <div class="page-header"> <h1>An example page <small>Rendered at 2017-05-17 16:57:14</ small></h1> </div> <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris consequat orci eget libero sollicitudin,…</p> </div> </div> <hx:include src="/footer"></hx:include> </div> -
$app->get('/header', function () use($app) { $response = new Response($app['twig']->render('header.twig'),200); $response
->setSharedMaxAge(500) ->setPublic(); return $response; })->bind('header'); $app->get('/footer', function () use($app) { $response = new Response($app['twig']->render('footer.twig'),200); $response ->setSharedMaxAge(500) ->setPublic(); return $response; })->bind('footer'); $app->get('/nav', function (Request $request) use($app) { if($app['session']->has('username')) { $loginLogoutUrl = $app['url_generator']->generate('logout'); $loginLogoutLabel = 'log_out'; } else { $loginLogoutUrl = $app['url_generator']->generate('login'); $loginLogoutLabel = 'log_in'; } $response = new Response($app['twig']->render('nav.twig', ['loginLogoutUrl'=>$loginLogoutUrl,'loginLogoutLabel'=>$loginLogoutLabel]),200); $response->headers->addCacheControlDirective('no-store', true); $response->headers->addCacheControlDirective('no-cache', true); $response ->setSharedMaxAge(0) ->setPrivate(); return $response; })->bind('nav'); Separate routes per block -
$app->after(function(Request $request, Response $response) use ($app) { $response->headers->set('Content-Length', strlen($response->getContent())); });
$app->after(function(Request $request, Response $response) use ($app) { $response->setVary('Accept-Language',false); $response->headers->set('Content-Length', strlen($response->getContent())); }); -
✓Navigation page ✓Private page Weak spots Not cached because of
stateful content -
JWT eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhZG1pb iIsImV4cCI6MTQ5NTUyODc1NiwibG9naW4iOnRydWV9.u4Idy- SYnrFdnH1h9_sNc4OasORBJcrh2fPo1EOTre8 ✓3 parts ✓Dot separated ✓Base64 encoded
JSON ✓Header ✓Payload ✓Signature (HMAC with secret) -
eyJzdWIiOiJhZG1pbiIsIm V4cCI6MTQ5NTUyODc1Niwi bG9naW4iOnRydWV9 { "alg": "HS256", "typ": "JWT" } {
"sub": "admin", "exp": 1495528756, "login": true } HMACSHA256( base64UrlEncode(header) + "." + base64UrlEncode(payload), secret ) eyJhbGciOiJIUzI1NiIsI nR5cCI6IkpXVCJ9 u4Idy- SYnrFdnH1h9_sNc4OasOR BJcrh2fPo1EOTre8 -
JWT Cookie:token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJz dWIiOiJhZG1pbiIsImV4cCI6MTQ5NTUyODc1NiwibG9naW4iOnRydW V9.u4Idy-SYnrFdnH1h9_sNc4OasORBJcrh2fPo1EOTre8 ✓Stored in a cookie ✓Can be
validated by Varnish ✓Payload can be processed by any language (e.g. Javascript) -
VCL vcl 4.0; import digest; import std; import cookie; import
var; backend default { .host = "localhost"; .port = "8080"; } Init -
VCL sub vcl_recv { var.set("key","SlowWebSitesSuck"); set req.url = std.querysort(req.url); if(req.http.accept-language
~ "^\s*(nl)") { set req.http.accept-language = regsub(req.http.accept- language,"^\s*(nl).*$","\1"); } else { set req.http.accept-language = "en"; } set req.http.Surrogate-Capability="key=ESI/1.0"; if ((req.method != "GET" && req.method != "HEAD") || req.http.Authorization) { return (pass); } call jwt; if(req.url == "/private" && req.http.X-Login != "true") { return(synth(302,"/logout")); } return(hash); } Receive request -
VCL sub vcl_backend_response { set beresp.http.x-host = bereq.http.host; set beresp.http.x-url
= bereq.url; if(beresp.http.Surrogate-Control~"ESI/1.0") { unset beresp.http.Surrogate-Control; set beresp.do_esi=true; return(deliver); } } sub vcl_deliver { unset resp.http.x-host; unset resp.http.x-url; unset resp.http.vary; } sub vcl_synth { if (resp.status == 301 || resp.status == 302) { set resp.http.location = resp.reason; set resp.reason = "Moved"; return (deliver); } } Receive backend response Strip headers Custom redirect logic -
sub jwt { if(req.http.cookie ~ "^([^;]+;[ ]*)*token=[^\.]+\.[^\.]+\.[^\.]+([ ]*;[^;]+)*$") { cookie.parse(req.http.cookie);
cookie.filter_except("token"); var.set("token", cookie.get("token")); var.set("header", regsub(var.get("token"),"([^\.]+)\.[^\.]+\.[^\.]+","\1")); var.set("type", regsub(digest.base64url_decode(var.get("header")),{"^.*?"typ"\s*: \s*"(\w+)".*?$"},"\1")); var.set("algorithm", regsub(digest.base64url_decode(var.get("header")),{"^.*?"alg"\s*: \s*"(\w+)".*?$"},"\1")); if(var.get("type") != "JWT" || var.get("algorithm") != "HS256") { return(synth(400, "Invalid token")); } var.set("rawPayload",regsub(var.get("token"),"[^\.]+\.([^\.]+)\.[^\.]+$","\1")); var.set("signature",regsub(var.get("token"),"^[^\.]+\.[^\.]+\.([^\.]+)$","\1")); var.set("currentSignature",digest.base64url_nopad_hex(digest.hmac_sha256(var.get("key"),var.ge t("header") + "." + var.get("rawPayload")))); var.set("payload", digest.base64url_decode(var.get("rawPayload"))); var.set("exp",regsub(var.get("payload"),{"^.*?"exp"\s*:\s*(\w+).*?$"},"\1")); var.set("username",regsub(var.get("payload"),{"^.*?"sub"\s*:\s*"(\w+)".*?$"},"\1")); if(var.get("signature") != var.get("currentSignature")) { return(synth(400, "Invalid token")); } if(var.get("username") ~ "^\w+$") { if(std.time(var.get("exp"),now) >= now) { set req.http.X-Login="true"; } else { set req.http.X-Login="false"; } } } } -
cookie.parse(req.http.cookie); cookie.filter_except("token"); var.set("token", cookie.get("token")); var.set("header", regsub(var.get("token"),"([^\.]+)\.[^\.]+\.[^\.]+","\1")); var.set("type", regsub(digest.base64url_decode(var.get("header")), {"^.*?"typ"\s*:\s*"(\w+)".*?$"},"\1")); var.set("algorithm",
regsub(digest.base64url_decode(var.get("header")), {"^.*?"alg"\s*:\s*"(\w+)".*?$"},"\1")); if(var.get("type") != "JWT" || var.get("algorithm") != "HS256") { return(synth(400, "Invalid token")); } Validate JWT header -
var.set("signature",regsub(var.get("token"),"^[^\.]+\.[^\.]+\.([^\.]+) $","\1")); var.set("currentSignature",digest.base64url_nopad_hex(digest.hmac_sha256(var. get("key"),var.get("header") + "." + var.get("rawPayload")))); if(var.get("signature") !=
var.get("currentSignature")) { return(synth(400, "Invalid token")); } Verify signature -
if(var.get("username") ~ "^\w+$") { if(std.time(var.get("exp"),now) >= now) { set req.http.X-Login="true";
} else { set req.http.X-Login="false"; } } Verify login action Check expiration Use X- Login in vcl_recv -
$app['jwtKey'] = 'SlowWebSitesSuck'; $app['jwtEncode'] = function() use ($app){ return function($username)
use ($app) { return JWT::encode([ 'sub'=>$username, 'exp'=>time() + 86400, 'login'=>true, ],$app['jwtKey']); }; }; $app['jwtValidate'] = function() use ($app) { return function($token) use ($app) { try { $data = JWT::decode($token,$app['jwtKey'],['HS256']); $data = (array)$data; if(!isset($app['credentials'][$data['sub']])) { return false; } return true; } catch(UnexpectedValueException $e) { return false; } }; }; The code Helper functions -
$app->get('/private', function (Request $request) use($app) { if(!$app['jwtValidate']($request->cookies->get('token'))) { return new
RedirectResponse($app['url_generator']->generate('login')); } $response = new Response($app['twig']->render('private.twig'),200); $response ->setSharedMaxAge(500) ->setPublic(); return $response; })->bind('private'); The code Redirect to login page -
$app->get('/nav', function (Request $request) use($app) { if($app['jwtValidate']($request->cookies->get('token'))) { $loginLogoutUrl =
$app['url_generator']->generate('logout'); $loginLogoutLabel = 'log_out'; } else { $loginLogoutUrl = $app['url_generator']->generate('login'); $loginLogoutLabel = 'log_in'; } $response = new Response($app['twig']->render('nav.twig', ['loginLogoutUrl'=>$loginLogoutUrl,'loginLogoutLabel'=>$loginLogoutLabel] ),200); $response ->setVary('X-Login',false) ->setSharedMaxAge(500) ->setPublic(); return $response; })->bind('nav'); The code Cache variation -
<script language="JavaScript"> function getCookie(name) { var value = "; "
+ document.cookie; var parts = value.split("; " + name + "="); if (parts.length == 2) return parts.pop().split(";").shift(); } function parseJwt (token) { var base64Url = token.split('.')[1]; var base64 = base64Url.replace('-', '+').replace('_', '/'); return JSON.parse(window.atob(base64)); }; $(document).ready(function(){ if ($.cookie('token') != null ){ var token = parseJwt($.cookie("token")); $("#usernameLabel").html(', ' + token.sub); } }); </script> Parse JWT in Javascript -
✓Application issues JWT ✓Varnish validates JWT ✓Varnish can verify login
✓Varnish can redirect to login page ✓No backend access required ✓Only POST /login hits the backend ✓Even works without Varnish ✓Even Javascript can parse JWT ✓Extra cache variation required JWT conclusion