Advanced fiddle workshop

Transcript

1. 1
   ©2019
   Getting the most out of
   Fastly Fiddle
   Andrew Betts
   Developer advocate
   

   View Slide

2. 2
   ©2019
   VCL compiles to native code
   Your application
   

   View Slide

3. 3
   ©2019
   VCL state machine
   Your programmable journey through Fastly's varnish
   RECV MISS
   HIT
   PASS
   FETCH DELIVER
   ERROR
   Receive request
   Deliver response
   Cache lookup Backend fetch
   LOG
   

   View Slide

4. 4
   ©2019
   We're working to bring WASM to Fastly
   Maintaining the speed of native code at the edge
   VCL TypeScript
   sub vcl_recv {
   if (!req.http.flags) {
   set req.http.tmpOrigUrl = req.url;
   set req.backend = F_PREFLIGHT_ORIGIN;
   set req.url = "/api/flags";
   } else {
   set req.backend = F_CONTENT_ORIGIN;
   }
   }
   sub vcl_deliver {
   if (resp.http.flags) {
   set req.http.flags = resp.http.flags;
   set req.url = req.http.tmpOrigUrl;
   unset req.http.tmpOrigUrl;
   restart;
   }
   }
   function recv(req: Request): Response {
   let pfReq = new Request("GET",
   F_PREFLIGHT_ORIGIN + "/api/flags"
   );
   let pfResp = pfReq.send();
   let req = new Request("GET",
   F_CONTENT_ORIGIN + req.url
   );
   req.set_header("flags",
   pfResp.get_header("flags")
   );
   let resp = bereq.send();
   return resp;
   }
   

   View Slide

5. 5
   ©2019
   Fastly Fiddle
   

   View Slide

6. 6
   ©2019
   Solution Fiddle features
   Vanilla service (no edge logic) Instrumentation; Server identification; Insights
   Image optimisation Shielding; Image output
   Geofencing Autocomplete; Header highlighting
   A/B testing Multiple requests; Cookie jar; Logging
   Conditional revalidation Surrogate keys; Clustering; timing metrics
   Threat intelligence Restarts; Synthetic responses; POST to GET rewriting
   GCS backend Follow redirects; Unit tests
   Solutions we will build today
   Learn about Fiddle, learn about cool things to do with Fastly too
   

   View Slide

7. 7
   ©2019
   Warning:
   ● Everything you enter into Fiddle is public
   ● Deploying code to thousands of servers is not
   instantaneous. Please be patient!
   ● We suggest you do each exercise in a different
   tab, so you still have them all to refer to at the
   end.
   ● Ask Fastly helpers if you get stuck
   

   View Slide

8. 8
   ©2019
   Vanilla service
   What does Fastly do just by default?
   Proxy and
   cache
   

   View Slide

9. 9
   ©2019
   Vanilla service
   What does Fastly do just by default?
   1. Open fiddle.fastlydemo.net in a new tab
   2. Give it a title, eg "Exercise 1: Vanilla"
   3. When the RUN button goes blue, click it
   

   View Slide

10. 10
    ©2019
    

    View Slide

11. 11
    ©2019
    Image optimisation
    Use the Fastly IO service to transform your images automatically
    Cache
    transformed
    image
    Process into
    desired
    format
    Cache original
    image
    

    View Slide

12. 12
    ©2019
    Image optimisation
    Use the Fastly IO service to transform your images automatically
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 2: Image optimisation"
    3. Set the origin server to https://triblondon.github.io
    4. Set the request path to /fastly-fiddle-examples/images/obama2.jpg
    5. Enable Shielding in the Options menu
    6. Add the following VCL to RECV:
    set req.http.X-Fastly-Imageopto-Api = "fastly";
    7. When the RUN button lights up, click it!
    

    View Slide

13. 13
    ©2019
    Image optimisation
    Use the Fastly IO service to transform your images automatically
    8. Add ?width=300 to the query string and RUN again
    9. Add &crop=16:9 to the query string and RUN again
    10. Change crop=16:9 to crop=16:9,smart and RUN again
    

    View Slide

14. 14
    ©2019
    

    View Slide

15. 15
    ©2019
    Geolocation
    Add Fastly-provided geolocation variables to a request
    + added geolocation data
    + Vary on response
    

    View Slide

16. 16
    ©2019
    Geolocation
    Add Fastly-provided geolocation variables to a request
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 3: Geolocation"
    3. Add the following VCL to RECV:
    set req.http.client-geo-continent = client.geo.continent_code;
    set req.http.client-geo-country = client.geo.country_code;
    set req.http.edge-geo-datacenter = server.datacenter;
    set req.http.client-source-network = client.as.number;
    4. Set the request path to:
    /response-headers?Vary=client-geo-country
    5. When the RUN button lights up, click it!
    

    View Slide

17. 17
    ©2019
    

    View Slide

18. 18
    ©2019
    A/B testing
    Select and assign buckets at the edge to cache test variations
    + added test bucket info
    + Vary on response
    

    View Slide

19. 19
    ©2019
    A/B testing
    Select and assign buckets at the edge to cache test variations
    0 100
    0 100
    hash( + "header") = 53
    hash( + "buttonSize") = 87
    "small" "medium" "large"
    "normal" "cute"
    buttonSize = "large"; header = "cute"
    

    View Slide

20. 20
    ©2019
    A/B testing
    Select and assign buckets at the edge to cache test variations
    1. Open fiddle.fastlydemo.net
    2. Add a title, eg "Exercise 4: A/B testing"
    3. Set origin server (replacing httpbin) to:
    https://fiddle-ab.glitch.me
    4. Set the request path to:
    /article/kittens
    5. In VCL_RECV, assign a User ID:
    set req.http.ab = if (
    req.http.Cookie:ab,
    req.http.Cookie:ab,
    uuid.version4()
    );
    6. In VCL_DELIVER, save the User ID into a cookie:
    if (!req.http.Cookie:ab) {
    add resp.http.Set-Cookie = ""
    "ab=" req.http.ab "; "
    "max-age=31536000; "
    "path=/; secure"
    ;
    set resp.http.Cache-Control = ""
    "no-store"
    ;
    }
    7. Click and choose "Add another" to
    copy the request
    

    View Slide

21. 21
    ©2019
    

    View Slide

22. 22
    ©2019
    A/B testing
    Select and assign buckets at the edge to cache test variations
    7. In VCL_RECV, calculate the first bucket assignment:
    declare local var.testKey STRING;
    declare local var.userTestSlot INTEGER;
    set var.testKey = req.http.ab "header";
    set var.userTestSlot = fastly.hash(var.testKey, 0, 1, 100);
    set req.http.ab-header = if (
    var.userTestSlot <= 50,
    "normal",
    "cute"
    );
    

    View Slide

23. 23
    ©2019
    A/B testing
    Select and assign buckets at the edge to cache test variations
    8. In VCL_RECV, add a second bucket assignment:
    set var.testKey = req.http.ab "buttonSize";
    set var.userTestSlot = fastly.hash(var.testKey, 0, 1, 100);
    set req.http.ab-buttonSize =
    if (var.userTestSlot <= 20, "small",
    if (var.userTestSlot <= 30, "medium",
    "large"
    ));
    

    View Slide

24. 24
    ©2019
    

    View Slide

25. 25
    ©2019
    Conditional revalidations
    Minimize traffic to origin
    Conditional GET
    304 Not Modified
    GET /thing
    GET /thing
    GET /thing
    200 OK
    Normal GET
    HIT
    

    View Slide

26. 26
    ©2019
    Conditional revalidation
    Keep a stale object in cache but revalidate with origin
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 5: Conditional revalidation"
    3. Set the request path to:
    /cache
    4. Add the following VCL to FETCH:
    set beresp.ttl = 1s;
    return(deliver);
    5. When the RUN button lights up, click it!
    6. Wait a couple of seconds, and run it again.
    

    View Slide

27. 27
    ©2019
    

    View Slide

28. 28
    ©2019
    Conditional revalidation
    Fastly Edge can revalidate with a Fastly shield
    7. Enable Shielding in the Options menu
    8. Remove line 1 of FETCH and replace with a TTL based on whether we are on the edge
    or the shield:
    set beresp.ttl = 1s;
    if (req.backend.is_shield) {
    set beresp.ttl = 1s;
    } else {
    set beresp.ttl = 10s;
    }
    9. When the RUN button lights up, SHIFT+click it (to run with empty cache)
    10. Wait a couple of seconds (no more than that), and run it again.
    

    View Slide

29. 29
    ©2019
    Shield
    Edge
    

    View Slide

30. 30
    ©2019
    Understanding cache depth
    When you say 'cache hit'...
    Edge POP
    Shield POP
    Origin
    

    View Slide

31. 31
    ©2019
    Wrapping up
    Like a gift parcel of VCL goodness
    • Explore more solutions at fastly.com/demos
    • Try Fiddle yourself at fiddle.fastlydemo.net
    • Got something good? Let us know in community.fastly.com
    or email me: abetts@fastly.com.
    

    View Slide

32. 32
    ©2019
    Thank you!
    Andrew Betts
    Tweet me at @triblondon
    abetts@fastly.com
    These slides are published at
    fastly.us/advanced-fiddle
    

    View Slide

33. 33
    ©2019
    GCS backend
    Use Google Cloud Storage to host your website more flexibly
    /resources /resources
    404 Not Found
    /resources/index.html
    200 OK
    308 Redirect
    /resources/
    200 OK
    /resources/index.html
    

    View Slide

34. 34
    ©2019
    GCS Backend
    Use Google Cloud Storage to host your website more flexibly
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 7: GCS backend"
    3. Replace the origin with:
    https://storage.googleapis.com
    4. Change the request path to:
    /resources
    5. Under request options, enable Follow Redirects
    6. Add the bucket prefix in RECV:
    if (req.restarts == 0) {
    set req.http.orig-req-url = req.url;
    set req.url = "/betts-gcp-gcs-fastly-tutorial" req.url;
    }
    

    View Slide

35. 35
    ©2019
    GCS Backend
    Use Google Cloud Storage to host your website more flexibly
    7. Still in RECV, add the index file suffix if the requested path is a directory:
    if (req.url ~ "\/$") {
    set req.url = req.url "index.html";
    }
    8. In DELIVER, detect when GCS says 'not found' for a non-directory path:
    if (resp.status == 404 && req.url !~ "\/index.html$") {
    set req.http.retry-for-dir = "1";
    restart;
    }
    9. Back in the middle of RECV, try appending a / to paths if we're retrying for a directory
    if (req.http.retry-for-dir) {
    set req.url = req.url "/";
    }
    

    View Slide

36. 36
    ©2019
    GCS Backend
    Use Google Cloud Storage to host your website more flexibly
    10. In FETCH, detect a successful response for a retry
    if (beresp.status == 200 && req.url ~ "\/index.html$" &&
    req.http.retry-for-dir) {
    error 901;
    }
    11. In ERROR, create the appropriate redirect if the retry succeeded:
    if (obj.status == 901) {
    set obj.status = 308;
    set obj.response = "Permanent redirect";
    set obj.http.Location = req.http.orig-req-url "/";
    synthetic "";
    return(deliver);
    }
    

    View Slide

37. 37
    ©2019
    

    View Slide

38. 38
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    Check with threat intel API
    Receive verdict
    Send request to normal origin
    

    View Slide

39. 39
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 6: Threat intelligence"
    3. Add an extra origin (in addition to httpbin):
    https://us-central1-rd---product.cloudfunctions.net
    4. Change the request path to:
    /post
    5. Set the request method to POST
    6. Add the following to the body data:
    username=andrew&password=apple
    

    View Slide

40. 40
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    7. Intercept risky requests in RECV and divert to the threat intel API:
    declare local var.cred STRING;
    set var.cred = subfield(req.postbody, "password", "&");
    if (req.method == "POST" && var.cred && !req.http.TI-Result) {
    set req.backend = F_origin_1;
    set req.http.Orig-URL = req.url;
    set req.http.Orig-Method = req.method;
    set req.http.TI-Key = digest.hash_sha1(var.cred);
    set req.url = "/threatIntelPOC?key=" substr(req.http.TI-Key, 0, 6);
    set req.method = "GET";
    log "Cred: " var.cred ", key: " req.http.TI-Key;
    }
    

    View Slide

41. 41
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    8. Capture the threat response in DELIVER:
    if (req.http.TI-Key && req.restarts == 0) {
    if (std.strstr(resp.http.Result, substr(req.http.TI-Key, 6))) {
    log "Credential is a known threat";
    set req.http.TI-Result = "FAIL";
    } else {
    log "Credential is OK";
    set req.http.TI-Result = "PASS";
    }
    restart;
    }
    

    View Slide

42. 42
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    9. Back in RECV, deal with restarted requests:
    if (req.http.TI-Result) {
    if (req.http.TI-Result != "PASS") {
    error 603;
    } else {
    set req.url = req.http.Orig-URL;
    set req.method = req.http.Orig-Method;
    }
    return (lookup);
    }
    

    View Slide

43. 43
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    10. Create the appropriate failure response synthetically in ERROR:
    if (obj.status == 603) {
    set obj.status = 403;
    set obj.response = "Forbidden";
    set obj.http.content-type = "text/plain";
    synthetic "Please contact us for assistance logging in";
    return (deliver);
    }
    11. Cleanup temporary headers in MISS:
    unset bereq.http.TI-Result;
    unset bereq.http.TI-Key;
    unset bereq.http.Orig-URL;
    unset bereq.http.Orig-Method;
    

    View Slide

44. 44
    ©2019
    

    View Slide

45. 45
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    12. Try changing password to something that isn't compromised
    username=andrew&password=apple-horse-coffee-tulip
    13. Try changing the request so it does not contain a password field:
    username=andrew
    

    View Slide