$30 off During Our Annual Pro Sale. View Details »

Advanced fiddle workshop

Andrew Betts
September 15, 2020

Advanced fiddle workshop

Andrew Betts

September 15, 2020
Tweet

More Decks by Andrew Betts

Other Decks in Technology

Transcript

  1. 1
    ©2019
    Getting the most out of
    Fastly Fiddle
    Andrew Betts
    Developer advocate

    View Slide

  2. 2
    ©2019
    VCL compiles to native code
    Your application

    View Slide

  3. 3
    ©2019
    VCL state machine
    Your programmable journey through Fastly's varnish
    RECV MISS
    HIT
    PASS
    FETCH DELIVER
    ERROR
    Receive request
    Deliver response
    Cache lookup Backend fetch
    LOG

    View Slide

  4. 4
    ©2019
    We're working to bring WASM to Fastly
    Maintaining the speed of native code at the edge
    VCL TypeScript
    sub vcl_recv {
    if (!req.http.flags) {
    set req.http.tmpOrigUrl = req.url;
    set req.backend = F_PREFLIGHT_ORIGIN;
    set req.url = "/api/flags";
    } else {
    set req.backend = F_CONTENT_ORIGIN;
    }
    }
    sub vcl_deliver {
    if (resp.http.flags) {
    set req.http.flags = resp.http.flags;
    set req.url = req.http.tmpOrigUrl;
    unset req.http.tmpOrigUrl;
    restart;
    }
    }
    function recv(req: Request): Response {
    let pfReq = new Request("GET",
    F_PREFLIGHT_ORIGIN + "/api/flags"
    );
    let pfResp = pfReq.send();
    let req = new Request("GET",
    F_CONTENT_ORIGIN + req.url
    );
    req.set_header("flags",
    pfResp.get_header("flags")
    );
    let resp = bereq.send();
    return resp;
    }

    View Slide

  5. 5
    ©2019
    Fastly Fiddle

    View Slide

  6. 6
    ©2019
    Solution Fiddle features
    Vanilla service (no edge logic) Instrumentation; Server identification; Insights
    Image optimisation Shielding; Image output
    Geofencing Autocomplete; Header highlighting
    A/B testing Multiple requests; Cookie jar; Logging
    Conditional revalidation Surrogate keys; Clustering; timing metrics
    Threat intelligence Restarts; Synthetic responses; POST to GET rewriting
    GCS backend Follow redirects; Unit tests
    Solutions we will build today
    Learn about Fiddle, learn about cool things to do with Fastly too

    View Slide

  7. 7
    ©2019
    Warning:
    ● Everything you enter into Fiddle is public
    ● Deploying code to thousands of servers is not
    instantaneous. Please be patient!
    ● We suggest you do each exercise in a different
    tab, so you still have them all to refer to at the
    end.
    ● Ask Fastly helpers if you get stuck

    View Slide

  8. 8
    ©2019
    Vanilla service
    What does Fastly do just by default?
    Proxy and
    cache

    View Slide

  9. 9
    ©2019
    Vanilla service
    What does Fastly do just by default?
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 1: Vanilla"
    3. When the RUN button goes blue, click it

    View Slide

  10. 10
    ©2019

    View Slide

  11. 11
    ©2019
    Image optimisation
    Use the Fastly IO service to transform your images automatically
    Cache
    transformed
    image
    Process into
    desired
    format
    Cache original
    image

    View Slide

  12. 12
    ©2019
    Image optimisation
    Use the Fastly IO service to transform your images automatically
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 2: Image optimisation"
    3. Set the origin server to https://triblondon.github.io
    4. Set the request path to /fastly-fiddle-examples/images/obama2.jpg
    5. Enable Shielding in the Options menu
    6. Add the following VCL to RECV:
    set req.http.X-Fastly-Imageopto-Api = "fastly";
    7. When the RUN button lights up, click it!

    View Slide

  13. 13
    ©2019
    Image optimisation
    Use the Fastly IO service to transform your images automatically
    8. Add ?width=300 to the query string and RUN again
    9. Add &crop=16:9 to the query string and RUN again
    10. Change crop=16:9 to crop=16:9,smart and RUN again

    View Slide

  14. 14
    ©2019

    View Slide

  15. 15
    ©2019
    Geolocation
    Add Fastly-provided geolocation variables to a request
    + added geolocation data
    + Vary on response

    View Slide

  16. 16
    ©2019
    Geolocation
    Add Fastly-provided geolocation variables to a request
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 3: Geolocation"
    3. Add the following VCL to RECV:
    set req.http.client-geo-continent = client.geo.continent_code;
    set req.http.client-geo-country = client.geo.country_code;
    set req.http.edge-geo-datacenter = server.datacenter;
    set req.http.client-source-network = client.as.number;
    4. Set the request path to:
    /response-headers?Vary=client-geo-country
    5. When the RUN button lights up, click it!

    View Slide

  17. 17
    ©2019

    View Slide

  18. 18
    ©2019
    A/B testing
    Select and assign buckets at the edge to cache test variations
    + added test bucket info
    + Vary on response

    View Slide

  19. 19
    ©2019
    A/B testing
    Select and assign buckets at the edge to cache test variations
    0 100
    0 100
    hash( + "header") = 53
    hash( + "buttonSize") = 87
    "small" "medium" "large"
    "normal" "cute"
    buttonSize = "large"; header = "cute"

    View Slide

  20. 20
    ©2019
    A/B testing
    Select and assign buckets at the edge to cache test variations
    1. Open fiddle.fastlydemo.net
    2. Add a title, eg "Exercise 4: A/B testing"
    3. Set origin server (replacing httpbin) to:
    https://fiddle-ab.glitch.me
    4. Set the request path to:
    /article/kittens
    5. In VCL_RECV, assign a User ID:
    set req.http.ab = if (
    req.http.Cookie:ab,
    req.http.Cookie:ab,
    uuid.version4()
    );
    6. In VCL_DELIVER, save the User ID into a cookie:
    if (!req.http.Cookie:ab) {
    add resp.http.Set-Cookie = ""
    "ab=" req.http.ab "; "
    "max-age=31536000; "
    "path=/; secure"
    ;
    set resp.http.Cache-Control = ""
    "no-store"
    ;
    }
    7. Click and choose "Add another" to
    copy the request

    View Slide

  21. 21
    ©2019

    View Slide

  22. 22
    ©2019
    A/B testing
    Select and assign buckets at the edge to cache test variations
    7. In VCL_RECV, calculate the first bucket assignment:
    declare local var.testKey STRING;
    declare local var.userTestSlot INTEGER;
    set var.testKey = req.http.ab "header";
    set var.userTestSlot = fastly.hash(var.testKey, 0, 1, 100);
    set req.http.ab-header = if (
    var.userTestSlot <= 50,
    "normal",
    "cute"
    );

    View Slide

  23. 23
    ©2019
    A/B testing
    Select and assign buckets at the edge to cache test variations
    8. In VCL_RECV, add a second bucket assignment:
    set var.testKey = req.http.ab "buttonSize";
    set var.userTestSlot = fastly.hash(var.testKey, 0, 1, 100);
    set req.http.ab-buttonSize =
    if (var.userTestSlot <= 20, "small",
    if (var.userTestSlot <= 30, "medium",
    "large"
    ));

    View Slide

  24. 24
    ©2019

    View Slide

  25. 25
    ©2019
    Conditional revalidations
    Minimize traffic to origin
    Conditional GET
    304 Not Modified
    GET /thing
    GET /thing
    GET /thing
    200 OK
    Normal GET
    HIT

    View Slide

  26. 26
    ©2019
    Conditional revalidation
    Keep a stale object in cache but revalidate with origin
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 5: Conditional revalidation"
    3. Set the request path to:
    /cache
    4. Add the following VCL to FETCH:
    set beresp.ttl = 1s;
    return(deliver);
    5. When the RUN button lights up, click it!
    6. Wait a couple of seconds, and run it again.

    View Slide

  27. 27
    ©2019

    View Slide

  28. 28
    ©2019
    Conditional revalidation
    Fastly Edge can revalidate with a Fastly shield
    7. Enable Shielding in the Options menu
    8. Remove line 1 of FETCH and replace with a TTL based on whether we are on the edge
    or the shield:
    set beresp.ttl = 1s;
    if (req.backend.is_shield) {
    set beresp.ttl = 1s;
    } else {
    set beresp.ttl = 10s;
    }
    9. When the RUN button lights up, SHIFT+click it (to run with empty cache)
    10. Wait a couple of seconds (no more than that), and run it again.

    View Slide

  29. 29
    ©2019
    Shield
    Edge

    View Slide

  30. 30
    ©2019
    Understanding cache depth
    When you say 'cache hit'...
    Edge POP
    Shield POP
    Origin

    View Slide

  31. 31
    ©2019
    Wrapping up
    Like a gift parcel of VCL goodness
    • Explore more solutions at fastly.com/demos
    • Try Fiddle yourself at fiddle.fastlydemo.net
    • Got something good? Let us know in community.fastly.com
    or email me: [email protected].

    View Slide

  32. 32
    ©2019
    Thank you!
    Andrew Betts
    Tweet me at @triblondon
    [email protected]
    These slides are published at
    fastly.us/advanced-fiddle

    View Slide

  33. 33
    ©2019
    GCS backend
    Use Google Cloud Storage to host your website more flexibly
    /resources /resources
    404 Not Found
    /resources/index.html
    200 OK
    308 Redirect
    /resources/
    200 OK
    /resources/index.html

    View Slide

  34. 34
    ©2019
    GCS Backend
    Use Google Cloud Storage to host your website more flexibly
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 7: GCS backend"
    3. Replace the origin with:
    https://storage.googleapis.com
    4. Change the request path to:
    /resources
    5. Under request options, enable Follow Redirects
    6. Add the bucket prefix in RECV:
    if (req.restarts == 0) {
    set req.http.orig-req-url = req.url;
    set req.url = "/betts-gcp-gcs-fastly-tutorial" req.url;
    }

    View Slide

  35. 35
    ©2019
    GCS Backend
    Use Google Cloud Storage to host your website more flexibly
    7. Still in RECV, add the index file suffix if the requested path is a directory:
    if (req.url ~ "\/$") {
    set req.url = req.url "index.html";
    }
    8. In DELIVER, detect when GCS says 'not found' for a non-directory path:
    if (resp.status == 404 && req.url !~ "\/index.html$") {
    set req.http.retry-for-dir = "1";
    restart;
    }
    9. Back in the middle of RECV, try appending a / to paths if we're retrying for a directory
    if (req.http.retry-for-dir) {
    set req.url = req.url "/";
    }

    View Slide

  36. 36
    ©2019
    GCS Backend
    Use Google Cloud Storage to host your website more flexibly
    10. In FETCH, detect a successful response for a retry
    if (beresp.status == 200 && req.url ~ "\/index.html$" &&
    req.http.retry-for-dir) {
    error 901;
    }
    11. In ERROR, create the appropriate redirect if the retry succeeded:
    if (obj.status == 901) {
    set obj.status = 308;
    set obj.response = "Permanent redirect";
    set obj.http.Location = req.http.orig-req-url "/";
    synthetic "";
    return(deliver);
    }

    View Slide

  37. 37
    ©2019

    View Slide

  38. 38
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    Check with threat intel API
    Receive verdict
    Send request to normal origin

    View Slide

  39. 39
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    1. Open fiddle.fastlydemo.net in a new tab
    2. Give it a title, eg "Exercise 6: Threat intelligence"
    3. Add an extra origin (in addition to httpbin):
    https://us-central1-rd---product.cloudfunctions.net
    4. Change the request path to:
    /post
    5. Set the request method to POST
    6. Add the following to the body data:
    username=andrew&password=apple

    View Slide

  40. 40
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    7. Intercept risky requests in RECV and divert to the threat intel API:
    declare local var.cred STRING;
    set var.cred = subfield(req.postbody, "password", "&");
    if (req.method == "POST" && var.cred && !req.http.TI-Result) {
    set req.backend = F_origin_1;
    set req.http.Orig-URL = req.url;
    set req.http.Orig-Method = req.method;
    set req.http.TI-Key = digest.hash_sha1(var.cred);
    set req.url = "/threatIntelPOC?key=" substr(req.http.TI-Key, 0, 6);
    set req.method = "GET";
    log "Cred: " var.cred ", key: " req.http.TI-Key;
    }

    View Slide

  41. 41
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    8. Capture the threat response in DELIVER:
    if (req.http.TI-Key && req.restarts == 0) {
    if (std.strstr(resp.http.Result, substr(req.http.TI-Key, 6))) {
    log "Credential is a known threat";
    set req.http.TI-Result = "FAIL";
    } else {
    log "Credential is OK";
    set req.http.TI-Result = "PASS";
    }
    restart;
    }

    View Slide

  42. 42
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    9. Back in RECV, deal with restarted requests:
    if (req.http.TI-Result) {
    if (req.http.TI-Result != "PASS") {
    error 603;
    } else {
    set req.url = req.http.Orig-URL;
    set req.method = req.http.Orig-Method;
    }
    return (lookup);
    }

    View Slide

  43. 43
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    10. Create the appropriate failure response synthetically in ERROR:
    if (obj.status == 603) {
    set obj.status = 403;
    set obj.response = "Forbidden";
    set obj.http.content-type = "text/plain";
    synthetic "Please contact us for assistance logging in";
    return (deliver);
    }
    11. Cleanup temporary headers in MISS:
    unset bereq.http.TI-Result;
    unset bereq.http.TI-Key;
    unset bereq.http.Orig-URL;
    unset bereq.http.Orig-Method;

    View Slide

  44. 44
    ©2019

    View Slide

  45. 45
    ©2019
    Threat intelligence
    Call out to an API to assess a risky request before sending it to origin
    12. Try changing password to something that isn't compromised
    username=andrew&password=apple-horse-coffee-tulip
    13. Try changing the request so it does not contain a password field:
    username=andrew

    View Slide