Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Accelerating OTT video platforms with Varnish - London Video Tech meetup 2020

Thijs Feryn
February 04, 2020

Accelerating OTT video platforms with Varnish - London Video Tech meetup 2020

Slides for my OTT video acceleration talk at the London Video Technology meetup February 2020 edition: https://feryn.eu/speaking/accelerating-ott-video-platforms-varnish-london-video-technology-meetup-2020/

Thijs Feryn

February 04, 2020
Tweet

More Decks by Thijs Feryn

Other Decks in Technology

Transcript

  1. ACCELERATING OTT
    VIDEO PLATFORMS
    WITH VARNISH
    BY THIJS FERYN

    View Slide

  2. I'M NOT A VIDEO EXPERT

    View Slide

  3. Slow websites
    SUCK

    View Slide

  4. View Slide

  5. is bollocks

    View Slide

  6. CONTENT DELIVERY CHALLENGES

    View Slide

  7. STABILITY

    View Slide

  8. REDUCE LATENCY

    View Slide

  9. SERVER
    CAPACITY

    View Slide

  10. NETWORK
    CAPACITY

    View Slide

  11. THROWING
    SERVERS
    AT THE PROBLEM

    View Slide

  12. MO' MONEY
    MO' SERVERS
    MO' PROBLEMS

    View Slide

  13. CACHE

    View Slide

  14. DON’T RECOMPUTE
    IF THE DATA
    HASN’T CHANGED

    View Slide

  15. HI, I'M THIJS

    View Slide

  16. I'M AN
    EVANGELIST
    AT

    View Slide

  17. View Slide

  18. View Slide

  19. +-5,000,000 WEBSITES
    20% OF THE TOP 10K WEBSITES

    View Slide

  20. View Slide

  21. MORE THAN 750M VIEWERS PER MONTH

    View Slide

  22. View Slide

  23. I'M @THIJSFERYN

    View Slide

  24. View Slide

  25. VIDEO IS BIG & BROAD INDUSTRY

    View Slide

  26. DELIVERY

    View Slide

  27. View Slide

  28. NORMALLY
    CLIENT SERVER

    View Slide

  29. WITH REVERSE CACHING PROXY
    CLIENT PROXY SERVER

    View Slide

  30. View Slide

  31. WHY NOT
    USE A CDN?

    View Slide

  32. VARNISH IS CDN SOFTWARE!

    View Slide

  33. VARNISH VARNISH VARNISH
    ORIGIN

    View Slide

  34. CDN CDN CDN
    ORIGIN
    VARNISH

    View Slide

  35. STREAM.M3U8
    STREAM_01.TS
    STREAM_02.TS
    STREAM_03.TS
    THESE WILL BE
    CACHED

    View Slide

  36. WHY IS VARNISH
    SO POWERFUL?

    View Slide

  37. WHY IS VARNISH SO POWERFUL?
    ✓ EXTREMELY LOW RESOURCE
    ✓ EXTREMELY STABLE
    ✓ 100 GBIT PER SERVER
    ✓ REQUEST COALESCING
    ✓ VARNISH CONFIGURATION LANGUAGE
    ✓ VMODS
    ✓ COMPLIES TO HTTP BEST PRACTICES

    View Slide

  38. ORIGIN
    VARNISH
    REQUEST
    COALESCING
    QUEUED
    CONNECTIONS
    SINGLE BACKEND
    FETCH

    View Slide

  39. ORIGIN
    VARNISH
    GRACE
    SERVE STALE
    OBJECT
    BACKGROUND
    FETCH

    View Slide

  40. Cache-control: public, max-age=3600,
    s-maxage=86400, stale-while-revalidate=100
    GRACE TTL

    View Slide

  41. vcl 4.1;
    backend default {
    .host = "origin";
    .port = "80";
    }
    sub vcl_backend_response {
    if(bereq.url == "/vod/master.m3u8" ){
    set beresp.grace = 1h;
    }
    }

    View Slide

  42. VARNISH CONFIGURATION LANGUAGE

    View Slide

  43. View Slide

  44. View Slide

  45. vcl 4.1;
    backend origin {
    .host = "1.2.3.4";
    .port = "80";
    }
    sub vcl_recv {
    if(req.url ~ "^/(live|vod)/") {
    unset req.http.authorization;
    unset req.http.cookie;
    }
    }
    sub vcl_backend_response {
    if(bereq.url ~ "^/(live|vod)/") {
    unset beresp.http.set-cookie;
    set beresp.grace = 5s;
    }
    if(bereq.url ~ "^/live/.*\.m3u8") {
    set beresp.ttl = 1ms;
    }
    if(bereq.url ~ "^/vod/.*\.m3u8") {
    set beresp.ttl = 3600s;
    }
    }

    View Slide

  46. LIVE STREAMING
    VS
    VIDEO ON DEMAND

    View Slide

  47. LIVE
    ✓ LOW LATENCY
    ✓ CONSTANT PLAYLIST UPDATES
    ✓ LOW TTL ON PLAYLISTS
    ✓ ONLY THE LAST X SEGMENTS ARE
    REQUIRED
    ✓ SEGMENT SIZE TRADEOFF

    View Slide

  48. VOD
    ✓ NO PLAYLIST UPDATES
    ✓ HIGH TTL ON PLAYLISTS
    ✓ PRE-FETCHING POSSIBLE
    ✓ ALL SEGMENTS ARE REQUIRED
    ✓ STORAGE REQUIREMENTS

    View Slide

  49. STORAGE

    View Slide

  50. HEAD VS LONG TAIL
    HEAD
    LONG TAIL

    View Slide

  51. WHERE VARNISH STORES ITS OBJECTS
    ✓ MEMORY
    ✓ DISK
    ✓ MASSIVE STORAGE ENGINE (ENTERPRISE ONLY)

    View Slide

  52. MSE
    ✓ MIX OF MEMORY AND DISK
    ✓ SMARTER LRU
    ✓ PRE-ALLOCATED LARGE FILES ON DISK
    ✓ LESS DISK FRAGMENTATION
    ✓ INTELLIGENT STORAGE ROUTING

    View Slide

  53. env: {
    id = "myenv";
    memcache_size = "100G";
    };
    MSE.CONF

    View Slide

  54. env: {
    id = "myenv";
    memcache_size = "100G";
    books = ( {
    id = "book1";
    directory = "/var/lib/mse/book1";
    database_size = "1G";
    stores = ( {
    id = "store-1-1";
    filename = "/var/lib/mse/stores/disk1/store-1-1.dat";
    size = "2T";
    tags = ( "slow", "sata" );
    }, {
    tags = ( "fast", "ssd" );
    id = "store-1-2";
    filename = "/var/lib/mse/stores/disk2/store-1-2.dat";
    size = "500G";
    } );
    });
    default_stores = "none";
    };
    MSE.CONF

    View Slide

  55. mkfs.mse -c /var/lib/mse/mse.conf
    CREATE FILES

    View Slide

  56. varnishd -s mse,/var/lib/mse/mse.conf
    ATTACH TO
    VARNISH

    View Slide

  57. ROUND-ROBIN STORE SELECTION

    View Slide

  58. vcl 4.1;
    import mse;
    sub vcl_backend_response {
    mse.set_weighting(size);
    }
    SIZE
    AVAILABLE
    SMOOTH

    View Slide

  59. vcl 4.1;
    import mse;
    sub vcl_backend_response {
    if (beresp.ttl < 120s) {
    mse.set_stores("none");
    } else {
    if (beresp.http.Transfer-Encoding ~ "chunked" ||
    beresp.http.Content-Length > 1M) {
    mse.set_stores("sata");
    } else {
    mse.set_stores("fast");
    }
    }
    }

    View Slide

  60. DECISION MAKING AT THE EDGE

    View Slide

  61. DIGITAL RIGHTS MANAGEMENT*

    View Slide

  62. #EXTM3U
    #EXT-X-VERSION:3
    #EXT-X-TARGETDURATION:5
    #EXT-X-MEDIA-SEQUENCE:15
    #EXT-X-KEY:METHOD=AES-128,URI="/live/
    enc.key",IV=0x230f26620bfafa3cd420cb68c0647415
    #EXTINF:3.200000,
    stream_015.ts
    #EXTINF:4.800000,
    stream_016.ts
    #EXTINF:3.200000,
    stream_017.ts
    #EXTINF:4.800000,
    stream_018.ts
    #EXTINF:1.088000,
    stream_019.ts
    #EXT-X-ENDLIST
    DECRYPTION KEY

    View Slide

  63. .TS FILES ARE ENCODED AND
    CAN ONLY BE DECODED BY THE PLAYER

    View Slide

  64. ENC.KEY SHOULD BE PROTECTED

    View Slide

  65. vcl 4.1;
    backend default {
    .host = "1.2.3.4";
    .port = "80";
    }
    sub vcl_recv {
    if(req.url ~ "^/live/enc.key") {
    if (req.http.Authorization != "Basic YWRtaW46YWRtaW4=") {
    return (synth(401, "Restricted"));
    }
    }
    unset req.http.Authorization;
    }
    sub vcl_synth {
    if (resp.status == 401) {
    set resp.http.WWW-Authenticate =
    {"Basic realm="Restricted area""};
    }
    }

    View Slide

  66. VALIDATE SESSION FROM REDIS

    View Slide

  67. vcl 4.1;
    import cookieplus;
    import redis;
    import kvstore;
    backend default {
    .host = "origin";
    .port = "80";
    }
    sub vcl_init
    {
    new authorized_sessions = kvstore.init();
    new db = redis.db(
    location="redis:6379",
    type=master,
    connection_timeout=500,
    shared_connections=false,
    max_connections=1);
    }

    View Slide

  68. sub vcl_recv
    {
    if(req.url ~ "^/live/enc.key") {
    if(authorized_sessions.get(cookieplus.get("PHPSESSID"),"empty") == "empty") {
    db.command("GET");
    db.push("PHPREDIS_SESSION:"+cookieplus.get("PHPSESSID"));
    db.execute();
    if(db.get_string_reply() !~ "authorized\|b\:1\;") {
    authorized_sessions.set(cookieplus.get("PHPSESSID"),"false");
    return (synth(401, "Not Authorized"));
    } else {
    authorized_sessions.set(cookieplus.get("PHPSESSID"),"true", 10s);
    }
    } elseif(authorized_sessions.get(cookieplus.get("PHPSESSID")) == "false") {
    return (synth(401, "Not Authorized"));
    }
    }
    }

    View Slide

  69. GEO FEATURES

    View Slide

  70. vcl 4.1;
    import mmdb;
    backend default { .host = "192.0.2.11"; .port = "8080"; }
    # create a database object
    sub vcl_init {
    new geodb = mmdb.init("/path/to/db");
    }
    sub vcl_recv {
    # retrieve the name of the request's origin
    set req.http.Country-Name = geodb.country_name(client.ip);
    # if the country doesn't come from Germany or Belgium, deny access
    if (req.http.Country-Name != "Germany" ||
    req.http.Country-Name != "Belgium") {
    return (synth(403, "Sorry, only available in Germany and Belgium"));
    }
    }
    Geo
    blocking

    View Slide

  71. vcl 4.1;
    import geodirector;
    import mmdb;
    backend us {
    .host = "us.example.com";
    .port = "80";
    }
    backend uk {
    .host = "uk.example.com";
    .port = "80";
    }
    backend be {
    .host = "be.example.com";
    .port = "80";
    }
    sub vcl_init {
    new geodb = mmdb.init("/path/to/db");
    }

    View Slide

  72. sub vcl_recv {
    set req.http.Country-Code = geodb.country_code(client.ip);
    if(req.http.Country-Code == "US") {
    set req.backend_hint = us;
    } elseif(req.http.Country-Code == "GB") {
    set req.backend_hint = uk;
    } else {
    set req.backend_hint = be;
    }
    }
    Geo
    backend
    selection

    View Slide

  73. NO MORE ORIGIN

    View Slide

  74. vcl 4.1;
    import file;
    backend default {
    .host = "origin";
    .port = "80";
    }
    sub vcl_init {
    new root = file.init("/var/www/html/");
    }
    sub vcl_backend_fetch {
    set bereq.backend = root.backend();
    }

    View Slide

  75. VIDEO SEGMENT PREFETCHING

    View Slide

  76. vcl 4.1;
    import http;
    backend default {
    .host = "origin";
    .port = "80";
    }
    sub vcl_recv {
    if (req.url ~ "^/vod/.+\.ts$") {
    http.init(0);
    http.req_set_max_loops(0,1);
    http.req_copy_headers(0);
    http.req_set_method(0, "HEAD");
    set req.http.x-next-url = http.prefetch_next_url();
    http.req_set_url(0, req.http.x-next-url);
    http.req_send_and_finish(0);
    }
    }

    View Slide

  77. RATE LIMITING & THROTTLING

    View Slide

  78. vcl 4.1;
    import vsthrottle;
    import tcp;
    backend default {
    .host = "origin";
    .port = "80";
    }
    sub vcl_recv {
    if (vsthrottle.is_denied(client.identity, 15, 10s, 60s)) {
    # Client has exceeded 15 reqs per 10s or get blocked for 60s
    return (synth(429, "Too Many Requests"));
    }
    # Download at 1 MB/s
    tcp.set_socket_pace(1000);
    }

    View Slide

  79. View Slide

  80. View Slide

  81. THE VMOD ECOSYSTEM
    HTTPS://VARNISH-CACHE.ORG/VMODS/
    HTTPS://DOCS.VARNISH-SOFTWARE.COM/VARNISH-CACHE-PLUS/VMODS/

    View Slide

  82. FROM $0.20/H + 14 DAYS FREE TRIAL

    View Slide

  83. View Slide

  84. View Slide

  85. HTTPS://FERYN.EU
    HTTPS://TWITTER.COM/THIJSFERYN
    HTTPS://INSTAGRAM.COM/THIJSFERYN

    View Slide