Accelerating OTT video platforms with Varnish - London Video Tech meetup 2020

Slides for my OTT video acceleration talk at the London Video Technology meetup February 2020 edition: https://feryn.eu/speaking/accelerating-ott-video-platforms-varnish-london-video-technology-meetup-2020/

Transcript

1. ACCELERATING OTT
   VIDEO PLATFORMS
   WITH VARNISH
   BY THIJS FERYN
   

   View Slide

2. I'M NOT A VIDEO EXPERT
   

   View Slide

3. Slow websites
   SUCK
   

   View Slide

4. View Slide

5. is bollocks
   

   View Slide

6. CONTENT DELIVERY CHALLENGES
   

   View Slide

7. STABILITY
   

   View Slide

8. REDUCE LATENCY
   

   View Slide

9. SERVER
   CAPACITY
   

   View Slide

10. NETWORK
    CAPACITY
    

    View Slide

11. THROWING
    SERVERS
    AT THE PROBLEM
    

    View Slide

12. MO' MONEY
    MO' SERVERS
    MO' PROBLEMS
    

    View Slide

13. CACHE
    

    View Slide

14. DON’T RECOMPUTE
    IF THE DATA
    HASN’T CHANGED
    

    View Slide

15. HI, I'M THIJS
    

    View Slide

16. I'M AN
    EVANGELIST
    AT
    

    View Slide

17. View Slide

18. View Slide

19. +-5,000,000 WEBSITES
    20% OF THE TOP 10K WEBSITES
    

    View Slide

20. View Slide

21. MORE THAN 750M VIEWERS PER MONTH
    

    View Slide

22. View Slide

23. I'M @THIJSFERYN
    

    View Slide

24. View Slide

25. VIDEO IS BIG & BROAD INDUSTRY
    

    View Slide

26. DELIVERY
    

    View Slide

27. View Slide

28. NORMALLY
    CLIENT SERVER
    

    View Slide

29. WITH REVERSE CACHING PROXY
    CLIENT PROXY SERVER
    

    View Slide

30. View Slide

31. WHY NOT
    USE A CDN?
    

    View Slide

32. VARNISH IS CDN SOFTWARE!
    

    View Slide

33. VARNISH VARNISH VARNISH
    ORIGIN
    

    View Slide

34. CDN CDN CDN
    ORIGIN
    VARNISH
    

    View Slide

35. STREAM.M3U8
    STREAM_01.TS
    STREAM_02.TS
    STREAM_03.TS
    THESE WILL BE
    CACHED
    

    View Slide

36. WHY IS VARNISH
    SO POWERFUL?
    

    View Slide

37. WHY IS VARNISH SO POWERFUL?
    ✓ EXTREMELY LOW RESOURCE
    ✓ EXTREMELY STABLE
    ✓ 100 GBIT PER SERVER
    ✓ REQUEST COALESCING
    ✓ VARNISH CONFIGURATION LANGUAGE
    ✓ VMODS
    ✓ COMPLIES TO HTTP BEST PRACTICES
    

    View Slide

38. ORIGIN
    VARNISH
    REQUEST
    COALESCING
    QUEUED
    CONNECTIONS
    SINGLE BACKEND
    FETCH
    

    View Slide

39. ORIGIN
    VARNISH
    GRACE
    SERVE STALE
    OBJECT
    BACKGROUND
    FETCH
    

    View Slide

40. Cache-control: public, max-age=3600,
    s-maxage=86400, stale-while-revalidate=100
    GRACE TTL
    

    View Slide

41. vcl 4.1;
    backend default {
    .host = "origin";
    .port = "80";
    }
    sub vcl_backend_response {
    if(bereq.url == "/vod/master.m3u8" ){
    set beresp.grace = 1h;
    }
    }
    

    View Slide

42. VARNISH CONFIGURATION LANGUAGE
    

    View Slide

43. View Slide

44. View Slide

45. vcl 4.1;
    backend origin {
    .host = "1.2.3.4";
    .port = "80";
    }
    sub vcl_recv {
    if(req.url ~ "^/(live|vod)/") {
    unset req.http.authorization;
    unset req.http.cookie;
    }
    }
    sub vcl_backend_response {
    if(bereq.url ~ "^/(live|vod)/") {
    unset beresp.http.set-cookie;
    set beresp.grace = 5s;
    }
    if(bereq.url ~ "^/live/.*\.m3u8") {
    set beresp.ttl = 1ms;
    }
    if(bereq.url ~ "^/vod/.*\.m3u8") {
    set beresp.ttl = 3600s;
    }
    }
    

    View Slide

46. LIVE STREAMING
    VS
    VIDEO ON DEMAND
    

    View Slide

47. LIVE
    ✓ LOW LATENCY
    ✓ CONSTANT PLAYLIST UPDATES
    ✓ LOW TTL ON PLAYLISTS
    ✓ ONLY THE LAST X SEGMENTS ARE
    REQUIRED
    ✓ SEGMENT SIZE TRADEOFF
    

    View Slide

48. VOD
    ✓ NO PLAYLIST UPDATES
    ✓ HIGH TTL ON PLAYLISTS
    ✓ PRE-FETCHING POSSIBLE
    ✓ ALL SEGMENTS ARE REQUIRED
    ✓ STORAGE REQUIREMENTS
    

    View Slide

49. STORAGE
    

    View Slide

50. HEAD VS LONG TAIL
    HEAD
    LONG TAIL
    

    View Slide

51. WHERE VARNISH STORES ITS OBJECTS
    ✓ MEMORY
    ✓ DISK
    ✓ MASSIVE STORAGE ENGINE (ENTERPRISE ONLY)
    

    View Slide

52. MSE
    ✓ MIX OF MEMORY AND DISK
    ✓ SMARTER LRU
    ✓ PRE-ALLOCATED LARGE FILES ON DISK
    ✓ LESS DISK FRAGMENTATION
    ✓ INTELLIGENT STORAGE ROUTING
    

    View Slide

53. env: {
    id = "myenv";
    memcache_size = "100G";
    };
    MSE.CONF
    

    View Slide

54. env: {
    id = "myenv";
    memcache_size = "100G";
    books = ( {
    id = "book1";
    directory = "/var/lib/mse/book1";
    database_size = "1G";
    stores = ( {
    id = "store-1-1";
    filename = "/var/lib/mse/stores/disk1/store-1-1.dat";
    size = "2T";
    tags = ( "slow", "sata" );
    }, {
    tags = ( "fast", "ssd" );
    id = "store-1-2";
    filename = "/var/lib/mse/stores/disk2/store-1-2.dat";
    size = "500G";
    } );
    });
    default_stores = "none";
    };
    MSE.CONF
    

    View Slide

55. mkfs.mse -c /var/lib/mse/mse.conf
    CREATE FILES
    

    View Slide

56. varnishd -s mse,/var/lib/mse/mse.conf
    ATTACH TO
    VARNISH
    

    View Slide

57. ROUND-ROBIN STORE SELECTION
    

    View Slide

58. vcl 4.1;
    import mse;
    sub vcl_backend_response {
    mse.set_weighting(size);
    }
    SIZE
    AVAILABLE
    SMOOTH
    

    View Slide

59. vcl 4.1;
    import mse;
    sub vcl_backend_response {
    if (beresp.ttl < 120s) {
    mse.set_stores("none");
    } else {
    if (beresp.http.Transfer-Encoding ~ "chunked" ||
    beresp.http.Content-Length > 1M) {
    mse.set_stores("sata");
    } else {
    mse.set_stores("fast");
    }
    }
    }
    

    View Slide

60. DECISION MAKING AT THE EDGE
    

    View Slide

61. DIGITAL RIGHTS MANAGEMENT*
    

    View Slide

62. #EXTM3U
    #EXT-X-VERSION:3
    #EXT-X-TARGETDURATION:5
    #EXT-X-MEDIA-SEQUENCE:15
    #EXT-X-KEY:METHOD=AES-128,URI="/live/
    enc.key",IV=0x230f26620bfafa3cd420cb68c0647415
    #EXTINF:3.200000,
    stream_015.ts
    #EXTINF:4.800000,
    stream_016.ts
    #EXTINF:3.200000,
    stream_017.ts
    #EXTINF:4.800000,
    stream_018.ts
    #EXTINF:1.088000,
    stream_019.ts
    #EXT-X-ENDLIST
    DECRYPTION KEY
    

    View Slide

63. .TS FILES ARE ENCODED AND
    CAN ONLY BE DECODED BY THE PLAYER
    

    View Slide

64. ENC.KEY SHOULD BE PROTECTED
    

    View Slide

65. vcl 4.1;
    backend default {
    .host = "1.2.3.4";
    .port = "80";
    }
    sub vcl_recv {
    if(req.url ~ "^/live/enc.key") {
    if (req.http.Authorization != "Basic YWRtaW46YWRtaW4=") {
    return (synth(401, "Restricted"));
    }
    }
    unset req.http.Authorization;
    }
    sub vcl_synth {
    if (resp.status == 401) {
    set resp.http.WWW-Authenticate =
    {"Basic realm="Restricted area""};
    }
    }
    

    View Slide

66. VALIDATE SESSION FROM REDIS
    

    View Slide

67. vcl 4.1;
    import cookieplus;
    import redis;
    import kvstore;
    backend default {
    .host = "origin";
    .port = "80";
    }
    sub vcl_init
    {
    new authorized_sessions = kvstore.init();
    new db = redis.db(
    location="redis:6379",
    type=master,
    connection_timeout=500,
    shared_connections=false,
    max_connections=1);
    }
    

    View Slide

68. sub vcl_recv
    {
    if(req.url ~ "^/live/enc.key") {
    if(authorized_sessions.get(cookieplus.get("PHPSESSID"),"empty") == "empty") {
    db.command("GET");
    db.push("PHPREDIS_SESSION:"+cookieplus.get("PHPSESSID"));
    db.execute();
    if(db.get_string_reply() !~ "authorized\|b\:1\;") {
    authorized_sessions.set(cookieplus.get("PHPSESSID"),"false");
    return (synth(401, "Not Authorized"));
    } else {
    authorized_sessions.set(cookieplus.get("PHPSESSID"),"true", 10s);
    }
    } elseif(authorized_sessions.get(cookieplus.get("PHPSESSID")) == "false") {
    return (synth(401, "Not Authorized"));
    }
    }
    }
    

    View Slide

69. GEO FEATURES
    

    View Slide

70. vcl 4.1;
    import mmdb;
    backend default { .host = "192.0.2.11"; .port = "8080"; }
    # create a database object
    sub vcl_init {
    new geodb = mmdb.init("/path/to/db");
    }
    sub vcl_recv {
    # retrieve the name of the request's origin
    set req.http.Country-Name = geodb.country_name(client.ip);
    # if the country doesn't come from Germany or Belgium, deny access
    if (req.http.Country-Name != "Germany" ||
    req.http.Country-Name != "Belgium") {
    return (synth(403, "Sorry, only available in Germany and Belgium"));
    }
    }
    Geo
    blocking
    

    View Slide

71. vcl 4.1;
    import geodirector;
    import mmdb;
    backend us {
    .host = "us.example.com";
    .port = "80";
    }
    backend uk {
    .host = "uk.example.com";
    .port = "80";
    }
    backend be {
    .host = "be.example.com";
    .port = "80";
    }
    sub vcl_init {
    new geodb = mmdb.init("/path/to/db");
    }
    

    View Slide

72. sub vcl_recv {
    set req.http.Country-Code = geodb.country_code(client.ip);
    if(req.http.Country-Code == "US") {
    set req.backend_hint = us;
    } elseif(req.http.Country-Code == "GB") {
    set req.backend_hint = uk;
    } else {
    set req.backend_hint = be;
    }
    }
    Geo
    backend
    selection
    

    View Slide

73. NO MORE ORIGIN
    

    View Slide

74. vcl 4.1;
    import file;
    backend default {
    .host = "origin";
    .port = "80";
    }
    sub vcl_init {
    new root = file.init("/var/www/html/");
    }
    sub vcl_backend_fetch {
    set bereq.backend = root.backend();
    }
    

    View Slide

75. VIDEO SEGMENT PREFETCHING
    

    View Slide

76. vcl 4.1;
    import http;
    backend default {
    .host = "origin";
    .port = "80";
    }
    sub vcl_recv {
    if (req.url ~ "^/vod/.+\.ts$") {
    http.init(0);
    http.req_set_max_loops(0,1);
    http.req_copy_headers(0);
    http.req_set_method(0, "HEAD");
    set req.http.x-next-url = http.prefetch_next_url();
    http.req_set_url(0, req.http.x-next-url);
    http.req_send_and_finish(0);
    }
    }
    

    View Slide

77. RATE LIMITING & THROTTLING
    

    View Slide

78. vcl 4.1;
    import vsthrottle;
    import tcp;
    backend default {
    .host = "origin";
    .port = "80";
    }
    sub vcl_recv {
    if (vsthrottle.is_denied(client.identity, 15, 10s, 60s)) {
    # Client has exceeded 15 reqs per 10s or get blocked for 60s
    return (synth(429, "Too Many Requests"));
    }
    # Download at 1 MB/s
    tcp.set_socket_pace(1000);
    }
    

    View Slide

79. View Slide

80. View Slide

81. THE VMOD ECOSYSTEM
    HTTPS://VARNISH-CACHE.ORG/VMODS/
    HTTPS://DOCS.VARNISH-SOFTWARE.COM/VARNISH-CACHE-PLUS/VMODS/
    

    View Slide

82. FROM $0.20/H + 14 DAYS FREE TRIAL
    

    View Slide

83. View Slide

84. View Slide

85. HTTPS://FERYN.EU
    HTTPS://TWITTER.COM/THIJSFERYN
    HTTPS://INSTAGRAM.COM/THIJSFERYN
    

    View Slide