Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building in-house CDN by Fredrik Widlund from SVT

Building in-house CDN by Fredrik Widlund from SVT

Fredrik Widlund from SVT sharing their experiences from building their own in-house CDN. Presented at Streaming Tech Sweden on Nov 22, 2017

E1f34f8ef7c9562f8c1a6606dbf2d968?s=128

Streaming Tech Sweden

November 22, 2017
Tweet

More Decks by Streaming Tech Sweden

Other Decks in Technology

Transcript

  1. None
  2. INTRODUCTION

  3. whoami • Programmer / Systems architect • Like to take

    things apart • Like to build on ideas and concepts • Security (Defcom) • Streaming (Qbrick, SVT) • AI / Computer Vision (OculusAI) • Big data analytics (Meltwater) • https://github.com/fredrikwidlund
  4. WHY BUILD A CDN

  5. 2010 • SVT uses Qbrick CDN for streaming • SVT

    is Qbrick’s largest customer • Losing that contract is a business risk
  6. 2010 • Could SVT build a CDN of their own

    + Content + Volume - Domain knowledge - Complexity
  7. 2015 • Qbrick lost the SVT contract to Akamai •

    SVT second in Sweden in terms of volume • Users online are about 6 times more expensive • Rapid cost growth • Innovation blockage
  8. Business proposition • Put together a team of 4-5 people

    • Full scale production in one year • Distribute 30% of content ourselves • Self-financing • Break trend with rapid cost increase • Address innovation issues
  9. HOW DO YOU BUILD A CDN

  10. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE

  11. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE PROVIDERS

  12. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE PROVIDERS

  13. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE PROVIDERS

  14. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE PROVIDERS

  15. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE PROVIDERS

  16. TRAFFIC NETWORK DISTRIBUTION PACKAGING PROVIDERS STORAGE General technology stack •

    Commodity hardware • Stable chipsets (Intel) • Persistence (Disk, Flash, RAM) • Networking (NICs, switches, routers) • Linux (CoreOS) • systemd • Docker
  17. TRAFFIC NETWORK DISTRIBUTION PACKAGING PROVIDERS STORAGE

  18. TRAFFIC NETWORK DISTRIBUTION PACKAGING PROVIDERS STORAGE - SAS / JBOD

    / ZFS / GLUSTER
  19. TRAFFIC NETWORK DISTRIBUTION PACKAGING PROVIDERS STORAGE - SAS / JBOD

    / ZFS / GLUSTER
  20. TRAFFIC NETWORK DISTRIBUTION PACKAGING - WOWZA PROVIDERS STORAGE - SAS

    / JBOD / ZFS / GLUSTER
  21. TRAFFIC NETWORK DISTRIBUTION PACKAGING - WOWZA PROVIDERS STORAGE - SAS

    / JBOD / ZFS / GLUSTER
  22. TRAFFIC NETWORK DISTRIBUTION - NGINX / NVMe PACKAGING - WOWZA

    PROVIDERS STORAGE - SAS / JBOD / ZFS / GLUSTER
  23. TRAFFIC DISTRIBUTION - NGINX / NVMe PACKAGING - WOWZA PROVIDERS

    STORAGE - SAS / JBOD / ZFS / GLUSTER NETWORK
  24. TRAFFIC - TRANSIT / PUBLIC PEERING / PRIVATE PEERING DISTRIBUTION

    - NGINX / NVMe PACKAGING - WOWZA PROVIDERS STORAGE - SAS / JBOD / ZFS / GLUSTER NETWORK
  25. Other things we worked on • Load balancing • Redundancy

    • API performance • Alerting and error handling • Quality measurements • Event and log management • Provisioning • HTTPS
  26. HOW DID WE DO

  27. Business value • In full production after a year •

    First year we handled 30% of the total volume in-house • Currently we handle about 50% • Distribution costs significantly lowered • Project was self-financed • Innovation boost • Knowledge boost
  28. Quality

  29. None
  30. Takeaways • Simplify when possible • Optimise when needed •

    Keep focus on business value • Don’t start from scratch
  31. QUESTIONS?