Building in-house CDN by Fredrik Widlund from SVT

Fredrik Widlund from SVT sharing their experiences from building their own in-house CDN. Presented at Streaming Tech Sweden on Nov 22, 2017

Transcript

2. INTRODUCTION
3. whoami • Programmer / Systems architect • Like to take things apart • Like to build on ideas and concepts • Security (Defcom) • Streaming (Qbrick, SVT) • AI / Computer Vision (OculusAI) • Big data analytics (Meltwater) • https://github.com/fredrikwidlund
4. WHY BUILD A CDN
5. 2010 • SVT uses Qbrick CDN for streaming • SVT is Qbrick’s largest customer • Losing that contract is a business risk
6. 2010 • Could SVT build a CDN of their own + Content + Volume - Domain knowledge - Complexity
7. 2015 • Qbrick lost the SVT contract to Akamai • SVT second in Sweden in terms of volume • Users online are about 6 times more expensive • Rapid cost growth • Innovation blockage
8. Business proposition • Put together a team of 4-5 people • Full scale production in one year • Distribute 30% of content ourselves • Self-financing • Break trend with rapid cost increase • Address innovation issues
9. HOW DO YOU BUILD A CDN
10. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE
11. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE PROVIDERS
12. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE PROVIDERS
13. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE PROVIDERS
14. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE PROVIDERS
15. TRAFFIC NETWORK DISTRIBUTION PACKAGING STORAGE PROVIDERS
16. TRAFFIC NETWORK DISTRIBUTION PACKAGING PROVIDERS STORAGE General technology stack • Commodity hardware • Stable chipsets (Intel) • Persistence (Disk, Flash, RAM) • Networking (NICs, switches, routers) • Linux (CoreOS) • systemd • Docker
17. TRAFFIC NETWORK DISTRIBUTION PACKAGING PROVIDERS STORAGE
18. TRAFFIC NETWORK DISTRIBUTION PACKAGING PROVIDERS STORAGE - SAS / JBOD / ZFS / GLUSTER
19. TRAFFIC NETWORK DISTRIBUTION PACKAGING PROVIDERS STORAGE - SAS / JBOD / ZFS / GLUSTER
20. TRAFFIC NETWORK DISTRIBUTION PACKAGING - WOWZA PROVIDERS STORAGE - SAS / JBOD / ZFS / GLUSTER
21. TRAFFIC NETWORK DISTRIBUTION PACKAGING - WOWZA PROVIDERS STORAGE - SAS / JBOD / ZFS / GLUSTER
22. TRAFFIC NETWORK DISTRIBUTION - NGINX / NVMe PACKAGING - WOWZA PROVIDERS STORAGE - SAS / JBOD / ZFS / GLUSTER
23. TRAFFIC DISTRIBUTION - NGINX / NVMe PACKAGING - WOWZA PROVIDERS STORAGE - SAS / JBOD / ZFS / GLUSTER NETWORK
24. TRAFFIC - TRANSIT / PUBLIC PEERING / PRIVATE PEERING DISTRIBUTION - NGINX / NVMe PACKAGING - WOWZA PROVIDERS STORAGE - SAS / JBOD / ZFS / GLUSTER NETWORK
25. Other things we worked on • Load balancing • Redundancy • API performance • Alerting and error handling • Quality measurements • Event and log management • Provisioning • HTTPS
26. HOW DID WE DO
27. Business value • In full production after a year • First year we handled 30% of the total volume in-house • Currently we handle about 50% • Distribution costs significantly lowered • Project was self-financed • Innovation boost • Knowledge boost
28. Quality
30. Takeaways • Simplify when possible • Optimise when needed • Keep focus on business value • Don’t start from scratch
31. QUESTIONS?