Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Engineering a Live Streaming Workflow for Super Bowl 53 at CBS

Engineering a Live Streaming Workflow for Super Bowl 53 at CBS

Presentation about how we built the video processing and delivery pipelines for Super Bowl 53. Presented at the Boston Video Technology Meetup in August, 2019.

https://www.meetup.com/BOS-Video-Technology/events/262652682/

5dc4abe3dc14c25021ff2295fc380d73?s=128

Flávio Ribeiro

August 26, 2019
Tweet

Transcript

  1. Engineering a Live Streaming Workflow for Super Bowl 53 at

    CBS
  2. Flavio Ribeiro Director of Engineering, Video Processing Team

  3. AGENDA • Signal Acquisition & Encoding ◦ Bitrate Ladder ◦

    Output Locking • Advertisement System • Delivery • Monitoring • Testing • Game Day
  4. MISSION: PROMOTE A GOOD EXPERIENCE FOR OUR USERS THROUGHOUT THE

    GAME
  5. SIGNAL ACQUISITION & ENCODING

  6. SIGNAL ACQUISITION & ENCODING

  7. SIGNAL ACQUISITION & ENCODING ← Master Encoder @ NYBC

  8. SIGNAL ACQUISITION & ENCODING ← TIMECODE EMBEDDER FRAME TIMECODE EMBEDDER

    CARD ➞
  9. BITRATE LADDER • 400x224, 29.97 @ 250 Kbps (QVBR, Quality

    Level: 7) • 512x288, 29.97 @ 500 Kbps (QVBR, Quality Level: 7) • 640x360, 29.97 @ 900 Kbps (QVBR, Quality Level: 7) • 960x540, 29.97 @ 1.8 Mbps (QVBR, Quality Level: 7) • 1280x720, 29.97 @ 3 Mbps (QVBR, Quality Level: 7) • 1280x720, 59.94 @ 6 Mbps (QVBR, Quality Level, 8)
  10. QVBR

  11. OUTPUT LOCKING More about it: https://d2908q01vomqb2.cloudfront.net/fb644351560d8296fe6da332236b1f8d61b2828a/2019/02/25/AWS_Product-Documentation_OUTPUT-LOCKING.pdf • Same output medatada

    across encoders in the pool ◦ PTS/DTS • Encoders talk via Multicast • Distributed Consensus ◦ No master node • Beware of Ad Breaks and SCTE/ID3/API integrations
  12. ADVERTISEMENT SYSTEM

  13. DELIVERY

  14. OUTCOME

  15. OUTCOME

  16. MONITORING • Mux.com

  17. MONITORING • Mux.com • TouchStream

  18. MONITORING • Mux.com • TouchStream • SumoLogic

  19. MONITORING • Mux.com • TouchStream • SumoLogic • Manifest Monitor

  20. MONITORING • Mux.com • TouchStream • SumoLogic • Manifest Monitor

    • MultiScreen Viewer ◦ Clappr + Clappr RTMP + Nginx
  21. MONITORING • Mux.com • TouchStream • SumoLogic • Manifest Monitor

    • MultiScreen Viewer ◦ Clappr + Clappr RTMP + Nginx • Slack Alerts
  22. TESTING • Streaming in Production prior to Game Day •

    Identifying Potential Points of Failure ◦ Simulate Chaos • Runbooks / Playbooks
  23. TESTING • Streaming in Production prior to Game Day •

    Identifying Potential Points of Failure ◦ Simulate Chaos • Runbooks / Playbooks
  24. TESTING • Streaming in Production prior to Game Day •

    Identifying Potential Points of Failure ◦ Simulate Chaos • Runbooks / Playbooks
  25. TESTING • Streaming in Production prior to Game Day •

    Identifying Potential Points of Failure ◦ Simulate Chaos • Runbooks / Playbooks • Tooling for Quickly Making Changes ◦ Must be well tested
  26. GAME DAY NY - ENCODING SF - DELIVERY FL -

    SITE
  27. None
  28. GAME DAY Streaming coverage of Super Bowl LIII set viewership

    records, (...). Across 7.5 million unique devices, (...) viewers consumed more than 560 million total minutes of live game coverage, up +19% from last year. The average minute audience of 2.6 million viewers during the game window is up +31% year-over-year. Reference: https://www.cbspressexpress.com/cbs-sports/releases/view?id=51928
  29. WE ARE HIRING!

  30. Thanks! @flavioribeiro