Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Designing a Playback Client for Large-Scale Live Sports Events

Designing a Playback Client for Large-Scale Live Sports Events

These are the presentation slides for Taiwan Multimedia Tech #9 held on 2023.07.19.
ABEMA is a Japanese Free Ad-Supported Streaming Television service, but it is also rapidly growing as a live streaming platform. As its video streaming engineers, we have a mission to design and develop a playback client to deliver a large number of viewers all at once for live events including those of the world’s largest sports. In this session, I describe our new playback client for our next evolution as an online video platform. The topics include how the playback client helps avoiding heavy load and what we intend to bring it up for the future.

Yusuke Goto

July 20, 2023
Tweet

More Decks by Yusuke Goto

Other Decks in Technology

Transcript

  1. Designing a Playback Client for Large-Scale Live Sports Events Yusuke

    Goto CyberAgent, Inc. / AbemaTV, Inc. 2023.07.19 Taiwan Multimedia Tech #9
  2. 五藤 佑典 YUSUKE GOTO https://ygoto3.com/ @ygoto3_ • Majored in Graphic

    Design at California State University, San Bernardino Career History 1. Graphic / Web Designer 2. Marketer 3. Web Engineer 4. Video Engineer 5. Product Manager • CyberAgent Developer Expert at CyberAgent, Inc. • Technical Product Manager at AbemaTV, Inc.
  3. Agenda • Background • Separation of Concerns ◦ ABEMA’s Team

    Topologies ◦ The “Fluffy” project ◦ The models for the new playback client • For “sports” ◦ Good quality picture frames / High frame rate ◦ Stronger content protection • For “large-scale” ◦ Multi-CDN ◦ Region Multiplexing • For “live” ◦ Easy license request spike ◦ Low-latency
  4. What is ABEMA? Hybrid of TV & video Allowing you

    to enjoy both 24/7 linear streaming and video on demand that allows you to watch missed or exclusive content at your preferred time without registration. Largest collection of original episodes in Japan Offering over 30,000 episodes available for streaming at any given time with the largest* collection of original episodes among domestic video services. (*as of January 2022, based on in-house research) 100% professional content Delivering high-quality content through a production system that leverages the respective strengths of CyberAgent and TV Asahi. Diverse lineup Broadcasting approximately 20 channels of diverse genres 24/7, including Japan’s only 24-hour news channel, as well as original dramas, romance programs, anime, sports, and more.
  5. 96 million DLs in 6 years and 8 months since

    the launch of ABEMA* April 2016 2017 2018 2019 2020 2021 March 2023 2022 Launched on April 11, 2016 96 mil DL Unit: DLs (downloads) *DLs: Total number of apps downloaded
  6. April 2016 2017 2018 2019 2020 2021 March 2023 2022

    Unit = 10 thousands Surpassed the record high of 34.09 million WAU *1 *1:WAU = Weekly Active Users
  7. Recently more live sports events on ABEMA https://abematv.co.jp/posts/18966021/ / https://www.cyberagent.co.jp/news/detail/id=27812

    / https://times.abema.tv/articles/-/10017349 https://www.cyberagent.co.jp/en/news/detail/id=27733 / https://times.abema.tv/articles/-/10075408
  8. Live sports events are huge “THE MATCH 2022" Sold Over

    500,000 PPV Tickets https://www.cyberagent.co.jp/en/news/detail/id=27733
  9. Live sports events are huge “FIFA World Cup Qatar 2022™”

    : the largest traffic since the launch https://www.cyberagent.co.jp/news/detail/id=28368
  10. Streaming large-scale live sports events requires For “sports” • Good

    quality picture frames • High frame rate • Stronger content protection For “large-scale” • Multi-CDN • Region multiplexing • Bandwidth management For “live” • Easy license request spike • Low-latency
  11. However, our old system has … For “sports” • Good

    quality picture frames • High frame rate • Stronger content protection 1080p VBR 8M ~ 30fps No live DRM
  12. However, our old system has … For “large-scale” • Multi-CDN

    • Region multiplexing • Bandwidth management Single CDN Single Region Not managed
  13. However, our old system has … For “live” • Easy

    license request spike • Low-latency No guard against a spike Normal latency
  14. Streaming large-scale live sports events requires For “sports” • Good

    quality picture frames • High frame rate • Stronger content protection For “large-scale” • Multi-CDN • Region multiplexing • Bandwidth management For “live” • Easy license request spike • Low-latency We needed a new live video streaming system to satisfy these
  15. Before we talk about the actual features … • We

    need to think about how we can keep designing our new playback client for long • We have a team dedicated to video playback engineering called “Streaming Client”
  16. What is Team Topologies? Team Topologies is an approach to

    • Optimize productivity with timeline in a team-first priority • Design an organization using the idea of 4 abstract team roles and 3 interaction modes between the teams https://teamtopologies.com/book
  17. The 4 team roles • Stream-aligned team ◦ Plays an

    essential part of the organization and its business domain • Enabling team ◦ Fills in the gap in a particular technical domain that is essential to the business, yet is not satisfactorily equipped enough to the stream-aligned team • Complicated Subsystem team ◦ Develops a part which requires significant technical expertise in the system • Platform team ◦ Provides an internal product that makes stream-aligned teams self-directively achieve relevant outcomes The main app dev teams Streaming Client
  18. Team Topologies around Streaming Client team Mobile App Development Team

    (Stream-aligned) Web Browser App Development Team (Stream-aligned) CTV App Development Team (Stream-aligned) Game Console App Development Team (Stream-aligned) Streaming Client Team (Complicated subsystem)
  19. The 3 inter-team interaction modes • Collaboration ◦ working together

    for a defined period of time to discover new things • X-as-a-Service ◦ one team provides and one team consumes something “as a Service” • Facilitation ◦ one team helps and mentors another team https://teamtopologies.com/key-concepts
  20. The 3 inter-team interaction modes • Collaboration ◦ working together

    for a defined period of time to discover new things • X-as-a-Service ◦ one team provides and one team consumes something “as a Service” • Facilitation ◦ one team helps and mentors another team https://teamtopologies.com/key-concepts
  21. The collaboration mode Mobile App Development Team (Stream-aligned) Web Browser

    App Development Team (Stream-aligned) CTV App Development Team (Stream-aligned) Game Console App Development Team (Stream-aligned) Streaming Client Team (Complicated subsystem) working together working together working together working together Takes more time to optimize the viewing experience
  22. Shifting from Colloboration to X-as-a-Service • Collaboration ◦ working together

    for a defined period of time to discover new things • X-as-a-Service ◦ one team provides and one team consumes something “as a Service” • Facilitation ◦ one team helps and mentors another team https://teamtopologies.com/key-concepts
  23. The X as a Service mode Mobile App Development Team

    (Stream-aligned) Web Browser App Development Team (Stream-aligned) CTV App Development Team (Stream-aligned) Game Console App Development Team (Stream-aligned) Streaming Client Team (Complicated subsystem) Uses the playback subsystem as a service Uses the playback subsystem as a service Uses the playback subsystem as a service Uses the playback subsystem as a service Unidirectional : optimization process gets faster
  24. The solution To develop a playback client that keeps evolving

    with the X-as-a-Service interaction mode The “Fluffy” project
  25. Team Topologies around the online video system development Stream-aligned teams

    Streaming Client Team (Complicated subsystem) Content Delivery Platform Team (Platform) Decisions on viewing quality are made independently between Streaming Client Team and Content Delivery Platform Team
  26. Team Topologies around the online video system development Stream-aligned teams

    Streaming Client Team (Complicated subsystem) Content Delivery Platform Team (Platform) How you separate the teams’ concerns is extremely important
  27. Separation of concerns b/w the Stream-aligned app and Fluffy The

    app’s concerns • What a user watches • How a user watches a content Content UseCase Hided by Fluffy Fluffy’s concerns • Playback stability • Resolution • In-stream ad insertion method • Content protection method Content Session
  28. The key concept of Fluffy’s separation of concerns Content Session

    • The session unit which starts when a user selects a content and ends when he/she leaves a content What is different than before? • We used to use a manifest as a unit of the playback session
  29. What is wrong with a manifest basis session • Which

    manifest you use deeply depends on streaming protocols ◦ Fluffy cannot freely decide which streaming protocol or player library is the best to use if the app specifies a manifest before it is passed to Fluffy • Sessions change even though one content has multiple options for manifest Many video streaming player libraries start playback session by manifest => Adopting the manifest basis session model is natural behaviour
  30. Expressing different use cases (MPEG-DASH) <MPD … type=”dynamic” …> …

    <SegmentTimeline> <S t=”882000” d=”750000” r=”7” /> <S t=”882000” d=”558000” /> </SegmentTimeline> … </MPD> <MPD … type=”dynamic” …> … <SegmentTimeline> <S t=”0” d=”750000” /> <S d=”750000” /> … </SegmentTimeline> … </MPD> <MPD … type=”static” …> … <SegmentTimeline> <S t=”0” d=”750000” /> <S d=”750000” /> … </SegmentTimeline> … </MPD> LIVE watch STARTOVER watch CATCHUP watch • The same session for the same encoded segments even in different manifests https://peach.blender.org/2008/04/the-use-of-violence-in-open-source-productions/
  31. Expressing different use cases (HLS) #EXTM3U #EXT-X-VERSION:6 #EXT-X-TARGETDURATION:9 #EXT-X-MEDIA-SEQUENCE:2 #EXT-X-MAP:URI="init.mp4"

    #EXTINF:8.333, segment_1632000.m4s #EXTINF:8.333, segment_2385000.m4s #EXTM3U #EXT-X-PLAYLIST-TYPE:EVENT #EXT-X-VERSION:6 #EXT-X-TARGETDURATION:9 #EXT-X-MEDIA-SEQUENCE:0 #EXT-X-MAP:URI="init.mp4" #EXTINF:8.333, segment_132000.m4s #EXTINF:8.333, segment_882000m4s … #EXTM3U #EXT-X-PLAYLIST-TYPE:VOD #EXT-X-VERSION:6 #EXT-X-TARGETDURATION:9 #EXT-X-MEDIA-SEQUENCE:0 #EXT-X-MAP:URI="init.mp4" #EXTINF:8.333, segment_132000.m4s #EXTINF:8.333, segment_882000.m4s … #EXT-X-ENDLIST LIVE watch STARTOVER watch CATCHUP watch • The same session for the same encoded segments even in different manifests https://peach.blender.org/2008/04/the-use-of-violence-in-open-source-productions/
  32. 1 session Use Case and Content Session • Switching Use

    Cases does not mean changing sessions Stream-aligned app Fluffy UseCase STARTOVER Content Session Live manifest Startover manifest Catch-up manifest Low-latency manifest UseCase LIVE 1 2 1 2 Content Session provides optimization options Such as caching data in common throughout the session
  33. Team Topologies around the online video system development Stream-aligned teams

    Streaming Client Team (Complicated subsystem) Content Delivery Platform Team (Platform) The “Playback Resource” allows Fluffy and the Content Delivery Platform to exchange information related to a viewing quality decision
  34. The “Playback Resource” Multiple resources for playback deriving from the

    identical video source • Viewing use cases • DRM key systems • Encoding patterns • In-stream ad insertion methods • CDN balancing methods message PlaybackResource {
 
 message Manifest {
 string arin = 1;
 url = 2;
 Manifest failover = 3;
 AdInsertion adInsertion = 4;
 CdnBalancing = 5;
 …
 }
 
 message Stream {
 Drm drm = 1;
 UseCase useCase = 2;
 FrameRate = frameRate = 3;
 repeated Manifest manifests = 4;
 …
 }
 
 string arin = 1;
 repeated Stream streams = 2;
 …
 
 }
 Fluffy decides which manifest is the best by the meta data
  35. Selecting a manifest from the “Playback Resource” Fluffy Stream-aligned app

    Content Delivery Platform Content Use Case ARIN Playback Resource Decides the best manifest

  36. Now we have the power to make things forward Fluffy,

    the new playback client, is the key to evolving the viewing experience Stream-aligned app Content UseCase Fluffy Content Delivery Platform Content identifier PlaybackResource We can make a lot of experiments and adopt new technologies to maximize the viewing experience without affecting the implementation in the Stream-aligned app Let’s see what we can do now
  37. Streaming large-scale live sports events requires For “sports” • Good

    quality picture frames • High frame rate • Stronger content protection For “large-scale” • Multi-CDN • Region multiplexing • Bandwidth management For “live” • Easy license request spike • Low-latency We will see what we do for each essence
  38. Resolution We love to watch sports in a high resolution

    • 4K / Good-quality Full HD • 60p - high framerate
  39. The tradeoff vs To deliver the top-quality video to the

    viewers To deliver information to every device As an entertainment platform An a social infrastructure The higher quality we deliver, the less devices benefit from it
  40. The tradeoff vs To deliver the top-quality video to the

    viewers To deliver information to every device As an entertainment platform An a social infrastructure The higher quality we deliver, the less devices benefit from it A playback client is unable to stably decode more data than that exceeding its device performance Required to balance between the device performance and the video quality
  41. The encoding patterns We classify video encoding use cases into

    3 categories to provide video encoding optimized to each device’s performance and use case • Defender ◦ Provides a video stream playable to any device ◦ Provides accessible information as a social infrastructure • Midfielder ◦ Provides video quality following the current standard fashion • Striker ◦ Provides the best possible quality and make the most of the content characteristics ▪ Examples: Sports for HFR, Music live concerts for immersive audio
  42. The quality composition Example: “FIFA World Cup Qatar 2022™” -

    Prepared HFR The “Striker” to make the most out of soccer’s video characteristics and entertainment essences Content-optimized composition a.k.a. Striker Coverage-prioritized composition a.k.a. Defender FPS 59.94 29.97 Rate Control Mode V:QVBR/A:VBR V/A:CBR Profile/Level [email protected] / <[email protected] Main4 1080p V:QVBR Max12M(Buf:24M) A:VBR 256k Stereo V:CBR 8M(Buf:auto) A:128k Stereo 720p V:QVBR Max8M(Buf:16M) A:VBR 256k Stereo V:;CBR 4M(Buf:auto) A:128k Stereo 480p V:QVBR Max4M(Buf:8M) A:VBR 256k Stereo V:CBR 2M(Buf:auto) A:128k Stereo 360p V:QVBR Maxim (Buf:4M) A:VBR 256k Stereo V:CBR 1M(Buf:auto) A:128k Stereo 240p V:CBR 0.5M(Buf:auto) A:128k Stereo V:CBR 0.5M(Buf:auto) A:128k Stereo
  43. “Large-scale” means high content value The more people watch the

    content, the higher value it has • A large-scale event requires even a stronger content protection method • The strongest content protection method is desirable, but DRM systems depend on streaming protocols
  44. Content protection depends on a streaming protocol ABEMA uses 2

    streaming protocols • HTTP Live Streaming a.k.a. HLS • MPEG-DASH
  45. The old adoption The old approach to ABEMA’s streaming protocol

    adoption Prefered HLS unless MPEG-DASH is the only option HLS • iOS app • tvOS app • Android mobile app • Android TV app • Nintendo Switch app • Web app on Safari (Desktop) • Web app on Safari (Mobile) • Web app on Chrome (Android mobile) • Google Cast receiver app MPEG-DASH • HTML5-based TV app • Web app on Edge • Web app on Chrome (Desktop) - Strong security in HTML Living Standard depends on EME(Encrypted Media Extensions)and Common Encryption - MPEG-DASH’s ecosystem has more advantage for EME and Common Encryption
  46. The new adoption Adopted MPEG-DASH on as many platforms as

    possible for the new system Leads to the most secure content protection and flexible expression on manifest HLS • iOS app • tvOS app • Nintendo Switch app • Web app on Safari (Desktop) • Web app on Safari (Mobile) MPEG-DASH • HTML5-based TV app • Web app on Edge • Web app on Chrome (Desktop) • Android mobile app • Android TV app • Web app on Chrome (Android mobile) • Google Cast receiver app - MPEG-DASH provides the other platforms with more security and flexibility - Apple HLS has far better advantage for products by Apple - The video ecosystem for the game domain is not mature enough
  47. Variations Streaming Protocols Secure content protection Making the most out

    of its platform’s standard protection methods DRM - Digital Rights Management Custom DRM Apple FairPlay Streaming • iOS app • tvOS app • Web app on Safari (Desktop) • Web app on Safari (Mobile) Google Widevine • HTML5-based TV app • Web app on Edge • Web app on Chrome (Desktop) • Android mobile app • Android TV app • Web app on Chrome (Android mobile) • Google Cast receiver app Original protection based on HLS AES-128 • Nintendo Switch app Apple products are most securely protected by FairPlay Streaming HLS AES-128 is relatively weak, so we limit the resolution up to SD Widevine is royalty-free covering the most products
  48. One final variations of streams mapped to device types HLS

    Striker FairPlay Streaming HLS Defender FairPlay Streaming HLS Defender Custom DRM (upto SD) MPEG-DASH Striker Widevine MPEG-DASH Defender Widevine iOS app Supports Supports - - - tvOS app Supports Supports - - - Android mobile app - - - Supports (Partially) Supports Android TV / Fire TV app - - - Supports (Partially) Supports Web desktop app on Chrome / Edge - - - - Supports Web desktop app on Safari - - - - Supports Web mobile app on Chrome - - - - Supports Web mobile app on Safari - - - - Supports Nintendo Switch app - - Supports - - Google Cast receiver app - - - - Supports Encoding Strategies Content Protection Methods
  49. Large-scale media traffic Streaming video’s large traffic is mostly made

    of tons of media segment data The first thing to care The most effective solution: Multi-CDN balancing
  50. Client-side multi-CDN balancing methods • Manifest basis balancing ◦ The

    player keeps fetching manifests and segments through the same CDN that the CDN decision server selects at the beginning of playback • Segment basis balancing ◦ The CDN decision server decides which CDN the player uses at the beginning of playback, but the player switches CDNs anytime during playback Balancing logic tends to be complicated, but adjusting load during playback is even possible
  51. • NPAW CDN Selector => Manifest basis balancing • NPAW

    CDN Active Switching w/ NPAW CDN Selector => Segment basis balancing NPAW CDN Balancing provides the 2 kinds
  52. • CDN Active Switching Features ◦ Control by the client-side

    SDK ◦ Standalone CDN health check ◦ Manipulates the manifest to switch CDNs segment by segment Testing NPAW CDN Active Switching Manifest manipulation makes an change unexpected for us in our manifest => Need more adjustments Result: Have not adopted Active Switching - Why?
  53. • NPAW CDN Selector uses tons of QoE data logged

    in NPAW Video Analytics, which we have been using for 5 years ◦ Can conduct a health check by playback quality data related to each CDN route Still, NPAW CDN Selector benefits us We adopted NPAW CDN Selector for “FIFA World Cup Qatar 2022™”
  54. The important features of CDN balancer • Weighting round-robin ◦

    To control CDN distribution based on the network condition, the price ranges, and so on • To apply a different rule to each viewing style ◦ Live playback ◦ Startover playback ◦ Catch-up playback • To apply a different rule to each angle ◦ The main angle is expected to be more redundant than the sub angles
  55. The important features of CDN balancer • Weighting round-robin ◦

    To control CDN distribution based on the network condition, the price ranges, and so on • To apply a different rule to each viewing style ◦ Live playback ◦ Startover playback ◦ Catch-up playback • To apply a different rule to each angle ◦ The main angle is expected to be more redundant than the sub angles Requires the performance data to decide weight based on the current playback quality
  56. The important features of CDN balancer • Weighting round-robin ◦

    To control CDN distribution based on the network condition, the price ranges, and so on • To apply a different rule to each viewing style ◦ Live playback ◦ Startover playback ◦ Catch-up playback • To apply a different rule to each angle ◦ The main angle is expected to be more redundant than the sub angles Priority changes when a viewing style changes • Live = temporary high load ◦ Performance > Price • Catch-up = distributed load ◦ Performance < Price
  57. The important features of CDN balancer • Weighting round-robin ◦

    To control CDN distribution based on the network condition, the price ranges, and so on • To apply a different rule to each viewing style ◦ Live playback ◦ Startover playback ◦ Catch-up playback • To apply a different rule to each angle ◦ The main angle is expected to be more redundant than the sub angles • The main angle has the most concurrent viewers in general • The main angle’s video quality is set possibly better than the sub angles ◦ More traffic volume per playback session
  58. Region multiplexing We use AWS Elemental MediaLive as the origin

    of our new online video system We multiplex public cloud regions in order to avoid a service outage due to a system failure on the cloud region
  59. How to switch regions Making use of the CDN balancing

    feature • Balancing sets of CDN providers and cloud regions using the CDN balancing feature Akamai for Tokyo Failure CloudFront for Seoul Failure Select another CDN / region CloudFront for Tokyo Failure Akamai for Seoul Example Select another CDN / region Select another CDN / region
  60. Easing a burst of region switching A burst of region

    switching possibly causes a service outage Assumable examples are • Playback clients start switching a request to the other CDN all at once when a failure occurs on one CDN • Playback clients start requesting to the other cloud region all at once when a failure occurs to the media origin (AWS Elemental MediaLive) on one cloud region or to the cloud region itself
  61. Prioritize switching regions over switching CDNs We simply switch regions

    first and switch CDNs later => There is higher risk that a number of concurrent viewers will fail playback possibly several times if the region is really in trouble CDN Group A CDN A for Region 1 CDN A for Region 2 CDN Group B CDN B for Region 1 CDN B for Region 2 Failure Failure Failure Failure There is no way to judge if a playback session failure comes from the CDN’s trouble or the region’s trouble
  62. The CDN and region balancing mechanism Tokyo Origin Fluffy Seoul

    Origin NPAW CDN Selector Original Balancer module Akamai CloudFront Akamai CloudFront z z z z z z z
  63. Resolution management is required because • The number of viewers

    is more important than the good resolution that a limited number of viewers receive We are to limit the resolution in case traffic gets heavy and let more people watch our contents
  64. The resolution management requirements • To limit the maximum resolution

    the playback client receives at any given time • To reflect the limit within minutes from the moment the request is called
  65. The resolution management mechanism Fluffy Feature Toggle System Feature Toggle

    SDK Bucketeer : • Sets the maximum resolution Bucketeer SDK: • Allows the playback client to consume a video stream up to the specified resolution • Reflects a setting in at most 10 minutes
  66. How the resolution management works 720p 480p 10 mins Sets

    the max resolution to 480p Most of the playback is limited to 480p
  67. License requests We support 3 kinds of DRM key systems

    DRM - Digital Rights Management Custom DRM Apple FairPlay Streaming • iOS app • tvOS app • Web app on Safari (Desktop) • Web app on Safari (Mobile) Google Widevine • HTML5-based TV app • Web app on Edge • Web app on Chrome (Desktop) • Android mobile app • Android TV app • Web app on Chrome (Android mobile) • Google Cast receiver app Original protection based on HLS AES-128 • Nintendo Switch app Each license server requires a different capacity plan More requests Less requests
  68. Spikes caused by ad breaks Spikes are caused by DRM

    license requests at the end of ad breaks • In-stream ad video is plaintext • The main content is DRM protected • The player needs a license challenge again entering the main content period The end of Ad break The end of Ad break The end of Ad break The end of Ad break
  69. Caching license key • The DRM packager applies the same

    license key to periods throughout the main content ◦ Key rotation costs too much for a several hour long content • The playback client caches the last license key in a playback session • The question is : Do the player libraries cache the license key? ◦ AVPlayer (iOS/tvOS) : OK ◦ ExoPlayer (Android OS) : OK ◦ THEOplayer (Web Browser) : NG with a problem under our particular condition ◦ CRI VOD Player (Nintendo Switch) : NG
  70. Case study for the custom DRM Public / commercial player

    libraries have no optimization feature for the custom DRM • CRI VOD Player, a streaming player library for game consoles, caches no key • We and CRI Middleware, the library developer, had expanded the library pluggable in order to add our custom DRM system feature • We developed one more plugin to cache the very last license key and reuse it while the specified key id is the same
  71. Case study for Google Widevine • Context: We have been

    using THEOplayer on Web since the launch of ABEMA • THEOplayer makes a license request each time the playhead entered a new period if useSeamlessPeriodSwitch = 'never' ◦ useSeamlessPeriodSwitch = 'never' means it disables the feature to switch periods seamlessly ◦ Why we disabled the feature - To provide a workaround to the issue that ugly dropped frames happened crossing the periods for ads that we had encoded (a compatibility issue) dash.js THEOplayer Luckily, no compatibility issue
  72. Hiding the details of low-latency protocol Still work in progress

    • Having difficulty providing low-latency live events ◦ Issues - flexible ad frequency management and DRM protection • However, experimental trials are easier to conduct with Fluffy Stream-aligned app Fluffy UseCase LOW_ LATENCY Content Session CMAF-CTE LL-HLS HESP WebRTC Available UseCases ? ? ? ?
  73. Takeaway • We designed a brand-new playback client to optimize

    the viewing experience for large-scale live sports events • The separation of concerns between the Stream-aligned apps and the video playback subsystem is the key to the optimization growth • The new playback client supports the essences of “sports,” “large-scale,“ and “live” with significant features