Spiderman's Uncle Ben and his advice for casting your Web(RTC)

Even those of us who have been building applications with WebRTC for over a decade still make mistakes because of the very nature of what makes WebRTC so powerful: it’s made up of many component parts. Forgetting about one of those parts may result in no noticeable difference when developing your application but if you add even a smidge of poor network conditions, your application will end up failing... badly. We’re now in an era of WebRTC where we have multiple WebRTC libraries that don’t use libWebRTC underneath and that’s fantastic, but as Uncle Ben from the Spiderman series once said, “With great power comes great responsibility” - never has a phrase been more true for WebRTC today.

Transcript

1. Dan Jenkins - RTC.ON Krakow, Poland, October 2023 SPIDER-MAN'S UNCLE BEN AND HIS ADVICE FOR CASTING YOUR WEB(RTC)
2. first, a little about me
3. • Founder of Everycast Labs, Nimble Ape • Creator of CommCon • Was the fi rst "Google Developer Expert" in WebRTC • Loves LEGO • Loves Real-Time Media • Loves developing for "the web" • Hasn't seen the latest Spider-Man movie • @dan_jenkins / @danjenkins@fosstodon.org Dan Jenkins
4. • Creators of Broadcast Bridge • A Platform as a Service for bringing in remote talent into production AV work fl ows • We work with WebRTC / SRT / NDI / Decklink & AJA cards. • Bridging Real-Time Comms with the "Broadcast" industry is fun! • hello@everycastlabs.uk Everycast Labs
5. • Real-Time Communication consultancy • Based in the UK • Work with Open Source Real- Time Comms • VoIP, WebRTC, Broadcast • Got a problem you want help with? Let us know • hello@nimblea.pe Nimble Ape
6. • Residential Event in the UK for "Open Media" - Real-Time or not • Everyone stays in the same hotel • Top quality production values • 5 Years worth of content on our YouTube Channel • commcon.xyz • (currently looking for sponsorship to make 2024 happen, let us know if you're interested) CommCon
7. Public Service Announcement All images were generated with bing image crator (Dall-E)
8. We don't ha ve anything to fear
9. AI Will not destroy us
10. Spider-Man's Uncle Ben and His Advice For Casting your Web(RTC)
12. uncle ben w as right
13. With Great Power Comes Great Responsibility
14. I'm going to assume basic WebRTC understanding After the talks this morning
15. A Little History Lesson...
16. Ma y 2011 Google Open Sourced WebRTC
17. Ma y 2011 Google Open Sourced WebRTC
18. libWebRTC
19. Used in all the Browsers
20. The browsers (mostly) join all the APIs together for "magic"
21. getUserMedia() new MediaStream() new RTCPeerConnection() pc.createOffer() pc.createAnswer() pc.createDataChannel() pc.setLocalDescription() pc.setRemoteDescription() W3C APIs
22. Extremely low latency Encrypted by default Peer to Peer Media/Data NAT Busting Made from many IETF specs
23. When you're talking to your clone... your clone understands you
25. MAGIC
26. But libWebRTC is a bit of a pain
27. It is huge
28. BUT libWebRTC != WebRTC https://dyte.io/blog/understanding-libwebrtc/
29. It's 2023 and we have other options
30. Pion WebRTC.rs Elixir WebRTC AIORTC pipE libDataChannel
31. They are all Variants of the same thing
33. Now you've got Flexibility
34. Build with the right tool for the job
35. In the right language you w ant
36. But Remember
37. With Great Power Comes Great Responsibility
38. If you're not careful...
39. You'll fall.
40. All those things you took for granted...
41. Are NOW your responsibility
42. I'm Excited!
43. We use Pion at Everycast Labs
44. It's part of our Broadcast Bridge Product (alongside Janus and GStreamer)
45. And we're guilty of "falling"
46. Let's take it back to basics
47. We fell into the "basic example" trap
48. Get Media from Janus into Pion...
49. Get Media from Pion into GStreamer...
50. Output Media from GStreamer in X Format...
51. And it worked!
52. Y A Y!
53. DONE!
54. Right !?
55. WRONG!
56. We didn't implement RTCP handling
57. Dan... WHAT IS RTCP?!
58. were you not listening to Lorenzo?
59. RTCP provides out-of-band statistics and control information for an RTP session
60. What happens without RTCP handling?
62. Media Quality Issues
63. There's no w a y for the receiver to sa y "I don't ha ve some data please send HELP"
67. And what did I do about it?
68. As a "Temporary" hack... I increased the number of Keyframes sent from Janus
69. "Bad media? No Problem! We'll get a new keyframe momentarily"
70. BAD DAN!
72. It actually worked Really Well... A Little too WELL
73. But Obviously...
74. I was Wrong.
75. RTCP exists for a reason.
76. Tsahi has a great article on this https://bloggeek.me/webrtc-media-resilience/
77. Why did I get it so wrong?
78. Examples and The D Word
79. Documentation
80. I'm not blaming Pion.
81. I know better.
82. WebRTC is difficult.
83. And complex.
84. WebRTC has so many moving parts
85. I think this is Where I went wrong
86. Taken from the Pion Website
87. This is talking about the W3C API
88. I Expected too much "Magic"
89. Lets Look at An Example
90. Take RTX & NACK handling.
91. Without NACK handling
92. WITH NACK handling
94. Would I ever do this in Ja v ascript?
95. NO.
96. I got complacent
97. I sa w code like this in the examples
99. Ah! "Send a PLI every 3 seconds... win!"
100. BUT THIS IS AN EXAMPLE!
101. This isn't production quality code
102. BUt examples are how people learn...
103. OOPS...
104. WebRTC is by definition a group of other specifications
105. ICE DTLS RTCP RTP (To name ONLY a few)
106. This means this could be an issue across any of these components
107. RTP, codecs, simulcast, svc, rtp extensions, buffering, etc etc etc
108. WebRTC Can be daunting
109. But its also hugely powerful
110. Add in hugely flexible client libs
111. Win!
112. However...
113. you need to understand the core parts of WebRTC
114. For our app, that means PROPERLY HANDLING RTCP!
115. In your app... that means properly handling RTCP too!
116. Always handle RTCP.
117. Don't leave it for another day.
118. Interop is hard.
119. But I Love That we have it.
120. A world of only libWebRTC is a pretty dull one.
121. Chad Hart has been following these trends for years now https://webrtchacks.com/webrtc-open-source-most-popular/
122. More Choice is only a good thing.
123. But Remember
124. Think of Uncle Ben.
125. With Great Power Comes Great Responsibility.
126. Thanks!
127. everycastlabs.uk broadcastbridge.app nimblea.pe commcon.xyz @dan_jenkins @danjenkins@fosstodon.org Thanks!