– Building ICEPerf – JanusCon 2024
Building
RelayPerf.com
1
Slide 2
Slide 2 text
– Building ICEPerf – JanusCon 2024
A Little About Us
2
Slide 3
Slide 3 text
– Building ICEPerf – JanusCon 2024
Dan Jenkins
3
Founder of Everycast Labs, Nimble Ape
Creator of CommCon
Was the
f
irst "Google Developer
Expert" in WebRTC
Loves LEGO
Loves Real-Time Media
Loves developing for "the web"
Over 10 years in Open Source RTC
@dan_jenkins /
@[email protected]
Slide 4
Slide 4 text
– Building ICEPerf – JanusCon 2024
Marco Vidonis
4
Software Engineer at
Nimble Ape
3 years @ Nimble Ape
Leeds, UK
OCRs and sailing
Half marathon
Slide 5
Slide 5 text
– Building ICEPerf – JanusCon 2024
Nimble Ape
5
Real-Time Communication
consultancy
Based in the UK
Work with Open Source Real-
Time Comms
VoIP, WebRTC, Broadcast/
Video delivery
[email protected]
Slide 6
Slide 6 text
– Building ICEPerf – JanusCon 2024
Everycast Labs
6
Creators of Broadcast Bridge
(broadcastbridge.app)
A Platform as a Service for
bringing in remote talent into
production AV work
f
lows
We work with WebRTC / SRT / NDI /
Decklink & AJA cards and more
[email protected]
Slide 7
Slide 7 text
– Building ICEPerf – JanusCon 2024
CommCon
7
"Open Media" - Real-Time or not
5 Years worth of content on our
YouTube Channel
commcon.xyz
London & San Francisco June
2024 - 1 Day Hackathon, 1 Day
Conference
Slide 8
Slide 8 text
– Building ICEPerf – JanusCon 2024
Building
RelayPerf.com
8
Slide 9
Slide 9 text
– Building ICEPerf – JanusCon 2024
Building
RelayPerf.com
9
Slide 10
Slide 10 text
– Building ICEPerf – JanusCon 2024
Building
ICEPerf.com
10
Slide 11
Slide 11 text
– Building ICEPerf – JanusCon 2024
11
Slide 12
Slide 12 text
– Building ICEPerf – JanusCon 2024
What Is It?
12
Slide 13
Slide 13 text
– Building ICEPerf – JanusCon 2024
ICE Is Vital for
WebRTC
13
Slide 14
Slide 14 text
– Building ICEPerf – JanusCon 2024
It's How We Make
Connections Between
WebRTC Peers
14
Slide 15
Slide 15 text
– Building ICEPerf – JanusCon 2024
And so STUN and
TURN Are Also Vital
15
Slide 16
Slide 16 text
– Building ICEPerf – JanusCon 2024
What's STUN?
16
Slide 17
Slide 17 text
– Building ICEPerf – JanusCon 2024
"Oi, Server in the
Cloud... What's my
Public IP and Port?"
17
Slide 18
Slide 18 text
– Building ICEPerf – JanusCon 2024
What's TURN?
18
Slide 19
Slide 19 text
– Building ICEPerf – JanusCon 2024
"Oi, Server, I Want To
Send my Traf
f
ic Through
You, How Can I Do That?"
19
Slide 20
Slide 20 text
– Building ICEPerf – JanusCon 2024
ICE Impacts How a
WebRTC Peer
Connection Is Setup
20
Slide 21
Slide 21 text
– Building ICEPerf – JanusCon 2024
So You Can See
Why...
21
Slide 22
Slide 22 text
– Building ICEPerf – JanusCon 2024
Delays in the ICE
Process Slow Down
Peer Connection
Creation
22
Slide 23
Slide 23 text
– Building ICEPerf – JanusCon 2024
What Public Data Is
There About the
Performance of STUN
and TURN Servers?
23
Slide 24
Slide 24 text
– Building ICEPerf – JanusCon 2024
None*
24
*Bold Statement without any hard evidence to back it up
Slide 25
Slide 25 text
– Building ICEPerf – JanusCon 2024
I Looked for 5 Minutes.
I Couldn't Find any
25
Slide 26
Slide 26 text
– Building ICEPerf – JanusCon 2024
Enter ICEPerf.com
26
Slide 27
Slide 27 text
– Building ICEPerf – JanusCon 2024
Originally Started
Out as RelayPerf.com
27
Slide 28
Slide 28 text
– Building ICEPerf – JanusCon 2024
We Wanted To Test
TURN Networks
28
Slide 29
Slide 29 text
– Building ICEPerf – JanusCon 2024
Who's the Best?
29
Slide 30
Slide 30 text
– Building ICEPerf – JanusCon 2024
After Buying Turnperf.Com
& Relayperf.Com We
Realised That Was Only
Half the Story
30
Slide 31
Slide 31 text
– Building ICEPerf – JanusCon 2024
So We Bought
ICEPerf.com
31
Slide 32
Slide 32 text
– Building ICEPerf – JanusCon 2024
So What Does It
Actually Do?
32
Slide 33
Slide 33 text
– Building ICEPerf – JanusCon 2024
And HOW?
33
Slide 34
Slide 34 text
– Building ICEPerf – JanusCon 2024
Firstly the What.
34
Slide 35
Slide 35 text
– Building ICEPerf – JanusCon 2024
We Test the TURN
Network's
Abilities
35
Slide 36
Slide 36 text
– Building ICEPerf – JanusCon 2024
As Well as the
Google STUN
Server
36
Slide 37
Slide 37 text
– Building ICEPerf – JanusCon 2024
And in the Future,
Open Source
Projects Too.
37
Slide 38
Slide 38 text
– Building ICEPerf – JanusCon 2024
The TURN
Networks
38
Slide 39
Slide 39 text
– Building ICEPerf – JanusCon 2024
Who Are They?
39
Slide 40
Slide 40 text
– Building ICEPerf – JanusCon 2024
Twilio, Xirsys,
Cloud
f
lare, Metered
and ExpressTURN
40
Slide 41
Slide 41 text
– Building ICEPerf – JanusCon 2024
Cloud
f
lare's
Announcement of a
TURN Network
Prompted This.
41
Slide 42
Slide 42 text
– Building ICEPerf – JanusCon 2024
Disclosure - I Own
Cloud
f
lare
Shares.
42
Slide 43
Slide 43 text
– Building ICEPerf – JanusCon 2024
I Want Cloud
f
lare
To Succeed.
43
Slide 44
Slide 44 text
– Building ICEPerf – JanusCon 2024
I Want the Others
To Succeed Just
As Much.
44
Slide 45
Slide 45 text
– Building ICEPerf – JanusCon 2024
For a Long Time
There Was Twilio
& Xirsys
45
Slide 46
Slide 46 text
– Building ICEPerf – JanusCon 2024
Not a Lot of
Choice.
46
Slide 47
Slide 47 text
– Building ICEPerf – JanusCon 2024
Then Subspace
Entered the Race.
47
Slide 48
Slide 48 text
– Building ICEPerf – JanusCon 2024
And Subsequently
Terminated
Operations.
48
Slide 49
Slide 49 text
– Building ICEPerf – JanusCon 2024
:Sadface:
49
Slide 50
Slide 50 text
– Building ICEPerf – JanusCon 2024
Back Down to Two
Networks.
50
Slide 51
Slide 51 text
– Building ICEPerf – JanusCon 2024
But Now We Have
5 Networks!
51
Slide 52
Slide 52 text
– Building ICEPerf – JanusCon 2024
Which Is Hugely
Exciting.
52
– Building ICEPerf – JanusCon 2024
Time To Receive a Candidate (STUN
& TURN)
Latency Through a TURN Server
Throughput Through a TURN Server
54
Slide 55
Slide 55 text
– Building ICEPerf – JanusCon 2024
1 Process
Handling 2 Peer
Connections
55
Slide 56
Slide 56 text
– Building ICEPerf – JanusCon 2024
Peer Connections
Connect With a
Speci
f
ic STUN/TURN
URI
56
Slide 57
Slide 57 text
– Building ICEPerf – JanusCon 2024
TURN Tests Force the
IceTransportPolicy of
Relay.
57
Slide 58
Slide 58 text
– Building ICEPerf – JanusCon 2024
TURN Tests Connect
Out via a TURN Server
and Back to the 2nd
Local Peer Connection
58
Slide 59
Slide 59 text
– Building ICEPerf – JanusCon 2024
All Tests Are Run
From a Home
Broadband
Connection
59
Slide 60
Slide 60 text
– Building ICEPerf – JanusCon 2024
Cloud Servers
Potentially Have
Much Better Peering
to Other "Clouds"
60
Slide 61
Slide 61 text
– Building ICEPerf – JanusCon 2024
Over to Marco To
Explain How It
Was Made
61
Slide 62
Slide 62 text
– Building ICEPerf – JanusCon 2024
How It’s
Made
62– Building ICEPerf – JanusCon 2024
Slide 63
Slide 63 text
– Building ICEPerf – JanusCon 2024
Written in GO
63– Building ICEPerf – JanusCon 2024
Slide 64
Slide 64 text
– Building ICEPerf – JanusCon 2024
Written in GO
63– Building ICEPerf – JanusCon 2024
Slide 65
Slide 65 text
– Building ICEPerf – JanusCon 2024
Written in GO
63– Building ICEPerf – JanusCon 2024
Slide 66
Slide 66 text
– Building ICEPerf – JanusCon 2024
BYOTSP
64– Building ICEPerf – JanusCon 2024
Slide 67
Slide 67 text
– Building ICEPerf – JanusCon 2024
BYOTSP
Bring Your Own TURN Server
Providers
65– Building ICEPerf – JanusCon 2024
Slide 68
Slide 68 text
– Building ICEPerf – JanusCon 2024
BYOTSP
Bring Your Own TURN Server
Providers
Plug-in providers
65– Building ICEPerf – JanusCon 2024
Slide 69
Slide 69 text
– Building ICEPerf – JanusCon 2024
BYOTSP
Bring Your Own TURN Server
Providers
Plug-in providers
Toggle providers and capabilities
on and o
f
65– Building ICEPerf – JanusCon 2024
Slide 70
Slide 70 text
– Building ICEPerf – JanusCon 2024
BYOTSP
Bring Your Own TURN Server
Providers
Plug-in providers
Toggle providers and capabilities
on and o
f
Save di
ff
erent con
f
igs
65– Building ICEPerf – JanusCon 2024
Slide 71
Slide 71 text
– Building ICEPerf – JanusCon 2024
BYOTSP
Bring Your Own TURN Server
Providers
Plug-in providers
Toggle providers and capabilities
on and o
f
Save di
ff
erent con
f
igs
Plan to expand
65– Building ICEPerf – JanusCon 2024
Slide 72
Slide 72 text
– Building ICEPerf – JanusCon 2024
con
f
ig.yaml
66– Building ICEPerf – JanusCon 2024
Slide 73
Slide 73 text
– Building ICEPerf – JanusCon 2024
Code design
Drivers
BDD (Learn Go With Tests https://
quii.gitbook.io/learn-go-with-
tests)
67– Building ICEPerf – JanusCon 2024
Slide 74
Slide 74 text
– Building ICEPerf – JanusCon 2024
Code
68– Building ICEPerf – JanusCon 2024
Slide 75
Slide 75 text
– Building ICEPerf – JanusCon 2024
69– Building ICEPerf – JanusCon 2024
Slide 76
Slide 76 text
– Building ICEPerf – JanusCon 2024
69– Building ICEPerf – JanusCon 2024
Slide 77
Slide 77 text
– Building ICEPerf – JanusCon 2024
70
Speci
f
ication
– Building ICEPerf – JanusCon 2024
Slide 78
Slide 78 text
– Building ICEPerf – JanusCon 2024
71
Test: Twilio
– Building ICEPerf – JanusCon 2024
Slide 79
Slide 79 text
– Building ICEPerf – JanusCon 2024
72
Implementation
– Building ICEPerf – JanusCon 2024
Slide 80
Slide 80 text
– Building ICEPerf – JanusCon 2024
73
Implementation
– Building ICEPerf – JanusCon 2024
Slide 81
Slide 81 text
– Building ICEPerf – JanusCon 2024
74
Implementation
– Building ICEPerf – JanusCon 2024
Slide 82
Slide 82 text
– Building ICEPerf – JanusCon 2024
75
Implementation
– Building ICEPerf – JanusCon 2024
Slide 83
Slide 83 text
– Building ICEPerf – JanusCon 2024
76
Speci
f
ication
– Building ICEPerf – JanusCon 2024
Slide 84
Slide 84 text
– Building ICEPerf – JanusCon 2024
77
Test: Cloud
f
lare
– Building ICEPerf – JanusCon 2024
Slide 85
Slide 85 text
– Building ICEPerf – JanusCon 2024
78
Implementation
– Building ICEPerf – JanusCon 2024
Slide 86
Slide 86 text
– Building ICEPerf – JanusCon 2024
79
Implementation
– Building ICEPerf – JanusCon 2024
Slide 87
Slide 87 text
– Building ICEPerf – JanusCon 2024
80
Client
– Building ICEPerf – JanusCon 2024
Slide 88
Slide 88 text
– Building ICEPerf – JanusCon 2024
81
Client
– Building ICEPerf – JanusCon 2024
Slide 89
Slide 89 text
– Building ICEPerf – JanusCon 2024
Loki
82– Building ICEPerf – JanusCon 2024
Slide 90
Slide 90 text
– Building ICEPerf – JanusCon 2024
Loki
82– Building ICEPerf – JanusCon 2024
Slide 91
Slide 91 text
– Building ICEPerf – JanusCon 2024
Loki
82– Building ICEPerf – JanusCon 2024
Slide 92
Slide 92 text
– Building ICEPerf – JanusCon 2024
Open Source
83– Building ICEPerf – JanusCon 2024
nimbleape/iceperf-website
nimbleape/iceperf-agent
Slide 93
Slide 93 text
– Building ICEPerf – JanusCon 2024
ICEPerf.com Is
Live!
84– Building ICEPerf – JanusCon 2024
Slide 94
Slide 94 text
– Building ICEPerf – JanusCon 2024
There Is Much
Work To Do.
85– Building ICEPerf – JanusCon 2024
Slide 95
Slide 95 text
– Building ICEPerf – JanusCon 2024
But Everything
Has a Beginning.
86– Building ICEPerf – JanusCon 2024
Slide 96
Slide 96 text
– Building ICEPerf – JanusCon 2024
Preliminary
Results
87– Building ICEPerf – JanusCon 2024
Slide 97
Slide 97 text
– Building ICEPerf – JanusCon 2024
Disclaimers
88– Building ICEPerf – JanusCon 2024
Slide 98
Slide 98 text
– Building ICEPerf – JanusCon 2024
Cloud
f
lare, Xirsys &
Twilio Gifted Us Credit
89– Building ICEPerf – JanusCon 2024
Metered & ExpressTURN
on Their Free Tiers
Slide 99
Slide 99 text
– Building ICEPerf – JanusCon 2024
90
Latency
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
– Building ICEPerf – JanusCon 2024
Slide 100
Slide 100 text
– Building ICEPerf – JanusCon 2024
91
Latency TURN UDP
– Building ICEPerf – JanusCon 2024
Slide 101
Slide 101 text
– Building ICEPerf – JanusCon 2024
92
Latency TURN TCP
– Building ICEPerf – JanusCon 2024
Slide 102
Slide 102 text
– Building ICEPerf – JanusCon 2024
93
Latency TURN TLS
– Building ICEPerf – JanusCon 2024
Slide 103
Slide 103 text
– Building ICEPerf – JanusCon 2024
94
First ICE candidate, STUN
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
– Building ICEPerf – JanusCon 2024
Slide 104
Slide 104 text
– Building ICEPerf – JanusCon 2024
95
First ICE candidate, TURN
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
– Building ICEPerf – JanusCon 2024
Slide 105
Slide 105 text
– Building ICEPerf – JanusCon 2024
96
First ICE candidate, TURN UDP
– Building ICEPerf – JanusCon 2024
Slide 106
Slide 106 text
– Building ICEPerf – JanusCon 2024
97
First ICE candidate, TURN TCP
– Building ICEPerf – JanusCon 2024
Slide 107
Slide 107 text
– Building ICEPerf – JanusCon 2024
98
First ICE candidate, TURN TLS
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
– Building ICEPerf – JanusCon 2024
Slide 108
Slide 108 text
– Building ICEPerf – JanusCon 2024
99
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
Max throughput
– Building ICEPerf – JanusCon 2024
Slide 109
Slide 109 text
– Building ICEPerf – JanusCon 2024
100
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
Max throughput TURN UDP
– Building ICEPerf – JanusCon 2024
Slide 110
Slide 110 text
– Building ICEPerf – JanusCon 2024
101
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
Max throughput TURN TCP
– Building ICEPerf – JanusCon 2024
Slide 111
Slide 111 text
– Building ICEPerf – JanusCon 2024
102
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
Max throughput TURN TLS
– Building ICEPerf – JanusCon 2024
Slide 112
Slide 112 text
– Building ICEPerf – JanusCon 2024
But Did You Know if You
Just Create a TURN URI
for a Network It Might
Just Work?
103
Slide 113
Slide 113 text
– Building ICEPerf – JanusCon 2024
104
Max throughput TURN TLS
– Building ICEPerf – JanusCon 2024
Slide 114
Slide 114 text
– Building ICEPerf – JanusCon 2024
105
Throughput
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
– Building ICEPerf – JanusCon 2024
Slide 115
Slide 115 text
– Building ICEPerf – JanusCon 2024
This Is a Graph
From Last Week
106
Slide 116
Slide 116 text
– Building ICEPerf – JanusCon 2024
107
Throughput
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
– Building ICEPerf – JanusCon 2024
Slide 117
Slide 117 text
– Building ICEPerf – JanusCon 2024
We Shared
Preliminary Data
With Cloud
f
lare
108
Slide 118
Slide 118 text
– Building ICEPerf – JanusCon 2024
109
Throughput
Tests completed over home broadband connection, 1 Gbps down, 100 Mbps up, hardwired
– Building ICEPerf – JanusCon 2024
Slide 119
Slide 119 text
– Building ICEPerf – JanusCon 2024
We've Already
Helped Fix an Issue.
110
Slide 120
Slide 120 text
– Building ICEPerf – JanusCon 2024
So What's Next?
111
Slide 121
Slide 121 text
– Building ICEPerf – JanusCon 2024
Lots of Things!
112
Slide 122
Slide 122 text
– Building ICEPerf – JanusCon 2024
Real-Time Access to
Data on the Website
113
Slide 123
Slide 123 text
– Building ICEPerf – JanusCon 2024
More Tests...
114
Slide 124
Slide 124 text
– Building ICEPerf – JanusCon 2024
Sending Media
115
Slide 125
Slide 125 text
– Building ICEPerf – JanusCon 2024
Measure Packet
Loss
116
Slide 126
Slide 126 text
– Building ICEPerf – JanusCon 2024
Max Concurrent
Throughput
117
Slide 127
Slide 127 text
– Building ICEPerf – JanusCon 2024
Separate Locations
for Offerer &
Answerer
118
Slide 128
Slide 128 text
– Building ICEPerf – JanusCon 2024
API Response Time
for STUN/TURN URIs
119
Slide 129
Slide 129 text
– Building ICEPerf – JanusCon 2024
Other WebRTC
Libraries Other Than
Pion
120
Slide 130
Slide 130 text
– Building ICEPerf – JanusCon 2024
Browser Based
Tests
121
Slide 131
Slide 131 text
– Building ICEPerf – JanusCon 2024
Multiple Locations
122
Slide 132
Slide 132 text
– Building ICEPerf – JanusCon 2024
And More!
123
Slide 133
Slide 133 text
– Building ICEPerf – JanusCon 2024
Cost Analysis
124
Slide 134
Slide 134 text
– Building ICEPerf – JanusCon 2024
Testing OSS
Projects
125
Slide 135
Slide 135 text
– Building ICEPerf – JanusCon 2024
Already Started
With Rel
126
https://github.com/elixir-webrtc/rel
Slide 136
Slide 136 text
– Building ICEPerf – JanusCon 2024
Adding in WebRTC
Metrics
127
Slide 137
Slide 137 text
– Building ICEPerf – JanusCon 2024
That's Going To Be
a Lot of Data...
128
Slide 138
Slide 138 text
– Building ICEPerf – JanusCon 2024
Lucky We're Friends
With Clever People
129
Slide 139
Slide 139 text
– Building ICEPerf – JanusCon 2024
130
Slide 140
Slide 140 text
– Building ICEPerf – JanusCon 2024
Pushing Data Through
TURN Networks
Becomes £££/$$$/€€€
131
Slide 141
Slide 141 text
– Building ICEPerf – JanusCon 2024
We're Going To Need
Some Kind of Support
To Make It Happen.
132
Slide 142
Slide 142 text
– Building ICEPerf – JanusCon 2024
That's Us
133
Slide 143
Slide 143 text
– Building ICEPerf – JanusCon 2024
Don't Forget To Get a
Sel
f
ie With Us for the
Chance of Winning a
CommCon UK/SF Ticket
134
Slide 144
Slide 144 text
– Building ICEPerf – JanusCon 2024
Thanks!
135
Slide 145
Slide 145 text
– Building ICEPerf – JanusCon 2024
OH!
Want This T-Shirt?
136