Slide 1

Slide 1 text

"Extreme" Tes+ng of WebRTC Applica+ons
 2015/11/4(Wed) @WebRTC Meetup Japan iwashi86(Yoshimasa Iwase) atakigawa(Akinori Takigawa)

Slide 2

Slide 2 text

■Name Yoshimasa Iwase / @iwashi86 ■Work NTT Communications, DevOps Engineer of SkyWay ■Name Akinori Takigawa / @kenkenkenken8 ■Work CTO of Nefrock Inc.

Slide 3

Slide 3 text

Copyright © NTT Communica+ons Corpora+on. All rights reserved. 3 SkyWay = WebRTC plaUorm (CPaaS)

Slide 4

Slide 4 text

TL; DR •  Tes+ng WebRTC plaUorm/app is super hard WebRTCプラットフォーム/アプリのテストは超大変 •  Let’s automate it! and we did it! そこで自動化したく、やってみた!

Slide 5

Slide 5 text

TL; DR •  Tes+ng WebRTC plaUorm/app is super hard WebRTCプラットフォーム/アプリのテストは超大変 •  Let’s automate it! and we did it! そこで自動化したく、やってみた!

Slide 6

Slide 6 text

Why so tough? なぜ大変?

Slide 7

Slide 7 text

Reason 1 : Manual opera+on (手動) Video 1 Video2 ① ② ③

Slide 8

Slide 8 text

Reason 1+α : Tes+ng signaling is not enough Video 1 Video2   Signaling Succeeded   Video s+ll black シグナリング単体じゃ不十分

Slide 9

Slide 9 text

Reason 2 : Various condi+ons (Clients) + iOS, Android, Na+ve … 条件多すぎ(クライアントだけで)

Slide 10

Slide 10 text

Reason 2+α : Aggressive update of clients like, gUM on HTTP just doesn’t work アップデート早すぎ(クライアントだけで) gUMはHTTPで動かなくなったり 6-8 weeks update!

Slide 11

Slide 11 text

Reason3 : Various condi+ons (Network) NAT-X ① Endpoint Independent Filtering X Endpoint Independent Mapping … over 80 cases (NATタイプだけで発着を考えると80種類超) 条件多すぎ(ネットワークの種類も)

Slide 12

Slide 12 text

Reason3 : Various condi+ons (Network) NAT-X NAT-Y UDP blocked! ① Endpoint Independent Filtering X Endpoint Independent Mapping … over 80 cases (NATタイプだけで発着を考えると80種類超) 条件多すぎ(ネットワークの種類も)

Slide 13

Slide 13 text

Reason3 : Various condi+ons (Network) NAT-X NAT-Y UDP blocked! ① Endpoint Independent Filtering X Endpoint Independent Mapping … over 80 cases (NATタイプだけで発着を考えると80種類超) 条件多すぎ(ネットワークの種類も) Just tes+ng in the same machine isn’t enough

Slide 14

Slide 14 text

TL; DR •  Tes+ng WebRTC plaUorm/app is super hard WebRTCプラットフォーム/アプリのテストは超大変 •  Let’s automate it! and we did it! そこで自動化したく、やってみた!

Slide 15

Slide 15 text

Ubuntu with Xvn Architecture Overview Ubuntu with Xvn NAT instance NAT instance Intranet1 Intranet2 Web Server (hosts test app page) Virtual Internet VPC Network Segment TURN server

Slide 16

Slide 16 text

Browser1 DataChannel Tes+ng Browser2

Slide 17

Slide 17 text

MediaChannel Tes+ng Browser1 Browser2 video (gUM) video (gUM)

Slide 18

Slide 18 text

Browser x Browser x NAT type x NAT type ・Browsers ・ Chrome Stable, Chrome Beta, Chrome Unstable ・ Firefox Stable (, Firefox Unstable) ・NAT types ・Theore+cally 9 types (RFC4787) ・some types not configurable with iptables -> wrote it from scratch ・4 * 4 * 9 * 9 = 1296 uh…

Slide 19

Slide 19 text

NAT types hup://www.netmanias.com/en/?m=view&id=techdocs&no=6065

Slide 20

Slide 20 text

Ubuntu with Xvn Architecture Overview Ubuntu with Xvn NAT instance NAT instance Intranet1 Intranet2 Web Server (hosts test app page) Virtual Internet VPC Network Segment TURN server

Slide 21

Slide 21 text

Ubuntu with Xvn Architecture Overview (Detailed ver.) Intranet1 Intranet2 Web Server (hosts test app page) Virtual Internet VPC Network Segment Ubuntu with Xvn Ubuntu with Xvn Ubuntu with Xvn Ubuntu with Xvn 16 clients NAT Type 9 NAT Type 8 NAT Type 7 NAT Type 6 NAT Type 5 NAT Type 4 NAT Type 3 NAT Type 2 NAT Type 1 Ubuntu with Xvn Ubuntu with Xvn Ubuntu with Xvn Ubuntu with Xvn Ubuntu with Xvn 16 clients NAT Type 9 NAT Type 8 NAT Type 7 NAT Type 6 NAT Type 5 NAT Type 4 NAT Type 3 NAT Type 2 NAT Type 1 TURN server

Slide 22

Slide 22 text

Test result and Packet visualiza+on see -> hup://status.skyway.io

Slide 23

Slide 23 text

Thanks!!