Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Extreme testing of WebRTC Applications

iwashi
November 29, 2022

Extreme testing of WebRTC Applications

This material is presented at WebRTC Meetup Japan:
https://44145dc9ce5b767cd2eb8b7683.doorkeeper.jp/events/33154

You can see video from:
https://youtu.be/02wAz0Ii-ZU?t=1h33m21s

iwashi

November 29, 2022
Tweet

More Decks by iwashi

Other Decks in Technology

Transcript

  1. "Extreme" Tes+ng of WebRTC Applica+ons

    2015/11/4(Wed) @WebRTC Meetup Japan
    iwashi86(Yoshimasa Iwase)
    atakigawa(Akinori Takigawa)

    View full-size slide

  2. ■Name
    Yoshimasa Iwase / @iwashi86
    ■Work
    NTT Communications,
    DevOps Engineer of SkyWay
    ■Name
    Akinori Takigawa / @kenkenkenken8
    ■Work
    CTO of Nefrock Inc.

    View full-size slide

  3. Copyright © NTT Communica+ons Corpora+on. All rights reserved. 3
    SkyWay = WebRTC plaUorm (CPaaS)

    View full-size slide

  4. TL; DR
    •  Tes+ng WebRTC plaUorm/app is super hard
    WebRTCプラットフォーム/アプリのテストは超大変
    •  Let’s automate it! and we did it!
    そこで自動化したく、やってみた!

    View full-size slide

  5. TL; DR
    •  Tes+ng WebRTC plaUorm/app is super hard
    WebRTCプラットフォーム/アプリのテストは超大変
    •  Let’s automate it! and we did it!
    そこで自動化したく、やってみた!

    View full-size slide

  6. Why so tough?
    なぜ大変?

    View full-size slide

  7. Reason 1 : Manual opera+on (手動)
    Video 1 Video2
    ① ② ③

    View full-size slide

  8. Reason 1+α : Tes+ng signaling is not enough
    Video 1 Video2
      Signaling Succeeded
      Video s+ll black
    シグナリング単体じゃ不十分

    View full-size slide

  9. Reason 2 : Various condi+ons (Clients)
    + iOS, Android, Na+ve …
    条件多すぎ(クライアントだけで)

    View full-size slide

  10. Reason 2+α : Aggressive update of clients
    like, gUM on HTTP just doesn’t work
    アップデート早すぎ(クライアントだけで)
    gUMはHTTPで動かなくなったり
    6-8 weeks update!

    View full-size slide

  11. Reason3 : Various condi+ons (Network)
    NAT-X
    ① Endpoint Independent Filtering X Endpoint Independent Mapping
    … over 80 cases (NATタイプだけで発着を考えると80種類超)
    条件多すぎ(ネットワークの種類も)

    View full-size slide

  12. Reason3 : Various condi+ons (Network)
    NAT-X
    NAT-Y
    UDP blocked!
    ① Endpoint Independent Filtering X Endpoint Independent Mapping
    … over 80 cases (NATタイプだけで発着を考えると80種類超)
    条件多すぎ(ネットワークの種類も)

    View full-size slide

  13. Reason3 : Various condi+ons (Network)
    NAT-X
    NAT-Y
    UDP blocked!
    ① Endpoint Independent Filtering X Endpoint Independent Mapping
    … over 80 cases (NATタイプだけで発着を考えると80種類超)
    条件多すぎ(ネットワークの種類も)
    Just tes+ng in the same machine isn’t enough

    View full-size slide

  14. TL; DR
    •  Tes+ng WebRTC plaUorm/app is super hard
    WebRTCプラットフォーム/アプリのテストは超大変
    •  Let’s automate it! and we did it!
    そこで自動化したく、やってみた!

    View full-size slide

  15. Ubuntu with Xvn
    Architecture Overview
    Ubuntu with Xvn
    NAT
    instance
    NAT
    instance
    Intranet1 Intranet2
    Web Server
    (hosts test
    app page)
    Virtual Internet
    VPC
    Network
    Segment
    TURN
    server

    View full-size slide

  16. Browser1
    DataChannel Tes+ng
    Browser2

    View full-size slide

  17. MediaChannel Tes+ng
    Browser1 Browser2
    video
    (gUM)
    video
    (gUM)

    View full-size slide

  18. Browser x Browser x NAT type x NAT type
    ・Browsers
    ・ Chrome Stable, Chrome Beta, Chrome Unstable
    ・ Firefox Stable (, Firefox Unstable)
    ・NAT types
    ・Theore+cally 9 types (RFC4787)
    ・some types not configurable with iptables
    -> wrote it from scratch
    ・4 * 4 * 9 * 9 = 1296
    uh…

    View full-size slide

  19. NAT types
    hup://www.netmanias.com/en/?m=view&id=techdocs&no=6065

    View full-size slide

  20. Ubuntu with Xvn
    Architecture Overview
    Ubuntu with Xvn
    NAT
    instance
    NAT
    instance
    Intranet1 Intranet2
    Web Server
    (hosts test
    app page)
    Virtual Internet
    VPC
    Network
    Segment
    TURN
    server

    View full-size slide

  21. Ubuntu
    with Xvn
    Architecture Overview (Detailed ver.)
    Intranet1 Intranet2
    Web Server
    (hosts test
    app page)
    Virtual Internet
    VPC
    Network
    Segment
    Ubuntu
    with Xvn
    Ubuntu
    with Xvn
    Ubuntu
    with Xvn
    Ubuntu
    with Xvn
    16 clients
    NAT
    Type
    9
    NAT
    Type
    8
    NAT
    Type
    7
    NAT
    Type
    6
    NAT
    Type
    5
    NAT
    Type
    4
    NAT
    Type
    3
    NAT
    Type
    2
    NAT
    Type
    1
    Ubuntu
    with Xvn
    Ubuntu
    with Xvn
    Ubuntu
    with Xvn
    Ubuntu
    with Xvn
    Ubuntu
    with Xvn
    16 clients
    NAT
    Type
    9
    NAT
    Type
    8
    NAT
    Type
    7
    NAT
    Type
    6
    NAT
    Type
    5
    NAT
    Type
    4
    NAT
    Type
    3
    NAT
    Type
    2
    NAT
    Type
    1
    TURN server

    View full-size slide

  22. Test result and Packet visualiza+on
    see -> hup://status.skyway.io

    View full-size slide