Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Statistics and Metrics in WebRTC

Varun Singh
November 04, 2015
580

Statistics and Metrics in WebRTC

WebRTC Meetup Yokohama,
Varun Singh
CEO callstats.io

Varun Singh

November 04, 2015
Tweet

Transcript

  1. Statistics in WebRTC
    Varun Singh
    CEO
    04. 11. 2015

    View full-size slide

  2. Announcement
    ¥ 0
    plan for developers
    2

    View full-size slide

  3. http://geek-and-poke.com/

    View full-size slide

  4. Why measure?
    4
    audio only call

    View full-size slide

  5. Performance Monitoring
    Endpoint Endpoint
    Middlebox
    MCU, SFU, TURN, …
    Monitor
    Monitor
    Monitor
    Third
    party
    Monitor
    Operations &
    Management
    System
    Implement RTCP
    Extension Reports
    http://tools.ietf.org/html/rfc6792
    https://tools.ietf.org/html/draft-ietf-xrblock-rtcweb-rtcp-xr-metrics

    View full-size slide

  6. HTML APIs
    • HTMLVideoElement Extensions
    • http://www.w3.org/TR/2014/CR-media-source-20140717/#htmlvideoelement-extensions
    • droppedVideoFrames, corruptedVideoFrames, totalFrameDelay
    • WebRTC Statistics API
    • (work in progress) http://www.w3.org/TR/2015/WD-webrtc-stats-20150206/
    • PeerConnection Metrics: RTP, datachannel
    • Transport Metrics: active and passive candidates
    • Security cipher suite

    View full-size slide

  7. Edge Stats (ORTC)
    Lots of data still
    set to zero. but
    encouraging to
    see the data

    View full-size slide

  8. cross browser
    • adapter.js is your friend

    View full-size slide

  9. Monitoring What?
    • Per-stream media quality
    • Transport quality
    • Annoyances

    View full-size slide

  10. What to measure?
    • Network metrics
    • bits per second, RTT, jitter, packet losses, …
    • Multimedia pipeline metrics
    • playout delay, frames metrics, …
    • Quality Models from metrics
    • Annoyances
    • resolution/frame rate changes, interface changes, …
    • failures (NATs, insufficient capacity for carrying media, …)
    • user feedback
    10 CAVEAT: not all WebRTC applications care about the same metrics

    View full-size slide

  11. Low hanging fruit
    • Endpoint monitoring (Traffic lights)
    • colors to show connection is active/inactive
    • off/online
    • good/bad/ugly

    View full-size slide

  12. Transport Metrics

    View full-size slide

  13. Diagnostics
    • characterise an impairment?
    • we start with an hypothesis…
    • implemented as decision trees
    • observe how often it occurs
    • within a call, across calls, geographies, …
    • How to characterise quality of the calls?
    13

    View full-size slide

  14. Disruptions
    Disruption: loss of connectivity when network
    interfaces change, low available capacity,or high
    delay
    The light grey vertical lines show disruption, highlighted by the red
    bounding boxes.

    View full-size slide

  15. Intermittent
    Disruptions

    View full-size slide

  16. Disruptions and user behaviour
    User Behaviour: The user tries to correct for the
    disruption by turning on and off video
    Zoom in to the events
    to view disruptions

    View full-size slide

  17. Endpoint Churn
    25% of the
    conferences
    has some
    users leave
    and
    rejoin.

    View full-size slide

  18. Session Failure

    View full-size slide

  19. Annoyances
    Setup times: waiting for the conference to
    begin: NAT traversal, negotiation, etc.
    Note: gathering and connectivity
    checks can happen concurrently

    View full-size slide

  20. • Analytics for WebRTC
    • 3 lines of code —> 5 minutes.
    1. include <br/>2. initialize() —> needs registration keys<br/>3. addNewFabric(pc, …) —> call started<br/>• optionally: send user events: audio muted/unmuted, video paused/resumed, call terminated, call held.<br/>• optionally: reportError() —> call failed to be setup due to SDP or Firewalls.<br/>• optionally: associateMstWithUserID() —> bridges using multiple media streams in a single peerconnection may<br/>want to correlate mediastream quality across endpoints.<br/>• optionally: collect and send user feedback<br/>http://www.callstats.io/api/<br/>

    View full-size slide