Statistics and Metrics in WebRTC

23d1ad8982616babb4d5b60470eea8a4?s=47 Varun Singh
November 04, 2015
310

Statistics and Metrics in WebRTC

WebRTC Meetup Yokohama,
Varun Singh
CEO callstats.io

23d1ad8982616babb4d5b60470eea8a4?s=128

Varun Singh

November 04, 2015
Tweet

Transcript

  1. Statistics in WebRTC Varun Singh CEO 04. 11. 2015

  2. Announcement ¥ 0 plan for developers 2

  3. http://geek-and-poke.com/

  4. Why measure? 4 audio only call

  5. Performance Monitoring Endpoint Endpoint Middlebox MCU, SFU, TURN, … Monitor

    Monitor Monitor Third party Monitor Operations & Management System Implement RTCP Extension Reports http://tools.ietf.org/html/rfc6792 https://tools.ietf.org/html/draft-ietf-xrblock-rtcweb-rtcp-xr-metrics
  6. HTML APIs • HTMLVideoElement Extensions • http://www.w3.org/TR/2014/CR-media-source-20140717/#htmlvideoelement-extensions • droppedVideoFrames, corruptedVideoFrames,

    totalFrameDelay • WebRTC Statistics API • (work in progress) http://www.w3.org/TR/2015/WD-webrtc-stats-20150206/ • PeerConnection Metrics: RTP, datachannel • Transport Metrics: active and passive candidates • Security cipher suite
  7. Edge Stats (ORTC) Lots of data still set to zero.

    but encouraging to see the data
  8. cross browser • adapter.js is your friend

  9. Monitoring What? • Per-stream media quality • Transport quality •

    Annoyances
  10. What to measure? • Network metrics • bits per second,

    RTT, jitter, packet losses, … • Multimedia pipeline metrics • playout delay, frames metrics, … • Quality Models from metrics • Annoyances • resolution/frame rate changes, interface changes, … • failures (NATs, insufficient capacity for carrying media, …) • user feedback 10 CAVEAT: not all WebRTC applications care about the same metrics
  11. Low hanging fruit • Endpoint monitoring (Traffic lights) • colors

    to show connection is active/inactive • off/online • good/bad/ugly
  12. Transport Metrics

  13. Diagnostics • characterise an impairment? • we start with an

    hypothesis… • implemented as decision trees • observe how often it occurs • within a call, across calls, geographies, … • How to characterise quality of the calls? 13
  14. Disruptions Disruption: loss of connectivity when network interfaces change, low

    available capacity,or high delay The light grey vertical lines show disruption, highlighted by the red bounding boxes.
  15. Intermittent Disruptions

  16. Disruptions and user behaviour User Behaviour: The user tries to

    correct for the disruption by turning on and off video Zoom in to the events to view disruptions
  17. Endpoint Churn 25% of the conferences has some users leave

    and rejoin.
  18. Session Failure

  19. Annoyances Setup times: waiting for the conference to begin: NAT

    traversal, negotiation, etc. Note: gathering and connectivity checks can happen concurrently
  20. • Analytics for WebRTC • 3 lines of code —>

    5 minutes. 1. include <script> 2. initialize() —> needs registration keys 3. addNewFabric(pc, …) —> call started • optionally: send user events: audio muted/unmuted, video paused/resumed, call terminated, call held. • optionally: reportError() —> call failed to be setup due to SDP or Firewalls. • optionally: associateMstWithUserID() —> bridges using multiple media streams in a single peerconnection may want to correlate mediastream quality across endpoints. • optionally: collect and send user feedback http://www.callstats.io/api/