Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Video Streaming Analysis

dynamis
December 10, 2019

Video Streaming Analysis

IIJ II seminar 201912 で使用したスライド
https://iijlab-seminars.connpass.com/event/157675/

dynamis

December 10, 2019
Tweet

More Decks by dynamis

Other Decks in Research

Transcript

  1. Collecting and analyzing video streaming
    services' quality data with Web Browser
    @ IIJLab seminar on 20191210
    WebDINO Japan - Tomoya ASAI

    View full-size slide

  2. about:speaker
    Tomoya ASAI - dynamis webdino.org
    CTO
    dynamis @ Community
    Twitter: @dynamitter
    facebook.com/dynamis
    speakerdeck.com/dynamis
    note: red panda (fire fox, lesser panda)
    2
    @

    View full-size slide

  3. about: WebDINO Japan
    Web, Browser, Open
    ( : )
    (Chromium, Firefox )
    ( : Gecko )
    Web & Open
    Web IoT
    : 2017 Mozilla Japan WebDINO Japan
    https://www.webdino.org/
    3

    View full-size slide

  4. Topics
    Web VideoMark Project
    Measurement and Open Data
    Early Data Analysis
    Next Steps
    4

    View full-size slide

  5. Acknowledgment
    Web VideoMark WebDINO Japan

    (QoE )
    NTT

    GitHub 貢
    https://vm.webdino.org/notice
    5

    View full-size slide

  6. Web VideoMark Project

    View full-size slide

  7. Web VideoMark
    Paravi, TVer, YouTube
    CQ IOT
    https://vm.webdino.org/opendata
    Web VideoMark Project by WebDINO Japan http://vm.webdino.org/
    7

    View full-size slide

  8. (58%, sandvine ) ( )
    (Chrome) (Web VideoMark) Android
    (VideoMark Browser)
    ( )
    8

    View full-size slide

  9. Web VideoMark - Endorsement
    IOT
    IIJ
    https://www.webdino.org/updates/news/201902051051/
    9

    View full-size slide

  10. ಈըαʔϏεͷ
    ਪఆମײ඼࣭஋
    ར༻ऀͷ

    ϒϥ΢βʔ
    ֶज़ݚڀ༻σʔλ

    ΍ௐࠪ෼ੳ݁Ռ͸

    Φʔϓϯσʔλʹ
    ʲΫϥΠΞϯτ৘ใʳ
    w ୺຤छผ

    1$PSϞόΠϧ

    w ϒϥ΢βόʔδϣϯ
    w 04όʔδϣϯ౳
    ஫ࢯ໊ɾॅॴ౳ͷݸਓ
    ಛఆ৘ใ͸ؚ·ͳ͍

    6TFS"HFOU͔Βͷਪଌ
    ʲωοτϫʔΫ৘ใʳ
    w ௨৴ࣄۀࣾɾ*41
    w ஍Ҭ ౎ಓ෎ݝ

    ஫ૹ৴ݩ*1ΞυϨεಗ
    ໊Խ ຤ඌΦΫςοτ
    ࡟আ
    ޙʹ(FP*1Ͱਪଌ
    ʲ࠶ੜ඼࣭ཁҼʳ
    w ಈը৘ใϏοτϨʔτɺ
    ղ૾౓ɺϑϨʔϜϨʔτ౳
    w ࠶ੜ৘ใ࠶ੜͷதஅɺ
    όοϑΝϦϯά౳
    w ૢ࡞৘ใಈըͷγʔΫɺ
    Ұ࣌ఀࢭɾ࠶։౳
    8FC7JEFP.BSL֦ுػೳ΍ϒϥ΢βʔͰࢹௌ৘ใΛऔಘ
    ಈը࠶ੜ࣌ͷ
    ମײ඼࣭஋Λ

    dͷ਺஋Ͱ

    ਪఆͯ͠ฦ͢
    2P&
    2VBMJUZPG&YQFSJFODF
    8FC
    7JEFP.BSL
    αʔόʔ
    ࢹௌ৘ใΛ෼ੳ
    ಈը഑৴αʔϏε΁ΞΫηε
    )5.-ɺಈըϑΝΠϧ౳Λड৴
    ௨৴ࣄۀऀɾ*41
    αʔϏεվળ༻ʹ
    ௐࠪσʔλΛఏڙ
    ʂ
    ಈըࢹௌ࣌ͷ
    2P&౳Λ֬ೝ
    ϓϥΠόγʔอޢॲཧ ֶज़ݚڀ༻σʔλ࡞੒
    ಗ໊Խॲཧ
    ಈը഑৴αʔϏεɾ
    $%/αʔόʔ
    Web VideoMark

    View full-size slide

  11. note: (QoE)
    (QoE: Quality of Experience)
    24
    ( )
    (1 5, MOS)
    QoE (ITU)
    ITU-T P SG12
    Web VideoMark ITU-T NTT
    11

    View full-size slide

  12. Web VideoMark ( )
    12
    GCP
    Internet
    sodium.js
    nginx/ uent-d


    sodium.js 

    PC Chrome

    +

    (Android)

    endpoint
    log file
    SodiumServer
    AggregateServer
    QoE 


    QoE 

    GeoIP 

    (IP )
    QoE Server
    QoE

    mongoDB

    QoE 

    QoE

    View full-size slide

  13. Web VideoMark ( )
    13
    GCP
    Internet
    sodium.js
    nginx/ uent-d


    sodium.js 

    PC Chrome

    +

    (Android)

    endpoint
    log file
    SodiumServer
    AggregateServer
    QoE 


    QoE 

    GeoIP 

    (IP )
    QoE Server
    QoE

    mongoDB
    mongoDB

    QoE 


    QoE

    View full-size slide

  14. Web VideoMark
    ( ) 

    ( )
    : 2019 12 10
    14

    View full-size slide

  15. Web VideoMark
    0. "videomark" ( )
    PC Chrome Web VideoMark

    Android VideoMark Browser
    1. VideoMark Chrome /
    2. YouTube
    3.
    15

    View full-size slide

  16. ( )
    ( )
    ( ) QoE
    15 QoE
    ⭐ 

    ( 1.0 ⭐ 5.0 ⭐⭐⭐⭐⭐)
    16

    View full-size slide

  17. ( MacBook )
    Paravi, TVer QoE (3.2) YouTube 3
    YouTube QoE 4,9, 3.5-3.6, 3.2 , Paravi TVer QoE 4.5
    :
    18

    View full-size slide

  18. Web VideoMark
    QoS QoE
    19

    View full-size slide

  19. QoE
    ISP, APN DL
    CM
    YouTube
    ( )
    20

    View full-size slide

  20. : API ( QoE)
    PC VM URL "stats"
    21

    View full-size slide

  21. ( )
    PC/Android Web VideoMark
    https://vm.webdino.org/
    ( )
    (Jetson Nano)
    LAN OK
    : https://vm.webdino.org/contact
    22

    View full-size slide

  22. https://vm.webdino.org
    23

    View full-size slide

  23. Measurement

    & Open Data

    View full-size slide

  24. https://vm.webdino.org/opendata
    IP URL
    Android UserAgent
    UA
    (a,b,c)
    https://vm.webdino.org/opendata
    25

    View full-size slide

  25. IP
    URL
    User Agent
    User Agent Android
    ID ( 1 )
    26

    View full-size slide

  26. IP ( )
    , , ISP (GeoIP )
    APN, PLMN (Android )
    URL
    CDN IP [*]
    URL [*]
    [*]
    ( )
    etc...
    Representation [**]
    Codec
    Representation
    /
    etc...
    QoE ( )
    (RTT) [*]
    [*]
    etc...
    [*]
    etc...
    CM [*]
    etc...
    [*] [**]
    27

    View full-size slide

  27. SIM ID
    GDPR
    IP
    GPS
    User Agent
    URL
    28

    View full-size slide

  28. ( )
    2019-04-07T06:38:58+00:00 {"date":"2019-04-07T06:38:58.253Z","startTime":
    297957833.29999995,"endTime":297962836.1000001,"session":"8b37d168-7343-45a1-a507-
    f8c230a0ebd4","location":"https://www.youtube.com","userAgent":"Mozilla/5.0 (Linux;
    Android 9; XXX) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.124 Mobile
    Safari/537.36","appVersion":"5.0 (Linux; Android 9; XXX) AppleWebKit/537.36 (KHTML,
    like Gecko) Chrome/70.0.3538.124 Mobile Safari/537.36","sequence":58463,"video":
    [{"property":{"uuid":"fd65101f-a8df-47c6-a804-3ac719e04ca3","id":"","class":
    ["video-stream","html5-main-video"],"src":"blob:https://www.youtube.com/a1551960-
    f053-492e-b945-84b075c8598f","domainName":"r7---sn-5n5ip-
    ioqe.googlevideo.com","width":0,"height":0,"videoWidth":1280,"videoHeight":
    720,"mediaSize":82.384398,"defaultPlaybackRate":1,"playbackRate":1,"playStartTime":
    1554511611888.0,"playEndTime":1554511694180.0,"currentPlayPos":
    82.384398,"currentPlayTime":82.245},"playback_quality":[],"event_play":
    [],"event_play_delta":[],"event_pause":[],"event_pause_delta":[],"event_seeking":
    [],"event_seeking_delta":[],"event_seeked":[],"event_seeked_delta":
    [],"event_ended":[],"event_ended_delta":[],"event_stalled":
    [],"event_stalled_delta":[],"event_progress":[],"event_progress_delta":
    [],"event_waiting":[],"event_waiting_delta":[],"event_canplay":
    [],"event_canplay_delta":[]}],"resource_timing":
    [],"REMOTE_ADDR":"180.235.59.0","country":"JP","subdivision":"13","isp":"Asahi
    Net"}
    5
    29

    View full-size slide

  29. ( , mongo DB )
    session:
    session id: uuid
    video id: uuid
    service: youtube
    user agent: XXXXXXX
    qoe: 4.2
    connection:
    - type: cellular
    effectiveType: 4g
    downlink: 1.55
    downlinkMax: 100
    rtt: 150
    apn: NTT DOCOMO
    plmn: NTT DOCOMO
    network: # IPͱGeoIP৘ใ
    - server host:
    xxx.yyy.com
    server ip: x.y.z.0
    client ip: x.y.z.0
    location: tokyo
    isp: ocn
    video:
    video id: xxxxxx
    duration: 160.3
    representations:
    - resolution:
    height: 1080
    width: 1920
    video codec: webm
    audio codec: opus
    video bitrate: 12345
    audio bitrate: 123
    play history:
    start time: 1234567890
    end time: 1234567890
    throughput history:
    # DL࣌ࠁͱ଎౓ɾRTT
    - dl time: 1234567890
    throughput: 123456
    rtt: 1234
    event history:
    - event type: seek
    time: 12.3
    representation
    history:
    # ৄࡉׂѪ
    frame drop history:
    # ৄࡉׂѪ
    buffer history:
    # ৄࡉׂѪ
    cm history:
    - duration: 12:3
    start time: 1234567890
    end time: 1234567890
    yml
    30

    View full-size slide

  30. https://vm.webdino.org
    31

    View full-size slide

  31. Initial Data Analysis

    View full-size slide

  32. 3/1 4/30 (2 )
    23645
    10 12K
    1000 ISP
    ISP
    :
    33

    View full-size slide

  33. QoE
    ISP QoE
    34

    View full-size slide

  34. ( )
    PC/Mobile
    ISP
    35

    View full-size slide

  35. ( )
    5
    3 9
    ( - ) / 1.0
    1.0 CM
    37

    View full-size slide

  36. ( )
    X 0.93-1.0 CM (CM )
    Y, Z 1.0 ( )
    Y 0-0.03 CM
    38
    Y
    X Z
    Y

    View full-size slide

  37. 1.0
    ( - ) /
    CM 1.0
    Y CM 1.0-1.15
    Z CM 1.0-1.15
    Y, Z 1.5
    39
    X Z
    Y

    View full-size slide

  38. PC
    PC 0.0
    CM
    40
    PC Mobile

    View full-size slide

  39. QoE
    ( ) (QoE) 

    ( )
    CM
    41

    View full-size slide

  40. /
    20 ( 130 )
    5, 10 5 5
    5,10,15... 5,10,15... ( )
    15 /
    42

    View full-size slide

  41. ( )
    (bps) ( )
    ( )
    44

    View full-size slide

  42. ( )
    QoE
    QoE
    QoE
    45

    View full-size slide

  43. B ( )
    C ( ) 

    ( )
    46
    A
    B
    C
    A
    B
    C

    View full-size slide

  44. B ( )
    C ( ) QoE
    47
    A
    B
    C

    View full-size slide

  45. ISP
    OCN ( ) Asahi Net ( )
    Asahi Net ( ) WiFi OCN ( )
    ( )
    48
    ISP X
    ISP Y
    ISP Z
    ISP X
    ISP Y
    ISP Z

    View full-size slide

  46. ISP
    ARTERIA ( ), Asahi Net ( )
    PC vs Mobile QoE ( ) ISP
    ( )
    49
    ISP X
    ISP Y
    ISP Z

    View full-size slide

  47. QoE ( , )
    QoE 1.0 5.0 30
    貢 1080p, 720p
    ABR
    50
    A
    720p30 1.5~2Mbsp
    480p30 1M
    1080p 2~3Mbps
    1080p 5Mbps

    View full-size slide

  48. QoE
    A PC
    PC QoE 5 1080p30 ( )
    PC 3.5 QoE PC
    51
    A (PC) A (mobile)

    View full-size slide

  49. QoE
    B, C QoE 4.6-4.73 720p
    ( )
    B QoE 4.73 1080p
    ( )
    52
    B C

    View full-size slide

  50. ISP QoE
    OCN PC
    4K, 8K, 60fps
    ARTERIA 

    53
    OCN ARTERIA

    View full-size slide

  51. ISP QoE
    ISP X QoE
    Asahi Net Android
    54
    Asahi Net
    ISP X

    View full-size slide

  52. ISP QoE
    ISP Y QoE 1 , Z QoE
    55
    ISP Z
    ISP Y

    View full-size slide

  53. ( , )
    2060p30, 1080p30, 720p30, 480p30
    ABR
    (ABR )
    57
    A

    View full-size slide

  54. A PC
    PC 1080p30 FSP
    PC 1080p30
    58
    A (PC) A (mobile)

    View full-size slide

  55. B, C B
    B 720p 2Mbps C 1080p 5Mbps
    59
    B C

    View full-size slide

  56. ISP
    OCN PC
    60
    OCN ARTERIA

    View full-size slide

  57. ISP
    Asahi Net Android
    61
    Asahi Net
    ISP X

    View full-size slide

  58. ISP
    ( )
    62
    ISP Z
    ISP Y

    View full-size slide

  59. A PC
    PC PC
    65
    A (PC) A (mobile)

    View full-size slide

  60. ISP
    ISP
    ( )
    67
    OCN ARTERIA

    View full-size slide

  61. ISP
    68
    Asahi Net
    ISP X

    View full-size slide

  62. ISP
    69
    ISP Z
    ISP Y

    View full-size slide

  63. QoE
    (
    )
    QoE
    QoE
    ( )
    70

    View full-size slide

  64. ISP ( )
    PC/Android
    : OSS
    71

    View full-size slide

  65. OSS
    YouTube
    (
    )
    72

    View full-size slide

  66. QoE
    QoE
    QoE
    QoE
    : 1080p, 30fps, audio 128kbps ( )

    video 3380kbps: 4.813 => 5000kbps: 4.882
    QoE (
    )
    :
    QoE
    73

    ...

    View full-size slide

  67. Web VideoMark
    ( )
    CM (
    CM CM )
    ISP
    QoE
    ISP
    74

    View full-size slide

  68. : APN PLMN etc.

    : RTT etc.
    CM
    : CM etc.
    CM
    QoE CM (
    )
    76

    View full-size slide

  69. 3 10
    YouTube +
    QoE
    77

    View full-size slide

  70. mongo DB
    ( )
    79

    View full-size slide

  71. etc...
    : https://vm.webdino.org/contact
    80

    View full-size slide

  72. ( )
    ( )
    PC/Android Web VideoMark
    https://vm.webdino.org/
    ( )
    (Jetson Nano)
    LAN OK
    : https://vm.webdino.org/contact
    81

    View full-size slide

  73. https://vm.webdino.org
    82

    View full-size slide

  74. vol.119 IEICE
    (IEICE )
    © 2019 IEICE , vol.119, no. 298, CQ-2019-92, Nov. 2019
    (
    )
    ®
    83

    View full-size slide