Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
VideoMark Analysis
dynamis
November 21, 2019
Research
1
110
VideoMark Analysis
ICICE 合同研究会 (CQ 研究会) 201911 月の招待講演にて利用したスライド
dynamis
November 21, 2019
Tweet
Share
More Decks by dynamis
See All by dynamis
Post COVID-19 Network Analysis
dynamis
0
72
Web Ecosystem for Embedded Linux.pdf
dynamis
0
93
Video Streaming Analysis
dynamis
0
110
React License Violation
dynamis
6
3.2k
Pi Zero IoT Workshop @ High School
dynamis
0
87
Open VideoMark Data
dynamis
1
260
AV1 Patent Issue
dynamis
5
2.8k
HTML5 on Renesas Linux Platform
dynamis
0
120
Web VideoMark Introduction
dynamis
0
410
Other Decks in Research
See All in Research
蓄電池経済効果シミュレーションを提示するとお客さんは販売会社を信頼するか?購入意欲はアップするか?最新調査結果
satoru_higuchi
0
1.6k
再帰化への認知的転回/the-turn-to-recursive-system
monochromegane
0
190
[IR Reading 2022春 論文紹介] Personalized Transfer of User Preferences for Cross-domain Recommendation (WSDM 2022) /IR-Reading-2022-spring
koheishinden
PRO
0
160
Making CRDTs Byzantine fault tolerant
ept
0
410
「Lean Interview」 誰でも、ほぼコストゼロ、1日でできるインタビュー法
shintokeimail
0
160
CGO'22 Tutorial Outro
chriscummins
0
650
PLDI '21論文読み会: DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion
ideininc
0
870
PLDI '21論文読み会: Quantum Abstract Interpretation
ideininc
0
740
PLDI '21論文読み会: Cyclic Program Synthesis
ideininc
0
800
みらい健康手帳アプリで健康寿命を延伸する試み
isabisi1484
0
190
Recent Findings on Density-Ratio Approachesin Machine Learning
masakat0
0
250
さくらの夕べ データ流通実証実験振り返りナイト
kikuzo
0
150
Featured
See All Featured
A Philosophy of Restraint
colly
192
15k
Fashionably flexible responsive web design (full day workshop)
malarkey
396
62k
We Have a Design System, Now What?
morganepeng
35
3k
Scaling GitHub
holman
451
140k
Fantastic passwords and where to find them - at NoRuKo
philnash
27
1.6k
Principles of Awesome APIs and How to Build Them.
keavy
113
15k
Six Lessons from altMBA
skipperchong
14
1.4k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
237
19k
The Power of CSS Pseudo Elements
geoffreycrofte
47
4k
How to name files
jennybc
40
63k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
37
3.3k
10 Git Anti Patterns You Should be Aware of
lemiorhan
638
52k
Transcript
[Invited Talk] Collecting and analyzing video streaming services' quality data
with Web browser. @ CQ 201911 WebDINO Japan - Tomoya ASAI NTT Network Technology Labs - Takuto KIMURA
about:speaker Tomoya ASAI - dynamis webdino.org CTO dynamis @ Community
Twitter: @dynamitter facebook.com/dynamis speakerdeck.com/dynamis note: red panda (fire fox, lesser panda) 2 @
about: WebDINO Japan Web, Browser, Open ( : ) (Chromium,
Firefox ) ( : Gecko ) Web & Open Web IoT : 2017 Mozilla Japan WebDINO Japan https://www.webdino.org/ 3
Topics Web VideoMark Project Measurement and Open Data Early Data
Analysis Next Steps 4
Acknowledgment Web VideoMark WebDINO Japan 貢 (QoE ) NTT 貢
GitHub 貢 https://vm.webdino.org/notice 5
Web VideoMark Project
ಈըαʔϏεͷ ਪఆମײ࣭ ར༻ऀͷ ϒϥβʔ ֶज़ݚڀ༻σʔλ ௐࠪੳ݁Ռ Φʔϓϯσʔλʹ ʲΫϥΠΞϯτใʳ w छผ
1$PSϞόΠϧ w ϒϥβόʔδϣϯ w 04όʔδϣϯ ࢯ໊ɾॅॴͷݸਓ ಛఆใؚ·ͳ͍ 6TFS"HFOU͔Βͷਪଌ ʲωοτϫʔΫใʳ w ௨৴ࣄۀࣾɾ*41 w Ҭ ಓݝ ૹ৴ݩ*1ΞυϨεಗ ໊Խ ඌΦΫςοτ আ ޙʹ(FP*1Ͱਪଌ ʲ࠶ੜ࣭ཁҼʳ w ಈըใϏοτϨʔτɺ ղ૾ɺϑϨʔϜϨʔτ w ࠶ੜใ࠶ੜͷதஅɺ όοϑΝϦϯά w ૢ࡞ใಈըͷγʔΫɺ Ұ࣌ఀࢭɾ࠶։ 8FC7JEFP.BSL֦ுػೳϒϥβʔͰࢹௌใΛऔಘ ಈը࠶ੜ࣌ͷ ମײ࣭Λ dͷͰ ਪఆͯ͠ฦ͢ 2P& 2VBMJUZPG&YQFSJFODF 8FC 7JEFP.BSL αʔόʔ ࢹௌใΛੳ ಈը৴αʔϏεΞΫηε )5.-ɺಈըϑΝΠϧΛड৴ ௨৴ࣄۀऀɾ*41 αʔϏεվળ༻ʹ ௐࠪσʔλΛఏڙ ʂ ಈըࢹௌ࣌ͷ 2P&Λ֬ೝ ϓϥΠόγʔอޢॲཧ ֶज़ݚڀ༻σʔλ࡞ ಗ໊Խॲཧ ಈը৴αʔϏεɾ $%/αʔόʔ Web VideoMark
Web VideoMark : 8 GCP Internet sodium.js nginx/ uent-d
sodium.js PC Chrome + (Android) endpoint log file SodiumServer AggregateServer QoE QoE GeoIP (IP ) QoE Server QoE mongoDB mongoDB QoE QoE
Web VideoMark Paravi, TVer, YouTube CQ IOT https://vm.webdino.org/opendata Web VideoMark
Project by WebDINO Japan http://vm.webdino.org/ 9
(58%, sandvine ) ( ) (Chrome) (Web VideoMark) Android (VideoMark
Browser) ( ) 10
CQ CQ https://www.webdino.org/updates/news/201902051051/ 11
Web VideoMark ( ) ( ) : 2019 10
12
( ) ( ) QoE 15 QoE ⭐ 13
14
( MacBook ) Paravi, TVer QoE (3.2) YouTube 3 YouTube
QoE 4,9, 3.5-3.6, 3.2 , Paravi TVer QoE 4.5 : 15
Web VideoMark QoS QoE 16
QoE ISP APN DL CM YouTube ( ) 17
note: (QoE) (QoE) (QoE: Quality of Experience) (1 5, MOS)
QoE QoE (ITU) Web VideoMark ITU (NTT ) 18
: API ( QoE) PC VM URL "stats" 19
( ) PC/Android Web VideoMark https://vm.webdino.org/ ( ) (Jetson Nano)
LAN OK : https://vm.webdino.org/contact 20
https://vm.webdino.org 21
Measurement & Open Data
https://vm.webdino.org/opendata IP URL Android UserAgent UA (a,b,c) https://vm.webdino.org/opendata 23
IP URL User Agent User Agent Android ID ( 1
) 24
IP ( ) , , ISP (GeoIP ) APN, PLMN
(Android ) URL CDN IP [*] URL [*] [*] ( ) etc... Representation [**] Codec Representation / etc... QoE ( ) (RTT) [*] [*] etc... [*] etc... CM [*] etc... [*] [**] 25
SIM ID GDPR IP GPS User Agent URL 26
( ) 2019-04-07T06:38:58+00:00 {"date":"2019-04-07T06:38:58.253Z","startTime": 297957833.29999995,"endTime":297962836.1000001,"session":"8b37d168-7343-45a1-a507- f8c230a0ebd4","location":"https://www.youtube.com","userAgent":"Mozilla/5.0 (Linux; Android 9; XXX)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.124 Mobile Safari/537.36","appVersion":"5.0 (Linux; Android 9; XXX) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.124 Mobile Safari/537.36","sequence":58463,"video": [{"property":{"uuid":"fd65101f-a8df-47c6-a804-3ac719e04ca3","id":"","class": ["video-stream","html5-main-video"],"src":"blob:https://www.youtube.com/a1551960- f053-492e-b945-84b075c8598f","domainName":"r7---sn-5n5ip- ioqe.googlevideo.com","width":0,"height":0,"videoWidth":1280,"videoHeight": 720,"mediaSize":82.384398,"defaultPlaybackRate":1,"playbackRate":1,"playStartTime": 1554511611888.0,"playEndTime":1554511694180.0,"currentPlayPos": 82.384398,"currentPlayTime":82.245},"playback_quality":[],"event_play": [],"event_play_delta":[],"event_pause":[],"event_pause_delta":[],"event_seeking": [],"event_seeking_delta":[],"event_seeked":[],"event_seeked_delta": [],"event_ended":[],"event_ended_delta":[],"event_stalled": [],"event_stalled_delta":[],"event_progress":[],"event_progress_delta": [],"event_waiting":[],"event_waiting_delta":[],"event_canplay": [],"event_canplay_delta":[]}],"resource_timing": [],"REMOTE_ADDR":"180.235.59.0","country":"JP","subdivision":"13","isp":"Asahi Net"} 5 27
( , mongo DB ) session: session id: uuid video
id: uuid service: youtube user agent: XXXXXXX qoe: 4.2 connection: - type: cellular effectiveType: 4g downlink: 1.55 downlinkMax: 100 rtt: 150 apn: NTT DOCOMO plmn: NTT DOCOMO network: # IPͱGeoIPใ - server host: xxx.yyy.com server ip: x.y.z.0 client ip: x.y.z.0 location: tokyo isp: ocn video: video id: xxxxxx duration: 160.3 representations: - resolution: height: 1080 width: 1920 video codec: webm audio codec: opus video bitrate: 12345 audio bitrate: 123 play history: start time: 1234567890 end time: 1234567890 throughput history: # DL࣌ࠁͱɾRTT - dl time: 1234567890 throughput: 123456 rtt: 1234 event history: - event type: seek time: 12.3 representation history: # ৄࡉׂѪ frame drop history: # ৄࡉׂѪ buffer history: # ৄࡉׂѪ cm history: - duration: 12:3 start time: 1234567890 end time: 1234567890 yml 28
https://vm.webdino.org 29
Initial Data Analysis
3/1 4/30 (2 ) 23645 10 12K 1000 ISP ISP
: 31
QoE ISP QoE 32
( ) PC/Mobile ISP 33
None
( ) 5 3 9 ( - ) / 1.0
1.0 CM 35
( ) X 0.93-1.0 CM (CM ) Y, Z 1.0
( ) Y 0-0.03 CM 36 Y X Z Y
1.0 ( - ) / CM 1.0 Y CM 1.0-1.15
Z CM 1.0-1.15 Y, Z 1.5 37 X Z Y
PC PC 0.0 CM 38 PC Mobile
QoE ( ) (QoE) ( ) CM 39
/ 20 ( 130 ) 5, 10 5 5 5,10,15...
5,10,15... ( ) 15 / 40
QoE
( ) (bps) ( ) 貢 ( ) 42
( ) QoE QoE QoE 43
B ( ) C ( ) ( ) 44
A B C A B C
B ( ) C ( ) QoE 45 A B
C
ISP OCN ( ) Asahi Net ( ) Asahi Net
( ) WiFi OCN ( ) ( ) 46 ISP X ISP Y ISP Z ISP X ISP Y ISP Z
ISP ARTERIA ( ), Asahi Net ( ) PC vs
Mobile QoE ( ) ISP ( ) 47 ISP X ISP Y ISP Z
QoE ( , ) QoE 1.0 5.0 30 貢 1080p,
720p ABR 48 A 720p30 1.5~2Mbsp 480p30 1M 1080p 2~3Mbps 1080p 5Mbps
QoE A PC PC QoE 5 1080p30 ( ) PC
3.5 QoE PC 49 A (PC) A (mobile)
QoE B, C QoE 4.6-4.73 720p ( ) B QoE
4.73 1080p ( ) 50 B C
ISP QoE OCN PC 4K, 8K, 60fps ARTERIA 51
OCN ARTERIA
ISP QoE ISP X QoE Asahi Net Android 52 Asahi
Net ISP X
ISP QoE ISP Y QoE 1 , Z QoE 53
ISP Z ISP Y
None
( , ) 2060p30, 1080p30, 720p30, 480p30 ABR (ABR )
55 A
A PC PC 1080p30 FSP PC 1080p30 56 A (PC)
A (mobile)
B, C B B 720p 2Mbps C 1080p 5Mbps 57
B C
ISP OCN PC 貢 58 OCN ARTERIA
ISP Asahi Net Android 59 Asahi Net ISP X
ISP ( ) 60 ISP Z ISP Y
None
( , ) 62 A
A PC PC PC 63 A (PC) A (mobile)
B C 64 B C
ISP ISP ( ) 65 OCN ARTERIA
ISP 66 Asahi Net ISP X
ISP 67 ISP Z ISP Y
QoE ( ) QoE QoE ( ) 68
ISP ( ) PC/Android : OSS 69
OSS YouTube ( ) 70
QoE QoE QoE QoE : 1080p, 30fps, audio 128kbps (
) video 3380kbps => 4.813, 5000kbps => 4.882 QoE ( ) : QoE 71
Web VideoMark ( ) CM ( CM CM ) ISP
QoE ISP 72
Next Steps
: APN PLMN etc. : RTT etc. CM : CM
etc. CM QoE CM ( ) 74
3 10 YouTube + QoE 75
( ) 較 Chrome 76
mongo DB ( ) 77
etc... : https://vm.webdino.org/contact 78
( ) ( ) PC/Android Web VideoMark https://vm.webdino.org/ ( )
(Jetson Nano) LAN OK : https://vm.webdino.org/contact 79
https://vm.webdino.org 80
vol.119 IEICE (IEICE ) © 2019 IEICE , vol.119, no.
298, CQ-2019-92, Nov. 2019 ( ) ® 貢 81