େن໛ϥΠϒ഑৴ MICHII Shunsuke | pixiv Technologies Inc. ImageFlux meetup #1 2018-01-25 ٕज़ηογϣϯ

MICHII Shunsuke | harukasan ImageFluxࣄۀ੹೚ऀ
 ϐΫγϒςΫϊϩδʔζגࣜձࣾࣥߦ໾һ ImageFluxͰ͸GoΛॻ͍ͨΓΠϯϑϥपΓΛ୲౰

ImageFlux Live • ϒϥ΢β͔ΒϓϥάΠϯෆཁͰϥΠϒ഑৴Λ։࢝ • HLSܗࣜͰϒϥ΢βɺϞόΠϧΞϓϦʹ഑৴ • hls.jsͳͲҰൠతͳHLSΫϥΠΞϯτͰ࠶ੜՄೳ • ޷͖ͳղ૾౓/඼࣭ʹεέʔϦϯάͯ͠഑৴

ࣄྫ: pixiv Sketch Live

ࣄྫ: pixiv ONE • ϐΫγϒॳͷϥΠϒυϩʔΠϯάΠϕϯτ • ಉ࣌ࢹௌऀ਺: 5,000ϢʔβʔҎ্ • ϒϥ΢β͔Βͷେن໛഑৴ࣄྫ

ImageFluxͱϥΠϒ഑৴ • ImageFluxͷϛογϣϯ:
 ϐΫγϒͷٕज़ΛεϐϯΞ΢τ͢Δ͜ͱͰϐΫγϒͷٕज़޲্Λ໨ࢦ͢ • pixiv Sketch LiveͷͨΊʹImageFluxνʔϜͰϥΠϒ഑৴ಈతม׵Λ։ൃ • ϥΠϒ഑৴ಈతม׵ΛαʔϏεͱͯ͠ఏڙ༧ఆ • ϐΫγϒͱͯ͠΋ImageFluxͱͯ͠΋ϥΠϒ഑৴ΛڧԽ

ImageFlux Live ഑৴ऀ ImageFlux Live WebRTC HLS ࢹௌऀ • WebRTCετϦʔϜͰड͚औͬͨಈըΛHLSʹม׵ͯ͠഑৴ • HLSܗࣜʹม׵͢Δ͜ͱͰɺ઀ଓ਺Λؾʹͤͣେن໛഑৴͕Մೳ • WebRTCͳͷͰ௿஗Ԇɺ͔ͭHLSͳͷͰେن໛഑৴Ͱ͖Δʂ

HLS (HTTP Live Streaming) • Apple͕։ൃͨ͠ϥΠϒετϦʔϛϯά࢓༷ • ϒϥ΢β/iOS/AndroidͰ࠶ੜՄೳ • iOS: ωΠςΟϒͰରԠ • Android: ExoPlayer • ϒϥ΢β: hls.js

HLS (HTTP Live Streaming) .m3u8 hls0.ts hls1.ts hls2.ts ϓϨΠϦετϑΝΠϧ ಈըετϦʔϜϑΝΠϧ #EXTM3U #EXT-X-TARGETDURATION:3 #EXT-X-VERSION:3 #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:3.00000, hls0.ts #EXTINF:3.00000, hls1.ts #EXTINF:3.00000, hls2.ts ϑΝΠϧͷ௕͞

HLS (HTTP Live Streaming) .m3u8 hls0.ts hls1.ts ϓϨΠϠʔ #EXTM3U #EXT-X-TARGETDURATION:3 #EXT-X-VERSION:3 #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:3.00000, hls0.ts #EXTINF:3.00000, hls1.ts ϙʔϦϯά

HLS (HTTP Live Streaming) .m3u8 hls0.ts hls1.ts ϓϨΠϠʔ ϙʔϦϯά #EXTM3U #EXT-X-TARGETDURATION:3 #EXT-X-VERSION:3 #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:3.00000, hls0.ts #EXTINF:3.00000, hls1.ts

HLS (HTTP Live Streaming) .m3u8 hls0.ts hls1.ts hls2.ts ϓϨΠϠʔ #EXTM3U #EXT-X-TARGETDURATION:3 #EXT-X-VERSION:3 #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:3.00000, hls0.ts #EXTINF:3.00000, hls1.ts #EXTINF:3.00000, hls2.ts ϙʔϦϯά

HLS (HTTP Live Streaming) .m3u8 hls0.ts hls1.ts hls2.ts ϓϨΠϠʔ #EXTM3U #EXT-X-TARGETDURATION:3 #EXT-X-VERSION:3 #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:3.00000, hls0.ts #EXTINF:3.00000, hls1.ts #EXTINF:3.00000, hls2.ts ϙʔϦϯά

HLSͱ஗Ԇ • ηάϝϯτϑΝΠϧͷ௕͞ʹ࠷୹ͷͳ͕͞ • EXT-X-STARTλάΛ࢖͏ͱϓϨΠϦετͷ࠶ੜΦϑηοτΛܾΊΒΕΔ • pixiv Sketch Liveͷઃఆͩͱ஗Ԇ͸10ʙඵ͘Β͍ • ͍·ͷͱ͜ΖؤுΔͱ3ඵ͘Β͍·ͰॖΊͦ͏

γεςϜߏ੒ Transcoder Transcoder Transcoder RTP stream Stream
 Manager Msgpack
 stream TS TS TS ഑৴ऀ WebRTC
 SFU WebRTC API Transcoder: H.264/AVC, OpusετϦʔϜΛ τϥϯείʔυ͠ɺMPEG2-TS ʹmuxingͯ͠ग़ྗ Stream Manager: Transcoderͷϓϩηε؅ཧ
 RTPετϦʔϜΛύʔε͠ɺॱং ੍ޚΛߦͬͯTranscoderʹ͓͘Δ Transcoder͕ग़ྗͨ͠TSϑΝΠϧ ΛHLSʹه࿥ API: ετϦʔϜΛ࢝ΊΔAPI ·ͩαʔϏε࢓༷͕ܾ·͍ͬͯ·ͤΜ

WebRTC SFU • WebRTC SFU / ηογϣϯ؅ཧʹWebRTC SFU SoraΛ࠾༻ • Sora͕ղআͨ͠RTPετϦʔϜΛStream Managerʹసૹ RTP stream WebRTC
 SFU WebRTC

Stream Manager Transcoder Stream
 Manager RTP stream UDP HTTP API TCP • GoͰॻ͔Εͨγϯάϧϓϩηεαʔό • ඞཁͳ਺ͷTranscoderϓϩηεΛىಈ • RTPετϦʔϜΛσίʔυɺύʔεͯ͠Msgpackʹ٧Ί௚ͯ͠τϥϯείʔμʹసૹ • RTPετϦʔϜ͸UDPͷͨΊॱং੍ޚɺόοϑΝϦϯά͕ඞཁʹͳΔ Msgpack

RTP • ϦΞϧλΠϜੑͷͨΊʹ࠶ૹ੍ޚͳͲΛল͍ͨϓϩτίϧ • UDPͳͷͰॱং੍ޚ͞Ε͍ͯͳ͍͕ɺγʔέϯε൪߸ؚ͕·Ε͍ͯΔͷͰ γʔέϯε൪߸Λϕʔεʹฒͼସ͑Δඞཁ͕͋Δ • RFC3350(RTP)Ͱ͸ϖΠϩʔυͷ಺༰·Ͱ͸ఆٛ͞Ε͍ͯͳ͍ • H.264: RFC6184 • Opus: RFC7587

Transcoder • RustͰॻ͔ΕͨCLIϓϩηε • bindgenΛར༻ͯ͠C++ͷSDKΛόΠϯυ • livavformatΛར༻ͯ͠Ի੠ͱಈըΛmuxing Tesla NVIDIA Video Codec SDK Transcoder Libavformat

NVDEC/NVENC • NVIDIA Tesla GPUʹ౥ࡌ͞Ε͍ͯΔNVDEC/NVENCΛར༻ https://developer.nvidia.com/nvidia-video-codec-sdk

NVDEC/NVENC • ࣮ࡍʹσίʔυ/ΤϯίʔυॲཧΛ΍Δͷ͸CUDAίΞͰ͸ͳ͘ઐ༻νοϓ • ઐ༻νοϓͷ਺ʹΑͬͯॲཧͰ͖ΔετϦʔϜ਺͕ܾ·Δ • decoder/encoder/cudaͦΕͧΕͷෛՙΛϞχλϦϯά͢Δͷ͕ॏཁ • CUDA͸εέʔϦϯάʹ͚ͩ࢖༻

NVDEC/NVENC Τϯίʔμෛՙ͕ߴΊ

Rust • ͍͞͠ΐʹࢼͯ͠΋ΒͬͨΞϧόΠτ͕Ұ൪͸΍͘ॻ͚Δͷ͕Rustͩͬͨ • ϝϞϦ·ΘΓͷॻ͖ํ͕C++ΑΓ҆͘͢͝શ/؆୯ • cgoͱҧͬͯbindgen͸όΠϯσΟϯάίʔυΛੜ੒ͯ͘͠ΕΔͷͰཧղ͠ ΍͍͢

ࠓޙͷల๬ • αʔϏε࢓༷ͷܾఆ • ύέοτϩε࣌ͷΤϥʔॲཧ޲্ • ϋʔυ΢ΣΞΤϯίʔμͷݕূ • αʔϏεͱͯ͠ϦϦʔεʂ

·ͱΊ • ImageFluxͰ͸ϥΠϒ഑৴΋ߦ͍͖ͬͯ·͢ʂ • ·ͩαʔϏε࢓༷͕ܾ·͍ͬͯͳ͍ͨΊώΞϦϯάΛߦ͍ͬͯΔஈ֊Ͱ͢ • ڵຯ͋Δํ͸ੋඇ͓੠͕͚͍ͩ͘͞