Go and on-demand video streaming / Goとオンデマンド動画配信

Lecture at university of Aizu / 2017/12/10

Transcript

1. Goͱ ΦϯσϚϯυಈը ഑৴ج൫ @__timakin__
2. • Seiji Takahashi • Github: timakin / Twitter: @__timakin__ • גࣜձࣾσΟʔɾΤψɾΤʔ • Perl / JavaScript • גࣜձࣾτϥϯεϦϛοτ • C++ (Cocos-2dx) • גࣜձࣾGunosy ৽نࣄۀ։ൃࣨ • Go / Swift ࣗݾ঺հ
3. גࣜձࣾGunosy ৽نࣄۀ։ൃࣨʹ͍ͭͯ
4. Gunosyͷ5ʙ10೥ޙͷ੒௕Λ࡞Δ΂͘ɺ VR΍ARɺԻ੠UIͳͲࠓޙීٴ͢ΔՄೳੑͷ͋Δ
 ৽͍͠σόΠε΍ٕज़Λݚڀ͠ɺ
 ৽نαʔϏεͷ্ཱͪ͛ͳͲΛߦ͏෦ॺ
5. Ϧαʔν • ւ֎ͷઌਐࣄྫͷ·ͱΊ • SDKͷެࣜDocs΍Github repoͷ
 ίʔυΛړͬͯ࠷৽ٕज़ͷ
 νϡʔτϦɾԠ༻ྫΛ୳Δ • SlackʹRSS௨஌ɺϨϙʔτڞ༗ ΍Δ͜ͱ
6. ։ൃ • ͓ன͍ͭͰʹνϡʔτϦΞϧ
 ϋοΧιϯ • ্ཱ͕ͪΓظ͸
 ۃྗখن໛νʔϜͰ ΍Δ͜ͱ
7. ྫ
8. ྫ
9. ษڧձ։࠵
10. ຊ೔ͷΞδΣϯμ
11. GoͱΦϯσϚϯυಈը഑৴ج൫ • ಈը഑৴ͱ͸ʁ • ഑৴·ͰͷྲྀΕ • GoΛͲ͏࢖͏͔ • ·ͱΊ ΞδΣϯμ
12. ΞδΣϯμ GoͰ࡞ΔΦϯσϚϯυಈը഑৴ج൫ • ಈը഑৴ͱ͸ʁ • ഑৴·ͰͷྲྀΕ • GoΛͲ͏࢖͏͔ • ·ͱΊ
13. େ͖͘෼͚ͯ ̎ छྨ ಈը഑৴ͱ͸ʁ
14. ϦΞϧλΠϜ഑৴ • ͍ΘΏΔϥΠϒಈը • WebSocketͷ
 ίωΫγϣϯͳͲΛ
 ௨ͯ͡ϒϩʔυΩϟετ ಈը഑৴ͷछྨ
15. ΦϯσϚϯυ഑৴ • ͋Β͔͡Ί࡞੒͞Εͨ
 ಈըΛݟΔ • ಈը͕ඞཁʹͳͬͨ
 λΠϛϯάͰࣗ༝ʹ
 ࢹௌͰ͖Δ ಈը഑৴ͷछྨ
16. GoͰ࡞ΔΦϯσϚϯυಈը഑৴ج൫ • ಈը഑৴ͱ͸ʁ • ഑৴·ͰͷྲྀΕ • GoΛͲ͏࢖͏͔ • ·ͱΊ ΞδΣϯμ
17. ಈը഑৴ʹඞཁͳ࡞ۀ Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ഑৴
18. ಈը഑৴ʹඞཁͳ࡞ۀ Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ഑৴ ετϨʔδ ഑৴ऀ AWSͷS3 Bucket΍
 GCPͷCloud Storage
 ͳͲʹΞοϓϩʔυ
19. ಈը഑৴ʹඞཁͳ࡞ۀ Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ഑৴ Τϯίʔμʔ ετϨʔδ ࣈນૠೖ΍ೖྗө૾ͷ
 ΤϯίʔσΟϯάʹՃ͑ͯɺ αϜωΠϧͷ੾ΓऔΓͳͲ
20. ಈը഑৴ʹඞཁͳ࡞ۀ Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ഑৴ ετϨʔδ ಈըΛ෼ׂͯ͠
 ޮ཰తʹ഑৴Մೳͳ
 ϑΝΠϧΛੜ੒͢Δɺ
 ҉߸Խ͢Δ ύοέʔδϯά"1*
21. ಈը഑৴ʹඞཁͳ࡞ۀ Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ഑৴ ഑৴"1*
    
    $%/ ετϨʔδ ड৴ऀ HTTPΫϥΠΞϯτʹରͯ͠ CDNΛ௨ͭͭ͠഑৴
22. શମ૾ ετϨʔδ ഑৴ऀ ө૾ॲཧɺ
 ύοέʔδϯά ഑৴"1*
    
    $%/ ड৴ऀ
23. GoͱΦϯσϚϯυಈը഑৴ج൫ • ಈը഑৴ͱ͸ʁ • ഑৴·ͰͷྲྀΕ • GoΛͲ͏࢖͏͔ • ·ͱΊ ΞδΣϯμ
24. Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ഑৴ Go͕ؔΘΔͱ͜Ζ ʁ
25. Go͕ؔΘΔͱ͜Ζ ө૾ॲཧ • ఏڙը࣭ʹ߹ΘͤͨϑϨʔϜ/ϏοτϨʔτͷௐ੔ • ϩεΛ཈͑ͨѹॖ ύοέʔδԽ • m3u8 + TSϑΝΠϧͷੜ੒ • ετϨʔδ΁ͷΞοϓϩʔυ ഑৴ • ಈըͷJSONΛฦ͢௨ৗͷAPIػೳ • αϜωΠϧ΍ؔ࿈ಈըੜ੒
26. Կ΋ߟ͑ͳ͍഑৴ mp4
27. Կ΋ߟ͑ͳ͍഑৴ mp4 Ϋιॏ͍
28. HLS഑৴ • HTTP Live Streaming • Apple͕iOS޲͚ʹ։ൃͨ͠ετϦʔϛϯάϓϩτίϧ • WindowsͳͲ͸࠶ੜ؀ڥ͕ݶఆ • ϫʔΫϑϩʔ • ಈըίϯςϯπͷΞοϓϩʔυ • ηάϝϯτԽ • ಉ࣌ʹαϜωΠϧੜ੒΍഑৴඼࣭ผͷੜ੒ͳͲ΋ • IndexFile(m3u8)Λ഑৴ͯ͠ɺΫϥΠΞϯτ͸
 ΠϯσοΫε͞ΕͨMPEG-2 TSϑΝΠϧΛಡΈʹߦ͘
29. HLSͷத਎ segmented media files (MPEG-2 TS) Index file (m3u8)
30. HLS഑৴ͷྲྀΕ NV )JHI NV .JE NV -PX
31. ؆୯ʹHLSΤϯίʔυ͢Δํ๏ ffmpeg Elastic Transcoder
32. ؆୯ʹHLSΤϯίʔυ͢Δํ๏ ffmpeg Elastic Transcoder
33. ffmpeg • ಈըͱԻ੠Λه࿥ɾม׵ɾ࠶ੜ͢ΔͨΊͷϑϦʔιϑτ ΢ΣΞ • ffmpegΛར༻ͨ͠Τϯίʔυɾσίʔυ͕Ͱ͖Δ
 ঎༻ιϑτ΢ΣΞ͸ஶ࡞ݖ৵֐໰୊ΛሃΉ…?
 https://blog.tai2.net/mpegla_and_ffmpeg.html
34. ffmpeg-based API • Docker multi-stage buildͰੜ੒͞ΕͨimageΛݩʹɺ execܦ༝ͰffmpegΛݺͿɻ • binding࣮૷͔ίϚϯυͷexecuteͰݺͿ΄͔ͳ͍ɻ • ݸਓతʹ͸GKEͰ্هimageΛݩʹͨ͠ίϯςφΛཱͯ ͯɺੜ੒݁ՌΛGoogle Cloud StorageʹΞοϓϩʔυ͢ ΔͳͲ͕ྑ͍ͱࢥͬͯΔɻ
35. ffmpeg-based API media file Kubernetes Engine Cloud Storage goroutine
36. ffmpeg encoding
37. ffmpeg encoding
38. Encoding command mp4ΛAACɺH.264ͰΤϯίʔυ͠ɺ 4ඵ͝ͱͷηάϝϯτʹ෼ׂͨ݁͠ՌΛอଘ͢Δίʔυɻ ಉҰσΟϨΫτϦʹm3u8ͱtsϑΝΠϧ͕ੜ੒͞ΕΔɻ
39. Exec Commands ffmpegͷbinding࣮૷(goav)Λ࢖͏ͷ΋͋Γɻ ͨͩɺॊೈͳΦϓγϣϯઃఆͳͲΛؑΈΔͱffmpegΛ
 ͦͷ··࣮ߦͨ͠ํ͕ྑ͍ɻ
40. ffmpeg bindingΛ࢖Θͳ͍৔߹͸۪௚ʹݺͿ͔͠ͳ͍ Exec Commands
41. goroutineΛར༻ͯ͠ੜ੒͞Εͨ ϑΝΠϧΛฒྻΞοϓϩʔυ Parallel upload
42. ffmpeg-based APIͷ·ͱΊ • αʔόʔʹΠϯετʔϧ͞ΕͨffmpegΛར༻͢Δɻ • GKEͳͲΛ࢖͍ͭͭɺDocker multi-stage buildͰੜ੒ ͞ΕͨimageΛݩʹɺexecܦ༝ͰffmpegΛݺͿํ๏΋
 ͋Γɻ • binding࣮૷ΑΓ͸ίϚϯυͷexecuteΛ͓͢͢Ίɻ
43. ؆୯ʹHLSΤϯίʔυ͢Δํ๏ ffmpeg Elastic Transcoder
44. Elastic Transcoder • AWS্ͷϚωʔδυಈըΤϯίʔμʔ • ίϯςϯπͷ࠶ੜ࣌ؒΛϕʔεʹ՝ۚ͞ΕΔ • ੜ੒݁Ռ͸S3ΦϒδΣΫτͱͯ͠อଘ͞ΕΔ
45. Elastic Transcoder input bucket media file Lambda Transcoder output bucket Transcoding Pipeline
46. Elastic Transcoder input bucket media file Lambda Transcoder output bucket Transcoding Pipeline
48. • goͰlambdaؔ਺Λॻ͚ΔϥΠϒϥϦ • όΠφϦͱͦΕΛݺͼग़͢index.jsΛੜ੒ͯ͠ɺ
 Node.jsϥϯλΠϜͰ࣮ߦ͢Δ go-apex
49. Elastic Transcoder input bucket media file Lambda Transcoder output bucket Transcoding Pipeline
50. S3 Observation
51. Elastic Transcoder input bucket media file Lambda Transcoder output bucket Transcoding Pipeline
52. Parameter Tuning
53. Elastic Transcoder ·ͱΊ • go-apexʹΑΔS3؂ࢹͱม׵ॲཧ • ࠷ۙLambda͕σϑΥϧτͰGoରԠ͢Δతͳൃදͨ͠ ͠஌ݟͨ·ͬͨΒͦͬͪʹ͠·͠ΐ͏ɻ • ϚωʔδυͰ৭ʑ΍ͬͯ͘ΕΔ෼ɺ෼ׂස౓ͳͲͷ
 ύϥϝʔλʔνϡʔχϯάʹ஫ҙ͠·͠ΐ͏ɻ
54. ʕ ◔ϖ◔ʔ < Thank You!!