Scalable Microservice for Media Transcoding / Transcoder

Rails Developers Meetup 2019の発表に利用した資料です。

Transcript

1. Scalable Microservice for Media Transcoding ञҪ ಞ @_atsushisakai גࣜձࣾϛΫγΟ Rails Developers Meetup 2019
2. ञҪ ಞ @_atsushisakai • Ո଒ΞϧόϜʮΈͯͶʯΛ։ൃ͍ͯ͠·͢ • SRE • Ruby on Rails • AWS • iOS/Android • Engineering Manager • iOSDC/DroidKaigi • RailsDM͸͸͡Ίͯ
3. Agenda ʮΈͯͶʯͰ࢖͏ɺಈըฤूɾτϥϯείʔυϚΠΫϩαʔϏεͷ͓࿩ɻ • ͳͥ࡞ͬͨͷ͔ʁ • ղܾ͍ͨ͠ཁ݅ɾ՝୊ͩͬͨͷ͔ʁ • Ͳ͏΍ͬͯղܾͨ͠ͷ͔ʁ
4. ·ͣ͸ͪ͜ΒͷಈըΛ͝ཡ͍ͩ͘͞!
5. ಈը(WIP)
6. ͜Ε͸Ұମ…ʁ • ʮ1ඵಈըʯͱ͍͏ࢠͲ΋ͷ੒௕ΛৼΓฦΔػೳ • େྔʹΞοϓϩʔυ͞Εͨಈը΍ࣸਅΛૉࡐͱͯ͠μΠδΣετಈըΛ࡞੒ • ൢച͢ΔDVDʹऩ࿥ͨ͠ΓɺΞϓϦʹ഑৴ͨ͠Γ͢Δ • ଟ༷ͳޮՌΛ෇͚ͯࢠҭͯͷࢥ͍ग़Λ࡞඼ʹ͢Δ • ௕͍ಈըΛ੾ͬͯɺผͷಈըͱ݁߹͢Δ • WatermarkΛ͚ͭΔʢ೔෇ͳͲͷը૾Λ͸Γ͚ͭΔʣ • ΤϑΣΫτΛ͚ͭͯܨ͛ΔʢΫϩεϑΣʔυͳͲʣ • BGMΛ͚ͭΔ
7. ಈը(WIP)
8. 1ඵಈըΛࣗಈͰେྔʹ࡞Δ
9. ͍͍ײ͡ʹͰ͖ΔϚωʔδυαʔϏεͳ͍Ͱ͔͢ʁ • ྫ͑͹ɺAWS ElasticTranscoder • Ͱ͖Δ͜ͱ • Watermark͸͚ͭΕΔ • ಈըͷ݁߹΋Ͱ͖Δ • Ͱ͖ͳ͍͜ͱ • ෼ׂͨ͠ΓɺΤϑΣΫτ͔͚ͨΓɺͱ͍͏͜ͱ͕Ͱ͖ͳ͍ • Ի੠ͱͷ߹੒ͱ͍͏ͷ΋Ͱ͖ͳ͍
10. ࡞Δ͔͠ͳ͍…
11. Ͳ͏΍ͬͯʁ ੾Γग़͠΍ ϦαΠζͳͲͷ લॲཧ ෳ਺ͷಈըΛ ݁߹ Իָͱ߹੒ ੒Ռ෺͕׬੒ʂ
12. Ͳ͏΍ͬͯʁ ੾Γग़͠΍ ϦαΠζͳͲͷ લॲཧ ෳ਺ͷಈըΛ ݁߹ Իָͱ߹੒ ੒Ռ෺͕׬੒ʂ ͍ΘΏΔύΠϓϥΠϯॲཧ%
13. ͜͜Ͱݴ͏ύΠϓϥΠϯॲཧͱ͸ • ffmpegʹΑΔม׵ॲཧΛ௚ྻʹ࿈݁ͨ͠΋ͷ • લͷॲཧͷΞ΢τϓοτ͕࣍ͷॲཧͷΠϯϓοτͱͳΔ
14. Έͳ͞ΜͳΒͲ͏࣮૷͠·͔͢❓
15. ࠷΋۪௚ͳ࣮૷
16. ࠷΋۪௚ͳ࣮૷ ffmpegΛॱ൪ʹ࣮ߦ͍ͯ͘͠εΫϦϓτΛͳΜΒ͔ͷݴޠͰॻ͍͍ͯ͘ • ύϑΥʔϚϯεͷ໰୊͕ग़ͯ͘Δ • ௚ྻͷॲཧΛಈը1݅ͣͭʹରͯ͠ॱ൪ʹ΍͍ͬͯͨΒ͕࣌ؒແݶʹඞཁ • ٸʹ഑৴ର৅ͷϢʔβʔ͕૿͑ͨΒͲ͏͢Δʁ • ίετͷ໰୊΋ग़ͯ͘Δ • CPU/ϝϞϦ͕༨ͬͨঢ়ଶͰ͸΋͍ͬͨͳ͍ • ޮ཰తͳϦτϥΠ͕Ͱ͖ͳ͍ • ్தͰࣦഊͨ͠Β࠷ॳ͔Β…
17. ฒྻ࣮ߦ͢ΔΑ͏ʹ࣮૷
18. ฒྻ࣮ߦ͢ΔΑ͏ʹ࣮૷ ύΠϓϥΠϯΛಉ࣌ʹෳ਺૸ΒͤΔͱɺ͖ͬͱύϑΥʔϚϯε͸ྑ͘ͳΔ • ϚϧνϓϩηεϓϩάϥϛϯάΊͬͪΌ೉ͦ͠͏ • ઈରʹόάΒͤΔࣗ৴͕͋Γ·͢ • ύϑΥʔϚϯε͸͋Δఔ౓Α͘ͳΔ͔΋͠Εͳ͍͚ͲνϡʔχϯάͰ͖ͳͦ͏ • Ҿ͖ଓ͖ϦτϥΠॲཧ΋೉ͦ͠͏
20. Sidekiq Ruby੡ͷ Job Queue γεςϜͰͱͯ΋༗໊ͳgemɻΦεεϝɻ ʮΈͯͶʯͰ͸100୆Ҏ্ͷSidekiq͕ಈ͘αʔόʔ͕ৗʹಈ͍͍ͯΔɻ • ύϑΥʔϚϯε͕ͱʹ͔͘ྑ͍ • Resque/DelayedJobͷ20ഒۙ͘ߴ଎ • https://github.com/mperham/sidekiq#performance • όʔδϣϯΞοϓʹΑΓͲΜͲΜύϑΥʔϚϯε͕ྑ͘ͳΔ • ଞͷGemґଘ͕ͱͯ΋গͳ͍ • ๛෋ͳެࣜυΩϡϝϯτɺ׆ൃͳ։ൃ • ActiveJobͰAdapter͕ఏڙ͞Ε͍ͯΔ
21. Sidekiq
22. Sidekiq Pro Sidekiqʹ՝ۚ͢Δͱྑ͍͜ͱ͕͋Γ·͢ɻ https://sidekiq.org/products/pro.html • ϝʔϧαϙʔτΛड͚ΒΕΔ • Job͕ϩετ͠ͳ͍ͨΊͷReliabilityػೳ • ෳࡶͳϫʔΫϑϩʔΛ࡞Δ͜ͱ͕Ͱ͖ΔBatchػೳ
23. Sidekiq Pro Sidekiqʹ՝ۚ͢Δͱྑ͍͜ͱ͕͋Γ·͢ɻ https://sidekiq.org/products/pro.html • ϝʔϧαϙʔτΛड͚ΒΕΔ • Job͕ϩετ͠ͳ͍ͨΊͷReliabilityػೳ • ෳࡶͳϫʔΫϑϩʔΛ࡞Δ͜ͱ͕Ͱ͖ΔBatchػೳ
24. Sidekiq Batch • Sidekiq Pro ݶఆͷػೳ • ୯७ͳWorkerͷॲཧʹґଘؔ܎Λ࡞Δ • ෳ਺ͷ Worker ΛଋͶͯɺͦͷ Batch ͝ͱʹॲཧΛਐΊ͍ͯ͘ • ͻͱͭͷ Batch ͕ऴΘΔͱ Callback ͕࣮ߦ͞Εɺ࣍ͷॲཧʹਐΉɺͱ͍͏ ॲཧΛهड़͢Δ͜ͱ͕Ͱ͖Δ • ͨͱ͑ D ͕ࣦഊͯ͠΋ D ͚ͩϦτϥΠ ͯ͘͠ΕΔ A B C D
26. ΋ͬͱෳࡶͳBatchॲཧ • A͕ऴΘΔͷΛ଴ͬͯBʙF͕ىಈ • BʙF͸ύϥϨϧʹىಈ͢Δ • BʙF͕ऴΘͬͨΒGΛىಈ͢Δ • … ͓΍ɺ͜Ε͸…ʁʁ https://github.com/mperham/sidekiq/wiki/Really-Complex-Workflows-with-Batches
27. Ͳ͏΍ͬͯ࡞Δʁ ੾Γग़͠΍ ϦαΠζͳͲͷ લॲཧ ෳ਺ͷಈըΛ ݁߹ Իָͱ߹੒ ੒Ռ෺͕׬੒ʂ Sidekiq BatchΛ࢖͏͜ͱͰ ໨తͷύΠϓϥΠϯ͕࣮૷Ͱ͖ͦ͏‼
28. ύΠϓϥΠϯ࣮૷Ͱ͖Δ͔΋ʁ ׂͱཧ૝ʹ͍ۙ☺ • جຊతʹ͸ͨͩͷSidekiq Workerͳͷ Ͱ͓खܰʹฒྻॲཧͰ͖Δ • ϓϩάϥϛϯά΋؆୯ • ޮ཰తͳϦτϥΠ͕ߦΘΕΔ • Sidekiqͷฒྻ౓ઃఆΛ্͛ΔͱCPU͕ ༨͍ͬͯΔݶΓߴ଎Խ͍ͯ͘͠ • CPU͕଍Γͳ͚Ε͹εέʔϧͤ͞Δ
29. Transcoder
30. ϚΠΫϩαʔϏεͱͯ͠ͷ࣮૷ • ผαʔϏε͔ΒTranscoder΁ͷϦΫΤετ͸AWS SQSΛܦ༝͢Δɻ • ඇಉظɾૄ݁߹ʹαʔϏεؒ࿈ܞ • ࣮ߦ݁Ռ΋AWS SQSʹ৐ͤͯฦ͢ • தؒϑΝΠϧ΍੒Ռ෺͸AWS S3ʹอଘ͢Δ • αʔϏεؒ͸ΦϒδΣΫτ໊ͷ΍ΓͱΓͷΈ • ࣮ϑΝΠϧ͸࢖͏ଆ͕औΓʹߦ͘
31. αʔϏεͷεέʔϥϏϦςΟ • ffmpeg͸ϞϦϞϦͱCPUΛ࢖͏ • ffmpegΛ࣮ߦ͢ΔWorker͕ͨ͘͞ΜQueueʹੵ·Ε͍ͯΔ࣌ʹίϯϐϡʔτϦ ιʔεΛΦʔτεέʔϧ͠ɺશମͷॲཧೳྗΛ্͛ΔʢAWS EC2ͷAutoScale ػೳͰ༰қʹରԠՄೳʣ
32. Transcoder γεςϜߏ੒ • શͯAWSͷαʔϏεΛར༻͠ ͨઃܭ • DBͱͷ΍ΓऔΓͷՕॴͰখن ໛ʹRails͕ಈ͘γεςϜ
33. Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 1 2 3 4 5 6
34. Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 2. SQSΛPolling͢ΔΠϯελϯ ε 1 2 3 4 5 6
35. Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 2. SQSΛPolling͢ΔΠϯελϯ ε 3. SQS͔ΒऔΓग़࣮ͨ͠ߦґཔ ΛSidekiq BatchͰॲཧ͢Δ 1 2 3 4 5 6
36. Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 2. SQSΛPolling͢ΔΠϯελϯ ε 3. SQS͔ΒऔΓग़࣮ͨ͠ߦґཔ ΛSidekiq BatchͰॲཧ͢Δ 4. RDSʹґཔ͞Εͨ಺༰Λอ ࣋ɾࢀর͢Δ ʢActiveRecordʣ 1 2 3 4 5 6
37. Transcoder γεςϜߏ੒ 5. Sidekiq ͷ Job ΛӬଓԽ͢Δ όοΫΤϯυͱͯ͠ͷRedis 1 2 3 4 5 6
38. Transcoder γεςϜߏ੒ 5. Sidekiq ͷ Job ΛӬଓԽ͢Δ όοΫΤϯυͱͯ͠ͷRedis 6. தؒੜ੒෺ɾ࠷ऴ੒Ռ෺Λอ ଘ͢ΔS3όέοτ 1 2 3 4 5 6
39. Transcoder ͷΞʔΩςΫνϟ JobͱResult • Job͸ɺSQSʹ৐ͤΔJSONͷ͜ͱ • ܕ͸ࣄલʹఆٛ͞Ε͍ͯΔ • SQSΛPolling͠ɺTranscoder͸ඇಉظʹॲཧΛ։࢝͢Δ • Result͸ɺSQSʹ৐ͤΔॲཧ݁ՌͷJSONͷ͜ͱ • ܕ͸ࣄલʹఆٛ͞Ε͍ͯΔ • SQSΛґཔͨ͠ଆ͕Pollingͯ݁͠ՌΛड͚ͱΔ • ݁Ռ͕Errorͷ৔߹ʹ͸ErrorResultΛฦ͢
43. Transcoder ͷΞʔΩςΫνϟ Pipeline • JobͷೖྗΛड͚ɺ࠷ޙʹResultΛग़ྗ͢Δ·ͰͷશମͷॲཧΛ࣮૷ • = Sidekiq Batchͷ࣮૷ͦͷ΋ͷɻ͜Εࣗମ΋Sidekiq Worker • ಺෦ʹෳ਺ͷStepΛ࣋ͭ • Pipeline͝ͱʹ੒Ռ෺͕ఆٛ͞Ε͍ͯΔ • ʮΞϓϦ഑৴ʯ༻ͷಈըΛͭ͘ΔPipeline • ʮDVDʯ༻ͷಈըΛͭ͘ΔPipeline
44. Transcoder ͷΞʔΩςΫνϟ Step • Pipeline͕ఆٛͨ͠ෳ਺ͷWorkerͷଋͷ͜ͱΛ͜͏ݺΜͰ͍Δ • = Sidekiq Batchʹ͓͚ΔBatchͷ͜ͱ • Step͝ͱʹCallbackΛΩοΫ͠ɺ࣍ͷStepʹҠΔ
45. Transcoder ͷΞʔΩςΫνϟ Worker • Step಺ͷͻͱͭͣͭͷSidekiq Workerͷ͜ͱ • ͢΂ͯSidekiq্Ͱඇಉظɾฒྻʹ࣮ߦ͞ΕΔ • جຊ͸ffmpegͷίϚϯυΛҰճ࣮ߦ͢Δཻ౓Ͱ࣮૷͢Δ • ffmpegͷ࣮ߦ݁Ռͷग़ྗϑΝΠϧ͸S3ʹอଘ͢Δ
46. Transcoder ͷΞʔΩςΫνϟ Command • WorkerͰ࣮ߦ͢ΔffmpegίϚϯυΛڞ௨ԽͷͨΊʹϥΠϒϥϦԽͨ͠΋ͷ • ୯ʹffmpegΛ࣮ߦ͠ɺͦͷ݁ՌΛϑΝΠϧͱͯ͠ड͚औΔ͚ͩ
47. Pipeline Step Step Step Worker + Command
48. Pipeline Step Step Step Worker + Command Sidekiq BatchͰղܾ
49. Sidekiq Batch࣮૷ͷίπ • Batch಺෦ͷWorker͸ґଘ౓Λ௿͘ɺཻ͔ͭ౓Λখ࣮͘͞૷͢Δɻ • εέʔϧ͠΍͍͢ɺϦτϥΠ͠΍͍͢ɺͳͲͷ໨తɻ • ෳࡶͳBatchΛ࡞Δ৔߹ɺPipelineΛදݱ͢Δίʔυ͸ͲΜͲΜංେԽ͢Δ • Callbackͷ࣮ߦQueue͸ͪΌΜͱઃఆͨ͠ํ͕ྑ͍ɻ • Callbackॲཧ΋࣮ମ͸Sidekiq Workerͱͯ͠ॳظԽ͞Εɺ࣮ߦ͞ΕΔɻ • େྔʹߴ଎ͳCallback͕࣮ߦ͞ΕΔͱRedisʹෛՙ͕ҟৗʹ͔͔ͬͯࢮ…
50. ͕࣌ؒͳͯ͘Ͱ͖ͳ͔ͬͨ͜ͱ • ಈըม׵ύϥϝʔλͷPresetػೳΛ࡞Γ͔ͨͬͨ • ઃఆϑΝΠϧΛ༻ҙ͓͚ͯ͠͹ɺJobͰPresetΛࢦఆ͢Δ͚ͩͰɺ೚ҙͷύϥ ϝʔλͰ͍ΖΜͳը࣭ɾαΠζͷ੒Ռ෺Λग़ͤΔΑ͏ʹ͔ͨͬͨ͠ • TV༻ɺεϚϗ༻ɺPC༻ͱ͔… • Pipelineͷࣗಈੜ੒ • Α͋͘ΔCIͷઃఆͷΑ͏ʹyamlʹPipelineͷॲཧΛఆ͍ٛͯͬͨ͠Βίʔυ Λࣗಈੜ੒͞ΕɺউखʹPipeline͕ग़དྷ্͕ΔΑ͏ʹ͔ͨͬͨ͠ • Pipelineͷ࣮૷͕ංେԽͯ͠΋ؾʹͳΒͳ͍
51. ޻෉ͨ͠ͱ͜Ζ 1ʢ࣌ؒ͋Ε͹ʁʣ • ʮલͷStepͷग़ྗΛɺ࣍ͷStepͷೖྗͱ͢ΔʯΛͲ͏΍ͬͯ΍Δ͔ʁ • Sidekiq BatchͷCallbackͰҾ͖ճ͍ͯ͘͜͠ͱͰղܾɻ
53. ޻෉ͨ͠ͱ͜Ζ 2ʢ࣌ؒ͋Ε͹ʁʣ • ffmpegίϚϯυͷೖग़ྗ࢓༷͕ಛघ͗͢Δ • ͻͱͭͷϑΝΠϧΛInput͠ɺͻͱͭͷϑΝΠϧΛOutput͢Δ • ෳ਺ͷϑΝΠϧΛInput͠ɺͻͱͭͷϑΝΠϧΛOutput͢Δ • ͔͠΋ೖྗͷॱং͸ॏཁ • CommandͷجఈΫϥεΛ࡞ͬͯந৅Խ • Pipeline → Step → Worker ΁ͱɺෳ਺ͷInputΛॱংΛอͪͳ͕Β͍͍ײ ͡ʹCommandʹ౉͢
55. ·ͱΊ • εέʔϦϯάՄೳͳಈըฤू Microservice ΛSidekiq BatchΛ࢖͍͍ͬͯײ͡ ʹ࡞ͬͨ • Sidekiq͸ͱͯ΋ྑ͍ • Sidekiq Pro͸΋ͬͱྑ͍ • Sidekiq BatchΛ࢖͏͜ͱͰෳࡶͳύΠϓϥΠϯॲཧΛεέʔϧͰ͖Δ • ಈըॲཧҎ֎ʹ΋ɺूܭɾղੳͱ͔ɺෳࡶͳcron scriptͱ͔… • ந৅తͳ࿩͕ଟ͘ͳͬͯ͠·ͬͨͷͰɺৄࡉ͸Sidekiq BatchΛੋඇ͓ࢼͩ͘͠ ͍͞ʢ$950/yearʣ
56. ·ͱΊ
58. THANK YOU!!