Scalable Microservice for Media Transcoding / Transcoder

Transcript

1. Scalable Microservice for Media Transcoding ञҪ ಞ @_atsushisakai גࣜձࣾϛΫγΟ Rails

   Developers Meetup 2019

2. ञҪ ಞ @_atsushisakai • Ո଒ΞϧόϜʮΈͯͶʯΛ։ൃ͍ͯ͠·͢ • SRE • Ruby on

   Rails • AWS • iOS/Android • Engineering Manager • iOSDC/DroidKaigi • RailsDM͸͸͡Ίͯ

3. Agenda ʮΈͯͶʯͰ࢖͏ɺಈըฤूɾτϥϯείʔυϚΠΫϩαʔϏεͷ͓࿩ɻ • ͳͥ࡞ͬͨͷ͔ʁ • ղܾ͍ͨ͠ཁ݅ɾ՝୊ͩͬͨͷ͔ʁ • Ͳ͏΍ͬͯղܾͨ͠ͷ͔ʁ

4. ·ͣ͸ͪ͜ΒͷಈըΛ͝ཡ͍ͩ͘͞!

5. ಈը(WIP)

6. ͜Ε͸Ұମ…ʁ • ʮ1ඵಈըʯͱ͍͏ࢠͲ΋ͷ੒௕ΛৼΓฦΔػೳ • େྔʹΞοϓϩʔυ͞Εͨಈը΍ࣸਅΛૉࡐͱͯ͠μΠδΣετಈըΛ࡞੒ • ൢച͢ΔDVDʹऩ࿥ͨ͠ΓɺΞϓϦʹ഑৴ͨ͠Γ͢Δ • ଟ༷ͳޮՌΛ෇͚ͯࢠҭͯͷࢥ͍ग़Λ࡞඼ʹ͢Δ •

   ௕͍ಈըΛ੾ͬͯɺผͷಈըͱ݁߹͢Δ • WatermarkΛ͚ͭΔʢ೔෇ͳͲͷը૾Λ͸Γ͚ͭΔʣ • ΤϑΣΫτΛ͚ͭͯܨ͛ΔʢΫϩεϑΣʔυͳͲʣ • BGMΛ͚ͭΔ

7. ಈը(WIP)

8. 1ඵಈըΛࣗಈͰେྔʹ࡞Δ

9. ͍͍ײ͡ʹͰ͖ΔϚωʔδυαʔϏεͳ͍Ͱ͔͢ʁ • ྫ͑͹ɺAWS ElasticTranscoder • Ͱ͖Δ͜ͱ • Watermark͸͚ͭΕΔ • ಈըͷ݁߹΋Ͱ͖Δ

   • Ͱ͖ͳ͍͜ͱ • ෼ׂͨ͠ΓɺΤϑΣΫτ͔͚ͨΓɺͱ͍͏͜ͱ͕Ͱ͖ͳ͍ • Ի੠ͱͷ߹੒ͱ͍͏ͷ΋Ͱ͖ͳ͍

10. ࡞Δ͔͠ͳ͍…

11. Ͳ͏΍ͬͯʁ ੾Γग़͠΍ ϦαΠζͳͲͷ લॲཧ ෳ਺ͷಈըΛ ݁߹ Իָͱ߹੒ ੒Ռ෺͕׬੒ʂ

12. Ͳ͏΍ͬͯʁ ੾Γग़͠΍ ϦαΠζͳͲͷ લॲཧ ෳ਺ͷಈըΛ ݁߹ Իָͱ߹੒ ੒Ռ෺͕׬੒ʂ ͍ΘΏΔύΠϓϥΠϯॲཧ%

13. ͜͜Ͱݴ͏ύΠϓϥΠϯॲཧͱ͸ • ffmpegʹΑΔม׵ॲཧΛ௚ྻʹ࿈݁ͨ͠΋ͷ • લͷॲཧͷΞ΢τϓοτ͕࣍ͷॲཧͷΠϯϓοτͱͳΔ

14. Έͳ͞ΜͳΒͲ͏࣮૷͠·͔͢❓

15. ࠷΋۪௚ͳ࣮૷

16. ࠷΋۪௚ͳ࣮૷ ffmpegΛॱ൪ʹ࣮ߦ͍ͯ͘͠εΫϦϓτΛͳΜΒ͔ͷݴޠͰॻ͍͍ͯ͘ • ύϑΥʔϚϯεͷ໰୊͕ग़ͯ͘Δ • ௚ྻͷॲཧΛಈը1݅ͣͭʹରͯ͠ॱ൪ʹ΍͍ͬͯͨΒ͕࣌ؒແݶʹඞཁ • ٸʹ഑৴ର৅ͷϢʔβʔ͕૿͑ͨΒͲ͏͢Δʁ • ίετͷ໰୊΋ग़ͯ͘Δ

    • CPU/ϝϞϦ͕༨ͬͨঢ়ଶͰ͸΋͍ͬͨͳ͍ • ޮ཰తͳϦτϥΠ͕Ͱ͖ͳ͍ • ్தͰࣦഊͨ͠Β࠷ॳ͔Β…

17. ฒྻ࣮ߦ͢ΔΑ͏ʹ࣮૷

18. ฒྻ࣮ߦ͢ΔΑ͏ʹ࣮૷ ύΠϓϥΠϯΛಉ࣌ʹෳ਺૸ΒͤΔͱɺ͖ͬͱύϑΥʔϚϯε͸ྑ͘ͳΔ • ϚϧνϓϩηεϓϩάϥϛϯάΊͬͪΌ೉ͦ͠͏ • ઈରʹόάΒͤΔࣗ৴͕͋Γ·͢ • ύϑΥʔϚϯε͸͋Δఔ౓Α͘ͳΔ͔΋͠Εͳ͍͚ͲνϡʔχϯάͰ͖ͳͦ͏ • Ҿ͖ଓ͖ϦτϥΠॲཧ΋೉ͦ͠͏

19. None

20. Sidekiq Ruby੡ͷ Job Queue γεςϜͰͱͯ΋༗໊ͳgemɻΦεεϝɻ ʮΈͯͶʯͰ͸100୆Ҏ্ͷSidekiq͕ಈ͘αʔόʔ͕ৗʹಈ͍͍ͯΔɻ • ύϑΥʔϚϯε͕ͱʹ͔͘ྑ͍ • Resque/DelayedJobͷ20ഒۙ͘ߴ଎

    • https://github.com/mperham/sidekiq#performance • όʔδϣϯΞοϓʹΑΓͲΜͲΜύϑΥʔϚϯε͕ྑ͘ͳΔ • ଞͷGemґଘ͕ͱͯ΋গͳ͍ • ๛෋ͳެࣜυΩϡϝϯτɺ׆ൃͳ։ൃ • ActiveJobͰAdapter͕ఏڙ͞Ε͍ͯΔ

21. Sidekiq

22. Sidekiq Pro Sidekiqʹ՝ۚ͢Δͱྑ͍͜ͱ͕͋Γ·͢ɻ https://sidekiq.org/products/pro.html • ϝʔϧαϙʔτΛड͚ΒΕΔ • Job͕ϩετ͠ͳ͍ͨΊͷReliabilityػೳ • ෳࡶͳϫʔΫϑϩʔΛ࡞Δ͜ͱ͕Ͱ͖ΔBatchػೳ

23. Sidekiq Pro Sidekiqʹ՝ۚ͢Δͱྑ͍͜ͱ͕͋Γ·͢ɻ https://sidekiq.org/products/pro.html • ϝʔϧαϙʔτΛड͚ΒΕΔ • Job͕ϩετ͠ͳ͍ͨΊͷReliabilityػೳ • ෳࡶͳϫʔΫϑϩʔΛ࡞Δ͜ͱ͕Ͱ͖ΔBatchػೳ

24. Sidekiq Batch • Sidekiq Pro ݶఆͷػೳ • ୯७ͳWorkerͷॲཧʹґଘؔ܎Λ࡞Δ • ෳ਺ͷ

    Worker ΛଋͶͯɺͦͷ Batch ͝ͱʹॲཧΛਐΊ͍ͯ͘ • ͻͱͭͷ Batch ͕ऴΘΔͱ Callback ͕࣮ߦ͞Εɺ࣍ͷॲཧʹਐΉɺͱ͍͏ ॲཧΛهड़͢Δ͜ͱ͕Ͱ͖Δ • ͨͱ͑ D ͕ࣦഊͯ͠΋ D ͚ͩϦτϥΠ ͯ͘͠ΕΔ A B C D
25. None

26. ΋ͬͱෳࡶͳBatchॲཧ • A͕ऴΘΔͷΛ଴ͬͯBʙF͕ىಈ • BʙF͸ύϥϨϧʹىಈ͢Δ • BʙF͕ऴΘͬͨΒGΛىಈ͢Δ • … ͓΍ɺ͜Ε͸…ʁʁ

    https://github.com/mperham/sidekiq/wiki/Really-Complex-Workflows-with-Batches

27. Ͳ͏΍ͬͯ࡞Δʁ ੾Γग़͠΍ ϦαΠζͳͲͷ લॲཧ ෳ਺ͷಈըΛ ݁߹ Իָͱ߹੒ ੒Ռ෺͕׬੒ʂ Sidekiq BatchΛ࢖͏͜ͱͰ

    ໨తͷύΠϓϥΠϯ͕࣮૷Ͱ͖ͦ͏‼

28. ύΠϓϥΠϯ࣮૷Ͱ͖Δ͔΋ʁ ׂͱཧ૝ʹ͍ۙ☺ • جຊతʹ͸ͨͩͷSidekiq Workerͳͷ Ͱ͓खܰʹฒྻॲཧͰ͖Δ • ϓϩάϥϛϯά΋؆୯ • ޮ཰తͳϦτϥΠ͕ߦΘΕΔ

    • Sidekiqͷฒྻ౓ઃఆΛ্͛ΔͱCPU͕ ༨͍ͬͯΔݶΓߴ଎Խ͍ͯ͘͠ • CPU͕଍Γͳ͚Ε͹εέʔϧͤ͞Δ

29. Transcoder

30. ϚΠΫϩαʔϏεͱͯ͠ͷ࣮૷ • ผαʔϏε͔ΒTranscoder΁ͷϦΫΤετ͸AWS SQSΛܦ༝͢Δɻ • ඇಉظɾૄ݁߹ʹαʔϏεؒ࿈ܞ • ࣮ߦ݁Ռ΋AWS SQSʹ৐ͤͯฦ͢ •

    தؒϑΝΠϧ΍੒Ռ෺͸AWS S3ʹอଘ͢Δ • αʔϏεؒ͸ΦϒδΣΫτ໊ͷ΍ΓͱΓͷΈ • ࣮ϑΝΠϧ͸࢖͏ଆ͕औΓʹߦ͘

31. αʔϏεͷεέʔϥϏϦςΟ • ffmpeg͸ϞϦϞϦͱCPUΛ࢖͏ • ffmpegΛ࣮ߦ͢ΔWorker͕ͨ͘͞ΜQueueʹੵ·Ε͍ͯΔ࣌ʹίϯϐϡʔτϦ ιʔεΛΦʔτεέʔϧ͠ɺશମͷॲཧೳྗΛ্͛ΔʢAWS EC2ͷAutoScale ػೳͰ༰қʹରԠՄೳʣ

32. Transcoder γεςϜߏ੒ • શͯAWSͷαʔϏεΛར༻͠ ͨઃܭ • DBͱͷ΍ΓऔΓͷՕॴͰখن ໛ʹRails͕ಈ͘γεςϜ

33. Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 1 2 3 4 5

    6

34. Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 2. SQSΛPolling͢ΔΠϯελϯ ε 1 2

    3 4 5 6

35. Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 2. SQSΛPolling͢ΔΠϯελϯ ε 3. SQS͔ΒऔΓग़࣮ͨ͠ߦґཔ

    ΛSidekiq BatchͰॲཧ͢Δ 1 2 3 4 5 6

36. Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 2. SQSΛPolling͢ΔΠϯελϯ ε 3. SQS͔ΒऔΓग़࣮ͨ͠ߦґཔ

    ΛSidekiq BatchͰॲཧ͢Δ 4. RDSʹґཔ͞Εͨ಺༰Λอ ࣋ɾࢀর͢Δ ʢActiveRecordʣ 1 2 3 4 5 6

37. Transcoder γεςϜߏ੒ 5. Sidekiq ͷ Job ΛӬଓԽ͢Δ όοΫΤϯυͱͯ͠ͷRedis 1 2

    3 4 5 6

38. Transcoder γεςϜߏ੒ 5. Sidekiq ͷ Job ΛӬଓԽ͢Δ όοΫΤϯυͱͯ͠ͷRedis 6. தؒੜ੒෺ɾ࠷ऴ੒Ռ෺Λอ

    ଘ͢ΔS3όέοτ 1 2 3 4 5 6

39. Transcoder ͷΞʔΩςΫνϟ JobͱResult • Job͸ɺSQSʹ৐ͤΔJSONͷ͜ͱ • ܕ͸ࣄલʹఆٛ͞Ε͍ͯΔ • SQSΛPolling͠ɺTranscoder͸ඇಉظʹॲཧΛ։࢝͢Δ •

    Result͸ɺSQSʹ৐ͤΔॲཧ݁ՌͷJSONͷ͜ͱ • ܕ͸ࣄલʹఆٛ͞Ε͍ͯΔ • SQSΛґཔͨ͠ଆ͕Pollingͯ݁͠ՌΛड͚ͱΔ • ݁Ռ͕Errorͷ৔߹ʹ͸ErrorResultΛฦ͢
40. None
41. None
42. None

43. Transcoder ͷΞʔΩςΫνϟ Pipeline • JobͷೖྗΛड͚ɺ࠷ޙʹResultΛग़ྗ͢Δ·ͰͷશମͷॲཧΛ࣮૷ • = Sidekiq Batchͷ࣮૷ͦͷ΋ͷɻ͜Εࣗମ΋Sidekiq Worker

    • ಺෦ʹෳ਺ͷStepΛ࣋ͭ • Pipeline͝ͱʹ੒Ռ෺͕ఆٛ͞Ε͍ͯΔ • ʮΞϓϦ഑৴ʯ༻ͷಈըΛͭ͘ΔPipeline • ʮDVDʯ༻ͷಈըΛͭ͘ΔPipeline

44. Transcoder ͷΞʔΩςΫνϟ Step • Pipeline͕ఆٛͨ͠ෳ਺ͷWorkerͷଋͷ͜ͱΛ͜͏ݺΜͰ͍Δ • = Sidekiq Batchʹ͓͚ΔBatchͷ͜ͱ •

    Step͝ͱʹCallbackΛΩοΫ͠ɺ࣍ͷStepʹҠΔ

45. Transcoder ͷΞʔΩςΫνϟ Worker • Step಺ͷͻͱͭͣͭͷSidekiq Workerͷ͜ͱ • ͢΂ͯSidekiq্Ͱඇಉظɾฒྻʹ࣮ߦ͞ΕΔ • جຊ͸ffmpegͷίϚϯυΛҰճ࣮ߦ͢Δཻ౓Ͱ࣮૷͢Δ

    • ffmpegͷ࣮ߦ݁Ռͷग़ྗϑΝΠϧ͸S3ʹอଘ͢Δ

46. Transcoder ͷΞʔΩςΫνϟ Command • WorkerͰ࣮ߦ͢ΔffmpegίϚϯυΛڞ௨ԽͷͨΊʹϥΠϒϥϦԽͨ͠΋ͷ • ୯ʹffmpegΛ࣮ߦ͠ɺͦͷ݁ՌΛϑΝΠϧͱͯ͠ड͚औΔ͚ͩ

47. Pipeline Step Step Step Worker + Command

48. Pipeline Step Step Step Worker + Command Sidekiq BatchͰղܾ

49. Sidekiq Batch࣮૷ͷίπ • Batch಺෦ͷWorker͸ґଘ౓Λ௿͘ɺཻ͔ͭ౓Λখ࣮͘͞૷͢Δɻ • εέʔϧ͠΍͍͢ɺϦτϥΠ͠΍͍͢ɺͳͲͷ໨తɻ • ෳࡶͳBatchΛ࡞Δ৔߹ɺPipelineΛදݱ͢Δίʔυ͸ͲΜͲΜංେԽ͢Δ • Callbackͷ࣮ߦQueue͸ͪΌΜͱઃఆͨ͠ํ͕ྑ͍ɻ

    • Callbackॲཧ΋࣮ମ͸Sidekiq Workerͱͯ͠ॳظԽ͞Εɺ࣮ߦ͞ΕΔɻ • େྔʹߴ଎ͳCallback͕࣮ߦ͞ΕΔͱRedisʹෛՙ͕ҟৗʹ͔͔ͬͯࢮ…

50. ͕࣌ؒͳͯ͘Ͱ͖ͳ͔ͬͨ͜ͱ • ಈըม׵ύϥϝʔλͷPresetػೳΛ࡞Γ͔ͨͬͨ • ઃఆϑΝΠϧΛ༻ҙ͓͚ͯ͠͹ɺJobͰPresetΛࢦఆ͢Δ͚ͩͰɺ೚ҙͷύϥ ϝʔλͰ͍ΖΜͳը࣭ɾαΠζͷ੒Ռ෺Λग़ͤΔΑ͏ʹ͔ͨͬͨ͠ • TV༻ɺεϚϗ༻ɺPC༻ͱ͔… • Pipelineͷࣗಈੜ੒

    • Α͋͘ΔCIͷઃఆͷΑ͏ʹyamlʹPipelineͷॲཧΛఆ͍ٛͯͬͨ͠Βίʔυ Λࣗಈੜ੒͞ΕɺউखʹPipeline͕ग़དྷ্͕ΔΑ͏ʹ͔ͨͬͨ͠ • Pipelineͷ࣮૷͕ංେԽͯ͠΋ؾʹͳΒͳ͍

51. ޻෉ͨ͠ͱ͜Ζ 1ʢ࣌ؒ͋Ε͹ʁʣ • ʮલͷStepͷग़ྗΛɺ࣍ͷStepͷೖྗͱ͢ΔʯΛͲ͏΍ͬͯ΍Δ͔ʁ • Sidekiq BatchͷCallbackͰҾ͖ճ͍ͯ͘͜͠ͱͰղܾɻ

52. None

53. ޻෉ͨ͠ͱ͜Ζ 2ʢ࣌ؒ͋Ε͹ʁʣ • ffmpegίϚϯυͷೖग़ྗ࢓༷͕ಛघ͗͢Δ • ͻͱͭͷϑΝΠϧΛInput͠ɺͻͱͭͷϑΝΠϧΛOutput͢Δ • ෳ਺ͷϑΝΠϧΛInput͠ɺͻͱͭͷϑΝΠϧΛOutput͢Δ • ͔͠΋ೖྗͷॱং͸ॏཁ

    • CommandͷجఈΫϥεΛ࡞ͬͯந৅Խ • Pipeline → Step → Worker ΁ͱɺෳ਺ͷInputΛॱংΛอͪͳ͕Β͍͍ײ ͡ʹCommandʹ౉͢
54. None

55. ·ͱΊ • εέʔϦϯάՄೳͳಈըฤू Microservice ΛSidekiq BatchΛ࢖͍͍ͬͯײ͡ ʹ࡞ͬͨ • Sidekiq͸ͱͯ΋ྑ͍ •

    Sidekiq Pro͸΋ͬͱྑ͍ • Sidekiq BatchΛ࢖͏͜ͱͰෳࡶͳύΠϓϥΠϯॲཧΛεέʔϧͰ͖Δ • ಈըॲཧҎ֎ʹ΋ɺूܭɾղੳͱ͔ɺෳࡶͳcron scriptͱ͔… • ந৅తͳ࿩͕ଟ͘ͳͬͯ͠·ͬͨͷͰɺৄࡉ͸Sidekiq BatchΛੋඇ͓ࢼͩ͘͠ ͍͞ʢ$950/yearʣ

56. ·ͱΊ

57. None

58. THANK YOU!!