Scalable Microservice for Media Transcoding / Transcoder

Slide 1

Slide 1 text

Scalable Microservice for Media Transcoding ञҪ ಞ @_atsushisakai גࣜձࣾϛΫγΟ Rails Developers Meetup 2019

Slide 2

Slide 2 text

ञҪ ಞ @_atsushisakai • Ո଒ΞϧόϜʮΈͯͶʯΛ։ൃ͍ͯ͠·͢ • SRE • Ruby on Rails • AWS • iOS/Android • Engineering Manager • iOSDC/DroidKaigi • RailsDM͸͸͡Ίͯ

Slide 3

Slide 3 text

Agenda ʮΈͯͶʯͰ࢖͏ɺಈըฤूɾτϥϯείʔυϚΠΫϩαʔϏεͷ͓࿩ɻ • ͳͥ࡞ͬͨͷ͔ʁ • ղܾ͍ͨ͠ཁ݅ɾ՝୊ͩͬͨͷ͔ʁ • Ͳ͏΍ͬͯղܾͨ͠ͷ͔ʁ

Slide 4

Slide 4 text

·ͣ͸ͪ͜ΒͷಈըΛ͝ཡ͍ͩ͘͞!

Slide 5

Slide 5 text

ಈը(WIP)

Slide 6

Slide 6 text

͜Ε͸Ұମ…ʁ • ʮ1ඵಈըʯͱ͍͏ࢠͲ΋ͷ੒௕ΛৼΓฦΔػೳ • େྔʹΞοϓϩʔυ͞Εͨಈը΍ࣸਅΛૉࡐͱͯ͠μΠδΣετಈըΛ࡞੒ • ൢച͢ΔDVDʹऩ࿥ͨ͠ΓɺΞϓϦʹ഑৴ͨ͠Γ͢Δ • ଟ༷ͳޮՌΛ෇͚ͯࢠҭͯͷࢥ͍ग़Λ࡞඼ʹ͢Δ • ௕͍ಈըΛ੾ͬͯɺผͷಈըͱ݁߹͢Δ • WatermarkΛ͚ͭΔʢ೔෇ͳͲͷը૾Λ͸Γ͚ͭΔʣ • ΤϑΣΫτΛ͚ͭͯܨ͛ΔʢΫϩεϑΣʔυͳͲʣ • BGMΛ͚ͭΔ

Slide 7

Slide 7 text

ಈը(WIP)

Slide 8

Slide 8 text

1ඵಈըΛࣗಈͰେྔʹ࡞Δ

Slide 9

Slide 9 text

͍͍ײ͡ʹͰ͖ΔϚωʔδυαʔϏεͳ͍Ͱ͔͢ʁ • ྫ͑͹ɺAWS ElasticTranscoder • Ͱ͖Δ͜ͱ • Watermark͸͚ͭΕΔ • ಈըͷ݁߹΋Ͱ͖Δ • Ͱ͖ͳ͍͜ͱ • ෼ׂͨ͠ΓɺΤϑΣΫτ͔͚ͨΓɺͱ͍͏͜ͱ͕Ͱ͖ͳ͍ • Ի੠ͱͷ߹੒ͱ͍͏ͷ΋Ͱ͖ͳ͍

Slide 10

Slide 10 text

࡞Δ͔͠ͳ͍…

Slide 11

Slide 11 text

Ͳ͏΍ͬͯʁ ੾Γग़͠΍ ϦαΠζͳͲͷ લॲཧ ෳ਺ͷಈըΛ ݁߹ Իָͱ߹੒ ੒Ռ෺͕׬੒ʂ

Slide 12

Slide 12 text

Ͳ͏΍ͬͯʁ ੾Γग़͠΍ ϦαΠζͳͲͷ લॲཧ ෳ਺ͷಈըΛ ݁߹ Իָͱ߹੒ ੒Ռ෺͕׬੒ʂ ͍ΘΏΔύΠϓϥΠϯॲཧ%

Slide 13

Slide 13 text

͜͜Ͱݴ͏ύΠϓϥΠϯॲཧͱ͸ • ffmpegʹΑΔม׵ॲཧΛ௚ྻʹ࿈݁ͨ͠΋ͷ • લͷॲཧͷΞ΢τϓοτ͕࣍ͷॲཧͷΠϯϓοτͱͳΔ

Slide 14

Slide 14 text

Έͳ͞ΜͳΒͲ͏࣮૷͠·͔͢❓

Slide 15

Slide 15 text

࠷΋۪௚ͳ࣮૷

Slide 16

Slide 16 text

࠷΋۪௚ͳ࣮૷ ffmpegΛॱ൪ʹ࣮ߦ͍ͯ͘͠εΫϦϓτΛͳΜΒ͔ͷݴޠͰॻ͍͍ͯ͘ • ύϑΥʔϚϯεͷ໰୊͕ग़ͯ͘Δ • ௚ྻͷॲཧΛಈը1݅ͣͭʹରͯ͠ॱ൪ʹ΍͍ͬͯͨΒ͕࣌ؒແݶʹඞཁ • ٸʹ഑৴ର৅ͷϢʔβʔ͕૿͑ͨΒͲ͏͢Δʁ • ίετͷ໰୊΋ग़ͯ͘Δ • CPU/ϝϞϦ͕༨ͬͨঢ়ଶͰ͸΋͍ͬͨͳ͍ • ޮ཰తͳϦτϥΠ͕Ͱ͖ͳ͍ • ్தͰࣦഊͨ͠Β࠷ॳ͔Β…

Slide 17

Slide 17 text

ฒྻ࣮ߦ͢ΔΑ͏ʹ࣮૷

Slide 18

Slide 18 text

ฒྻ࣮ߦ͢ΔΑ͏ʹ࣮૷ ύΠϓϥΠϯΛಉ࣌ʹෳ਺૸ΒͤΔͱɺ͖ͬͱύϑΥʔϚϯε͸ྑ͘ͳΔ • ϚϧνϓϩηεϓϩάϥϛϯάΊͬͪΌ೉ͦ͠͏ • ઈରʹόάΒͤΔࣗ৴͕͋Γ·͢ • ύϑΥʔϚϯε͸͋Δఔ౓Α͘ͳΔ͔΋͠Εͳ͍͚ͲνϡʔχϯάͰ͖ͳͦ͏ • Ҿ͖ଓ͖ϦτϥΠॲཧ΋೉ͦ͠͏

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

Sidekiq Ruby੡ͷ Job Queue γεςϜͰͱͯ΋༗໊ͳgemɻΦεεϝɻ ʮΈͯͶʯͰ͸100୆Ҏ্ͷSidekiq͕ಈ͘αʔόʔ͕ৗʹಈ͍͍ͯΔɻ • ύϑΥʔϚϯε͕ͱʹ͔͘ྑ͍ • Resque/DelayedJobͷ20ഒۙ͘ߴ଎ • https://github.com/mperham/sidekiq#performance • όʔδϣϯΞοϓʹΑΓͲΜͲΜύϑΥʔϚϯε͕ྑ͘ͳΔ • ଞͷGemґଘ͕ͱͯ΋গͳ͍ • ๛෋ͳެࣜυΩϡϝϯτɺ׆ൃͳ։ൃ • ActiveJobͰAdapter͕ఏڙ͞Ε͍ͯΔ

Slide 21

Slide 21 text

Sidekiq

Slide 22

Slide 22 text

Sidekiq Pro Sidekiqʹ՝ۚ͢Δͱྑ͍͜ͱ͕͋Γ·͢ɻ https://sidekiq.org/products/pro.html • ϝʔϧαϙʔτΛड͚ΒΕΔ • Job͕ϩετ͠ͳ͍ͨΊͷReliabilityػೳ • ෳࡶͳϫʔΫϑϩʔΛ࡞Δ͜ͱ͕Ͱ͖ΔBatchػೳ

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Sidekiq Batch • Sidekiq Pro ݶఆͷػೳ • ୯७ͳWorkerͷॲཧʹґଘؔ܎Λ࡞Δ • ෳ਺ͷ Worker ΛଋͶͯɺͦͷ Batch ͝ͱʹॲཧΛਐΊ͍ͯ͘ • ͻͱͭͷ Batch ͕ऴΘΔͱ Callback ͕࣮ߦ͞Εɺ࣍ͷॲཧʹਐΉɺͱ͍͏ ॲཧΛهड़͢Δ͜ͱ͕Ͱ͖Δ • ͨͱ͑ D ͕ࣦഊͯ͠΋ D ͚ͩϦτϥΠ ͯ͘͠ΕΔ A B C D

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

΋ͬͱෳࡶͳBatchॲཧ • A͕ऴΘΔͷΛ଴ͬͯBʙF͕ىಈ • BʙF͸ύϥϨϧʹىಈ͢Δ • BʙF͕ऴΘͬͨΒGΛىಈ͢Δ • … ͓΍ɺ͜Ε͸…ʁʁ https://github.com/mperham/sidekiq/wiki/Really-Complex-Workﬂows-with-Batches

Slide 27

Slide 27 text

Ͳ͏΍ͬͯ࡞Δʁ ੾Γग़͠΍ ϦαΠζͳͲͷ લॲཧ ෳ਺ͷಈըΛ ݁߹ Իָͱ߹੒ ੒Ռ෺͕׬੒ʂ Sidekiq BatchΛ࢖͏͜ͱͰ ໨తͷύΠϓϥΠϯ͕࣮૷Ͱ͖ͦ͏‼

Slide 28

Slide 28 text

ύΠϓϥΠϯ࣮૷Ͱ͖Δ͔΋ʁ ׂͱཧ૝ʹ͍ۙ☺ • جຊతʹ͸ͨͩͷSidekiq Workerͳͷ Ͱ͓खܰʹฒྻॲཧͰ͖Δ • ϓϩάϥϛϯά΋؆୯ • ޮ཰తͳϦτϥΠ͕ߦΘΕΔ • Sidekiqͷฒྻ౓ઃఆΛ্͛ΔͱCPU͕ ༨͍ͬͯΔݶΓߴ଎Խ͍ͯ͘͠ • CPU͕଍Γͳ͚Ε͹εέʔϧͤ͞Δ

Slide 29

Slide 29 text

Transcoder

Slide 30

Slide 30 text

ϚΠΫϩαʔϏεͱͯ͠ͷ࣮૷ • ผαʔϏε͔ΒTranscoder΁ͷϦΫΤετ͸AWS SQSΛܦ༝͢Δɻ • ඇಉظɾૄ݁߹ʹαʔϏεؒ࿈ܞ • ࣮ߦ݁Ռ΋AWS SQSʹ৐ͤͯฦ͢ • தؒϑΝΠϧ΍੒Ռ෺͸AWS S3ʹอଘ͢Δ • αʔϏεؒ͸ΦϒδΣΫτ໊ͷ΍ΓͱΓͷΈ • ࣮ϑΝΠϧ͸࢖͏ଆ͕औΓʹߦ͘

Slide 31

Slide 31 text

αʔϏεͷεέʔϥϏϦςΟ • ffmpeg͸ϞϦϞϦͱCPUΛ࢖͏ • ffmpegΛ࣮ߦ͢ΔWorker͕ͨ͘͞ΜQueueʹੵ·Ε͍ͯΔ࣌ʹίϯϐϡʔτϦ ιʔεΛΦʔτεέʔϧ͠ɺશମͷॲཧೳྗΛ্͛ΔʢAWS EC2ͷAutoScale ػೳͰ༰қʹରԠՄೳʣ

Slide 32

Slide 32 text

Transcoder γεςϜߏ੒ • શͯAWSͷαʔϏεΛར༻͠ ͨઃܭ • DBͱͷ΍ΓऔΓͷՕॴͰখن ໛ʹRails͕ಈ͘γεςϜ

Slide 33

Slide 33 text

Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 1 2 3 4 5 6

Slide 34

Slide 34 text

Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 2. SQSΛPolling͢ΔΠϯελϯ ε 1 2 3 4 5 6

Slide 35

Slide 35 text

Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 2. SQSΛPolling͢ΔΠϯελϯ ε 3. SQS͔ΒऔΓग़࣮ͨ͠ߦґཔ ΛSidekiq BatchͰॲཧ͢Δ 1 2 3 4 5 6

Slide 36

Slide 36 text

Transcoder γεςϜߏ੒ 1. ଞͷαʔϏεͱͷೖग़ྗΠϯ λʔϑΣʔε͸SQSΛ༻͍Δ 2. SQSΛPolling͢ΔΠϯελϯ ε 3. SQS͔ΒऔΓग़࣮ͨ͠ߦґཔ ΛSidekiq BatchͰॲཧ͢Δ 4. RDSʹґཔ͞Εͨ಺༰Λอ ࣋ɾࢀর͢Δ ʢActiveRecordʣ 1 2 3 4 5 6

Slide 37

Slide 37 text

Transcoder γεςϜߏ੒ 5. Sidekiq ͷ Job ΛӬଓԽ͢Δ όοΫΤϯυͱͯ͠ͷRedis 1 2 3 4 5 6

Slide 38

Slide 38 text

Transcoder γεςϜߏ੒ 5. Sidekiq ͷ Job ΛӬଓԽ͢Δ όοΫΤϯυͱͯ͠ͷRedis 6. தؒੜ੒෺ɾ࠷ऴ੒Ռ෺Λอ ଘ͢ΔS3όέοτ 1 2 3 4 5 6

Slide 39

Slide 39 text

Transcoder ͷΞʔΩςΫνϟ JobͱResult • Job͸ɺSQSʹ৐ͤΔJSONͷ͜ͱ • ܕ͸ࣄલʹఆٛ͞Ε͍ͯΔ • SQSΛPolling͠ɺTranscoder͸ඇಉظʹॲཧΛ։࢝͢Δ • Result͸ɺSQSʹ৐ͤΔॲཧ݁ՌͷJSONͷ͜ͱ • ܕ͸ࣄલʹఆٛ͞Ε͍ͯΔ • SQSΛґཔͨ͠ଆ͕Pollingͯ݁͠ՌΛड͚ͱΔ • ݁Ռ͕Errorͷ৔߹ʹ͸ErrorResultΛฦ͢

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

Transcoder ͷΞʔΩςΫνϟ Pipeline • JobͷೖྗΛड͚ɺ࠷ޙʹResultΛग़ྗ͢Δ·ͰͷશମͷॲཧΛ࣮૷ • = Sidekiq Batchͷ࣮૷ͦͷ΋ͷɻ͜Εࣗମ΋Sidekiq Worker • ಺෦ʹෳ਺ͷStepΛ࣋ͭ • Pipeline͝ͱʹ੒Ռ෺͕ఆٛ͞Ε͍ͯΔ • ʮΞϓϦ഑৴ʯ༻ͷಈըΛͭ͘ΔPipeline • ʮDVDʯ༻ͷಈըΛͭ͘ΔPipeline

Slide 44

Slide 44 text

Transcoder ͷΞʔΩςΫνϟ Step • Pipeline͕ఆٛͨ͠ෳ਺ͷWorkerͷଋͷ͜ͱΛ͜͏ݺΜͰ͍Δ • = Sidekiq Batchʹ͓͚ΔBatchͷ͜ͱ • Step͝ͱʹCallbackΛΩοΫ͠ɺ࣍ͷStepʹҠΔ

Slide 45

Slide 45 text

Transcoder ͷΞʔΩςΫνϟ Worker • Step಺ͷͻͱͭͣͭͷSidekiq Workerͷ͜ͱ • ͢΂ͯSidekiq্Ͱඇಉظɾฒྻʹ࣮ߦ͞ΕΔ • جຊ͸ffmpegͷίϚϯυΛҰճ࣮ߦ͢Δཻ౓Ͱ࣮૷͢Δ • ffmpegͷ࣮ߦ݁Ռͷग़ྗϑΝΠϧ͸S3ʹอଘ͢Δ

Slide 46

Slide 46 text

Transcoder ͷΞʔΩςΫνϟ Command • WorkerͰ࣮ߦ͢ΔffmpegίϚϯυΛڞ௨ԽͷͨΊʹϥΠϒϥϦԽͨ͠΋ͷ • ୯ʹffmpegΛ࣮ߦ͠ɺͦͷ݁ՌΛϑΝΠϧͱͯ͠ड͚औΔ͚ͩ

Slide 47

Slide 47 text

Pipeline Step Step Step Worker + Command

Slide 48

Slide 48 text

Pipeline Step Step Step Worker + Command Sidekiq BatchͰղܾ

Slide 49

Slide 49 text

Sidekiq Batch࣮૷ͷίπ • Batch಺෦ͷWorker͸ґଘ౓Λ௿͘ɺཻ͔ͭ౓Λখ࣮͘͞૷͢Δɻ • εέʔϧ͠΍͍͢ɺϦτϥΠ͠΍͍͢ɺͳͲͷ໨తɻ • ෳࡶͳBatchΛ࡞Δ৔߹ɺPipelineΛදݱ͢Δίʔυ͸ͲΜͲΜංେԽ͢Δ • Callbackͷ࣮ߦQueue͸ͪΌΜͱઃఆͨ͠ํ͕ྑ͍ɻ • Callbackॲཧ΋࣮ମ͸Sidekiq Workerͱͯ͠ॳظԽ͞Εɺ࣮ߦ͞ΕΔɻ • େྔʹߴ଎ͳCallback͕࣮ߦ͞ΕΔͱRedisʹෛՙ͕ҟৗʹ͔͔ͬͯࢮ…

Slide 50

Slide 50 text

͕࣌ؒͳͯ͘Ͱ͖ͳ͔ͬͨ͜ͱ • ಈըม׵ύϥϝʔλͷPresetػೳΛ࡞Γ͔ͨͬͨ • ઃఆϑΝΠϧΛ༻ҙ͓͚ͯ͠͹ɺJobͰPresetΛࢦఆ͢Δ͚ͩͰɺ೚ҙͷύϥ ϝʔλͰ͍ΖΜͳը࣭ɾαΠζͷ੒Ռ෺Λग़ͤΔΑ͏ʹ͔ͨͬͨ͠ • TV༻ɺεϚϗ༻ɺPC༻ͱ͔… • Pipelineͷࣗಈੜ੒ • Α͋͘ΔCIͷઃఆͷΑ͏ʹyamlʹPipelineͷॲཧΛఆ͍ٛͯͬͨ͠Βίʔυ Λࣗಈੜ੒͞ΕɺউखʹPipeline͕ग़དྷ্͕ΔΑ͏ʹ͔ͨͬͨ͠ • Pipelineͷ࣮૷͕ංେԽͯ͠΋ؾʹͳΒͳ͍

Slide 51

Slide 51 text

޻෉ͨ͠ͱ͜Ζ 1ʢ࣌ؒ͋Ε͹ʁʣ • ʮલͷStepͷग़ྗΛɺ࣍ͷStepͷೖྗͱ͢ΔʯΛͲ͏΍ͬͯ΍Δ͔ʁ • Sidekiq BatchͷCallbackͰҾ͖ճ͍ͯ͘͜͠ͱͰղܾɻ

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

޻෉ͨ͠ͱ͜Ζ 2ʢ࣌ؒ͋Ε͹ʁʣ • ffmpegίϚϯυͷೖग़ྗ࢓༷͕ಛघ͗͢Δ • ͻͱͭͷϑΝΠϧΛInput͠ɺͻͱͭͷϑΝΠϧΛOutput͢Δ • ෳ਺ͷϑΝΠϧΛInput͠ɺͻͱͭͷϑΝΠϧΛOutput͢Δ • ͔͠΋ೖྗͷॱং͸ॏཁ • CommandͷجఈΫϥεΛ࡞ͬͯந৅Խ • Pipeline → Step → Worker ΁ͱɺෳ਺ͷInputΛॱংΛอͪͳ͕Β͍͍ײ ͡ʹCommandʹ౉͢

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

·ͱΊ • εέʔϦϯάՄೳͳಈըฤू Microservice ΛSidekiq BatchΛ࢖͍͍ͬͯײ͡ ʹ࡞ͬͨ • Sidekiq͸ͱͯ΋ྑ͍ • Sidekiq Pro͸΋ͬͱྑ͍ • Sidekiq BatchΛ࢖͏͜ͱͰෳࡶͳύΠϓϥΠϯॲཧΛεέʔϧͰ͖Δ • ಈըॲཧҎ֎ʹ΋ɺूܭɾղੳͱ͔ɺෳࡶͳcron scriptͱ͔… • ந৅తͳ࿩͕ଟ͘ͳͬͯ͠·ͬͨͷͰɺৄࡉ͸Sidekiq BatchΛੋඇ͓ࢼͩ͘͠ ͍͞ʢ$950/yearʣ