Video decode on iOS

Video decode on iOS

iOS上でビデオコンテナのデコード手法をAssetReaderとVTDecompressSessionの比較を通して学びます。

3e66b3ccfa617b78c59d9dec865aa29f?s=128

noppefoxwolf

June 21, 2018
Tweet

Transcript

  1. 7JEFPEFDPEFPOJ04 ŠQPUBUPUJQT ! !OPQQFGPYXPMG  

  2. OPQQF ‣ !OPQQFGPYXPMG ‣ ! ͖ͭͶ޷͖ ‣ J04ΞϓϦ։ൃ̔೥໨ %F/"೥໨ ‣

    ιʔγϟϧϥΠϒࣄۀ෦ ! !OPQQFGPYXPMG  
  3. ! !OPQQFGPYXPMG  

  4. OPQQFGPYXPMG,JUTVOFCJ ಁաಈը࠶ੜϏϡʔϥΠϒϥϦ IUUQTHJUIVCDPNOPQQFGPYXPMG,JUTVOFCJ ! !OPQQFGPYXPMG  

  5. ! !OPQQFGPYXPMG  

  6. ,JUTVOFCJͷ࢓૊Έ ϏσΦͷσίʔυˠϚεΫ߹੒ˠඳը ! !OPQQFGPYXPMG  

  7. ,JUTVOFCJͷ࢓૊Έ ϏσΦͷσίʔυˠϚεΫ߹੒ˠඳը ࠓ೔͸͜ͷลͷ࿩Λ͠·͢ ! !OPQQFGPYXPMG  

  8. ϏσΦϑΝΠϧͷલఏ஌ࣝ ྫ͑͹ɺNQNPWXNW ෳ਺ͷϝσΟΞίϯςφͷू·ΓɺओʹಈըͱԻ੠͕֨ ೲ͞Ε͍ͯΔͷ͕ҰൠతͳϑΥʔϚοτ ଞʹ΋ࣈນ΍੩ࢭըͷίϯςφؚ͕·ΕΔ͜ͱ΋͋Δ ! !OPQQFGPYXPMG  

  9. ϏσΦϑΝΠϧͷલఏ஌ࣝ ! !OPQQFGPYXPMG  

  10. ϏσΦϑΝΠϧͷσίʔυͱ͸ ϝσΟΞίϯςφΛ࠶ੜग़དྷΔঢ়ଶʹ෮ݩ͢Δ ‣ ϏσΦίϯςφ͔Β̍ϑϨʔϜ͝ͱͷը૾Λऔಘ ‣ Ի੠ίϯςφ͔ΒԻ೾৘ใΛऔಘ ‣ ࣈນίϯςφ͔ΒจࣈྻΛऔಘ FUD ͜ΕΒΛߦͬͯಈըͱͯ͠࠶ੜ͢Δ

    ! !OPQQFGPYXPMG  
  11. ϏσΦͷίʔσοΫ ϏσΦίϯςφͷූ߸ԽʢΤϯίʔυʣํ๏͸ෳ਺͋ Δɻ ྫ͑͹ɺII71 ίʔσοΫͷҧ͍ʹΑͬͯѹॖ཰΍ॲཧʹඞཁͳύϫʔ ͕มΘΔɻ ! !OPQQFGPYXPMG  

  12. J04ͰͷϏσΦίϯςφͷσίʔυํ๏ ‣ "7"TTFU3FBEFS ‣ 75%FDPNQSFTTJPO4FTTJPO ‣ ⒎NQFH ! !OPQQFGPYXPMG 

    
  13. ରԠίʔσοΫͷ֬ೝ ͜ΕΒͷϑϨʔϜϫʔΫ͸಺෦Ͱ04୺຤ͷσίʔμ ͷ࣮૷Λར༻͍ͯ͠Δɻ ಉ͡NQͰ΋)͔)Έ͍ͨͳίʔσοΫͷҧ ͍͕͋Γɺ04΍୺຤ʹΑͬͯαϙʔτ͕ҟͳΔͷͰ஫ ҙɻ ! !OPQQFGPYXPMG  

  14. ྫ͑͹)&7$͸Ұ෦ͷνοϓͰ͔͠ϋΠύϑΥʔϚϯεͳσίʔυΛ͢Δ͜ͱ͕Ͱ͖ͳ͍ɻ ! !OPQQFGPYXPMG  

  15. ରԠίʔσοΫͷ֬ೝ ެࣜαΠτͷ4QFDΈΕ͹αϙʔτ͍ͯ͠ΔίʔσοΫ ͕Θ͔Δɻ ରԠίʔσοΫΛऔಘ͢Δϝιου͸ݟ͔ͭΒͳ͔ͬ ͨɻ ! IUUQTXXXBQQMFDPNKQJQIPOFYTQFDTͱ͔ ! !OPQQFGPYXPMG 

    
  16. "7"TTFU3FBEFS let reader = try! AVAssetReader(asset: asset) let output =

    AVAssetReaderTrackOutput(track: asset.videoTrack) reader.add(output) reader.startReading() output.copyNextSampleBuffer() output.copyNextSampleBuffer() output.copyNextSampleBuffer() ... ͱͯ΋؆୯ DPQZ/FYU4BNQMF#V⒎FSΛݺͿͨͼʹऔΕΔϑϨʔϜ ͷλΠϜελϯϓ͕ਐΉ ! !OPQQFGPYXPMG  
  17. "7"TTFU3FBEFSͷಛ௃ ‣ ରԠίʔσοΫͰ͋Ε͹ίʔσοΫΛؾʹͤͣʹσ ίʔυͰ͖Δ ‣ ඇৗʹ؆୯ͳΠϯλʔϑΣΠε ‣ ࣍ͷTBNQMF#V⒎FS͔͠औΕͳ͍ ! !OPQQFGPYXPMG

     
  18. 75%FDPNQSFTTJPO4FTTJPOͷಛ௃ ௿ϨΠϠʔͷϏσΦσίʔυ༻ϑϨʔϜϫʔΫ ଟ෼"7"TTFU3FBEFS"7"TTFU3FBEFS5SBDL0VUQVU͸ ͜ΕΛ࢖͍ͬͯΔʁ ΍΍ෳࡶͰ໘౗ͳ୅ΘΓʹॊೈੑ͕ߴ͍ ! !OPQQFGPYXPMG  

  19. 75%FDPNQSFTTJPO4FTTJPOͷྲྀΕ  IBWDΛ෼ׂ  ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢  75%FDPNQSFTTJPO4FTTJPOΛੜ੒  νϟϯΫΛ4FTTJPOʹ౉͢ 

    σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞ αϯϓϧͰ͸ྫͱͯ͠ΩʔϑϨʔϜΛऔΓग़͍ͯ͠Δ ! !OPQQFGPYXPMG  
  20. IBWDΛ෼ׂ /"-pMFGPSNBU ! !OPQQFGPYXPMG  

  21. ! !OPQQFGPYXPMG  

  22. ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢ /"-ͷϔομΛࢀরͯ͠Yͷ΋ͷ͕ΩʔϑϨʔϜΛ ͍࣋ͬͯΔɻ ࠩ෼৘ใ͕ෆཁͳ׬શͳϑϨʔϜ ! !OPQQFGPYXPMG  

  23. OBMVOJUUZQF *40*&$Ͱఆٛ IUUQXXXTUBSPDFBOTPSHFCPPL*40QEG ! !OPQQFGPYXPMG  

  24. 75%FDPNQSFTTJPO4FTTJPOΛੜ੒ VTDecompressionSessionCreate(_:, _:, _:, _:, _:, _:) Ҿ਺ʹ͸ίʔϧόοΫઌ΍ϑϨʔϜͷ৘ใ QJYFM'PSNBU ΍)ͷύϥϝʔλ

    ͕ඞཁ ! !OPQQFGPYXPMG  
  25. νϟϯΫΛ4FTTJPOʹ౉͢ CMSampleBufferCreateReady(sampleBuffer) VTDecompressionSessionDecodeFrame(_:, _:, _:, _:, &sampleBuffer) CZUF഑ྻΛ௚઀౉͢ࣄ͸ग़དྷͳ͍ͷͰɺCZUF഑ྻ͔Β ࡞ͬͨ#MPDL#V⒎FSΛݩʹ4BNQMF#V⒎FSΛ࡞Γ౉͢ɻ !

    !OPQQFGPYXPMG  
  26. σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞Ε Δ callback.decompressionOutputCallback = { (_, _, _, _, imageBuffer,

    _, _) in // imageBuffer } 4FTTJPOʹొ࿥ͨ͠DBMMCBDL΁σίʔυ͞Εͨ JNBHF#V⒎FS͕ฦͬͯ͘Δ ! !OPQQFGPYXPMG  
  27. ! !OPQQFGPYXPMG  

  28. 1SPT$POT &BTZ 4USFBNJOH 1FSGPSNBODF $VTUPNJ[F "7"TTFU3FB EFS P Y P

    ˚ 75%FDPNQSF TTJPO4FTTJPO Y P P P ϦϞʔτͷΞηοτͷQJYFM#V⒎FS͸औΕͳ͍ ! !OPQQFGPYXPMG  
  29. 75%FDPNQSFTTJPO4FTTJPO ‣ ετϦʔϛϯάͰ΋Ωϟϓνϟ͕औΕΔ ‣ "7"TTFU3FBEFSʹόά͕͋ͬͯ΋େৎ෉ ‣ ෼ࢄͯ͠Τϯίʔυ͍ͨ࣌͠ͱ͔ʹศར ‣ ΤϥʔνΣοΫͱ͔ͪΌΜͱ͠ͳ͍ͱ͍͚ͳ͍ !

    !OPQQFGPYXPMG  
  30. )%FDPEF4BNQMF IUUQTHJUIVCDPNOPQQFGPYXPMG)%FDPEF4BNQMF ! !OPQQFGPYXPMG  

  31. OPQQFGPYXPMG,JUTVOFCJ VTDecompressionSession͸ະαϙʔτ ύϑΥʔϚϯεʹҰఆͷޮՌ͕͋Γͦ͏Ͱ͋Ε͹࣮૷༧ ఆ ! !OPQQFGPYXPMG  

  32. J04%$ͷ$G1ग़ͯ͠Έ·ͨ͠ɻ ࠓճͷ࿩ͷ΄͔ɺ,JUTVOFCJΛ.FUBMରԠͤͯ͞Ͳ͏ͳ Δͷ͔ͱ͔ൺֱͯ͠Έ͍ͨͱࢥ͍ͬͯ·͢ ! !OPQQFGPYXPMG