Video decode on iOS

iOS上でビデオコンテナのデコード手法をAssetReaderとVTDecompressSessionの比較を通して学びます。

Transcript

1. 7JEFP
   EFDPEF
   PO
   J04 Š
   QPUBUPUJQT !
   !OPQQFGPYXPMG
    
2. OPQQF ‣
   !OPQQFGPYXPMG ‣
   !
   ͖ͭͶ޷͖ ‣
   J04ΞϓϦ։ൃ̔೥໨ 
   
   %F/"೥໨ ‣
   ιʔγϟϧϥΠϒࣄۀ෦
    !
   !OPQQFGPYXPMG
    
3. !
   !OPQQFGPYXPMG
    
4. OPQQFGPYXPMG,JUTVOFCJ ಁաಈը࠶ੜϏϡʔϥΠϒϥϦ 
   IUUQTHJUIVCDPNOPQQFGPYXPMG,JUTVOFCJ !
   !OPQQFGPYXPMG
    
5. !
   !OPQQFGPYXPMG
    
6. ,JUTVOFCJͷ࢓૊Έ ϏσΦͷσίʔυ
   ˠ
   ϚεΫ߹੒
   ˠ
   ඳը !
   !OPQQFGPYXPMG
    
7. ,JUTVOFCJͷ࢓૊Έ ϏσΦͷσίʔυ
   ˠ
   ϚεΫ߹੒
   ˠ
   ඳը ࠓ೔͸͜ͷลͷ࿩Λ͠·͢ !
   !OPQQFGPYXPMG
    
8. ϏσΦϑΝΠϧͷલఏ஌ࣝ ྫ͑͹ɺNQ
   NPW
   XNW ෳ਺ͷϝσΟΞίϯςφͷू·ΓɺओʹಈըͱԻ੠͕֨ ೲ͞Ε͍ͯΔͷ͕ҰൠతͳϑΥʔϚοτ ଞʹ΋ࣈນ΍੩ࢭըͷίϯςφؚ͕·ΕΔ͜ͱ΋͋Δ !
   !OPQQFGPYXPMG
    
9. ϏσΦϑΝΠϧͷલఏ஌ࣝ !
   !OPQQFGPYXPMG
    
10. ϏσΦϑΝΠϧͷσίʔυͱ͸ ϝσΟΞίϯςφΛ࠶ੜग़དྷΔঢ়ଶʹ෮ݩ͢Δ ‣
    ϏσΦίϯςφ͔Β̍ϑϨʔϜ͝ͱͷը૾Λऔಘ ‣
    Ի੠ίϯςφ͔ΒԻ೾৘ใΛऔಘ ‣
    ࣈນίϯςφ͔ΒจࣈྻΛऔಘ FUD ͜ΕΒΛߦͬͯಈըͱͯ͠࠶ੜ͢Δ !
    !OPQQFGPYXPMG
     
11. ϏσΦͷίʔσοΫ ϏσΦίϯςφͷූ߸ԽʢΤϯίʔυʣํ๏͸ෳ਺͋ Δɻ ྫ͑͹ɺII71 ίʔσοΫͷҧ͍ʹΑͬͯѹॖ཰΍ॲཧʹඞཁͳύϫʔ ͕มΘΔɻ !
    !OPQQFGPYXPMG
     
12. J04ͰͷϏσΦίϯςφͷσίʔυํ๏ ‣
    "7"TTFU3FBEFS ‣
    75%FDPNQSFTTJPO4FTTJPO ‣
    ⒎NQFH !
    !OPQQFGPYXPMG
     
13. ରԠίʔσοΫͷ֬ೝ ͜ΕΒͷϑϨʔϜϫʔΫ͸಺෦Ͱ04୺຤ͷσίʔμ ͷ࣮૷Λར༻͍ͯ͠Δɻ ಉ͡NQͰ΋)͔)Έ͍ͨͳίʔσοΫͷҧ ͍͕͋Γɺ04΍୺຤ʹΑͬͯαϙʔτ͕ҟͳΔͷͰ஫ ҙɻ !
    !OPQQFGPYXPMG
     
14. ྫ͑͹)&7$͸Ұ෦ͷνοϓͰ͔͠ϋΠύϑΥʔϚϯεͳσίʔυΛ͢Δ͜ͱ͕Ͱ͖ͳ͍ɻ !
    !OPQQFGPYXPMG
     
15. ରԠίʔσοΫͷ֬ೝ ެࣜαΠτͷ4QFDΈΕ͹αϙʔτ͍ͯ͠ΔίʔσοΫ ͕Θ͔Δɻ ରԠίʔσοΫΛऔಘ͢Δϝιου͸ݟ͔ͭΒͳ͔ͬ ͨɻ ! 
    IUUQTXXXBQQMFDPNKQJQIPOFYTQFDT
    ͱ͔ !
    !OPQQFGPYXPMG
     
16. "7"TTFU3FBEFS let reader = try! AVAssetReader(asset: asset) let output = AVAssetReaderTrackOutput(track: asset.videoTrack) reader.add(output) reader.startReading() output.copyNextSampleBuffer() output.copyNextSampleBuffer() output.copyNextSampleBuffer() ... ͱͯ΋؆୯ DPQZ/FYU4BNQMF#V⒎FSΛݺͿͨͼʹऔΕΔϑϨʔϜ ͷλΠϜελϯϓ͕ਐΉ !
    !OPQQFGPYXPMG
     
17. "7"TTFU3FBEFSͷಛ௃ ‣
    ରԠίʔσοΫͰ͋Ε͹ίʔσοΫΛؾʹͤͣʹσ ίʔυͰ͖Δ ‣
    ඇৗʹ؆୯ͳΠϯλʔϑΣΠε ‣
    ࣍ͷTBNQMF#V⒎FS͔͠औΕͳ͍ !
    !OPQQFGPYXPMG
     
18. 75%FDPNQSFTTJPO4FTTJPOͷಛ௃ ௿ϨΠϠʔͷϏσΦσίʔυ༻ϑϨʔϜϫʔΫ ଟ෼"7"TTFU3FBEFS"7"TTFU3FBEFS5SBDL0VUQVU͸ ͜ΕΛ࢖͍ͬͯΔʁ ΍΍ෳࡶͰ໘౗ͳ୅ΘΓʹॊೈੑ͕ߴ͍ !
    !OPQQFGPYXPMG
     
19. 75%FDPNQSFTTJPO4FTTJPOͷྲྀΕ  IBWDΛ෼ׂ  ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢  75%FDPNQSFTTJPO4FTTJPOΛੜ੒  νϟϯΫΛ4FTTJPOʹ౉͢  σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞ 
    αϯϓϧͰ͸ྫͱͯ͠ΩʔϑϨʔϜΛऔΓग़͍ͯ͠Δ !
    !OPQQFGPYXPMG
     
20. IBWDΛ෼ׂ /"-
    pMF
    GPSNBU !
    !OPQQFGPYXPMG
     
21. !
    !OPQQFGPYXPMG
     
22. ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢ /"-ͷϔομΛࢀরͯ͠Yͷ΋ͷ͕ΩʔϑϨʔϜΛ ͍࣋ͬͯΔɻ 
    ࠩ෼৘ใ͕ෆཁͳ׬શͳϑϨʔϜ !
    !OPQQFGPYXPMG
     
23. OBM
    VOJU
    UZQF *40*&$
    Ͱఆٛ 
    IUUQXXXTUBSPDFBOTPSHFCPPL*40QEG !
    !OPQQFGPYXPMG
     
24. 75%FDPNQSFTTJPO4FTTJPOΛੜ੒ VTDecompressionSessionCreate(_:, _:, _:, _:, _:, _:) Ҿ਺ʹ͸ίʔϧόοΫઌ΍ϑϨʔϜͷ৘ใ QJYFM'PSNBU ΍)ͷύϥϝʔλ ͕ඞཁ !
    !OPQQFGPYXPMG
     
25. νϟϯΫΛ4FTTJPOʹ౉͢ CMSampleBufferCreateReady(sampleBuffer) VTDecompressionSessionDecodeFrame(_:, _:, _:, _:, &sampleBuffer) CZUF഑ྻΛ௚઀౉͢ࣄ͸ग़དྷͳ͍ͷͰɺCZUF഑ྻ͔Β ࡞ͬͨ#MPDL#V⒎FSΛݩʹ4BNQMF#V⒎FSΛ࡞Γ౉͢ɻ !
    !OPQQFGPYXPMG
     
26. σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞Ε Δ callback.decompressionOutputCallback = { (_, _, _, _, imageBuffer, _, _) in // imageBuffer } 4FTTJPOʹొ࿥ͨ͠DBMMCBDL΁σίʔυ͞Εͨ JNBHF#V⒎FS͕ฦͬͯ͘Δ !
    !OPQQFGPYXPMG
     
27. !
    !OPQQFGPYXPMG
     
28. 1SPT
    $POT &BTZ 4USFBNJOH 1FSGPSNBODF $VTUPNJ[F "7"TTFU3FB EFS P Y P ˚ 75%FDPNQSF TTJPO4FTTJPO Y P P P 
    ϦϞʔτͷΞηοτͷQJYFM#V⒎FS͸औΕͳ͍ !
    !OPQQFGPYXPMG
     
29. 75%FDPNQSFTTJPO4FTTJPO ‣
    ετϦʔϛϯάͰ΋Ωϟϓνϟ͕औΕΔ ‣
    "7"TTFU3FBEFSʹόά͕͋ͬͯ΋େৎ෉ ‣
    ෼ࢄͯ͠Τϯίʔυ͍ͨ࣌͠ͱ͔ʹศར ‣
    ΤϥʔνΣοΫͱ͔ͪΌΜͱ͠ͳ͍ͱ͍͚ͳ͍ !
    !OPQQFGPYXPMG
     
30. )%FDPEF4BNQMF IUUQTHJUIVCDPNOPQQFGPYXPMG)%FDPEF4BNQMF !
    !OPQQFGPYXPMG
     
31. OPQQFGPYXPMG,JUTVOFCJ VTDecompressionSession͸ະαϙʔτ
    ύϑΥʔϚϯεʹҰఆͷޮՌ͕͋Γͦ͏Ͱ͋Ε͹࣮૷༧ ఆ !
    !OPQQFGPYXPMG
     
32. J04%$ͷ$G1ग़ͯ͠Έ·ͨ͠ɻ ࠓճͷ࿩ͷ΄͔ɺ,JUTVOFCJΛ.FUBMରԠͤͯ͞Ͳ͏ͳ Δͷ͔ͱ͔ൺֱͯ͠Έ͍ͨͱࢥ͍ͬͯ·͢
    !
    !OPQQFGPYXPMG