Video decode on iOS

iOS上でビデオコンテナのデコード手法をAssetReaderとVTDecompressSessionの比較を通して学びます。

Transcript

1. 7JEFPEFDPEFPOJ04
   ŠQPUBUPUJQT
   !
   !OPQQFGPYXPMG
   

   View Slide

2. OPQQF
   ‣ !OPQQFGPYXPMG
   ‣
   !
   ͖ͭͶ޷͖
   ‣ J04ΞϓϦ։ൃ̔೥໨
   %F/"೥໨
   ‣ ιʔγϟϧϥΠϒࣄۀ෦
   !
   !OPQQFGPYXPMG
   

   View Slide

3. !
   !OPQQFGPYXPMG
   

   View Slide

4. OPQQFGPYXPMG,JUTVOFCJ
   ಁաಈը࠶ੜϏϡʔϥΠϒϥϦ
   IUUQTHJUIVCDPNOPQQFGPYXPMG,JUTVOFCJ
   !
   !OPQQFGPYXPMG
   

   View Slide

5. !
   !OPQQFGPYXPMG
   

   View Slide

6. ,JUTVOFCJͷ࢓૊Έ
   ϏσΦͷσίʔυˠϚεΫ߹੒ˠඳը
   !
   !OPQQFGPYXPMG
   

   View Slide

7. ,JUTVOFCJͷ࢓૊Έ
   ϏσΦͷσίʔυˠϚεΫ߹੒ˠඳը
   ࠓ೔͸͜ͷลͷ࿩Λ͠·͢
   !
   !OPQQFGPYXPMG
   

   View Slide

8. ϏσΦϑΝΠϧͷલఏ஌ࣝ
   ྫ͑͹ɺNQNPWXNW
   ෳ਺ͷϝσΟΞίϯςφͷू·ΓɺओʹಈըͱԻ੠͕֨
   ೲ͞Ε͍ͯΔͷ͕ҰൠతͳϑΥʔϚοτ
   ଞʹ΋ࣈນ΍੩ࢭըͷίϯςφؚ͕·ΕΔ͜ͱ΋͋Δ
   !
   !OPQQFGPYXPMG
   

   View Slide

9. ϏσΦϑΝΠϧͷલఏ஌ࣝ
   !
   !OPQQFGPYXPMG
   

   View Slide

10. ϏσΦϑΝΠϧͷσίʔυͱ͸
    ϝσΟΞίϯςφΛ࠶ੜग़དྷΔঢ়ଶʹ෮ݩ͢Δ
    ‣ ϏσΦίϯςφ͔Β̍ϑϨʔϜ͝ͱͷը૾Λऔಘ
    ‣ Ի੠ίϯςφ͔ΒԻ೾৘ใΛऔಘ
    ‣ ࣈນίϯςφ͔ΒจࣈྻΛऔಘ
    FUD
    ͜ΕΒΛߦͬͯಈըͱͯ͠࠶ੜ͢Δ
    !
    !OPQQFGPYXPMG
    

    View Slide

11. ϏσΦͷίʔσοΫ
    ϏσΦίϯςφͷූ߸ԽʢΤϯίʔυʣํ๏͸ෳ਺͋
    Δɻ
    ྫ͑͹ɺII71
    ίʔσοΫͷҧ͍ʹΑͬͯѹॖ཰΍ॲཧʹඞཁͳύϫʔ
    ͕มΘΔɻ
    !
    !OPQQFGPYXPMG
    

    View Slide

12. J04ͰͷϏσΦίϯςφͷσίʔυํ๏
    ‣ "7"TTFU3FBEFS
    ‣ 75%FDPNQSFTTJPO4FTTJPO
    ‣ ⒎NQFH
    !
    !OPQQFGPYXPMG
    

    View Slide

13. ରԠίʔσοΫͷ֬ೝ
    ͜ΕΒͷϑϨʔϜϫʔΫ͸಺෦Ͱ04୺຤ͷσίʔμ
    ͷ࣮૷Λར༻͍ͯ͠Δɻ
    ಉ͡NQͰ΋)͔)Έ͍ͨͳίʔσοΫͷҧ
    ͍͕͋Γɺ04΍୺຤ʹΑͬͯαϙʔτ͕ҟͳΔͷͰ஫
    ҙɻ
    !
    !OPQQFGPYXPMG
    

    View Slide

14. ྫ͑͹)&7$͸Ұ෦ͷνοϓͰ͔͠ϋΠύϑΥʔϚϯεͳσίʔυΛ͢Δ͜ͱ͕Ͱ͖ͳ͍ɻ
    !
    !OPQQFGPYXPMG
    

    View Slide

15. ରԠίʔσοΫͷ֬ೝ
    ެࣜαΠτͷ4QFDΈΕ͹αϙʔτ͍ͯ͠ΔίʔσοΫ
    ͕Θ͔Δɻ
    ରԠίʔσοΫΛऔಘ͢Δϝιου͸ݟ͔ͭΒͳ͔ͬ
    ͨɻ
    !
    IUUQTXXXBQQMFDPNKQJQIPOFYTQFDTͱ͔
    !
    !OPQQFGPYXPMG
    

    View Slide

16. "7"TTFU3FBEFS
    let reader = try! AVAssetReader(asset: asset)
    let output = AVAssetReaderTrackOutput(track: asset.videoTrack)
    reader.add(output)
    reader.startReading()
    output.copyNextSampleBuffer()
    output.copyNextSampleBuffer()
    output.copyNextSampleBuffer()
    ...
    ͱͯ΋؆୯
    DPQZ/FYU4BNQMF#V⒎FSΛݺͿͨͼʹऔΕΔϑϨʔϜ
    ͷλΠϜελϯϓ͕ਐΉ
    !
    !OPQQFGPYXPMG
    

    View Slide

17. "7"TTFU3FBEFSͷಛ௃
    ‣ ରԠίʔσοΫͰ͋Ε͹ίʔσοΫΛؾʹͤͣʹσ
    ίʔυͰ͖Δ
    ‣ ඇৗʹ؆୯ͳΠϯλʔϑΣΠε
    ‣ ࣍ͷTBNQMF#V⒎FS͔͠औΕͳ͍
    !
    !OPQQFGPYXPMG
    

    View Slide

18. 75%FDPNQSFTTJPO4FTTJPOͷಛ௃
    ௿ϨΠϠʔͷϏσΦσίʔυ༻ϑϨʔϜϫʔΫ
    ଟ෼"7"TTFU3FBEFS"7"TTFU3FBEFS5SBDL0VUQVU͸
    ͜ΕΛ࢖͍ͬͯΔʁ
    ΍΍ෳࡶͰ໘౗ͳ୅ΘΓʹॊೈੑ͕ߴ͍
    !
    !OPQQFGPYXPMG
    

    View Slide

19. 75%FDPNQSFTTJPO4FTTJPOͷྲྀΕ
    IBWDΛ෼ׂ
    ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢
    75%FDPNQSFTTJPO4FTTJPOΛੜ੒
    νϟϯΫΛ4FTTJPOʹ౉͢
    σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞
    αϯϓϧͰ͸ྫͱͯ͠ΩʔϑϨʔϜΛऔΓग़͍ͯ͠Δ
    !
    !OPQQFGPYXPMG
    

    View Slide

20. IBWDΛ෼ׂ
    /"-pMFGPSNBU
    !
    !OPQQFGPYXPMG
    

    View Slide

21. !
    !OPQQFGPYXPMG
    

    View Slide

22. ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢
    /"-ͷϔομΛࢀরͯ͠Yͷ΋ͷ͕ΩʔϑϨʔϜΛ
    ͍࣋ͬͯΔɻ
    ࠩ෼৘ใ͕ෆཁͳ׬શͳϑϨʔϜ
    !
    !OPQQFGPYXPMG
    

    View Slide

23. OBMVOJUUZQF
    *40*&$Ͱఆٛ
    IUUQXXXTUBSPDFBOTPSHFCPPL*40QEG
    !
    !OPQQFGPYXPMG
    

    View Slide

24. 75%FDPNQSFTTJPO4FTTJPOΛੜ੒
    VTDecompressionSessionCreate(_:, _:, _:, _:, _:, _:)
    Ҿ਺ʹ͸ίʔϧόοΫઌ΍ϑϨʔϜͷ৘ใ QJYFM'PSNBU
    ΍)ͷύϥϝʔλ
    ͕ඞཁ
    !
    !OPQQFGPYXPMG
    

    View Slide

25. νϟϯΫΛ4FTTJPOʹ౉͢
    CMSampleBufferCreateReady(sampleBuffer)
    VTDecompressionSessionDecodeFrame(_:, _:, _:, _:, &sampleBuffer)
    CZUF഑ྻΛ௚઀౉͢ࣄ͸ग़དྷͳ͍ͷͰɺCZUF഑ྻ͔Β
    ࡞ͬͨ#MPDL#V⒎FSΛݩʹ4BNQMF#V⒎FSΛ࡞Γ౉͢ɻ
    !
    !OPQQFGPYXPMG
    

    View Slide

26. σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞Ε
    Δ
    callback.decompressionOutputCallback = { (_, _, _, _, imageBuffer, _, _) in
    // imageBuffer
    }
    4FTTJPOʹొ࿥ͨ͠DBMMCBDL΁σίʔυ͞Εͨ
    JNBHF#V⒎FS͕ฦͬͯ͘Δ
    !
    !OPQQFGPYXPMG
    

    View Slide

27. !
    !OPQQFGPYXPMG
    

    View Slide

28. 1SPT$POT
    &BTZ 4USFBNJOH 1FSGPSNBODF $VTUPNJ[F
    "7"TTFU3FB
    EFS
    P Y P ˚
    75%FDPNQSF
    TTJPO4FTTJPO
    Y P P P
    ϦϞʔτͷΞηοτͷQJYFM#V⒎FS͸औΕͳ͍
    !
    !OPQQFGPYXPMG
    

    View Slide

29. 75%FDPNQSFTTJPO4FTTJPO
    ‣ ετϦʔϛϯάͰ΋Ωϟϓνϟ͕औΕΔ
    ‣ "7"TTFU3FBEFSʹόά͕͋ͬͯ΋େৎ෉
    ‣ ෼ࢄͯ͠Τϯίʔυ͍ͨ࣌͠ͱ͔ʹศར
    ‣ ΤϥʔνΣοΫͱ͔ͪΌΜͱ͠ͳ͍ͱ͍͚ͳ͍
    !
    !OPQQFGPYXPMG
    

    View Slide

30. )%FDPEF4BNQMF
    IUUQTHJUIVCDPNOPQQFGPYXPMG)%FDPEF4BNQMF
    !
    !OPQQFGPYXPMG
    

    View Slide

31. OPQQFGPYXPMG,JUTVOFCJ
    VTDecompressionSession͸ະαϙʔτ
    ύϑΥʔϚϯεʹҰఆͷޮՌ͕͋Γͦ͏Ͱ͋Ε͹࣮૷༧
    ఆ
    !
    !OPQQFGPYXPMG
    

    View Slide

32. J04%$ͷ$G1ग़ͯ͠Έ·ͨ͠ɻ
    ࠓճͷ࿩ͷ΄͔ɺ,JUTVOFCJΛ.FUBMରԠͤͯ͞Ͳ͏ͳ
    Δͷ͔ͱ͔ൺֱͯ͠Έ͍ͨͱࢥ͍ͬͯ·͢
    !
    !OPQQFGPYXPMG
    

    View Slide