$30 off During Our Annual Pro Sale. View Details »

Video decode on iOS

Video decode on iOS

iOS上でビデオコンテナのデコード手法をAssetReaderとVTDecompressSessionの比較を通して学びます。

noppefoxwolf

June 21, 2018
Tweet

More Decks by noppefoxwolf

Other Decks in Programming

Transcript

  1. 7JEFPEFDPEFPOJ04
    ŠQPUBUPUJQT
    !
    !OPQQFGPYXPMG

    View Slide

  2. OPQQF
    ‣ !OPQQFGPYXPMG

    !
    ͖ͭͶ޷͖
    ‣ J04ΞϓϦ։ൃ̔೥໨
    %F/"೥໨
    ‣ ιʔγϟϧϥΠϒࣄۀ෦
    !
    !OPQQFGPYXPMG

    View Slide

  3. !
    !OPQQFGPYXPMG

    View Slide

  4. OPQQFGPYXPMG,JUTVOFCJ
    ಁաಈը࠶ੜϏϡʔϥΠϒϥϦ
    IUUQTHJUIVCDPNOPQQFGPYXPMG,JUTVOFCJ
    !
    !OPQQFGPYXPMG

    View Slide

  5. !
    !OPQQFGPYXPMG

    View Slide

  6. ,JUTVOFCJͷ࢓૊Έ
    ϏσΦͷσίʔυˠϚεΫ߹੒ˠඳը
    !
    !OPQQFGPYXPMG

    View Slide

  7. ,JUTVOFCJͷ࢓૊Έ
    ϏσΦͷσίʔυˠϚεΫ߹੒ˠඳը
    ࠓ೔͸͜ͷลͷ࿩Λ͠·͢
    !
    !OPQQFGPYXPMG

    View Slide

  8. ϏσΦϑΝΠϧͷલఏ஌ࣝ
    ྫ͑͹ɺNQNPWXNW
    ෳ਺ͷϝσΟΞίϯςφͷू·ΓɺओʹಈըͱԻ੠͕֨
    ೲ͞Ε͍ͯΔͷ͕ҰൠతͳϑΥʔϚοτ
    ଞʹ΋ࣈນ΍੩ࢭըͷίϯςφؚ͕·ΕΔ͜ͱ΋͋Δ
    !
    !OPQQFGPYXPMG

    View Slide

  9. ϏσΦϑΝΠϧͷલఏ஌ࣝ
    !
    !OPQQFGPYXPMG

    View Slide

  10. ϏσΦϑΝΠϧͷσίʔυͱ͸
    ϝσΟΞίϯςφΛ࠶ੜग़དྷΔঢ়ଶʹ෮ݩ͢Δ
    ‣ ϏσΦίϯςφ͔Β̍ϑϨʔϜ͝ͱͷը૾Λऔಘ
    ‣ Ի੠ίϯςφ͔ΒԻ೾৘ใΛऔಘ
    ‣ ࣈນίϯςφ͔ΒจࣈྻΛऔಘ
    FUD
    ͜ΕΒΛߦͬͯಈըͱͯ͠࠶ੜ͢Δ
    !
    !OPQQFGPYXPMG

    View Slide

  11. ϏσΦͷίʔσοΫ
    ϏσΦίϯςφͷූ߸ԽʢΤϯίʔυʣํ๏͸ෳ਺͋
    Δɻ
    ྫ͑͹ɺII71
    ίʔσοΫͷҧ͍ʹΑͬͯѹॖ཰΍ॲཧʹඞཁͳύϫʔ
    ͕มΘΔɻ
    !
    !OPQQFGPYXPMG

    View Slide

  12. J04ͰͷϏσΦίϯςφͷσίʔυํ๏
    ‣ "7"TTFU3FBEFS
    ‣ 75%FDPNQSFTTJPO4FTTJPO
    ‣ ⒎NQFH
    !
    !OPQQFGPYXPMG

    View Slide

  13. ରԠίʔσοΫͷ֬ೝ
    ͜ΕΒͷϑϨʔϜϫʔΫ͸಺෦Ͱ04୺຤ͷσίʔμ
    ͷ࣮૷Λར༻͍ͯ͠Δɻ
    ಉ͡NQͰ΋)͔)Έ͍ͨͳίʔσοΫͷҧ
    ͍͕͋Γɺ04΍୺຤ʹΑͬͯαϙʔτ͕ҟͳΔͷͰ஫
    ҙɻ
    !
    !OPQQFGPYXPMG

    View Slide

  14. ྫ͑͹)&7$͸Ұ෦ͷνοϓͰ͔͠ϋΠύϑΥʔϚϯεͳσίʔυΛ͢Δ͜ͱ͕Ͱ͖ͳ͍ɻ
    !
    !OPQQFGPYXPMG

    View Slide

  15. ରԠίʔσοΫͷ֬ೝ
    ެࣜαΠτͷ4QFDΈΕ͹αϙʔτ͍ͯ͠ΔίʔσοΫ
    ͕Θ͔Δɻ
    ରԠίʔσοΫΛऔಘ͢Δϝιου͸ݟ͔ͭΒͳ͔ͬ
    ͨɻ
    !
    IUUQTXXXBQQMFDPNKQJQIPOFYTQFDTͱ͔
    !
    !OPQQFGPYXPMG

    View Slide

  16. "7"TTFU3FBEFS
    let reader = try! AVAssetReader(asset: asset)
    let output = AVAssetReaderTrackOutput(track: asset.videoTrack)
    reader.add(output)
    reader.startReading()
    output.copyNextSampleBuffer()
    output.copyNextSampleBuffer()
    output.copyNextSampleBuffer()
    ...
    ͱͯ΋؆୯
    DPQZ/FYU4BNQMF#V⒎FSΛݺͿͨͼʹऔΕΔϑϨʔϜ
    ͷλΠϜελϯϓ͕ਐΉ
    !
    !OPQQFGPYXPMG

    View Slide

  17. "7"TTFU3FBEFSͷಛ௃
    ‣ ରԠίʔσοΫͰ͋Ε͹ίʔσοΫΛؾʹͤͣʹσ
    ίʔυͰ͖Δ
    ‣ ඇৗʹ؆୯ͳΠϯλʔϑΣΠε
    ‣ ࣍ͷTBNQMF#V⒎FS͔͠औΕͳ͍
    !
    !OPQQFGPYXPMG

    View Slide

  18. 75%FDPNQSFTTJPO4FTTJPOͷಛ௃
    ௿ϨΠϠʔͷϏσΦσίʔυ༻ϑϨʔϜϫʔΫ
    ଟ෼"7"TTFU3FBEFS"7"TTFU3FBEFS5SBDL0VUQVU͸
    ͜ΕΛ࢖͍ͬͯΔʁ
    ΍΍ෳࡶͰ໘౗ͳ୅ΘΓʹॊೈੑ͕ߴ͍
    !
    !OPQQFGPYXPMG

    View Slide

  19. 75%FDPNQSFTTJPO4FTTJPOͷྲྀΕ
    IBWDΛ෼ׂ
    ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢
    75%FDPNQSFTTJPO4FTTJPOΛੜ੒
    νϟϯΫΛ4FTTJPOʹ౉͢
    σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞
    αϯϓϧͰ͸ྫͱͯ͠ΩʔϑϨʔϜΛऔΓग़͍ͯ͠Δ
    !
    !OPQQFGPYXPMG

    View Slide

  20. IBWDΛ෼ׂ
    /"-pMFGPSNBU
    !
    !OPQQFGPYXPMG

    View Slide

  21. !
    !OPQQFGPYXPMG

    View Slide

  22. ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢
    /"-ͷϔομΛࢀরͯ͠Yͷ΋ͷ͕ΩʔϑϨʔϜΛ
    ͍࣋ͬͯΔɻ
    ࠩ෼৘ใ͕ෆཁͳ׬શͳϑϨʔϜ
    !
    !OPQQFGPYXPMG

    View Slide

  23. OBMVOJUUZQF
    *40*&$Ͱఆٛ
    IUUQXXXTUBSPDFBOTPSHFCPPL*40QEG
    !
    !OPQQFGPYXPMG

    View Slide

  24. 75%FDPNQSFTTJPO4FTTJPOΛੜ੒
    VTDecompressionSessionCreate(_:, _:, _:, _:, _:, _:)
    Ҿ਺ʹ͸ίʔϧόοΫઌ΍ϑϨʔϜͷ৘ใ QJYFM'PSNBU
    ΍)ͷύϥϝʔλ
    ͕ඞཁ
    !
    !OPQQFGPYXPMG

    View Slide

  25. νϟϯΫΛ4FTTJPOʹ౉͢
    CMSampleBufferCreateReady(sampleBuffer)
    VTDecompressionSessionDecodeFrame(_:, _:, _:, _:, &sampleBuffer)
    CZUF഑ྻΛ௚઀౉͢ࣄ͸ग़དྷͳ͍ͷͰɺCZUF഑ྻ͔Β
    ࡞ͬͨ#MPDL#V⒎FSΛݩʹ4BNQMF#V⒎FSΛ࡞Γ౉͢ɻ
    !
    !OPQQFGPYXPMG

    View Slide

  26. σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞Ε
    Δ
    callback.decompressionOutputCallback = { (_, _, _, _, imageBuffer, _, _) in
    // imageBuffer
    }
    4FTTJPOʹొ࿥ͨ͠DBMMCBDL΁σίʔυ͞Εͨ
    JNBHF#V⒎FS͕ฦͬͯ͘Δ
    !
    !OPQQFGPYXPMG

    View Slide

  27. !
    !OPQQFGPYXPMG

    View Slide

  28. 1SPT$POT
    &BTZ 4USFBNJOH 1FSGPSNBODF $VTUPNJ[F
    "7"TTFU3FB
    EFS
    P Y P ˚
    75%FDPNQSF
    TTJPO4FTTJPO
    Y P P P
    ϦϞʔτͷΞηοτͷQJYFM#V⒎FS͸औΕͳ͍
    !
    !OPQQFGPYXPMG

    View Slide

  29. 75%FDPNQSFTTJPO4FTTJPO
    ‣ ετϦʔϛϯάͰ΋Ωϟϓνϟ͕औΕΔ
    ‣ "7"TTFU3FBEFSʹόά͕͋ͬͯ΋େৎ෉
    ‣ ෼ࢄͯ͠Τϯίʔυ͍ͨ࣌͠ͱ͔ʹศར
    ‣ ΤϥʔνΣοΫͱ͔ͪΌΜͱ͠ͳ͍ͱ͍͚ͳ͍
    !
    !OPQQFGPYXPMG

    View Slide

  30. )%FDPEF4BNQMF
    IUUQTHJUIVCDPNOPQQFGPYXPMG)%FDPEF4BNQMF
    !
    !OPQQFGPYXPMG

    View Slide

  31. OPQQFGPYXPMG,JUTVOFCJ
    VTDecompressionSession͸ະαϙʔτ
    ύϑΥʔϚϯεʹҰఆͷޮՌ͕͋Γͦ͏Ͱ͋Ε͹࣮૷༧

    !
    !OPQQFGPYXPMG

    View Slide

  32. J04%$ͷ$G1ग़ͯ͠Έ·ͨ͠ɻ
    ࠓճͷ࿩ͷ΄͔ɺ,JUTVOFCJΛ.FUBMରԠͤͯ͞Ͳ͏ͳ
    Δͷ͔ͱ͔ൺֱͯ͠Έ͍ͨͱࢥ͍ͬͯ·͢
    !
    !OPQQFGPYXPMG

    View Slide