Video decode on iOS

7JEFPEFDPEFPOJ04 QPUBUPUJQT ! !OPQQFGPYXPMG

OPQQF ‣ !OPQQFGPYXPMG ‣ ! ͖ͭͶ޷͖ ‣ J04ΞϓϦ։ൃ̔೥໨ %F/"೥໨ ‣
ιʔγϟϧϥΠϒࣄۀ෦ ! !OPQQFGPYXPMG

! !OPQQFGPYXPMG

OPQQFGPYXPMG,JUTVOFCJ ಁաಈը࠶ੜϏϡʔϥΠϒϥϦ IUUQTHJUIVCDPNOPQQFGPYXPMG,JUTVOFCJ ! !OPQQFGPYXPMG

! !OPQQFGPYXPMG

,JUTVOFCJͷ࢓૊Έ ϏσΦͷσίʔυˠϚεΫ߹੒ˠඳը ! !OPQQFGPYXPMG

,JUTVOFCJͷ࢓૊Έ ϏσΦͷσίʔυˠϚεΫ߹੒ˠඳը ࠓ೔͸͜ͷลͷ࿩Λ͠·͢ ! !OPQQFGPYXPMG

ϏσΦϑΝΠϧͷલఏ஌ࣝ ྫ͑͹ɺNQNPWXNW ෳ਺ͷϝσΟΞίϯςφͷू·ΓɺओʹಈըͱԻ੠͕֨ ೲ͞Ε͍ͯΔͷ͕ҰൠతͳϑΥʔϚοτ ଞʹ΋ࣈນ΍੩ࢭըͷίϯςφؚ͕·ΕΔ͜ͱ΋͋Δ ! !OPQQFGPYXPMG

ϏσΦϑΝΠϧͷલఏ஌ࣝ ! !OPQQFGPYXPMG

ϏσΦϑΝΠϧͷσίʔυͱ͸ ϝσΟΞίϯςφΛ࠶ੜग़དྷΔঢ়ଶʹ෮ݩ͢Δ ‣ ϏσΦίϯςφ͔Β̍ϑϨʔϜ͝ͱͷը૾Λऔಘ ‣ Ի੠ίϯςφ͔ΒԻ೾৘ใΛऔಘ ‣ ࣈນίϯςφ͔ΒจࣈྻΛऔಘ FUD ͜ΕΒΛߦͬͯಈըͱͯ͠࠶ੜ͢Δ
! !OPQQFGPYXPMG

ϏσΦͷίʔσοΫ ϏσΦίϯςφͷූ߸ԽʢΤϯίʔυʣํ๏͸ෳ਺͋ Δɻ ྫ͑͹ɺII71 ίʔσοΫͷҧ͍ʹΑͬͯѹॖ཰΍ॲཧʹඞཁͳύϫʔ ͕มΘΔɻ ! !OPQQFGPYXPMG

J04ͰͷϏσΦίϯςφͷσίʔυํ๏ ‣ "7"TTFU3FBEFS ‣ 75%FDPNQSFTTJPO4FTTJPO ‣ ⒎NQFH ! !OPQQFGPYXPMG

ରԠίʔσοΫͷ֬ೝ ͜ΕΒͷϑϨʔϜϫʔΫ͸಺෦Ͱ04୺຤ͷσίʔμ ͷ࣮૷Λར༻͍ͯ͠Δɻ ಉ͡NQͰ΋)͔)Έ͍ͨͳίʔσοΫͷҧ ͍͕͋Γɺ04΍୺຤ʹΑͬͯαϙʔτ͕ҟͳΔͷͰ஫ ҙɻ ! !OPQQFGPYXPMG

ྫ͑͹)&7$͸Ұ෦ͷνοϓͰ͔͠ϋΠύϑΥʔϚϯεͳσίʔυΛ͢Δ͜ͱ͕Ͱ͖ͳ͍ɻ ! !OPQQFGPYXPMG

ରԠίʔσοΫͷ֬ೝ ެࣜαΠτͷ4QFDΈΕ͹αϙʔτ͍ͯ͠ΔίʔσοΫ ͕Θ͔Δɻ ରԠίʔσοΫΛऔಘ͢Δϝιου͸ݟ͔ͭΒͳ͔ͬ ͨɻ ! IUUQTXXXBQQMFDPNKQJQIPOFYTQFDTͱ͔ ! !OPQQFGPYXPMG

"7"TTFU3FBEFS let reader = try! AVAssetReader(asset: asset) let output =
AVAssetReaderTrackOutput(track: asset.videoTrack) reader.add(output) reader.startReading() output.copyNextSampleBuffer() output.copyNextSampleBuffer() output.copyNextSampleBuffer() ... ͱͯ΋؆୯ DPQZ/FYU4BNQMF#V⒎FSΛݺͿͨͼʹऔΕΔϑϨʔϜ ͷλΠϜελϯϓ͕ਐΉ ! !OPQQFGPYXPMG

"7"TTFU3FBEFSͷಛ௃ ‣ ରԠίʔσοΫͰ͋Ε͹ίʔσοΫΛؾʹͤͣʹσ ίʔυͰ͖Δ ‣ ඇৗʹ؆୯ͳΠϯλʔϑΣΠε ‣ ࣍ͷTBNQMF#V⒎FS͔͠औΕͳ͍ ! !OPQQFGPYXPMG

75%FDPNQSFTTJPO4FTTJPOͷಛ௃ ௿ϨΠϠʔͷϏσΦσίʔυ༻ϑϨʔϜϫʔΫ ଟ෼"7"TTFU3FBEFS"7"TTFU3FBEFS5SBDL0VUQVU͸ ͜ΕΛ࢖͍ͬͯΔʁ ΍΍ෳࡶͰ໘౗ͳ୅ΘΓʹॊೈੑ͕ߴ͍ ! !OPQQFGPYXPMG

75%FDPNQSFTTJPO4FTTJPOͷྲྀΕ IBWDΛ෼ׂ ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢ 75%FDPNQSFTTJPO4FTTJPOΛੜ੒ νϟϯΫΛ4FTTJPOʹ౉͢
σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞ αϯϓϧͰ͸ྫͱͯ͠ΩʔϑϨʔϜΛऔΓग़͍ͯ͠Δ ! !OPQQFGPYXPMG

IBWDΛ෼ׂ /"-pMFGPSNBU ! !OPQQFGPYXPMG

! !OPQQFGPYXPMG

ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢ /"-ͷϔομΛࢀরͯ͠Yͷ΋ͷ͕ΩʔϑϨʔϜΛ ͍࣋ͬͯΔɻ ࠩ෼৘ใ͕ෆཁͳ׬શͳϑϨʔϜ ! !OPQQFGPYXPMG

OBMVOJUUZQF *40*&$Ͱఆٛ IUUQXXXTUBSPDFBOTPSHFCPPL*40QEG ! !OPQQFGPYXPMG

75%FDPNQSFTTJPO4FTTJPOΛੜ੒ VTDecompressionSessionCreate(_:, _:, _:, _:, _:, _:) Ҿ਺ʹ͸ίʔϧόοΫઌ΍ϑϨʔϜͷ৘ใ QJYFM'PSNBU ΍)ͷύϥϝʔλ
͕ඞཁ ! !OPQQFGPYXPMG

νϟϯΫΛ4FTTJPOʹ౉͢ CMSampleBufferCreateReady(sampleBuffer) VTDecompressionSessionDecodeFrame(_:, _:, _:, _:, &sampleBuffer) CZUF഑ྻΛ௚઀౉͢ࣄ͸ग़དྷͳ͍ͷͰɺCZUF഑ྻ͔Β ࡞ͬͨ#MPDL#V⒎FSΛݩʹ4BNQMF#V⒎FSΛ࡞Γ౉͢ɻ !
!OPQQFGPYXPMG

σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞Ε Δ callback.decompressionOutputCallback = { (_, _, _, _, imageBuffer,
_, _) in // imageBuffer } 4FTTJPOʹొ࿥ͨ͠DBMMCBDL΁σίʔυ͞Εͨ JNBHF#V⒎FS͕ฦͬͯ͘Δ ! !OPQQFGPYXPMG

! !OPQQFGPYXPMG

1SPT$POT &BTZ 4USFBNJOH 1FSGPSNBODF $VTUPNJ[F "7"TTFU3FB EFS P Y P
˚ 75%FDPNQSF TTJPO4FTTJPO Y P P P ϦϞʔτͷΞηοτͷQJYFM#V⒎FS͸औΕͳ͍ ! !OPQQFGPYXPMG

75%FDPNQSFTTJPO4FTTJPO ‣ ετϦʔϛϯάͰ΋Ωϟϓνϟ͕औΕΔ ‣ "7"TTFU3FBEFSʹόά͕͋ͬͯ΋େৎ෉ ‣ ෼ࢄͯ͠Τϯίʔυ͍ͨ࣌͠ͱ͔ʹศར ‣ ΤϥʔνΣοΫͱ͔ͪΌΜͱ͠ͳ͍ͱ͍͚ͳ͍ !
!OPQQFGPYXPMG

)%FDPEF4BNQMF IUUQTHJUIVCDPNOPQQFGPYXPMG)%FDPEF4BNQMF ! !OPQQFGPYXPMG

OPQQFGPYXPMG,JUTVOFCJ VTDecompressionSession͸ະαϙʔτ ύϑΥʔϚϯεʹҰఆͷޮՌ͕͋Γͦ͏Ͱ͋Ε͹࣮૷༧ ఆ ! !OPQQFGPYXPMG

J04%$ͷ$G1ग़ͯ͠Έ·ͨ͠ɻ ࠓճͷ࿩ͷ΄͔ɺ,JUTVOFCJΛ.FUBMରԠͤͯ͞Ͳ͏ͳ Δͷ͔ͱ͔ൺֱͯ͠Έ͍ͨͱࢥ͍ͬͯ·͢ ! !OPQQFGPYXPMG

Video decode on iOS

Video decode on iOS

noppefoxwolf

More Decks by noppefoxwolf

Other Decks in Programming

Featured

Transcript

7JEFPEFDPEFPOJ04 QPUBUPUJQT ! !OPQQFGPYXPMG

OPQQF ‣ !OPQQFGPYXPMG ‣ ! ͖ͭͶ޷͖ ‣ J04ΞϓϦ։ൃ̔೥໨ %F/"೥໨ ‣

! !OPQQFGPYXPMG

OPQQFGPYXPMG,JUTVOFCJ ಁաಈը࠶ੜϏϡʔϥΠϒϥϦ IUUQTHJUIVCDPNOPQQFGPYXPMG,JUTVOFCJ ! !OPQQFGPYXPMG

! !OPQQFGPYXPMG

,JUTVOFCJͷ࢓૊Έ ϏσΦͷσίʔυˠϚεΫ߹੒ˠඳը ! !OPQQFGPYXPMG

,JUTVOFCJͷ࢓૊Έ ϏσΦͷσίʔυˠϚεΫ߹੒ˠඳը ࠓ೔͸͜ͷลͷ࿩Λ͠·͢ ! !OPQQFGPYXPMG

ϏσΦϑΝΠϧͷલఏ஌ࣝ ྫ͑͹ɺNQNPWXNW ෳ਺ͷϝσΟΞίϯςφͷू·ΓɺओʹಈըͱԻ੠͕֨ ೲ͞Ε͍ͯΔͷ͕ҰൠతͳϑΥʔϚοτ ଞʹ΋ࣈນ΍੩ࢭըͷίϯςφؚ͕·ΕΔ͜ͱ΋͋Δ ! !OPQQFGPYXPMG

ϏσΦϑΝΠϧͷલఏ஌ࣝ ! !OPQQFGPYXPMG

ϏσΦϑΝΠϧͷσίʔυͱ͸ ϝσΟΞίϯςφΛ࠶ੜग़དྷΔঢ়ଶʹ෮ݩ͢Δ ‣ ϏσΦίϯςφ͔Β̍ϑϨʔϜ͝ͱͷը૾Λऔಘ ‣ Ի੠ίϯςφ͔ΒԻ೾৘ใΛऔಘ ‣ ࣈນίϯςφ͔ΒจࣈྻΛऔಘ FUD ͜ΕΒΛߦͬͯಈըͱͯ͠࠶ੜ͢Δ

ϏσΦͷίʔσοΫ ϏσΦίϯςφͷූ߸ԽʢΤϯίʔυʣํ๏͸ෳ਺͋ Δɻ ྫ͑͹ɺII71 ίʔσοΫͷҧ͍ʹΑͬͯѹॖ཰΍ॲཧʹඞཁͳύϫʔ ͕มΘΔɻ ! !OPQQFGPYXPMG

J04ͰͷϏσΦίϯςφͷσίʔυํ๏ ‣ "7"TTFU3FBEFS ‣ 75%FDPNQSFTTJPO4FTTJPO ‣ ⒎NQFH ! !OPQQFGPYXPMG

ରԠίʔσοΫͷ֬ೝ ͜ΕΒͷϑϨʔϜϫʔΫ͸಺෦Ͱ04୺຤ͷσίʔμ ͷ࣮૷Λར༻͍ͯ͠Δɻ ಉ͡NQͰ΋)͔)Έ͍ͨͳίʔσοΫͷҧ ͍͕͋Γɺ04΍୺຤ʹΑͬͯαϙʔτ͕ҟͳΔͷͰ஫ ҙɻ ! !OPQQFGPYXPMG

ྫ͑͹)&7$͸Ұ෦ͷνοϓͰ͔͠ϋΠύϑΥʔϚϯεͳσίʔυΛ͢Δ͜ͱ͕Ͱ͖ͳ͍ɻ ! !OPQQFGPYXPMG

ରԠίʔσοΫͷ֬ೝ ެࣜαΠτͷ4QFDΈΕ͹αϙʔτ͍ͯ͠ΔίʔσοΫ ͕Θ͔Δɻ ରԠίʔσοΫΛऔಘ͢Δϝιου͸ݟ͔ͭΒͳ͔ͬ ͨɻ ! IUUQTXXXBQQMFDPNKQJQIPOFYTQFDTͱ͔ ! !OPQQFGPYXPMG

"7"TTFU3FBEFS let reader = try! AVAssetReader(asset: asset) let output =

"7"TTFU3FBEFSͷಛ௃ ‣ ରԠίʔσοΫͰ͋Ε͹ίʔσοΫΛؾʹͤͣʹσ ίʔυͰ͖Δ ‣ ඇৗʹ؆୯ͳΠϯλʔϑΣΠε ‣ ࣍ͷTBNQMF#V⒎FS͔͠औΕͳ͍ ! !OPQQFGPYXPMG

75%FDPNQSFTTJPO4FTTJPOͷಛ௃ ௿ϨΠϠʔͷϏσΦσίʔυ༻ϑϨʔϜϫʔΫ ଟ෼"7"TTFU3FBEFS"7"TTFU3FBEFS5SBDL0VUQVU͸ ͜ΕΛ࢖͍ͬͯΔʁ ΍΍ෳࡶͰ໘౗ͳ୅ΘΓʹॊೈੑ͕ߴ͍ ! !OPQQFGPYXPMG

75%FDPNQSFTTJPO4FTTJPOͷྲྀΕ IBWDΛ෼ׂ ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢ 75%FDPNQSFTTJPO4FTTJPOΛੜ੒ νϟϯΫΛ4FTTJPOʹ౉͢

IBWDΛ෼ׂ /"-pMFGPSNBU ! !OPQQFGPYXPMG

! !OPQQFGPYXPMG

ө૾ؚ͕·ΕΔνϟϯΫΛऔΓग़͢ /"-ͷϔομΛࢀরͯ͠Yͷ΋ͷ͕ΩʔϑϨʔϜΛ ͍࣋ͬͯΔɻ ࠩ෼৘ใ͕ෆཁͳ׬શͳϑϨʔϜ ! !OPQQFGPYXPMG

OBMVOJUUZQF 40&$Ͱఆٛ IUUQXXXTUBSPDFBOTPSHFCPPL*40QEG ! !OPQQFGPYXPMG

75%FDPNQSFTTJPO4FTTJPOΛੜ੒ VTDecompressionSessionCreate(_:, _:, _:, _:, _:, _:) Ҿ਺ʹ͸ίʔϧόοΫઌ΍ϑϨʔϜͷ৘ใ QJYFM'PSNBU ΍)ͷύϥϝʔλ

νϟϯΫΛ4FTTJPOʹ౉͢ CMSampleBufferCreateReady(sampleBuffer) VTDecompressionSessionDecodeFrame(_:, _:, _:, _:, &sampleBuffer) CZUF഑ྻΛ௚઀౉͢ࣄ͸ग़དྷͳ͍ͷͰɺCZUF഑ྻ͔Β ࡞ͬͨ#MPDL#V⒎FSΛݩʹ4BNQMF#V⒎FSΛ࡞Γ౉͢ɻ !

σίʔυࡁΈͷ*NBHF#V⒎FS͕ίʔϧόοΫʹฦ͞Ε Δ callback.decompressionOutputCallback = { (_, _, _, _, imageBuffer,

! !OPQQFGPYXPMG

1SPT$POT &BTZ 4USFBNJOH 1FSGPSNBODF $VTUPNJ[F "7"TTFU3FB EFS P Y P

75%FDPNQSFTTJPO4FTTJPO ‣ ετϦʔϛϯάͰ΋Ωϟϓνϟ͕औΕΔ ‣ "7"TTFU3FBEFSʹόά͕͋ͬͯ΋େৎ෉ ‣ ෼ࢄͯ͠Τϯίʔυ͍ͨ࣌͠ͱ͔ʹศར ‣ ΤϥʔνΣοΫͱ͔ͪΌΜͱ͠ͳ͍ͱ͍͚ͳ͍ !

)%FDPEF4BNQMF IUUQTHJUIVCDPNOPQQFGPYXPMG)%FDPEF4BNQMF ! !OPQQFGPYXPMG

OPQQFGPYXPMG,JUTVOFCJ VTDecompressionSession͸ະαϙʔτ ύϑΥʔϚϯεʹҰఆͷޮՌ͕͋Γͦ͏Ͱ͋Ε͹࣮૷༧ ఆ ! !OPQQFGPYXPMG

J04%$ͷ$G1ग़ͯ͠Έ·ͨ͠ɻ ࠓճͷ࿩ͷ΄͔ɺ,JUTVOFCJΛ.FUBMରԠͤͯ͞Ͳ͏ͳ Δͷ͔ͱ͔ൺֱͯ͠Έ͍ͨͱࢥ͍ͬͯ·͢ ! !OPQQFGPYXPMG