iOSでmp4をデコードして自分だけの動画プレーヤーを作ろう！/orecon

by noppefoxwolf

Slide 1

Slide 1 text

J04ͰNQΛσίʔυͯࣗ͠෼ͩ ͚ͷಈըϓϨʔϠʔΛ࡞Ζ͏ʂ Զίϯ4VNNFS%BZ$ PSFDPO@JPTD

Slide 2

Slide 2 text

J04%$͓ർΕ༷Ͱͨ͠ʂʂ PSFDPO@JPTD

Slide 3

Slide 3 text

ϥΠϒ഑৴ΞϓϦͷΞΠςϜ࠶ੜΛ.FUBMͰ ࣮૷͢Δࣄʹͳͬͨ࿩ J04%$Ͱ݄೔ʹొஃ͠·ͨ͠ɻ ֤ํ໘ʹ͋Γ͕ͱ͏͍͟͝·͢ʂ IUUQTHPPHM#:X%/# PSFDPO@JPTD

Slide 4

Slide 4 text

OPQQF ! גࣜձࣾσΟʔɾΤψɾΤʔ " ͖ͭͶ͔Θ͍͍ # ࠓ೔஀ੜ೔ PSFDPO@JPTD

Slide 5

Slide 5 text

"(&/%" w ಈըͷ࠶ੜํ๏ͱಛ௃ w ϏσΦͷߏ଄ཧղ w 4XJGUͰσίʔυॲཧΛॻ͘ w ·ͱΊ PSFDPO@JPTD

Slide 6

Slide 6 text

Ұ൪؆୯ͳํ๏ PSFDPO@JPTD

Slide 7

Slide 7 text

"71MBZFS$POUSPMMFS w "7,JUGSBNFXPSL w "71MBZFSΛ౉ͤ͹ద౰ʹ࠶ੜͯ͘͠ΕΔ w J04 PSFDPO@JPTD

Slide 8

Slide 8 text

"71MBZFS$POUSPMMFS ࢖͍ํ let vc = AVPlayerViewController() vc.player = AVPlayer(url: url) present(vc, animated: true, completion: nil) PSFDPO@JPTD

Slide 9

Slide 9 text

"71MBZFS$POUSPMMFS ! w 04΍σόΠε͝ͱʹ࠷దͳσβΠϯ w "JSQMBZͳͲʹ΋αϙʔτ w ૢ࡞6*͕ఏڙ͞ΕΔ ! w 6*΍ػೳͷΧελϚΠζੑ͸ͳ͍ PSFDPO@JPTD

Slide 10

Slide 10 text

6*ΛΧελϚΠζ͍ͨ͠ͱ͖͸ʁ PSFDPO@JPTD

Slide 11

Slide 11 text

"71MBZFS-BZFS w "7'PVOEBUJPOGSBNFXPSL w "71MBZFSΛ౉ͯ͠ૢ࡞ w J04 PSFDPO@JPTD

Slide 12

Slide 12 text

"71MBZFS-BZFS let player = AVPlayer(url: url) player.play() let layer = AVPlayerLayer(player: player) layer.frame = frame view.layer.addSublayer(layer) PSFDPO@JPTD

Slide 13

Slide 13 text

"71MBZFS-BZFS ! w $"-BZFSͱͯ͠ѻ͑ΔͷͰॊೈ ! w ಛʹͳ͠ PSFDPO@JPTD

Slide 14

Slide 14 text

ಛʹͳ͠ PSFDPO@JPTD

Slide 15

Slide 15 text

ಛʹͳ͠ʁ PSFDPO@JPTD

Slide 16

Slide 16 text

"71MBZFS-BZFSͰࣄ଍Γͳ͍࣌ w 73ΰʔάϧͰݟΕΔΑ͏ʹͯ͠ʈʈ w ө૾͔Βਓ͚ͩൈ͖ग़ͯ͠࠶ੜ͍ͨ͠ʙ w ಈըʹϦΞϧλΠϜͰΤϑΣΫτΛ͔͚͍ͨ PSFDPO@JPTD

Slide 17

Slide 17 text

"71MBZFS-BZFSͰࣄ଍Γͳ͍࣌ ө૾ࣗମΛ࿪·ͤͨΓɺϑΟϧλΛ͔͚Δʹ͸ "71MBZFS-BZFSͰ͸Ͱ͖ͳ͍ɻ ಛʹϦΞϧλΠϜͷॲཧʹͳΔ΄Ͳ೉͍͠ɻ PSFDPO@JPTD

Slide 18

Slide 18 text

PSFDPO@JPTD

Slide 19

Slide 19 text

"7"TTFU3FBEFS w "7"TTFU3FBEFS5SBDL0VUQVUͱ૊Έ߹ΘͤΔͱຖ ϑϨʔϜͷ$.4BNQMF#V⒎FS͕औΕΔ PSFDPO@JPTD

Slide 20

Slide 20 text

"7"TTFU3FBEFS let asset = AVURLAsset(url: url) let video = asset.tracks(withMediaType: .video).first! let reader = try! AVAssetReader(asset: asset) let output = AVAssetReaderTrackOutput(track: video) if reader.canAdd(output) { reader.add(output) } output.copyNextSampleBuffer() output.copyNextSampleBuffer() output.copyNextSampleBuffer() PSFDPO@JPTD

Slide 21

Slide 21 text

PSFDPO@JPTD

Slide 22

Slide 22 text

"7"TTFU3FBEFS ඳըઌ͸ࣗ༝ w 6**NBHF7JFX let image = CIImage(cvPixelBuffer: pb) imageView.image = UIImage(ciimage: image) w "74BNQMF#V⒎FS%JTQMBZ-BZFS let sb = output.copyNextSampleBuffer() displayLayer.enqueue(sb) PSFDPO@JPTD

Slide 23

Slide 23 text

"7"TTFU3FBEFS w .FUBM CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, textureCache, imageBuffer, nil, pixelFormat, width, height, 0, &imageTexture) // ~ লུ commandBuffer.present(drawable) commandBuffer.commit() PSFDPO@JPTD

Slide 24

Slide 24 text

"7"TTFU3FBEFS ! w ຖϑϨʔϜͷ4BNQMFS#V⒎FS͕ ಘΒΕΔ ! w Ի੠͸ผͰॲཧ PSFDPO@JPTD

Slide 25

Slide 25 text

͜ΕͰԿͰ΋ग़དྷͦ͏ PSFDPO@JPTD

Slide 26

Slide 26 text

͜ΕͰԿͰ΋ग़དྷͦ͏ʁ PSFDPO@JPTD

Slide 27

Slide 27 text

ྫ͑͹TOBQTIPUͷ໰୊ PSFDPO@JPTD

Slide 28

Slide 28 text

TOBQTIPU໰୊ ωοτϫʔΫϦιʔεͷ63-Λࢦఆͯ͠࠶ੜ͢Δͱɺ ى͜Δ໰୊ɻ ࠶ੜ͍ͯ͠ΔಈըϦιʔεͷεφοϓγϣοτ͕ࡱΕͳ ͍ɻ PSFDPO@JPTD

Slide 29

Slide 29 text

TOBQTIPU໰୊ TOBQTIPU7JFX"GUFS4DSFFO6QEBU FT ਅͬ҉ "7"TTFU*NBHF(FOFSBUPS ϩʔΧϧͷΈ "7"TTFU3FBEFS PVUQVUΛ઀ଓͰ͖ͳ͍ 04ͷεΫγϣ ࡱΕΔ PSFDPO@JPTD

Slide 30

Slide 30 text

٧Μͩ PSFDPO@JPTD

Slide 31

Slide 31 text

٧ΜͰͳ͍ʂʂʂ PSFDPO@JPTD

Slide 32

Slide 32 text

ࣗ෼Ͱө૾Λσίʔυ͢Ε͹0, PSFDPO@JPTD

Slide 33

Slide 33 text

PSFDPO@JPTD

Slide 34

Slide 34 text

5*14 w ϋʔυ΢ΣΞσίʔμ ϋʔυͱͯ͠νοϓʹ౥ࡌ͞Ε͍ͯΔσίʔμ ߴ଎ʹσίʔυՄೳ w ιϑτ΢ΣΞσίʔμ ιϑτ΢ΣΞͰ࣮૷͞Εͨσίʔμ جຊతʹ͸஗͍ɺ$16ϦιʔεΛ৯͏ ϋʔυ΢ΣΞ͕ରԠ͍ͯ͠ͳ͍ίʔσοΫΛ࠶ੜͰ͖Δ PSFDPO@JPTD

Slide 35

Slide 35 text

ιϑτ΢ΣΞσίʔμͷྫ w CSJPO0(7,JU 7171ͳͲͷιϑτ΢ΣΞσίʔμ͕࣮૷͞Ε͍ͯΔ w POFWDBU"1/(,JU BQOHΛιϑτ΢ΣΞσίʔμΛ࢖ͬͯ$(*NBHFʹσ ίʔυ PSFDPO@JPTD

Slide 36

Slide 36 text

75%FDPNQSFTTJPO4FTTJPO w 7JEFP5PPMCPYGSBNFXPSL w J04 w ϋʔυ΢ΣΞσίʔμͷΠϯλʔϑΣΠε w σίʔυࡁΈͷ$71JYFM#V⒎FSΛฦ͢ PSFDPO@JPTD

Slide 37

Slide 37 text

.1ͷύʔε

Slide 38

Slide 38 text

PSFDPO@JPTD

Slide 39

Slide 39 text

PSFDPO@JPTD

Slide 40

Slide 40 text

PSFDPO@JPTD

Slide 41

Slide 41 text

TXJGUͰNQΛCPY͝ͱʹύʔε͢Δ *OQVU4USFBNͰ CPY4J[Fͷॱ൪ͰಡΈଓ͚Ε͹ 0, ಛఆͷCPYͰ͸ɺࣗ਎ͷCPYʹରͯ͠࠶౓ݕࡧΛߦ͏ let stream = InputStream(url: url) stream.open() while stream.hasBytesAvailable { let size = stream.readUint32() let type = stream.readAscii(4) let data = stream.read(size - 8) } IUUQTHJUIVCDPNOPQQFGPYXPMG.1#PY%VNQFSTXJGU PSFDPO@JPTD

Slide 42

Slide 42 text

NQͷߏ଄ ftyp(24) moov(52997) trak(19399) trak(33461) ... mdat(8754890) ߏ଄ྫ PSFDPO@JPTD

Slide 43

Slide 43 text

GUZQ CZUF͝ͱʹαϙʔτ͍ͯ͠Δϒϥϯυ͕ฒͿ ઌ಄͸Major Brandɺͦͷ͋ͱ͸Compatible Brand Major Brand: mp42 Compatible Brand: isom Compatible Brand: mp42 .BKPSͱ$PNQBUJCMFͷؒʹ͸CZUFۭ͘ IUUQXXXGUZQTDPN PSFDPO@JPTD

Slide 44

Slide 44 text

NEBU ಈըͱԻ੠ͷ࣮σʔλ͕֨ೲ͞Ε͍ͯΔɻ PSFDPO@JPTD

Slide 45

Slide 45 text

NPPW ଟ͘ͷCPYͰߏ੒͞ΕΔCPY NEBUͷͲͷҐஔʹͲͷσʔλ͕ೖ͍ͬͯΔ͔ͳͲ΋ɺ ͜ͷCPYͷதʹ͋ΔCPYͰఆٛ͞Ε͍ͯΔ PSFDPO@JPTD

Slide 46

Slide 46 text

5*14 'BJS1MBZ %3. NEBUʹ҉߸Խ͞ΕͨΦʔσΟΦσʔλ ϚελʔΩʔΛ҉߸Խͯ͠ίϯςφʹอଘ ϚελʔΩʔͷෳ߹Ωʔ͸ΞΧ΢ϯτͱ)8ࣝผࢠͰ؅ ཧ IUUQTEFWFMPQFSBQQMFDPNTUSFBNJOHGQT PSFDPO@JPTD

Slide 47

Slide 47 text

.1͔Βө૾σʔλΛऔΓग़͢ NPPWʹؚ·ΕΔ৘ใΛݩʹɺNEBU͔Β୳͢ɻ ͷͰ͕͢ʜ PSFDPO@JPTD

Slide 48

Slide 48 text

ࠓ೔͸ɺࣄલʹऔΓग़ͨ͠΋ͷΛ༻ҙ͠·ͨ͠ʜ $ mp4box -raw 1 movie.mp4 Extracting MPEG-4 AVC-H264 stream to h264 PSFDPO@JPTD

Slide 49

Slide 49 text

Iͷߏ଄ͱσίʔυ IUUQTTQFBLFSEFDLDPNOPQQFGPYXPMGWJEFPEFDPEFPOJPT PSFDPO@JPTD

Slide 50

Slide 50 text

IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED

Slide 51

Slide 51 text

ొ৔ਓ෺ w *ϑϨʔϜ w #1ϑϨʔϜ w 414114 ͜ΕΒ͕/"-ͱݺ͹ΕΔߏ଄ʹೖͬͯฒΜͰ͍Δ PSFDPO@JPTD

Slide 52

Slide 52 text

*ϑϨʔϜ w I͸ɺ̍ຕͷը૾ͱͦͷࠩ෼Ͱ਺ඵͷө૾Λදݱ ͢Δ w *%31JDUVSF͸ͦͷݩʹͳΔ̍ຕ w ΩʔϑϨʔϜͱ΋ݴ͏ PSFDPO@JPTD

Slide 53

Slide 53 text

1#ϑϨʔϜ w *ϑϨʔϜ͔Βͷࠩ෼৘ใ w 1͕̍ຕ͔Β༧ଌͨ͠εϥΠε w #͕̎ຕ͔Β༧ଌͨ͠εϥΠε #J 1SFEJDUJWF༧ଌ PSFDPO@JPTD

Slide 54

Slide 54 text

414 4FRVFODF1BSBNFUFS4FU ϓϩϑΝΠϧɺϨϕϧ 114 1JDUVSF1BSBNFUFS4FU ϐΫνϟͷූ߸Խ PSFDPO@JPTD

Slide 55

Slide 55 text

/"-ͷऔΓग़͠ํ

Slide 56

Slide 56 text

PSFDPO@JPTD

Slide 57

Slide 57 text

/"-6IFBEFS /"-ͷछྨ͕ॻ͔Ε͍ͯΔ YdYεϥΠε Y*ϑϨʔϜ PSFDPO@JPTD

Slide 58

Slide 58 text

75%FDPNQSFTT4FTTJPO ࣮ࡍ͸$ͷΠϯλʔϑΣΠεͳͷͰɺ4XJGUΠϯλʔ ϑΣΠεΛ࢖ͬͨٙࣅίʔυͰղઆ PSFDPO@JPTD

Slide 59

Slide 59 text

75%FDPNQSFTT4FTTJPOͷॳظԽ let desc: CMVideoFormatDescription = ~~ let callback = { (imageBuffer: CVImageBuffer?) in } let attrib = [.pixelFormat : .gbra] let session = VTDecompressSession(desc, params, attrib, callback) PSFDPO@JPTD

Slide 60

Slide 60 text

$.7JEFP'PSNBU%FTDSJQUJPO let pps: [UInt8] = ~~ let sps: [UInt8] = ~~ let desc = CMVideoFormatDescriptionCreateFromH264ParameterSets([pps, pps]) PSFDPO@JPTD

Slide 61

Slide 61 text

75%FDPNQSFTT4FTTJPO session.decodeFrame(sampleBuffer) PSFDPO@JPTD

Slide 62

Slide 62 text

σίʔμʹ౉͢4BNQMF#V⒎FS let packet = (IϑϨʔϜ΍B/PϑϨʔϜɾεϥΠε) let blockBuffer = CMBlockBuffer(packet) let sampleBuffer = CMSampleBuffer(blockBuffer) PSFDPO@JPTD

Slide 63

Slide 63 text

ԻपΓ "VEJP5PPMCPY PSFDPO@JPTD

Slide 64

Slide 64 text

"VEJP4FSWJDF2VFVF PSFDPO@JPTD

Slide 65

Slide 65 text

"VEJP4FSWJDF2VFVFͷॳظԽ let format: AudioStreamBasicDescription = file[kAudioFilePropertyDataFormat] let callback = { } let audioQueue = AudioQueue(format, callback) PSFDPO@JPTD

Slide 66

Slide 66 text

"VEJP2VFVF΁ͷFORVFVF let buffer = audioQueue.allocate(size) buffer.read(packet) audioQueue.enqueue(buffer) PSFDPO@JPTD

Slide 67

Slide 67 text

·ͱΊ PSFDPO@JPTD

Slide 68

Slide 68 text

75"5͸Ͳ͏͍͏࣌ʹ࢖͏΂͖ʁ w "7,JU"7'PVOEBUJPOͰ࣮૷ग़དྷͳ͍࣌ͷ࠷ऴฌث w SUNQͷετϦʔϛϯάͷදࣔͱ͔ PSFDPO@JPTD

Slide 69

Slide 69 text

ษڧํ๏ w TIPHP)BJTIJO,JUTXJGU NQͷCPYߏ଄͕Ϋϥεఆٛ͞Ε͍ͯͨΓ͢ΔͷͰɺ࣮૷ϕʔεͰཧղ͠΍͍͢ w վగࡾ൛)"7$ڭՊॻ ΠϯϓϨεඪ४ڭՊॻγϦʔζ ූ߸Խͷ࿩ͱߏ଄ͷ࿩͕೔ຊޠͰॻ͍ͯ͋Δɻগ͠ཧղ͔ͯ͠ΒಡΉͱಡΊͦ ͏ɻʢ8*1ʣ w ͋ͱ࢓༷ॻͱ͔ʜ PSFDPO@JPTD