Upgrade to Pro — share decks privately, control downloads, hide ads and more …

iOSでmp4をデコードして自分だけの動画プレーヤーを作ろう!/orecon

3e66b3ccfa617b78c59d9dec865aa29f?s=47 noppefoxwolf
September 12, 2018

 iOSでmp4をデコードして自分だけの動画プレーヤーを作ろう!/orecon

3e66b3ccfa617b78c59d9dec865aa29f?s=128

noppefoxwolf

September 12, 2018
Tweet

Transcript

  1. J04ͰNQΛσίʔυͯࣗ͠෼ͩ ͚ͷಈըϓϨʔϠʔΛ࡞Ζ͏ʂ Զίϯ4VNNFS%BZ$ PSFDPO@JPTD 

  2. J04%$͓ർΕ༷Ͱͨ͠ʂʂ PSFDPO@JPTD 

  3. ϥΠϒ഑৴ΞϓϦͷΞΠςϜ࠶ੜΛ.FUBMͰ ࣮૷͢Δࣄʹͳͬͨ࿩ J04%$Ͱ݄೔ʹొஃ͠·ͨ͠ɻ ֤ํ໘ʹ͋Γ͕ͱ͏͍͟͝·͢ʂ IUUQTHPPHM#:X%/# PSFDPO@JPTD 

  4. OPQQF ! גࣜձࣾσΟʔɾΤψɾΤʔ " ͖ͭͶ͔Θ͍͍ # ࠓ೔஀ੜ೔ PSFDPO@JPTD 

  5. "(&/%" w ಈըͷ࠶ੜํ๏ͱಛ௃ w ϏσΦͷߏ଄ཧղ w 4XJGUͰσίʔυॲཧΛॻ͘ w ·ͱΊ PSFDPO@JPTD

    
  6. Ұ൪؆୯ͳํ๏ PSFDPO@JPTD 

  7. "71MBZFS$POUSPMMFS w "7,JUGSBNFXPSL w "71MBZFSΛ౉ͤ͹ద౰ʹ࠶ੜͯ͘͠ΕΔ w J04 PSFDPO@JPTD 

  8. "71MBZFS$POUSPMMFS ࢖͍ํ let vc = AVPlayerViewController() vc.player = AVPlayer(url: url)

    present(vc, animated: true, completion: nil) PSFDPO@JPTD 
  9. "71MBZFS$POUSPMMFS ! w 04΍σόΠε͝ͱʹ࠷దͳσβΠϯ w "JSQMBZͳͲʹ΋αϙʔτ w ૢ࡞6*͕ఏڙ͞ΕΔ ! w

    6*΍ػೳͷΧελϚΠζੑ͸ͳ͍ PSFDPO@JPTD 
  10. 6*ΛΧελϚΠζ͍ͨ͠ͱ͖͸ʁ PSFDPO@JPTD 

  11. "71MBZFS-BZFS w "7'PVOEBUJPOGSBNFXPSL w "71MBZFSΛ౉ͯ͠ૢ࡞ w J04 PSFDPO@JPTD 

  12. "71MBZFS-BZFS let player = AVPlayer(url: url) player.play() let layer =

    AVPlayerLayer(player: player) layer.frame = frame view.layer.addSublayer(layer) PSFDPO@JPTD 
  13. "71MBZFS-BZFS ! w $"-BZFSͱͯ͠ѻ͑ΔͷͰॊೈ ! w ಛʹͳ͠ PSFDPO@JPTD 

  14. ಛʹͳ͠ PSFDPO@JPTD 

  15. ಛʹͳ͠ʁ PSFDPO@JPTD 

  16. "71MBZFS-BZFSͰࣄ଍Γͳ͍࣌ w 73ΰʔάϧͰݟΕΔΑ͏ʹͯ͠ʈʈ w ө૾͔Βਓ͚ͩൈ͖ग़ͯ͠࠶ੜ͍ͨ͠ʙ w ಈըʹϦΞϧλΠϜͰΤϑΣΫτΛ͔͚͍ͨ PSFDPO@JPTD 

  17. "71MBZFS-BZFSͰࣄ଍Γͳ͍࣌ ө૾ࣗମΛ࿪·ͤͨΓɺϑΟϧλΛ͔͚Δʹ͸ "71MBZFS-BZFSͰ͸Ͱ͖ͳ͍ɻ ಛʹϦΞϧλΠϜͷॲཧʹͳΔ΄Ͳ೉͍͠ɻ PSFDPO@JPTD 

  18. PSFDPO@JPTD 

  19. "7"TTFU3FBEFS w "7"TTFU3FBEFS5SBDL0VUQVUͱ૊Έ߹ΘͤΔͱຖ ϑϨʔϜͷ$.4BNQMF#V⒎FS͕औΕΔ PSFDPO@JPTD 

  20. "7"TTFU3FBEFS let asset = AVURLAsset(url: url) let video = asset.tracks(withMediaType:

    .video).first! let reader = try! AVAssetReader(asset: asset) let output = AVAssetReaderTrackOutput(track: video) if reader.canAdd(output) { reader.add(output) } output.copyNextSampleBuffer() output.copyNextSampleBuffer() output.copyNextSampleBuffer() PSFDPO@JPTD 
  21. PSFDPO@JPTD 

  22. "7"TTFU3FBEFS ඳըઌ͸ࣗ༝ w 6**NBHF7JFX let image = CIImage(cvPixelBuffer: pb) imageView.image

    = UIImage(ciimage: image) w "74BNQMF#V⒎FS%JTQMBZ-BZFS let sb = output.copyNextSampleBuffer() displayLayer.enqueue(sb) PSFDPO@JPTD 
  23. "7"TTFU3FBEFS w .FUBM CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, textureCache, imageBuffer, nil, pixelFormat, width, height,

    0, &imageTexture) // ~ লུ commandBuffer.present(drawable) commandBuffer.commit() PSFDPO@JPTD 
  24. "7"TTFU3FBEFS ! w ຖϑϨʔϜͷ4BNQMFS#V⒎FS͕ ಘΒΕΔ ! w Ի੠͸ผͰॲཧ PSFDPO@JPTD 

  25. ͜ΕͰԿͰ΋ग़དྷͦ͏ PSFDPO@JPTD 

  26. ͜ΕͰԿͰ΋ग़དྷͦ͏ʁ PSFDPO@JPTD 

  27. ྫ͑͹TOBQTIPUͷ໰୊ PSFDPO@JPTD 

  28. TOBQTIPU໰୊ ωοτϫʔΫϦιʔεͷ63-Λࢦఆͯ͠࠶ੜ͢Δͱɺ ى͜Δ໰୊ɻ ࠶ੜ͍ͯ͠ΔಈըϦιʔεͷεφοϓγϣοτ͕ࡱΕͳ ͍ɻ PSFDPO@JPTD 

  29. TOBQTIPU໰୊ TOBQTIPU7JFX"GUFS4DSFFO6QEBU FT ਅͬ҉ "7"TTFU*NBHF(FOFSBUPS ϩʔΧϧͷΈ "7"TTFU3FBEFS PVUQVUΛ઀ଓͰ͖ͳ͍ 04ͷεΫγϣ ࡱΕΔ

    PSFDPO@JPTD 
  30. ٧Μͩ PSFDPO@JPTD 

  31. ٧ΜͰͳ͍ʂʂʂ PSFDPO@JPTD 

  32. ࣗ෼Ͱө૾Λσίʔυ͢Ε͹0, PSFDPO@JPTD 

  33. PSFDPO@JPTD 

  34. 5*14 w ϋʔυ΢ΣΞσίʔμ ϋʔυͱͯ͠νοϓʹ౥ࡌ͞Ε͍ͯΔσίʔμ ߴ଎ʹσίʔυՄೳ w ιϑτ΢ΣΞσίʔμ ιϑτ΢ΣΞͰ࣮૷͞Εͨσίʔμ جຊతʹ͸஗͍ɺ$16ϦιʔεΛ৯͏ ϋʔυ΢ΣΞ͕ରԠ͍ͯ͠ͳ͍ίʔσοΫΛ࠶ੜͰ͖Δ

    PSFDPO@JPTD 
  35. ιϑτ΢ΣΞσίʔμͷྫ w CSJPO0(7,JU 7171ͳͲͷιϑτ΢ΣΞσίʔμ͕࣮૷͞Ε͍ͯΔ w POFWDBU"1/(,JU BQOHΛιϑτ΢ΣΞσίʔμΛ࢖ͬͯ$(*NBHFʹσ ίʔυ PSFDPO@JPTD 

  36. 75%FDPNQSFTTJPO4FTTJPO w 7JEFP5PPMCPYGSBNFXPSL w J04 w ϋʔυ΢ΣΞσίʔμͷΠϯλʔϑΣΠε w σίʔυࡁΈͷ$71JYFM#V⒎FSΛฦ͢ PSFDPO@JPTD

    
  37. .1ͷύʔε 

  38. PSFDPO@JPTD 

  39. PSFDPO@JPTD 

  40. PSFDPO@JPTD 

  41. TXJGUͰNQΛCPY͝ͱʹύʔε͢Δ *OQVU4USFBNͰ  CPY4J[Fͷॱ൪ͰಡΈଓ͚Ε͹ 0, ಛఆͷCPYͰ͸ɺࣗ਎ͷCPYʹରͯ͠࠶౓ݕࡧΛߦ͏ let stream = InputStream(url:

    url) stream.open() while stream.hasBytesAvailable { let size = stream.readUint32() let type = stream.readAscii(4) let data = stream.read(size - 8) } IUUQTHJUIVCDPNOPQQFGPYXPMG.1#PY%VNQFSTXJGU PSFDPO@JPTD 
  42. NQͷߏ଄ ftyp(24) moov(52997) trak(19399) trak(33461) ... mdat(8754890) ߏ଄ྫ PSFDPO@JPTD 

  43. GUZQ CZUF͝ͱʹαϙʔτ͍ͯ͠Δϒϥϯυ͕ฒͿ ઌ಄͸Major Brandɺͦͷ͋ͱ͸Compatible Brand Major Brand: mp42 Compatible Brand:

    isom Compatible Brand: mp42 .BKPSͱ$PNQBUJCMFͷؒʹ͸CZUFۭ͘ IUUQXXXGUZQTDPN PSFDPO@JPTD 
  44. NEBU ಈըͱԻ੠ͷ࣮σʔλ͕֨ೲ͞Ε͍ͯΔɻ PSFDPO@JPTD 

  45. NPPW ଟ͘ͷCPYͰߏ੒͞ΕΔCPY NEBUͷͲͷҐஔʹͲͷσʔλ͕ೖ͍ͬͯΔ͔ͳͲ΋ɺ ͜ͷCPYͷதʹ͋ΔCPYͰఆٛ͞Ε͍ͯΔ PSFDPO@JPTD 

  46. 5*14 'BJS1MBZ %3.  NEBUʹ҉߸Խ͞ΕͨΦʔσΟΦσʔλ ϚελʔΩʔΛ҉߸Խͯ͠ίϯςφʹอଘ ϚελʔΩʔͷෳ߹Ωʔ͸ΞΧ΢ϯτͱ)8ࣝผࢠͰ؅ ཧ IUUQTEFWFMPQFSBQQMFDPNTUSFBNJOHGQT PSFDPO@JPTD

    
  47. .1͔Βө૾σʔλΛऔΓग़͢ NPPWʹؚ·ΕΔ৘ใΛݩʹɺNEBU͔Β୳͢ɻ ͷͰ͕͢ʜ PSFDPO@JPTD 

  48. ࠓ೔͸ɺࣄલʹऔΓग़ͨ͠΋ͷΛ༻ҙ͠·ͨ͠ʜ $ mp4box -raw 1 movie.mp4 Extracting MPEG-4 AVC-H264 stream

    to h264 PSFDPO@JPTD 
  49. Iͷߏ଄ͱσίʔυ IUUQTTQFBLFSEFDLDPNOPQQFGPYXPMGWJEFPEFDPEFPOJPT PSFDPO@JPTD 

  50. IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED 

  51. ొ৔ਓ෺ w *ϑϨʔϜ w #1ϑϨʔϜ w 414114 ͜ΕΒ͕/"-ͱݺ͹ΕΔߏ଄ʹೖͬͯฒΜͰ͍Δ PSFDPO@JPTD 

  52. *ϑϨʔϜ w I͸ɺ̍ຕͷը૾ͱͦͷࠩ෼Ͱ਺ඵͷө૾Λදݱ ͢Δ w *%31JDUVSF͸ͦͷݩʹͳΔ̍ຕ w ΩʔϑϨʔϜͱ΋ݴ͏ PSFDPO@JPTD 

  53. 1#ϑϨʔϜ w *ϑϨʔϜ͔Βͷࠩ෼৘ใ w 1͕̍ຕ͔Β༧ଌͨ͠εϥΠε w #͕̎ຕ͔Β༧ଌͨ͠εϥΠε  #J 1SFEJDUJWF༧ଌ

    PSFDPO@JPTD 
  54. 414 4FRVFODF1BSBNFUFS4FU ϓϩϑΝΠϧɺϨϕϧ 114 1JDUVSF1BSBNFUFS4FU ϐΫνϟͷූ߸Խ PSFDPO@JPTD 

  55. /"-ͷऔΓग़͠ํ 

  56. PSFDPO@JPTD 

  57. /"-6IFBEFS /"-ͷछྨ͕ॻ͔Ε͍ͯΔ YdYεϥΠε Y*ϑϨʔϜ PSFDPO@JPTD 

  58. 75%FDPNQSFTT4FTTJPO ࣮ࡍ͸$ͷΠϯλʔϑΣΠεͳͷͰɺ4XJGUΠϯλʔ ϑΣΠεΛ࢖ͬͨٙࣅίʔυͰղઆ PSFDPO@JPTD 

  59. 75%FDPNQSFTT4FTTJPOͷॳظԽ let desc: CMVideoFormatDescription = ~~ let callback = {

    (imageBuffer: CVImageBuffer?) in } let attrib = [.pixelFormat : .gbra] let session = VTDecompressSession(desc, params, attrib, callback) PSFDPO@JPTD 
  60. $.7JEFP'PSNBU%FTDSJQUJPO let pps: [UInt8] = ~~ let sps: [UInt8] =

    ~~ let desc = CMVideoFormatDescriptionCreateFromH264ParameterSets([pps, pps]) PSFDPO@JPTD 
  61. 75%FDPNQSFTT4FTTJPO session.decodeFrame(sampleBuffer) PSFDPO@JPTD 

  62. σίʔμʹ౉͢4BNQMF#V⒎FS let packet = (IϑϨʔϜ΍B/PϑϨʔϜɾεϥΠε) let blockBuffer = CMBlockBuffer(packet) let

    sampleBuffer = CMSampleBuffer(blockBuffer) PSFDPO@JPTD 
  63. ԻपΓ "VEJP5PPMCPY PSFDPO@JPTD 

  64. "VEJP4FSWJDF2VFVF PSFDPO@JPTD 

  65. "VEJP4FSWJDF2VFVFͷॳظԽ let format: AudioStreamBasicDescription = file[kAudioFilePropertyDataFormat] let callback = {

    } let audioQueue = AudioQueue(format, callback) PSFDPO@JPTD 
  66. "VEJP2VFVF΁ͷFORVFVF let buffer = audioQueue.allocate(size) buffer.read(packet) audioQueue.enqueue(buffer) PSFDPO@JPTD 

  67. ·ͱΊ PSFDPO@JPTD 

  68. 75"5͸Ͳ͏͍͏࣌ʹ࢖͏΂͖ʁ w "7,JU"7'PVOEBUJPOͰ࣮૷ग़དྷͳ͍࣌ͷ࠷ऴฌث w SUNQͷετϦʔϛϯάͷදࣔͱ͔ PSFDPO@JPTD 

  69. ษڧํ๏ w TIPHP)BJTIJO,JUTXJGU NQͷCPYߏ଄͕Ϋϥεఆٛ͞Ε͍ͯͨΓ͢ΔͷͰɺ࣮૷ϕʔεͰཧղ͠΍͍͢ w վగࡾ൛)"7$ڭՊॻ ΠϯϓϨεඪ४ڭՊॻγϦʔζ ූ߸Խͷ࿩ͱߏ଄ͷ࿩͕೔ຊޠͰॻ͍ͯ͋Δɻগ͠ཧղ͔ͯ͠ΒಡΉͱಡΊͦ ͏ɻʢ8*1ʣ w

    ͋ͱ࢓༷ॻͱ͔ʜ PSFDPO@JPTD 
  70. Ҏ্Ͱ͢ʂʢͪΐͬͱએ఻ʣ PSFDPO@JPTD 

  71. ٕज़ॻయ̑ OPQQFMBC͑ ຊग़͠·͢ʂʢ8*1ʣ PSFDPO@JPTD 

  72. 1PDPDIB ഑৴αʔϏεʹڵຯͷ͋ΔΤϯδχΞืूதʂ PSFDPO@JPTD 

  73. ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ ࠓޙͱ΋ΑΖ͓͘͠ئ͍͠·͢ PSFDPO@JPTD