Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
iOSでmp4をデコードして自分だけの動画プレーヤーを作ろう!/orecon
Search
noppefoxwolf
September 12, 2018
Programming
2
10k
iOSでmp4をデコードして自分だけの動画プレーヤーを作ろう!/orecon
noppefoxwolf
September 12, 2018
Tweet
Share
More Decks by noppefoxwolf
See All by noppefoxwolf
既存アプリをvisionOS対応してリリースした話/visionOS LT vol5
noppefoxwolf
0
140
UIのブラックボックスを探る/iOSDC23
noppefoxwolf
3
3.1k
CoreGraphicsでドット絵を描こう/iOSDC22
noppefoxwolf
0
1.8k
ランタイムデバッグのススメ/iOSDC21
noppefoxwolf
1
4k
google/mediapipe で始めるARアプリ開発/iOSDC2020
noppefoxwolf
1
1.2k
モバイルファーストなアプリを作るためにvearがしたこと/xRDCC
noppefoxwolf
0
74
ソーシャルライブサービスにおけるデジタル化粧の仕組みと実装/iOSDC19
noppefoxwolf
4
5k
Limited import clarification and its effect/tryswift2019
noppefoxwolf
2
1k
立ち上げ時のライブ配信アプリに最適な開発環境・技術的ノウハウとは/PocoDevMeetup-1
noppefoxwolf
0
1.1k
Other Decks in Programming
See All in Programming
App Router への移行は「改善」となり得るのか?/ Can migration to App Router be an improvement
takefumiyoshii
8
2.1k
Micro Frontends for Java Microservices - Devnexus 2024
mraible
PRO
0
400
Doctrine ORMでValue Objectを扱う方法4選 #phpstudy / 4 ways to handle Value Objects with Doctrine ORM
77web
4
110
AWS Application Composerで始める、 サーバーレスなデータ基盤構築 / 20240406-jawsug-hokuriku-shinkansen
kasacchiful
1
240
OpenAPI を守るのは難しい
ohmori_yusuke
2
760
受託開発でGitLab CI を活用していく
xiombatsg
1
260
オブジェクト指向は必要なのか / Is object-oriented needed?
kishida
31
21k
Rubyでたのしむクリエイティブコーディング/Enjoy Creative coding with Ruby
chobishiba
1
160
データアナリストが行うDatabricksを活用したETLの自動化事例
shinoa
0
240
GitHub Actionsで泣かないためにやっておきたい設定 / Recommended GHA settings to avoid crying
pinkumohikan
3
450
Site Reliability Engineering for GMO
pyama86
4
560
1BRC--Nerd Sniping the Java Community
gunnarmorling
0
300
Featured
See All Featured
StorybookのUI Testing Handbookを読んだ
zakiyama
10
4.5k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
124
32k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
15
1.4k
Creatively Recalculating Your Daily Design Routine
revolveconf
209
11k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
39
4.4k
[RailsConf 2023] Rails as a piece of cake
palkan
21
3.9k
KATA
mclloyd
14
12k
GraphQLの誤解/rethinking-graphql
sonatard
49
9.2k
Building a Scalable Design System with Sketch
lauravandoore
455
32k
GitHub's CSS Performance
jonrohan
1023
450k
Principles of Awesome APIs and How to Build Them.
keavy
119
16k
A better future with KSS
kneath
230
16k
Transcript
J04ͰNQΛσίʔυͯࣗͩ͠ ͚ͷಈըϓϨʔϠʔΛ࡞Ζ͏ʂ Զίϯ4VNNFS%BZ$ PSFDPO@JPTD
J04%$͓ർΕ༷Ͱͨ͠ʂʂ PSFDPO@JPTD
ϥΠϒ৴ΞϓϦͷΞΠςϜ࠶ੜΛ.FUBMͰ ࣮͢Δࣄʹͳͬͨ J04%$Ͱ݄ʹొஃ͠·ͨ͠ɻ ֤ํ໘ʹ͋Γ͕ͱ͏͍͟͝·͢ʂ IUUQTHPPHM#:X%/# PSFDPO@JPTD
OPQQF ! גࣜձࣾσΟʔɾΤψɾΤʔ " ͖ͭͶ͔Θ͍͍ # ࠓੜ PSFDPO@JPTD
"(&/%" w ಈըͷ࠶ੜํ๏ͱಛ w ϏσΦͷߏཧղ w 4XJGUͰσίʔυॲཧΛॻ͘ w ·ͱΊ PSFDPO@JPTD
Ұ൪؆୯ͳํ๏ PSFDPO@JPTD
"71MBZFS$POUSPMMFS w "7,JUGSBNFXPSL w "71MBZFSΛͤదʹ࠶ੜͯ͘͠ΕΔ w J04 PSFDPO@JPTD
"71MBZFS$POUSPMMFS ͍ํ let vc = AVPlayerViewController() vc.player = AVPlayer(url: url)
present(vc, animated: true, completion: nil) PSFDPO@JPTD
"71MBZFS$POUSPMMFS ! w 04σόΠε͝ͱʹ࠷దͳσβΠϯ w "JSQMBZͳͲʹαϙʔτ w ૢ࡞6*͕ఏڙ͞ΕΔ ! w
6*ػೳͷΧελϚΠζੑͳ͍ PSFDPO@JPTD
6*ΛΧελϚΠζ͍ͨ͠ͱ͖ʁ PSFDPO@JPTD
"71MBZFS-BZFS w "7'PVOEBUJPOGSBNFXPSL w "71MBZFSΛͯ͠ૢ࡞ w J04 PSFDPO@JPTD
"71MBZFS-BZFS let player = AVPlayer(url: url) player.play() let layer =
AVPlayerLayer(player: player) layer.frame = frame view.layer.addSublayer(layer) PSFDPO@JPTD
"71MBZFS-BZFS ! w $"-BZFSͱͯ͠ѻ͑ΔͷͰॊೈ ! w ಛʹͳ͠ PSFDPO@JPTD
ಛʹͳ͠ PSFDPO@JPTD
ಛʹͳ͠ʁ PSFDPO@JPTD
"71MBZFS-BZFSͰࣄΓͳ͍࣌ w 73ΰʔάϧͰݟΕΔΑ͏ʹͯ͠ʈʈ w ө૾͔Βਓ͚ͩൈ͖ग़ͯ͠࠶ੜ͍ͨ͠ʙ w ಈըʹϦΞϧλΠϜͰΤϑΣΫτΛ͔͚͍ͨ PSFDPO@JPTD
"71MBZFS-BZFSͰࣄΓͳ͍࣌ ө૾ࣗମΛ·ͤͨΓɺϑΟϧλΛ͔͚Δʹ "71MBZFS-BZFSͰͰ͖ͳ͍ɻ ಛʹϦΞϧλΠϜͷॲཧʹͳΔ΄Ͳ͍͠ɻ PSFDPO@JPTD
PSFDPO@JPTD
"7"TTFU3FBEFS w "7"TTFU3FBEFS5SBDL0VUQVUͱΈ߹ΘͤΔͱຖ ϑϨʔϜͷ$.4BNQMF#V⒎FS͕औΕΔ PSFDPO@JPTD
"7"TTFU3FBEFS let asset = AVURLAsset(url: url) let video = asset.tracks(withMediaType:
.video).first! let reader = try! AVAssetReader(asset: asset) let output = AVAssetReaderTrackOutput(track: video) if reader.canAdd(output) { reader.add(output) } output.copyNextSampleBuffer() output.copyNextSampleBuffer() output.copyNextSampleBuffer() PSFDPO@JPTD
PSFDPO@JPTD
"7"TTFU3FBEFS ඳըઌࣗ༝ w 6**NBHF7JFX let image = CIImage(cvPixelBuffer: pb) imageView.image
= UIImage(ciimage: image) w "74BNQMF#V⒎FS%JTQMBZ-BZFS let sb = output.copyNextSampleBuffer() displayLayer.enqueue(sb) PSFDPO@JPTD
"7"TTFU3FBEFS w .FUBM CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, textureCache, imageBuffer, nil, pixelFormat, width, height,
0, &imageTexture) // ~ লུ commandBuffer.present(drawable) commandBuffer.commit() PSFDPO@JPTD
"7"TTFU3FBEFS ! w ຖϑϨʔϜͷ4BNQMFS#V⒎FS͕ ಘΒΕΔ ! w ԻผͰॲཧ PSFDPO@JPTD
͜ΕͰԿͰग़དྷͦ͏ PSFDPO@JPTD
͜ΕͰԿͰग़དྷͦ͏ʁ PSFDPO@JPTD
ྫ͑TOBQTIPUͷ PSFDPO@JPTD
TOBQTIPU ωοτϫʔΫϦιʔεͷ63-Λࢦఆͯ͠࠶ੜ͢Δͱɺ ى͜Δɻ ࠶ੜ͍ͯ͠ΔಈըϦιʔεͷεφοϓγϣοτ͕ࡱΕͳ ͍ɻ PSFDPO@JPTD
TOBQTIPU TOBQTIPU7JFX"GUFS4DSFFO6QEBU FT ਅͬ҉ "7"TTFU*NBHF(FOFSBUPS ϩʔΧϧͷΈ "7"TTFU3FBEFS PVUQVUΛଓͰ͖ͳ͍ 04ͷεΫγϣ ࡱΕΔ
PSFDPO@JPTD
٧Μͩ PSFDPO@JPTD
٧ΜͰͳ͍ʂʂʂ PSFDPO@JPTD
ࣗͰө૾Λσίʔυ͢Ε0, PSFDPO@JPTD
PSFDPO@JPTD
5*14 w ϋʔυΣΞσίʔμ ϋʔυͱͯ͠νοϓʹࡌ͞Ε͍ͯΔσίʔμ ߴʹσίʔυՄೳ w ιϑτΣΞσίʔμ ιϑτΣΞͰ࣮͞Εͨσίʔμ جຊతʹ͍ɺ$16ϦιʔεΛ৯͏ ϋʔυΣΞ͕ରԠ͍ͯ͠ͳ͍ίʔσοΫΛ࠶ੜͰ͖Δ
PSFDPO@JPTD
ιϑτΣΞσίʔμͷྫ w CSJPO0(7,JU 7171ͳͲͷιϑτΣΞσίʔμ͕࣮͞Ε͍ͯΔ w POFWDBU"1/(,JU BQOHΛιϑτΣΞσίʔμΛͬͯ$(*NBHFʹσ ίʔυ PSFDPO@JPTD
75%FDPNQSFTTJPO4FTTJPO w 7JEFP5PPMCPYGSBNFXPSL w J04 w ϋʔυΣΞσίʔμͷΠϯλʔϑΣΠε w σίʔυࡁΈͷ$71JYFM#V⒎FSΛฦ͢ PSFDPO@JPTD
.1ͷύʔε
PSFDPO@JPTD
PSFDPO@JPTD
PSFDPO@JPTD
TXJGUͰNQΛCPY͝ͱʹύʔε͢Δ *OQVU4USFBNͰ CPY4J[Fͷॱ൪ͰಡΈଓ͚Ε 0, ಛఆͷCPYͰɺࣗͷCPYʹରͯ͠࠶ݕࡧΛߦ͏ let stream = InputStream(url:
url) stream.open() while stream.hasBytesAvailable { let size = stream.readUint32() let type = stream.readAscii(4) let data = stream.read(size - 8) } IUUQTHJUIVCDPNOPQQFGPYXPMG.1#PY%VNQFSTXJGU PSFDPO@JPTD
NQͷߏ ftyp(24) moov(52997) trak(19399) trak(33461) ... mdat(8754890) ߏྫ PSFDPO@JPTD
GUZQ CZUF͝ͱʹαϙʔτ͍ͯ͠Δϒϥϯυ͕ฒͿ ઌ಄Major Brandɺͦͷ͋ͱCompatible Brand Major Brand: mp42 Compatible Brand:
isom Compatible Brand: mp42 .BKPSͱ$PNQBUJCMFͷؒʹCZUFۭ͘ IUUQXXXGUZQTDPN PSFDPO@JPTD
NEBU ಈըͱԻͷ࣮σʔλ͕֨ೲ͞Ε͍ͯΔɻ PSFDPO@JPTD
NPPW ଟ͘ͷCPYͰߏ͞ΕΔCPY NEBUͷͲͷҐஔʹͲͷσʔλ͕ೖ͍ͬͯΔ͔ͳͲɺ ͜ͷCPYͷதʹ͋ΔCPYͰఆٛ͞Ε͍ͯΔ PSFDPO@JPTD
5*14 'BJS1MBZ %3. NEBUʹ҉߸Խ͞ΕͨΦʔσΟΦσʔλ ϚελʔΩʔΛ҉߸Խͯ͠ίϯςφʹอଘ ϚελʔΩʔͷෳ߹ΩʔΞΧϯτͱ)8ࣝผࢠͰ ཧ IUUQTEFWFMPQFSBQQMFDPNTUSFBNJOHGQT PSFDPO@JPTD
.1͔Βө૾σʔλΛऔΓग़͢ NPPWʹؚ·ΕΔใΛݩʹɺNEBU͔Β୳͢ɻ ͷͰ͕͢ʜ PSFDPO@JPTD
ࠓɺࣄલʹऔΓग़ͨ͠ͷΛ༻ҙ͠·ͨ͠ʜ $ mp4box -raw 1 movie.mp4 Extracting MPEG-4 AVC-H264 stream
to h264 PSFDPO@JPTD
Iͷߏͱσίʔυ IUUQTTQFBLFSEFDLDPNOPQQFGPYXPMGWJEFPEFDPEFPOJPT PSFDPO@JPTD
IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED
ొਓ w *ϑϨʔϜ w #1ϑϨʔϜ w 414114 ͜ΕΒ͕/"-ͱݺΕΔߏʹೖͬͯฒΜͰ͍Δ PSFDPO@JPTD
*ϑϨʔϜ w Iɺ̍ຕͷը૾ͱͦͷࠩͰඵͷө૾Λදݱ ͢Δ w *%31JDUVSFͦͷݩʹͳΔ̍ຕ w ΩʔϑϨʔϜͱݴ͏ PSFDPO@JPTD
1#ϑϨʔϜ w *ϑϨʔϜ͔Βͷࠩใ w 1͕̍ຕ͔Β༧ଌͨ͠εϥΠε w #͕̎ຕ͔Β༧ଌͨ͠εϥΠε #J 1SFEJDUJWF༧ଌ
PSFDPO@JPTD
414 4FRVFODF1BSBNFUFS4FU ϓϩϑΝΠϧɺϨϕϧ 114 1JDUVSF1BSBNFUFS4FU ϐΫνϟͷූ߸Խ PSFDPO@JPTD
/"-ͷऔΓग़͠ํ
PSFDPO@JPTD
/"-6IFBEFS /"-ͷछྨ͕ॻ͔Ε͍ͯΔ YdYεϥΠε Y*ϑϨʔϜ PSFDPO@JPTD
75%FDPNQSFTT4FTTJPO ࣮ࡍ$ͷΠϯλʔϑΣΠεͳͷͰɺ4XJGUΠϯλʔ ϑΣΠεΛͬͨٙࣅίʔυͰղઆ PSFDPO@JPTD
75%FDPNQSFTT4FTTJPOͷॳظԽ let desc: CMVideoFormatDescription = ~~ let callback = {
(imageBuffer: CVImageBuffer?) in } let attrib = [.pixelFormat : .gbra] let session = VTDecompressSession(desc, params, attrib, callback) PSFDPO@JPTD
$.7JEFP'PSNBU%FTDSJQUJPO let pps: [UInt8] = ~~ let sps: [UInt8] =
~~ let desc = CMVideoFormatDescriptionCreateFromH264ParameterSets([pps, pps]) PSFDPO@JPTD
75%FDPNQSFTT4FTTJPO session.decodeFrame(sampleBuffer) PSFDPO@JPTD
σίʔμʹ͢4BNQMF#V⒎FS let packet = (IϑϨʔϜB/PϑϨʔϜɾεϥΠε) let blockBuffer = CMBlockBuffer(packet) let
sampleBuffer = CMSampleBuffer(blockBuffer) PSFDPO@JPTD
ԻपΓ "VEJP5PPMCPY PSFDPO@JPTD
"VEJP4FSWJDF2VFVF PSFDPO@JPTD
"VEJP4FSWJDF2VFVFͷॳظԽ let format: AudioStreamBasicDescription = file[kAudioFilePropertyDataFormat] let callback = {
} let audioQueue = AudioQueue(format, callback) PSFDPO@JPTD
"VEJP2VFVFͷFORVFVF let buffer = audioQueue.allocate(size) buffer.read(packet) audioQueue.enqueue(buffer) PSFDPO@JPTD
·ͱΊ PSFDPO@JPTD
75"5Ͳ͏͍͏࣌ʹ͏͖ʁ w "7,JU"7'PVOEBUJPOͰ࣮ग़དྷͳ͍࣌ͷ࠷ऴฌث w SUNQͷετϦʔϛϯάͷදࣔͱ͔ PSFDPO@JPTD
ษڧํ๏ w TIPHP)BJTIJO,JUTXJGU NQͷCPYߏ͕Ϋϥεఆٛ͞Ε͍ͯͨΓ͢ΔͷͰɺ࣮ϕʔεͰཧղ͍͢͠ w վగࡾ൛)"7$ڭՊॻ ΠϯϓϨεඪ४ڭՊॻγϦʔζ ූ߸Խͷͱߏͷ͕ຊޠͰॻ͍ͯ͋Δɻগ͠ཧղ͔ͯ͠ΒಡΉͱಡΊͦ ͏ɻʢ8*1ʣ w
͋ͱ༷ॻͱ͔ʜ PSFDPO@JPTD
Ҏ্Ͱ͢ʂʢͪΐͬͱએʣ PSFDPO@JPTD
ٕज़ॻయ̑ OPQQFMBC͑ ຊग़͠·͢ʂʢ8*1ʣ PSFDPO@JPTD
1PDPDIB ৴αʔϏεʹڵຯͷ͋ΔΤϯδχΞืूதʂ PSFDPO@JPTD
͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ ࠓޙͱΑΖ͓͘͠ئ͍͠·͢ PSFDPO@JPTD