Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MediaCodecで動画編集をしてみよう

Masayuki Suda
February 08, 2018

 MediaCodecで動画編集をしてみよう

DroidKaigi 2018でプレゼンさせていただいたスライドです。

Masayuki Suda

February 08, 2018
Tweet

More Decks by Masayuki Suda

Other Decks in Programming

Transcript

  1. // ΢ΥʔλʔϚʔΫ new Mp4Composer(srcMp4Path, destMp4Path) .filter( new GlWatermarkFilter(watermarkBitmap, Position.RIGHT_BOTTOM) )

    .start(); // ηϐΞϑΟϧλʔ new Mp4Composer(srcMp4Path, destMp4Path) .filter(new GlSepiaFilter()) .start(); .Q$PNQPTFSBOESPJE IUUQTHJUIVCDPN.BTBZVLJ4VEB.Q$PNQPTFSBOESPJE
  2. .FEJB&YUSBDUPSͷ େࡶ೺ͳίʔυͷྲྀΕ // Πϯελϯεੜ੒ mediaExtractor = new MediaExtractor(); // ಈըฤूݩͷϑΝΠϧΛFileDescriptorܗࣜͰηοτ͢Δɻ

    mediaExtractor.setDataSource(inputFileDescriptor); // Decoder(MediaCodec)ͷ࡞੒ MediaFormat inputFormat = mediaExtractor.getTrackFormat(trackIndex); decoder = MediaCodec.createDecoderByType(inputFormat.getString(MediaFormat.KEY_MIME)); // Decoderʹ౉͢ByteBufferΛ࡞੒ decoderInputBuffers = decoder.getInputBuffers(); // sample (MediaCodecͰѻ͑Δ1୯Ґ)ͷαΠζΛऔಘ int result = decoder.dequeueInputBuffer(0); int sampleSize = mediaExtractor.readSampleData(decoderInputBuffers[result], 0); // DecoderʹDataΛ౉͢ boolean isKeyFrame = (mediaExtractor.getSampleFlags() & MediaExtractor.SAMPLE_FLAG_SYNC) != 0; decoder.queueInputBuffer(result, 0, sampleSize, mediaExtractor.getSampleTime(), isKeyFrame ? MediaCodec.BUFFER_FLAG_SYNC_FRAME : 0); // ࣍ͷsampleσʔλʹਐΉ mediaExtractor.advance(); ᶅ.FEJB$PEFDͷϙΠϯτ
  3. // MediaExtractor͔ΒडऔɺॲཧࡁΈͷOutput BufferΛऔಘ int result = decoder.dequeueOutputBuffer(bufferInfo, 0); // 2ͭΊͷҾ਺͸SurfaceʹtextureΛඳը͢Δ͔Ͳ͏͔

    decoder.releaseOutputBuffer(result, true); // ඳը͞Εͨtextureͷ౸ண଴ͪ decoderSurface.awaitNewImage(); // textureΛOpenGLͰඳը decoderSurface.drawImage(); // ඳը͞ΕΔ࣌ؒΛࢦఆ encoderSurface.setPresentationTime(bufferInfo.presentationTimeUs * 1000); // ݱࡏඳը͞Ε͍ͯΔ಺༰ΛEncoderʹૹ৴ encoderSurface.swapBuffers(); // encoderͰඳը৘ใΛडऔΔ encoder.dequeueOutputBuffer(bufferInfo, 0); encoderOutputBuffers = encoder.getOutputBuffers(); // encoderͷoutputBufferΛMediaMuxerʹ౉ͯ͠ɺmp4ϑΝΠϧʹ͢Δ muxRender.writeSampleData(MuxRender.SampleType.VIDEO, encoderOutputBuffers[result], bufferInfo); ᶅ.FEJB$PEFDͷϙΠϯτ .FEJB$PEFDɹ.FEJB.VYFSͷ େࡶ೺ͳίʔυͷྲྀΕ
  4. .1 .FEJB &YUSBDUPS .FEJB$PEFD %FDPEFS 0QFO(-&4$POUFYU 'JMUFS %FDPEFS4VSGBDF &ODPEFS4VSGBDF .FEJB$PEFD

    &ODPEFS .FEJB .VYFS .1 4VSGBDF5FYUVSF 4XBQ#V⒎FS ESBX #ZUF#V⒎FS #ZUF#V⒎FS ᶆ0QFO(-&4ͷϙΠϯτ
  5. ᶆ0QFO(-&4ͷϙΠϯτ 0QFO(-&4Ͱͷඳը·ͰͷྲྀΕ ௖఺৘ใΛ༻ҙ private final float[] triangleVerticesData = { //

    X, Y, Z, U, V -1.0f, -1.0f, 0, 0.f, 0.f, 1.0f, -1.0f, 0, 1.f, 0.f, -1.0f, 1.0f, 0, 0.f, 1.f, 1.0f, 1.0f, 0, 1.f, 1.f, };
  6. (-1,1,0) +y +x (1,1,0) (1,-1,0) (-1,-1,0) +z (0,0) (1,0) (0,1)

    67࠲ඪ ௖఺࠲ඪ ᶆ0QFO(-&4ͷϙΠϯτ 0QFO(-&4Ͱͷඳը·ͰͷྲྀΕ ௖఺৘ใΛ༻ҙ ը૾࡞੒ࢀߟݩɹ ͚Μ͝ͷ͓԰ෑɹςΫενϟϚοϐϯά IUUQULFOHPHJUIVCJPCMPHPQFOHMFTELOPXMFEHF
  7. 7FSUFY4IBEFS VOJGPSNNBUV.71.BUSJY֦େճసͳͲͷม׵࠲ඪ VOJGPSNNBUV45.BUSJYσίʔυͨ͠ςΫενϟͷߦྻ BUUSJCVUFWFDB1PTJUJPOY Z [ͷ࠲ඪ BUUSJCVUFWFDB5FYUVSF$PPSEV W࠲ඪ WBSZJOHWFDW5FYUVSF$PPSE'SBHNFOU4IBEFSʹ౉͢࠲ඪ WPJENBJO

    \ HM@1PTJUJPOV.71.BUSJY B1PTJUJPO W5FYUVSF$PPSE V45.BUSJY B5FYUVSF$PPSE YZ ^ ᶆ0QFO(-&4ͷϙΠϯτ ɾม׵ߦྻͱ௖఺࠲ඪΛܭࢉ͠ɺ௖఺ҐஔΛܾఆ ɾςΫενϟͷ67࠲ඪͷܾఆΛ'SBHNFOU4IBEFS΁౉͢
  8. ᶆ0QFO(-&4ͷϙΠϯτ ม׵ߦྻͱ͸ private float[] MVPMatrix = new float[16]; // ୯Ґߦྻͷੜ੒

    Matrix.setIdentityM(MVPMatrix, 0); // ฒߦҠಈ Matrix.translateM(MVPMatrix, 0, x:0, y:0, z:0f); // ֦େɾॖখ Matrix.scaleM(MVPMatrix, 0, x:1, y:1, z:1); // ճస Matrix.rotateM(MVPMatrix, 0, rotation:0, x:0.f, y:0.f, z:1.f);
  9. .1 .FEJB &YUSBDUPS .FEJB$PEFD %FDPEFS 0QFO(-&4$POUFYU 'JMUFS %FDPEFS4VSGBDF &ODPEFS4VSGBDF .FEJB$PEFD

    &ODPEFS .FEJB .VYFS .1 4VSGBDF5FYUVSF 4XBQ#V⒎FS ESBX #ZUF#V⒎FS #ZUF#V⒎FS ᶆ0QFO(-&4ͷϙΠϯτ ᶃ%FDPEFS4VSGBDFͰ4VSGBDF5FYUVSFͱͯ͠ը૾Λड͚औΔɻ 4DBMF΍ճస༻ͷม׵ߦྻΛ؅ཧɻ ᶄ'JMUFSͰ4VSGBDF5FYUVSFʹ৭Λ͚ͭͨΓɺ΢ΥʔλʔϚʔΫΛ͚ͭͨΓ͢Δ ᶅ&ODPEFS4VSGBDFͰ.FEJB$PEFDͷ&ODPEFʹॻ͖ࠐΈΛߦ͏
  10. // ඳը͞Εͨtextureͷ౸ண଴ͪ decoderSurface.awaitNewImage(); // textureΛOpenGLͰඳը ಺෦ͰFilter͕drawΛߦͬͯ·͢ɻ decoderSurface.drawImage(); // ඳը͞ΕΔ࣌ؒΛࢦఆ encoderSurface.setPresentationTime(bufferInfo.presentationTimeUs

    * 1000); // ݱࡏඳը͞Ε͍ͯΔ಺༰ΛEncoderʹૹ৴ encoderSurface.swapBuffers(); // encoderͰඳը৘ใΛडऔΔ encoder.dequeueOutputBuffer(bufferInfo, 0); encoderOutputBuffers = encoder.getOutputBuffers(); // encoderͷoutputBufferΛMediaMuxerʹ౉ͯ͠ɺmp4ϑΝΠϧʹ͢Δ muxRender.writeSampleData(MuxRender.SampleType.VIDEO, encoderOutputBuffers[result], bufferInfo); ᶆ0QFO(-&4ͷϙΠϯτ %FDPEFS4VSGBDFͱ&ODPEFS 4VSGBDFͷେࡶ೺ͳίʔυͷྲྀΕ
  11. (MBZ4DBMFͷ'SBHNFOU4IBEFS FYUFOTJPO(-@0&4@&(-@JNBHF@FYUFSOBMSFRVJSF QSFDJTJPONFEJVNQqPBU WBSZJOHWFDW5FYUVSF$PPSE VOJGPSNTBNQMFS&YUFSOBM0&4T5FYUVSF DPOTUIJHIQWFDXFJHIUWFD    

    WPJENBJO \ qPBUMVNJOBODFEPU UFYUVSF% T5FYUVSF W5FYUVSF$PPSE SHC XFJHIU  HM@'SBH$PMPSWFD WFD MVNJOBODF   ^ ᶇಈըʹϑΟϧλʔΛ͔͚Δ
  12. 4FQJBͷ'SBHNFOU4IBEFS FYUFOTJPO(-@0&4@&(-@JNBHF@FYUFSOBMSFRVJSF QSFDJTJPONFEJVNQqPBU WBSZJOHWFDW5FYUVSF$PPSE VOJGPSNTBNQMFS&YUFSOBM0&4T5FYUVSF WPJENBJO \ WFDGSBH$PMPSUFYUVSF% T5FYUVSF W5FYUVSF$PPSE

     HM@'SBH$PMPSSEPU GSBH$PMPSSHC WFD     HM@'SBH$PMPSHEPU GSBH$PMPSSHC WFD     HM@'SBH$PMPSCEPU GSBH$PMPSSHC WFD     ^ ᶇಈըʹϑΟϧλʔΛ͔͚Δ
  13. 'SBHNFOU4IBEFSͰΞϧϑΝ߹੒ FYUFOTJPO(-@0&4@&(-@JNBHF@FYUFSOBMSFRVJSF QSFDJTJPONFEJVNQqPBU WBSZJOHWFDW5FYUVSF$PPSE VOJGPSNTBNQMFS&YUFSOBM0&4T5FYUVSF VOJGPSNMPXQTBNQMFS%P5FYUVSFΞοϓϩʔυͨ͠UFYUVSF WPJENBJO \ MPXQWFDDUFYUVSF% T5FYUVSF

    W5FYUVSF$PPSE  MPXQWFDDUFYUVSF% P5FYUVSF W5FYUVSF$PPSE  MPXQWFDPVUQVU$PMPS PVUQVU$PMPSSDS DS DB  DB  PVUQVU$PMPSHDH DH DB  DB  PVUQVU$PMPSCDC DC DB  DB  PVUQVU$PMPSBDB DB  DB  HM@'SBH$PMPSPVUQVU$PMPS ^ ᶈಈըʹ΢ΥʔλʔϚʔΫΛ͚ͭΔ