Vitamio Hardware decoder

Vitamio is an open multimedia framework for Android and iOS, with full and real hardware accelerated decoder and renderer

Transcript

1. Vitamio Hardware decoder
2. Agenda • 音视频解码流程 • Android multimedia framework • Vitamio hardware decoder
3. 音视频解码流程
4. Android multimedia framework
6. Android MediaPlayer ➔ Supported File Type(s) / Container Formats 3GPP MPEG-4 MPEG-TS (not seekable, Android 3.0+) Matroska (.mkv, Android 4.0+) WebM ➔ Network protocols RTSP HTTP/HTTPS Http live streaming (Android 3.0+)
7. What to do ? 1. Support Android 2.3 or before 2. Support more file types 3. Support more network protocols 4. Better support supported file 5. Support more devices 6. Support subtitles
8. Android MediaPlayer
9. Vitamio 1. FFmpeg demuxer 2. libstagefright 私有 API 调用 OpenMAX 硬解 3. Android 4.1+ MediaCodec API
10. Vitamio
11. FFmpeg AVFormatContext *ic = NULL; av_register_all(); avformat_network_init(); ic = avformat_alloc_context(); // open file avformat_open_input(&ic, player->url, NULL, NULL); av_dump_format(ic, 0, player->url, 0); avformat_find_stream_info(ic, NULL);
12. FFmpeg AVCodecContext *codecCtx = formatCtx->streams[stream_index]->codec; enum AVCodecID id = codecCtx->codec_id; // create FFmpegSource : MediaSource AVPacket packet; ffmpeg->av_read_frame(formatCtx, &packet); if (packet.stream_index == is->video_index) { ffmpeg->av_bitstream_filter_filter(converter, codecCtx, NULL, &new_pkt.data, &new_pkt.size, packet.data, packet.size, packet.flags & AV_PKT_FLAG_KEY); }
13. OMXCodec // Get ANativeWindow sp<ANativeWindow> mNativeWindow = getNativeWindowFromSurface(); // init video decoder OMXClient mClient; mClient.connect(); sp<MediaSource> mOmxSource = OMXCodec::Create(mClient.interface(), mVideoSource- >getFormat(), false, mVideoSource, NULL, 0, mNativeWindow); mOmxSource->start(); // loop read decoded frame MediaBuffer *mVideoBuffer; // AVPacket status_t err = mVideoDecoder->read(&mVideoBuffer, &options); err = mNativeWindow->queueBuffer(mNativeWindow.get(), mVideoBuffer->graphicBuffer().get());
14. OMXCodec 1. 适合大部分 2.3+ 系统, 比较稳定 2. Android 不同版本 私有API 可能会变化, 需要针对不同的 版本编译 3. 不同芯片的硬解码输出可能不同, 需要单独处理
15. MediaCodec API New Low-Level Media APIs in Android MediaCodec class can be used to access low-level media codec, i.e. encoder/decoder components. https://developers.google.com/events/io/2012/sessions/gooio2012/117/
16. MediaCodec // create MediaFormat MediaFormat trackFormat = MediaFormat.createVideoFormat(mime, width, height); // create MediaCodec MediaCodec videoCodec = null; final String mimeType = trackFormat.getString(MediaFormat.KEY_MIME); if (mimeType.contains("video/")) { videoCodec = MediaCodec.createDecoderByType(mimeType); videoCodec.configure(trackFormat, surface, null, 0); videoCodec.start(); mInputBuffers = codec.getInputBuffers(); mOutputBuffers = codec.getOutputBuffers(); }
17. MediaCodec for (;;) { int inputBufferIndex = codec.dequeueInputBuffer(timeoutUs); if (inputBufferIndex >= 0) { // fill inputBuffers[inputBufferIndex] with valid data ... codec.queueInputBuffer(inputBufferIndex, ...); } int outputBufferIndex = codec.dequeueOutputBuffer(timeoutUs); if (outputBufferIndex >= 0) { // outputBuffer is ready to be processed or rendered. codec.releaseOutputBuffer(outputBufferIndex, ...); } … }
18. MediaCodec API 1. 性能不错, 1080p 毫无压力 2. 只支持 Android 4.1+, 在 Android 4.3 得到改进 3. 无法保证所有的 4.1+ 手机可以正常使用和显示 http://bigflake.com/mediacodec/
19. Thanks GitHub:@crossle Twitter:@songxiaoming