Tweet
Tweet

Slide 1

Slide 1 text

2019 DevDay Looking for a Silver Bullet To Edit Videos on Android. > Seungwon Lee > LINE Plus AppDev5 Android Engineer

Slide 2

Slide 2 text

Agenda > Why Do We Need a Silver Bullet? > What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting

Slide 3

Slide 3 text

Agenda > Why Do We Need a Silver Bullet? > What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting

Slide 4

Slide 4 text

When You Upload a Video in LINE Chatroom Timeline Video Profile

Slide 5

Slide 5 text

Processing Video Client-side Overlay stickers Adjust video size …

Slide 6

Slide 6 text

Why Do We Need a Silver Bullet?

Slide 7

Slide 7 text

How to Resolve the Issues Device A Device B Device C

Slide 8

Slide 8 text

Blacklist > List of devices that had problems during video processing. Device A Device B Device C Device D …

Slide 9

Slide 9 text

Blacklist > Blacklisted devices were redirected so that client-side video processing would not occur Bypass Device A Device B Device C Device D …

Slide 10

Slide 10 text

Blacklisted Devices *H*-E*** *H*-0** S*-0** S*-0** R**** **** * R**** **** * *H*-E*** V**-* *-0**

Slide 11

Slide 11 text

Blacklisted Devices *H*-E*** *H*-0** S*-0** S*-0** R**** **** * R**** **** * *H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H

Slide 12

Slide 12 text

Blacklisted Devices *H*-E*** *H*-0** S*-0** S*-0** R**** **** * R**** **** * *H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H **F-***0 **F-***0 **F-**9 **F-***0 **A_***0 **R-***0 **Y-*L **R-***0 **E-***0 **E-***2 **R-***0 **Y-* *E-***0 **A-**1 **A-**1 ****W *7-**0 ****W **V-***L **E-**2 **E-**4 **E-**1 **E-**3 **E-***0

Slide 13

Slide 13 text

Android Wars: A New Hope {Episode 4) > android-transcoder made by Yuya Tanaka > https://github.com/ypresto/android-transcoder > Written with pure Java without codec licensing issues. > Supports OpenGL renderer

Slide 14

Slide 14 text

Android Wars: A New Hope {Episode 4) Legacy Video Processor in LINE android- transcoder *A**** P* X O *A**** H***** X O *A**** P* X O *A**** H***** X X *A**** M***** O O > android-transcoder made by Yuya Tanaka > https://github.com/ypresto/android-transcoder > Written with pure Java without codec licensing issues. > Supports OpenGL renderer

Slide 15

Slide 15 text

Agenda > Why Do We Need a Silver Bullet? > What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

Video Format : MP4 Video Codec : H.264 Audio Codec : AAC

Slide 18

Slide 18 text

Video Frame

Slide 19

Slide 19 text

Codec Encoding (H.264)

Slide 20

Slide 20 text

Codec Encoding (H.264) Decoding (H.264)

Slide 21

Slide 21 text

Codec Encoding (AAC) Decoding (AAC)

Slide 22

Slide 22 text

Audio Track (AAC) Video Track (H.264) MP4 Container Video Format(Container) Reference: - https://en.wikipedia.org/wiki/Video_file_format - https://en.wikipedia.org/wiki/Codec

Slide 23

Slide 23 text

Transcoding Trans - coding Transfer - Encoding

Slide 24

Slide 24 text

Vorbis Audio VP8 Video 3GP Container Friend A Mine Transcoding Vorbis, VP8 (X) AAC, H.264 (O)

Slide 25

Slide 25 text

AAC Audio H.264 Video MP4 Container Vorbis Audio VP8 Video 3GP Container Encoder (AAC) Decoder (Vorbis) Friend A Mine Transcoding Encoder (H.264) Decoder (VP8) Vorbis, VP8 (X) AAC, H.264 (O)

Slide 26

Slide 26 text

Agenda > Why Do We Need a Silver Bullet? > What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting

Slide 27

Slide 27 text

android-transcoder Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder

Slide 28

Slide 28 text

android-transcoder Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder

Slide 29

Slide 29 text

android-transcoder Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Reference: https://github.com/ypresto/android-transcoder

Slide 30

Slide 30 text

android-transcoder Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Reference: https://github.com/ypresto/android-transcoder

Slide 31

Slide 31 text

How to Extract/Decode a Video Extractor android.media.MediaExtractor Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder

Slide 32

Slide 32 text

> Query the metadata of each track > Read the encoded data stream > Demux data streams into each track android.media.MediaExtractor Media Extractor Audio info: AAC Video info: H.264 Audio Track (AAC) Video Track (H.264) MP4 Container

Slide 33

Slide 33 text

> Query the metadata of each track > Read the encoded data stream > Demux data streams into each track android.media.MediaExtractor Media Extractor Audio info: AAC Video info: H.264 Audio Track (AAC) Video Track (H.264) MP4 Container Audio Decoder Video Decoder

Slide 34

Slide 34 text

How to Extract Video and Audio Media Extractor Time Audio Track (AAC) Video Track (H.264) MP4 Container val size = mediaExtractor.readSampleData(byteBuf, offset) val presentationTimeUs = mediaExtractor.getSampleTime()

Slide 35

Slide 35 text

How to Extract Video and Audio Data Media Extractor mediaExtractor.advance() val size = mediaExtractor.readSampleData(byteBuf, offset) End of Stream Time Audio Track (AAC) Video Track (H.264) MP4 Container End of Stream … …

Slide 36

Slide 36 text

How to Decode a Video Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor android.media.MediaCodec Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder

Slide 37

Slide 37 text

How to Decode a Video > Wrapper class for media codecs (i.e. encoder and decoder) > Use together with android.media.MediaExtractor, android.media.MediaMuxer AAC Audio Decoder H.264 Video Decoder

Slide 38

Slide 38 text

How to Extract/Decode a Video Extracting, Decoding Rendering Encoding, Muxing Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder

Slide 39

Slide 39 text

How to Extract/Decode a Video Media Extractor Audio Track (AAC) Video Track (H.264) MP4 Container …

Slide 40

Slide 40 text

How to Extract/Decode a Video Video Decoder Media Extractor Audio Track (AAC) Video Track (H.264) MP4 Container

Slide 41

Slide 41 text

How to Extract/Decode a Video Video Decoder Media Extractor Audio Track (AAC) Video Track (H.264) MP4 Container Empty Input Buffer

Slide 42

Slide 42 text

How to Extract/Decode a Video Video Decoder Media Extractor Audio Track (AAC) Video Track (H.264) MP4 Container Empty Input Buffer Fill Input Buffer

Slide 43

Slide 43 text

How to Extract/Decode a Video Video Decoder Output Surface Media Extractor Audio Track (AAC) Video Track (H.264) MP4 Container Empty Input Buffer Fill Input Buffer Video Renderer

Slide 44

Slide 44 text

How to Extract/Decode a Video Video Decoder Output Surface Media Extractor Audio Track (AAC) Video Track (H.264) MP4 Container Video Renderer onFrameAvailable Empty Input Buffer Fill Input Buffer

Slide 45

Slide 45 text

How to Extract/Decode a Video Video Decoder Output Surface Media Extractor Audio Track (AAC) Video Track (H.264) MP4 Container Video Renderer onFrameAvailable Released Buffer in Surface Empty Input Buffer Fill Input Buffer

Slide 46

Slide 46 text

Rendering Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Reference: https://github.com/ypresto/android-transcoder

Slide 47

Slide 47 text

How to Render a Video Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Output Surface Input Surface Frame Buffer Reference: https://github.com/ypresto/android-transcoder

Slide 48

Slide 48 text

How to Render a Video Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Output Surface Input Surface Frame Buffer Render Reference: https://github.com/ypresto/android-transcoder

Slide 49

Slide 49 text

How to Render a Video Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Output Surface Input Surface Frame Buffer Render Render Reference: https://github.com/ypresto/android-transcoder

Slide 50

Slide 50 text

How to Render a Video > OutputSurface and InputSurface will be bound with the current OpenGL environment Output Surface from Decoder FrameBuffer Input Surface to Encoder

Slide 51

Slide 51 text

How to Render a Video > Pixel data of a video frame is drawn into OutputSurface of the decoder 720 X 1280 Output Surface from Decoder FrameBuffer Input Surface to Encoder

Slide 52

Slide 52 text

How to Render a Video Output Surface from Decoder FrameBuffer Input Surface to Encoder > We can reduce a video frame by drawing it on FrameBuffer in OpenGL 720 X 1280 540 X 960

Slide 53

Slide 53 text

How to Render a Video > We pass the resized frame to InputSurface of the encoder 540 X 960 720 X 1280 540 X 960 Output Surface from Decoder FrameBuffer Input Surface to Encoder

Slide 54

Slide 54 text

How to Encode/Mux a Video Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Reference: https://github.com/ypresto/android-transcoder

Slide 55

Slide 55 text

How to Extract/Decode a Video Video OpenGL Renderer Input Surface Video Encoder

Slide 56

Slide 56 text

Output Buffer How to Extract/Decode a Video Video OpenGL Renderer Request Encoding Input Surface Video Encoder

Slide 57

Slide 57 text

Muxer How to Extract/Decode a Video Video OpenGL Renderer Request Encoding Audio Track Video Track Input Surface Output Buffer Release Output Buffer Video Encoder

Slide 58

Slide 58 text

How to Encode/Mux a Video Muxer android.media.MediaMuxer Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder

Slide 59

Slide 59 text

android.media.MediaMuxer > Mux encoded streams of audio/video > Supports MP4, WebM and 3GP files AAC Encoder H.264 Encoder Media Muxer Audio Track (AAC) Video Track (H.264) Video Container Reference: - https://developer.android.com/reference/android/media/MediaMuxer - https://developer.android.com/guide/topics/media/media-formats

Slide 60

Slide 60 text

android.media.MediaMuxer Media Muxer mediaMuxer.writeSampleData(trackIndex, byteBuf, bufferInfo) -> Audio Track (AAC) Video Track (H.264) Video Container Time

Slide 61

Slide 61 text

android.media.MediaMuxer Media Muxer mediaMuxer.writeSampleData(trackIndex, byteBuf, bufferInfo) -> Audio Track (AAC) Video Track (H.264) Video Container End of stream Time End of stream … …

Slide 62

Slide 62 text

How Does android-transcoder Work Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder

Slide 63

Slide 63 text

I think you completely understand it

Slide 64

Slide 64 text

Adopting android-transcoder to LINE

Slide 65

Slide 65 text

Issues with adopting android-transcoder to LINE

Slide 66

Slide 66 text

Audio Features Didn’t Exist Audio sampler Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container

Slide 67

Slide 67 text

Overlaying Stickers Didn’t Exist Audio sampler Overlay Sticker Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container

Slide 68

Slide 68 text

Renovate to new project: Cruiser

Slide 69

Slide 69 text

Agenda > Why Do We Need a Silver Bullet? > What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting

Slide 70

Slide 70 text

Added Features to Cruiser v1.0 Container MP4 Video Codec H.264 Bitrate 2 Mbit/s Resolution 720*1280 Trim range 1200 ms ~ 250300 ms Audio Codec AAC Sample Rate 44.1 kHz ... ... > Adjust sampling rate of audio > Add APIs for a trancoding policy including codec, resolution, trim ranges, and etc.

Slide 71

Slide 71 text

Audio Sampler Audio Sampler > Adjust sampling rate of audio Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container

Slide 72

Slide 72 text

Audio Sampler Audio Sampler AAC Decoder AAC Encoder > Based on libsamplerate made by erikd > https://github.com/erikd/libsamplerate > A library for performing sample rate conversion of audio data 320 kbit/s 96 kbit/s

Slide 73

Slide 73 text

Cruiser v1.0 Features > Adjusts video size and sampling audio with a simple policy for transcoding > Trims video Container MP4 Video Codec H.264 Bitrate 2 Mbit/s Resolution 720*1280 Trim range 1200 ms ~ 250300 ms Audio Codec AAC Samplerate 44.1 kHz

Slide 74

Slide 74 text

Cruiser v1.0 Features > Adjusts video size and sampling audio with a simple policy for transcoding > Trims video > Mutes audio X Container MP4 Video Codec H.264 Bitrate 2 Mbit/s Resolution 720*1280 Trim range 1200 ms ~ 250300 ms Audio Codec AAC Samplerate 44.1 kHz

Slide 75

Slide 75 text

Cruiser v1.1 Feature > Overlays LINE static and animated stickers on a video

Slide 76

Slide 76 text

Cruiser v1.1 Main Feature Audio sampler Overlay Sticker Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container

Slide 77

Slide 77 text

How Does Cruiser Overlay A Sticker > OutputSurface and InputSurface will bind with the current OpenGL environment Output Surface from Decoder FrameBuffer Input Surface to Encoder

Slide 78

Slide 78 text

How Does Cruiser Overlay A Sticker > Pixel data of a video frame is copied into OutputSurface of the decoder Output Surface from Decoder FrameBuffer Input Surface to Encoder

Slide 79

Slide 79 text

How Does Cruiser Overlay A Sticker > We load a sticker as a Bitmap instance and replace it to a texture that OpenGL can recognize Teuxture Output Surface from Decoder FrameBuffer Input Surface to Encoder

Slide 80

Slide 80 text

How Does Cruiser Overlay A Sticker > We can draw additionally overlay a sticker on FrameBuffer in OpenGL Output Surface from Decoder FrameBuffer Input Surface to Encoder

Slide 81

Slide 81 text

How Does Cruiser Overlay A Sticker > We pass this frame to InputSurface of the encoder Output Surface from Decoder FrameBuffer Input Surface to Encoder

Slide 82

Slide 82 text

How Does Cruiser Overlay A Sticker > Like this, We can overlay serval stickers on a video frame Output Surface from Decoder FrameBuffer Input Surface to Encoder

Slide 83

Slide 83 text

Blacklisted Devices *H*-E*** *H*-0** S*-0** S*-0** R**** **** * R**** **** * *H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H **F-***0 **F-***0 **F-**9 **F-***0 **A_***0 **R-***0 **Y-*L **R-***0 **E-***0 **E-***2 **R-***0 **Y-* *E-***0 **A-**1 **A-**1 ****W *7-**0 ****W **V-***L **E-**2 **E-**4 **E-**1 **E-**3 **E-***0

Slide 84

Slide 84 text

*H*-E*** *H*-0** S*-0** S*-0** R**** **** * R**** **** * *H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H **F-***0 **F-***0 **F-**9 **F-***0 **A_***0 **R-***0 **Y-*L **R-***0 **E-***0 **E-***2 **R-***0 **Y-* *E-***0 **A-**1 **A-**1 ****W *7-**0 ****W **V-***L **E-**2 **E-**4 **E-**1 **E-**3 **E-***0 Blacklist On Cruiser X

Slide 85

Slide 85 text

*H*-E*** *H*-0** S*-0** S*-0** R**** **** * R**** **** * *H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H **F-***0 **F-***0 **F-**9 **F-***0 **A_***0 **R-***0 **Y-*L **R-***0 **E-***0 **E-***2 **R-***0 **Y-* *E-***0 **A-**1 **A-**1 ****W *7-**0 ****W **V-***L **E-**2 **E-**4 **E-**1 **E-**3 **E-***0 Blacklist On Cruiser X !

Slide 86

Slide 86 text

Can't beat Android fragmentation

Slide 87

Slide 87 text

Agenda > Why Do We Need a Silver Bullet? > What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting

Slide 88

Slide 88 text

Troubleshooting Video issues Quality issue Orientation issue Audio issue 0 bit/s Issue Out of Sync Audio and Video Issue Audio Distortion Issue

Slide 89

Slide 89 text

Troubleshooting Video Issues > Quality Issue Orientation Issue Audio Issue 0 bit/s Issue Out of Sync Audio and Video Issue Audio Distortion Issue

Slide 90

Slide 90 text

Low Video Quality Original Video Transcoded Video with a Video Filter

Slide 91

Slide 91 text

A Video Profile Configuration Video Encoder Video Encoder Cruiser Device Camera High Profile Video High Profile Video

Slide 92

Slide 92 text

A Video Profile Configuration Video Encoder Video Encoder Cruiser Device Camera High Profile Video Base Profile Video

Slide 93

Slide 93 text

Low Quality Due to Video Profiles Base Profile Video High Profile Video

Slide 94

Slide 94 text

Low Quality Due to Video Profiles Video Encoder Cruiser High Profile Video > Don’t leave a device to choose a video profile High Profile Video Encoder Cruiser Base Profile Video Base Profile

Slide 95

Slide 95 text

Troubleshooting Video Issues Quality Issue > Orientation Issue Audio Issue 0 bit/s Issue Out of Sync Audio and Video Issue Audio Distortion Issue

Slide 96

Slide 96 text

Rotated Video Original Video Transcoded Video

Slide 97

Slide 97 text

Orientation Setting to Encoder Video Encoder Muxer 90 Orientation = 90 Extractor 90 90

Slide 98

Slide 98 text

Orientation Setting to Encoder X 90 Ignored Video Encoder Muxer 90 Orientation = 90 Extractor 90

Slide 99

Slide 99 text

Video Rotated Issue 90 Video Encoder Muxer 90 Orientation = 90 Extractor 90

Slide 100

Slide 100 text

Video Rotated Issue > Don’t set the orientation value of a video to an encoder Video Encoder Muxer 90 Orientation = 90 Extractor 90 X

Slide 101

Slide 101 text

Troubleshooting Video Issues Quality Issue Orientation Issue Audio Issue > 0 bit/s Issue Out of Sync Audio and Video Issue Audio Distortion Issue

Slide 102

Slide 102 text

Audio Distortion Original Video Transcoded Video

Slide 103

Slide 103 text

0 bit/s From MediaExtractor Audio bitrate = 128 kbit/s

Slide 104

Slide 104 text

0 bit/s Issue Extract the bitrate Audio bitrate = 128 kbit/s Extractor 128 kbit/s

Slide 105

Slide 105 text

0 bit/s Issue 128 kbit/s 96 kbit/s 96 kbit/s Audio Format Video Format Container Audio Decoder Audio Encoder Muxer Extractor Audio Down Sampler Audio Format Container Video Format 96 kbit/s 48 kbit/s 48 kbit/s 48 kbit/s 48 kbit/s Audio Codec AAC Sample Rate 44.1 kHz Bitrate 96 kbit/s

Slide 106

Slide 106 text

0 bit/s Issue Audio bitrate = 128 kbit/s Extract the bitrate Extractor 0 kbit/s

Slide 107

Slide 107 text

0 bit/s Issue > Don’t forget to check 0 values 0 kbit/s 0 kbit/s Audio Format Video Format Container Audio Decoder Audio Encoder Muxer Extractor Audio Down Sampler Audio Format Container Video Format 0 kbit/s 0 kbit/s

Slide 108

Slide 108 text

0 bit/s Issue > Don’t forget to check 0 values 0 kbit/s 96 kbit/s 96 kbit/s Audio Format Video Format Container Audio Decoder Audio Encoder Muxer Extractor Audio Down Sampler Audio Format Container Video Format 96 kbit/s

Slide 109

Slide 109 text

Troubleshooting Video Issues Quality Issue Orientation Issue Audio Issue 0 bit/s Issue > Out of Sync Audio and Video Issue Audio Distortion Issue

Slide 110

Slide 110 text

Out of Sync Audio and Video Issue Original Video Transcoded video

Slide 111

Slide 111 text

Waiting of Muxer Initialization Muxer Audio Track Video Track Output Buffer Video Encoder

Slide 112

Slide 112 text

Waiting of Muxer Initialization Muxer Audio Track Video Track Output Buffer Release Output Buffer Video Encoder

Slide 113

Slide 113 text

Waiting of Muxer Initialization Muxer Audio Track Video Track Output Buffer Release Output Buffer Video Encoder

Slide 114

Slide 114 text

Waiting of Muxer Initialization Muxer Audio Track Video Track Output Buffer Release Output Buffer Video Encoder

Slide 115

Slide 115 text

Waiting of Muxer Initialization Muxer Audio Track Video Track Output Buffer Video Encoder X MUXER_STATE_UNINITIALIZED

Slide 116

Slide 116 text

Waiting of Muxer Initialization Muxer Audio Track Video Track Queue Video Encoder Output Buffer Release Output Buffer Shallow copy MUXER_STATE_UNINITIALIZED

Slide 117

Slide 117 text

Waiting of Muxer Initialization Muxer Audio Track Video Track Video Encoder MUXER_STATE_STARTED Output Buffer Release Output buffer Pass Output Buffer Queue

Slide 118

Slide 118 text

Waiting of Muxer Initialization Muxer Audio Track Video Track Queue Video Encoder MUXER_STATE_STARTED Output Buffer Release Output buffer

Slide 119

Slide 119 text

Waiting of Muxer Initialization Muxer Audio Track Video Track Video Encoder MUXER_STATE_STARTED Output Buffer Release Output buffer Pass Output Buffer Queue

Slide 120

Slide 120 text

Out of Sync Issue Muxer Audio Track Video Track Pass Output Buffer Queue Video Encoder MUXER_STATE_STARTED Output Buffer Release Output buffer Shallow copy

Slide 121

Slide 121 text

Out of Sync Issue Muxer Audio Track Video Track Pass Output Buffer Queue Video Encoder MUXER_STATE_STARTED Output Buffer Release Output Buffer Deep copy > Don’t rely on release timing of the device

Slide 122

Slide 122 text

Troubleshooting Video Issues Quality Issue Orientation Issue Audio Issue 0 bit/s Issue Out of Sync Audio and Video Issue > Audio Distortion Issue

Slide 123

Slide 123 text

Audio Distortion Problem Original Video Transcoded Video

Slide 124

Slide 124 text

Original Audio Bitrate Audio Distortion Problem Extract the bitrate Extractor 96 kbit/s

Slide 125

Slide 125 text

Audio Distortion Problem After Trimming Extract the bitrate File Analysis 123 kbit/s Extract the bitrate Extractor 24 kbit/s

Slide 126

Slide 126 text

Audio Distortion Problem Audio Format Video Format Container Audio Decoder Audio Encoder Muxer Extractor Audio Down Sampler Audio Format Container Video Format > We transcoded the audio track and resampled the ByteBuffer of the audio > We don’t need to resample this problematic audio track 24 kbit/s (123 kbit/s) 24 kbit/s (123 kbit/s) 24 kbit/s (123 kbit/s) 24 kbit/s (123 kbit/s)

Slide 127

Slide 127 text

Audio Distortion Problem > Cruiser can detect this issue by extracting 24 kbit/s > Cruiser bypasses the resampling process if it has 24 kbit/s Audio Format Video Format Container Audio Decoder Audio Encoder Muxer Extractor Audio Down Sampler Audio Format Container Video Format X X X Audio Stream Bypass 24 kbit/s (123 kbit/s) 24 kbit/s (123 kbit/s)

Slide 128

Slide 128 text

Troubleshooting Video issues Quality issue Orientation issue Audio issue 0 bit/s Issue Out of Sync Audio and Video Issue Audio Distortion Issue

Slide 129

Slide 129 text

Do not count on Android devices You're the only hope

Slide 130

Slide 130 text

Have I found a silver bullet?

Slide 131

Slide 131 text

Blacklisted Devices *H*-E*** *H*-0** S*-0** S*-0** R**** **** * R**** **** * *H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H **F-***0 **F-***0 **F-**9 **F-***0 **A_***0 **R-***0 **Y-*L **R-***0 **E-***0 **E-***2 **R-***0 **Y-* *E-***0 **A-**1 **A-**1 ****W *7-**0 ****W **V-***L **E-**2 **E-**4 **E-**1 **E-**3 **E-***0 X

Slide 132

Slide 132 text

In-progress

Slide 133

Slide 133 text

Cruiser v1.2 features > Make a video with a static image Cruiser v1.3 features > Crop and rotate a video What‘s Next? Cruiser v2.0 features > Each component runs on multiple threads

Slide 134

Slide 134 text

No content

Slide 135

Slide 135 text

Thank you