Video Processing on Android

An increasing number of social media apps such as Instagram, Snapchat and Messenger are using video features. With modern Android devices having enough processing power to run video processing algorithms locally, there’s no need to send videos through expensive internet bandwidth to be processed on the backend. Overlays, transcoding and cropping are just a few of the challenges you face when working with video on-device. In this talk I will go through my experience experimenting with built-in as well as third party libraries for video manipulation on Android. We will look at the benefits of two leading media processing libraries, and how you can leverage them to enhance Android’s native MediaCodec API to accomplish these tasks. You will walk away with a head start on how to tackle the most common challenges with videos on Android.

Transcript

1. V I D E O P R O C E S S I N G O N A N D R O I D N A M R ATA B A N D E K A R
2. W H O A M I • OANDA • raywenderlich.com • Ginger!
3. V I D E O P R O C E S S I N G I N T R O D U C T I O N T O
4. M O B I L E A P P S P O P U L A R U S E I N
5. C O M M O N F E AT U R E S • Special video effects • Merging videos • Cropping and trimming
6. M Y S E C O N D J O B !
7. W E AT H E R G I F
8. F E AT U R E L I S T ?
10. I O S W E AT H E R G I F
11. V I D E O E D I T I N G O N D E V I C E
12. A N D R O I D S U P P O R T • MediaCodec • MediaExtractor • MediaMuxer
14. W H AT N E X T ?
15. F F M P E G
16. W H Y F F M P E G ? • Swiss Army Knife for video • Filters • Subtitles
17. F F M P E G O N A N D R O I D • Executable binary • Shared object library
18. F F M P E G E X E C U TA B L E B U I L D I N G T H E
19. D E P E N D E N C I E S • autoconf • automake • libtool
20. G E T T I N G S TA R T E D Step 1: Download and unpack NDK
21. G E T T I N G S TA R T E D Step 2: Download FFmpeg source code https://github.com/FFmpeg/FFmpeg
22. G E T T I N G S TA R T E D Step 3: Download sources for other libraries like yasm and libvpx
23. G E T T I N G S TA R T E D Step 4: Put FFmpeg and other libraries under ndk/sources
24. C O N F I G U R I N G …
26. C O N F I G U R I N G
27. C O N F I G U R I N G
28. C O N F I G U R I N G
29. C O N F I G U R I N G
30. C O M P I L I N G
31. F F M P E G E X E C U TA B L E I N Y O U R A N D R O I D P R O J E C T
32. C H A N G E P E R M I S S I O N S
33. P R O C E S S B U I L D E R
34. F I LT E R C H A I N I N G
35. P E R F O R M A N C E
36. S H A R E D O B J E C T L I B R A RY B U I L D I N G F F M P E G A S A
37. S E T U P F O R F F M P E G . S O Step 1: JNI project same folder as NDK
38. S E T U P F O R F F M P E G . S O Step 2: Android.mk file
39. S E T U P F O R F F M P E G . S O Step 3: Application.mk file
40. S E T U P F O R F F M P E G . S O Step 4: Place native code in app/jni
41. N D K M A G I C
42. L O A D S H A R E D L I B R A RY
43. W E AT H E R G I F D E M O
45. L I C E N S I N G
47. L I C E N S E C O M P L I A N C E
48. L I C E N S E C O M P L I A N C E • Dynamic linking • FFmpeg source • Configure script
49. L I C E N S E C O M P L I A N C E
50. L I C E N S E C O M P L I A N C E
51. F F M P E G P R O S • Modular • Support for codecs • Versatile • Well documented
52. F F M P E G C H A L L E N G E S • Licensing • Slow • Large
53. M P 4 PA R S E R
54. M P 4 PA R S E R F E AT U R E S • Concatenation • Trimming • Muxing • Demuxing
55. M P 4 PA R S E R U S A G E
56. M P 4 PA R S E R D E M O
58. M P 4 PA R S E R P R O S • Java library • Fast • Clean API
59. M P 4 PA R S E R C H A L L E N G E S • Limited functionality • No encoding and decoding • Concatenates only if inputs have same format
60. R E S O U R C E S • https://github.com/guardianproject/android- ffmpeg • https://github.com/Yelp/ffmpeg-android • https://github.com/inFullMobile/videokit- ffmpeg-android • https://www.ffmpeg.org/legal.html • https://github.com/sannies/mp4parser • https://github.com/TeamWanari/RxMp4Parser
61. O A N D A I S H I R I N G ! jobs.oanda.com
62. T H A N K Y O U ! Q U E S T I O N S ? @ n a m r a t a C o d e s https://github.com/namrata-b/talks/