Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Video Processing on Android

Video Processing on Android

An increasing number of social media apps such as Instagram, Snapchat and Messenger are using video features. With modern Android devices having enough processing power to run video processing algorithms locally, there’s no need to send videos through expensive internet bandwidth to be processed on the backend. Overlays, transcoding and cropping are just a few of the challenges you face when working with video on-device. In this talk I will go through my experience experimenting with built-in as well as third party libraries for video manipulation on Android. We will look at the benefits of two leading media processing libraries, and how you can leverage them to enhance Android’s native MediaCodec API to accomplish these tasks. You will walk away with a head start on how to tackle the most common challenges with videos on Android.

F4d08154505cf87987b38f4d596cdf15?s=128

Namrata Bandekar

July 13, 2017
Tweet

Transcript

  1. V I D E O P R O C E

    S S I N G O N A N D R O I D N A M R ATA B A N D E K A R
  2. W H O A M I • OANDA • raywenderlich.com

    • Ginger!
  3. V I D E O P R O C E

    S S I N G I N T R O D U C T I O N T O
  4. M O B I L E A P P S

    P O P U L A R U S E I N
  5. C O M M O N F E AT U

    R E S • Special video effects • Merging videos • Cropping and trimming
  6. M Y S E C O N D J O

    B !
  7. W E AT H E R G I F

  8. F E AT U R E L I S T

    ?
  9. None
  10. I O S W E AT H E R G

    I F
  11. V I D E O E D I T I

    N G O N D E V I C E
  12. A N D R O I D S U P

    P O R T • MediaCodec • MediaExtractor • MediaMuxer
  13. None
  14. W H AT N E X T ?

  15. F F M P E G

  16. W H Y F F M P E G ?

    • Swiss Army Knife for video • Filters • Subtitles
  17. F F M P E G O N A N

    D R O I D • Executable binary • Shared object library
  18. F F M P E G E X E C

    U TA B L E B U I L D I N G T H E
  19. D E P E N D E N C I

    E S • autoconf • automake • libtool
  20. G E T T I N G S TA R

    T E D Step 1: Download and unpack NDK
  21. G E T T I N G S TA R

    T E D Step 2: Download FFmpeg source code https://github.com/FFmpeg/FFmpeg
  22. G E T T I N G S TA R

    T E D Step 3: Download sources for other libraries like yasm and libvpx
  23. G E T T I N G S TA R

    T E D Step 4: Put FFmpeg and other libraries under ndk/sources
  24. C O N F I G U R I N

    G …
  25. None
  26. C O N F I G U R I N

    G
  27. C O N F I G U R I N

    G
  28. C O N F I G U R I N

    G
  29. C O N F I G U R I N

    G
  30. C O M P I L I N G

  31. F F M P E G E X E C

    U TA B L E I N Y O U R A N D R O I D P R O J E C T
  32. C H A N G E P E R M

    I S S I O N S
  33. P R O C E S S B U I

    L D E R
  34. F I LT E R C H A I N

    I N G
  35. P E R F O R M A N C

    E
  36. S H A R E D O B J E

    C T L I B R A RY B U I L D I N G F F M P E G A S A
  37. S E T U P F O R F F

    M P E G . S O Step 1: JNI project same folder as NDK
  38. S E T U P F O R F F

    M P E G . S O Step 2: Android.mk file
  39. S E T U P F O R F F

    M P E G . S O Step 3: Application.mk file
  40. S E T U P F O R F F

    M P E G . S O Step 4: Place native code in app/jni
  41. N D K M A G I C

  42. L O A D S H A R E D

    L I B R A RY
  43. W E AT H E R G I F D

    E M O
  44. None
  45. L I C E N S I N G

  46. None
  47. L I C E N S E C O M

    P L I A N C E
  48. L I C E N S E C O M

    P L I A N C E • Dynamic linking • FFmpeg source • Configure script
  49. L I C E N S E C O M

    P L I A N C E
  50. L I C E N S E C O M

    P L I A N C E
  51. F F M P E G P R O S

    • Modular • Support for codecs • Versatile • Well documented
  52. F F M P E G C H A L

    L E N G E S • Licensing • Slow • Large
  53. M P 4 PA R S E R

  54. M P 4 PA R S E R F E

    AT U R E S • Concatenation • Trimming • Muxing • Demuxing
  55. M P 4 PA R S E R U S

    A G E
  56. M P 4 PA R S E R D E

    M O
  57. None
  58. M P 4 PA R S E R P R

    O S • Java library • Fast • Clean API
  59. M P 4 PA R S E R C H

    A L L E N G E S • Limited functionality • No encoding and decoding • Concatenates only if inputs have same format
  60. R E S O U R C E S •

    https://github.com/guardianproject/android- ffmpeg • https://github.com/Yelp/ffmpeg-android • https://github.com/inFullMobile/videokit- ffmpeg-android • https://www.ffmpeg.org/legal.html • https://github.com/sannies/mp4parser • https://github.com/TeamWanari/RxMp4Parser
  61. O A N D A I S H I R

    I N G ! jobs.oanda.com
  62. T H A N K Y O U ! Q

    U E S T I O N S ? @ n a m r a t a C o d e s https://github.com/namrata-b/talks/