Video Editing on Android - Yelp Engineering

The Android framework makes it easy to play videos. But did you know you your app can support editing videos as well? With fairly simple setup it’s possible to apply filters, perform transformations and color adjustments, add subtitles, and even paint over videos!

In this talk you’ll learn how to create videos directly within your app. We’ll start by building a few video editing libraries, incorporate them into the app build, and finally we’ll look at how to implement video trimming as an example application. Look forward to unleashing your users’ creative powers!

Transcript

1. @MGhareeb32 Muhammad Siddeek Video Editing on Android
2. Yelp’s Mission Connecting people with great local businesses.
7. Generate trimmed video Compile video editing library Implement trimmer screen
8. Compile Video Editing Library
16. input.mp4 greyscaled.mp4
17. input.mp4 cropped.mp4
18. input.mp4 reversed.mp4
19. input.mp4 overlayed.mp4
20. input.mp4 + palette.bmp giffed.gif
21. input.mp4 + nyan.gif nyaned.mp4
22. https://github.com/Yelp/ffmpeg-android
23. https://github.com/Yelp/ffmpeg-android !
24. ffmpeg
25. ?
26. Universal Apk armeabi-v7a x86 arm64-v8a armeabi x86_64 mips, mips64
27. Universal Apk armeabi-v7a Most common x86 arm64-v8a armeabi x86_64 mips, mips64
28. Universal Apk armeabi-v7a Most common x86 All Intel devices arm64-v8a armeabi x86_64 mips, mips64
29. Universal Apk armeabi-v7a Most common x86 All Intel devices arm64-v8a Can run armeabi-v7a armeabi x86_64 mips, mips64
30. Universal Apk armeabi-v7a Most common x86 All Intel devices arm64-v8a Can run armeabi-v7a armeabi API Level < 19 x86_64 mips, mips64
31. Universal Apk armeabi-v7a Most common x86 All Intel devices arm64-v8a Can run armeabi-v7a armeabi API Level < 19 x86_64 No Android devices mips, mips64 No Android devices
32. Universal Apk armeabi-v7a Most common x86 All Intel devices arm64-v8a Can run armeabi-v7a armeabi API Level < 19 x86_64 No Android devices mips, mips64 No Android devices WWFD?
33. Universal Apk armeabi-v7a Most common x86 All Intel devices arm64-v8a Can run armeabi-v7a armeabi API Level < 19 x86_64 No Android devices mips, mips64 No Android devices
34. Universal Apk Option #1 armeabi-v7a Most common x86 All Intel devices arm64-v8a Can run armeabi-v7a armeabi API Level < 19 x86_64 No Android devices mips, mips64 No Android devices
35. ABI Split android { splits { abi { enable true reset() include "x86", "armeabi-v7a" universalApk false } } }
36. android { splits { abi { enable true reset() include "x86", "armeabi-v7a" universalApk false } } } ABI Split WWGD?
37. android { splits { abi { enable true reset() include "x86", "armeabi-v7a" universalApk false } } } ABI Split
38. android { splits { abi { enable true reset() include "x86", "armeabi-v7a" universalApk false } } } ABI Split
39. ABI Split android { splits { abi { enable true reset() include "x86", "armeabi-v7a" universalApk false } } } Option #2
40. ABI Split Universal Apk
41. ABI Split Universal Apk
42. ABI Split Universal Apk
43. ABI Split Universal Apk
44. ABI Split Universal Apk
45. W W YD? ABI Split Universal Apk
46. Universal Apk armeabi-v7a Most common x86 All Intel devices arm64-v8a Can run armeabi-v7a armeabi API Level < 19 x86_64 No Android devices mips, mips64 No Android devices
47. Generate trimmed video Compile video editing library Implement trimmer screen
48. Implement Trimmer Screen
49. Trim
50. Trim frame
51. Trim frame trim window
52. Trim frame play-head trim window
53. Trim frame play-head trim window seek()
54. TextureView Trim
55. TextureView MediaPlayer Trim
56. TextureView MediaPlayer start() stop() Trim
57. TextureView MediaPlayer Trim TimelineView
58. TextureView MediaPlayer Trim TimelineView seekTo(time) getCurrentPoisition()
59. TextureView MediaPlayer Trim TimelineView seekTo(time) getCurrentPoisition() VideoMetaDataRetriever getFrameAt(time)
60. TextureView MediaPlayer Trim TimelineView VideoMetaDataRetriever
61. TextureView MediaPlayer Trim TimelineView start() VideoMetaDataRetriever
62. TextureView MediaPlayer Trim TimelineView start() VideoMetaDataRetriever // updates
63. TextureView MediaPlayer Trim TimelineView start() VideoMetaDataRetriever getFram eAt(tim e) // updates
64. TextureView MediaPlayer Trim TimelineView start() // updates VideoMetaDataRetriever getFram eAt(tim e) getCurrentPoisition()
65. TextureView MediaPlayer Trim TimelineView start() // updates VideoMetaDataRetriever getFram eAt(tim e) getCurrentPoisition()
66. TextureView MediaPlayer Trim TimelineView start() // updates VideoMetaDataRetriever getFram eAt(tim e) getCurrentPoisition() seekTo(begin)
67. TextureView MediaPlayer Trim TimelineView start() // updates VideoMetaDataRetriever getFram eAt(tim e) getCurrentPoisition() seekTo(begin) // interacts
77. sync sync sync
78. sync sync sync diff diff diff diff diff
79. sync sync sync diff diff diff diff diff seekTo( )
80. sync sync sync diff diff diff diff diff seekTo( , SEEK_PREVIOUS_SYNC)
81. sync sync sync diff diff diff diff diff
82. sync sync sync diff diff diff diff diff
83. sync sync sync diff diff diff diff diff
84. sync sync sync diff diff diff diff diff // ”freeze” the display
85. sync sync sync diff diff diff diff diff // ”freeze” the display TextureView ImageView
86. sync sync sync diff diff diff diff diff // ”freeze” the display TextureView ImageView // copy Bitmap
87. sync sync sync diff diff diff diff diff // ”freeze” the display // ”uncover” display TextureView ImageView
88. sync sync sync diff diff diff diff diff // ”freeze” the display // ”uncover” display Option #1 TextureView ImageView
97. Option #2
98. Freeze Display Double Buffering
102. Generate trimmed video Compile video editing library Implement trimmer screen
103. Generate Trimmed Video
104. input.mp4 trimmed.mp4
105. Runtime.getRuntime().exec( // your command goes here ).waitFor();
106. Runtime.getRuntime().exec( "ffmpeg –i " + inputFile + " -ss " + startTime + " –to " + endTime + " –y " + outputFile ).waitFor();
107. final Process process = Runtime.getRuntime().exec( "ffmpeg –i " + inputFile + " -ss " + startTime + " –to " + endTime + " –y " + outputFile ); process.getInputStream(); // stdout process.getErrorStream(); // stderr process.getOutputStream(); // stdin process.waitFor();
108. final Process process = Runtime.getRuntime().exec( "ffmpeg –i " + inputFile + " -ss " + startTime + " –to " + endTime + " –y " + outputFile ); process.getInputStream(); // stdout process.getErrorStream(); // stderr process.getOutputStream(); // stdin process.waitFor();
109. final Process process = Runtime.getRuntime().exec( "ffmpeg –i " + inputFile + " -ss " + startTime + " –to " + endTime + " –y " + outputFile ); process.getInputStream(); // stdout process.getErrorStream(); // stderr process.getOutputStream(); // stdin process.waitFor();
111. CPU
112. context.getFilesDir() + "ffmpeg" /data/data/com.video.msiddeek/ffmpeg CPU
113. context.getFilesDir() + "ffmpeg" /data/data/com.video.msiddeek/ffmpeg CPU chmod +x
114. context.getFilesDir() + "ffmpeg" /data/data/com.video.msiddeek/ffmpeg CPU chmod +x Runtime.getRuntime().exec( "/data/data/com.video.msiddeek/ffmpeg -version” ).waitFor();
115. Permissions <?xml version="1.0" encoding="utf-8"?> <manifest …> … ??? </manifest>
116. Permissions <?xml version="1.0" encoding="utf-8"?> <manifest …> … </manifest>
117. input.mp4 output.mp4 trim input.mp4
118. input.mp4 output.mp4 trim input.mp4 Option #1
119. input.mp4 part0 part1
120. input.mp4 part0 part1 trim part0.mp4 trim part1.mp4
121. input.mp4 part0 part1 trim part0.mp4 trim part1.mp4 concat output.mp4
122. input.mp4 part0 part1 trim part0.mp4 trim part1.mp4 concat output.mp4 concurrent
123. input.mp4 part0 part1 trim part0.mp4 trim part1.mp4 copy concat output.mp4 concurrent cheap
124. input.mp4 part0 part1 trim part0.mp4 trim part1.mp4 copy concat output.mp4 concurrent cheap Option #2
125. Generate trimmed video Compile video editing library Implement trimmer screen
126. • https://github.com/Yelp/ffmpeg-android • irc://irc.freenode.net/ffmpeg • https://tldrlegal.com • http://apkmirror.com • https://android.jlelse.eu/controlling-apk-size-when- using-native-libraries-45c6c0e5b70a Useful Links
127. www.yelp.com/careers We're Hiring!
128. @YelpEngineering fb.com/YelpEngineers engineeringblog.yelp.com github.com/yelp