When MediaProjection Isn't Enough - Building a Custom In-App Session Recorder (Droidcon Berlin 2025)

Debugging user issues gets so much easier when you can see exactly what they experienced. While off-the-shelf solutions exist to integrate screen recording into our apps, they didn’t give us the quality, flexibility, or control we needed. So, our team built our own native, in-app session recording tool for Android.

In this talk, we’ll share why we decided to go in-house, how we tackled the technical and UX hurdles along the way, and what we learned about recording high-quality video and audio data.

We’ll dive into tricky bits like exploring - and discarding - the MediaProjection API, handling codecs and muxers across devices, designing recording controls that stay out of the user’s way, playing back recordings, and managing background uploads without draining your users’ battery or patience.

Whether you’re exploring advanced media capture, better bug reporting, or just want a peek behind a surprisingly complex Android feature, you’ll come away with practical insights to help you build your own solutions with more confidence.

Join us for a real-world look at building complex media pipelines on Android!

By the end of the talk, you’ll have a better grasp on:

- How robust in-app screen recording can improve QA and support
- UX ideas for recording controls that stay out of your users’ way
- Tips for working with the MediaProjection API, manual screen and audio recording, encoding, and muxing
- How to handle tricky edge cases caused by device and OS fragmentation in terms of screen recording
- Ideas for future features that in-app screen recording can unlock

Transcript

1. When MediaProjection Isn ' t Enough Building a Custom In-App Session Recorder Francisco Franco Staff Android Engineer István Juhos Staff Android Engineer
2. Who we are
3. • Last-mile delivery management platform • Providing efficient route optimization to save time on deliveries • Building apps and services tailored to the use cases of last-mile delivery drivers and courier companies Who we are
4. • Circuit Route Planner • Mobile app built for delivery drivers that helps them finish work earlier Our apps and services
5. • Circuit Route Planner • Mobile app built for delivery drivers that helps them finish work earlier • Circuit for Teams • Delivery management for multi-driver teams • Web app for depot managers and dispatchers • Mobile app for drivers Our apps and services
6. Our apps and services
7. Our apps and services
8. • Treating our users as a community What we also do
9. • Treating our users as a community • Listening to our community What we also do
10. • Treating our users as a community • Listening to our community • Prioritizing, planning and implementing requested features What we also do
11. • Treating our users as a community • Listening to our community • Prioritizing, planning and implementing requested features • Fixing reported issues as soon as possible What we also do
12. User feedback
13. • Direct contact with sales and PMs Receiving feedback
14. • Direct contact with sales and PMs • Discord Driver Community with 2500+ members Receiving feedback
15. • Direct contact with sales and PMs • Discord Driver Community with 2500+ members • In-app, via Intercom Receiving feedback
16. • Intercom • Support team monitoring and triaging reported issues and suggestions, handing them over to product and devs • We encourage users to send screenshots and screen recordings describing issues Handling feedback
17. • Intercom • Support team monitoring and triaging reported issues and suggestions, handing them over to product and devs • We encourage users to send screenshots and screen recordings describing issues Handling feedback
18. • Many options available for recording Users sharing screen recordings
19. • Many options available for recording • 3rd party apps supporting Android 5+ Users sharing screen recordings
20. • Many options available for recording • 3rd party apps supporting Android 5+ • Video recording app -> start recording -> reproduce issue in Circuit app -> stop recording -> attach file in Intercom 😵💫 Users sharing screen recordings
21. • Built-in screen recording available from Android 11 / API 30 Users sharing screen recordings
22. Users sharing screen recordings
23. Let ' s improve the video feedback experience! 💡
24. • Initial goals • Reduce friction • Enable recording of entire delivery sessions Improving the video feedback experience
25. Let ' s use what Android has to offer
26. Let ' s use what Android has to offer developer.android.com/media/grow/media-projection
27. • Introduced in Android 5 ( API 21 ) to capture the contents of the device ' s screen MediaProjection API
28. • Introduced in Android 5 ( API 21 ) to capture the contents of the device ' s screen • Our minApi is 23 ✅ MediaProjection API
29. • Introduced in Android 5 ( API 21 ) to capture the contents of the device ' s screen • Our minApi is 23 ✅ • But... MediaProjection API
30. • Introduced in Android 5 ( API 21 ) to capture the contents of the device ' s screen • Whole-screen recording • Limited audio capture (mic only) • A dialog is shown to start every session 😕 MediaProjection API
31. • Introduced in Android 5 ( API 21 ) to capture the contents of the device ' s screen • Whole-screen recording • Limited audio capture (mic only) • A dialog is shown to start every session 😕 MediaProjection API
32. MediaProjection API • Android 10 ( API 29 ) • Added system audio capture (AudioPlaybackCaptureConfig)
33. MediaProjection API • Android 10 ( API 29 ) • Added system audio capture (AudioPlaybackCaptureConfig) • Android 12 ( API 31 ) • Restrictions tighten: foreground service requirement • Privacy indicators • Audio capture opt-in per app
34. MediaProjection API • Android 14 ( API 34 ) • Stronger privacy • Blur sensitive content • Watermarking with some OEMs • Behavior changes per vendor (Samsung vs Pixel vs Chinese OEMs)
35. • Android 14 QPR2 ( API 34 ) • single app sharing 🥳 MediaProjection API
36. • Android 14 QPR2 ( API 34 ) • single app sharing... but... 😕 MediaProjection API
37. • Android 14 QPR2 ( API 34 ) • single app sharing... but... 😕 MediaProjection API
38. • Android 14 QPR2 ( API 34 ) • single app sharing... but... 😕 MediaProjection API
39. MediaProjection API • Android 15 QPR1 ( API 35 ) • Indicator chip on the status bar
40. MediaProjection API
41. • Inconsistent UX across OS versions 😕 MediaProjection API - conclusions
42. • Inconsistent UX across OS versions 😕 • Uses system dialogs MediaProjection API - conclusions
43. • Inconsistent UX across OS versions 😕 • Uses system dialogs • Limited configuration MediaProjection API - conclusions
44. • Inconsistent UX across OS versions 😕 • Uses system dialogs • Limited configuration • More friction 😵💫 MediaProjection API - conclusions
45. • Inconsistent UX across OS versions 😕 • Uses system dialogs • Limited configuration • More friction 😵💫 MediaProjection API - conclusions
46. Building the custom recorder 🏗
47. • New technical requirements • Only record our screens to improve privacy Building the custom recorder
48. • New technical requirements • Only record our screens to improve privacy • Audio capture is essential, but voluntary Building the custom recorder
49. • New technical requirements • Only record our screens to improve privacy • Audio capture is essential, but voluntary • Taps should show up on the recording Building the custom recorder
50. • New technical requirements • Only record our screens to improve privacy • Audio capture is essential, but voluntary • Taps should show up on the recording • Good quality video Building the custom recorder
51. • New technical requirements • Only record our screens to improve privacy • Audio capture is essential, but voluntary • Taps should show up on the recording • Good quality video • Optimized video file size Building the custom recorder
52. • UX requirements • Deep link support to start the recording Building the custom recorder
53. • UX requirements • Deep link support to start the recording • Built-in UI flow Building the custom recorder
54. • UX requirements • Deep link support to start the recording • Built-in UI flow Building the custom recorder
55. • Single-activity* app (*mostly) • Subclass and add recording capabilities • Touch event recording as a bonus ✅ • Convert legacy Views to Compose and capture the composition everywhere Figuring out the entry point
56. • Capture touches • Capture audio from the mic - with the ability to pause anytime • Capture the composition into a bitmap @ 60fps The routine is "simple" 👇 🎤 🎦
57. Combining everything together 👇 🎤 🎦 🎞
58. Combining everything together 🎞 👇 🎤 🎦
59. Combining everything together 🎞 👇 🎤 🎦
60. Combining everything together 🎤 🎤 🎤 🚫
61. Combining everything together 🎞 👇 🎤 🎦
62. Combining everything together 🎞 👇 🎤 🎦
65. Reflection! 🪞😱 - WindowManagerGlobal fun getWmgReflection(): WmgReflection? { wmgReflection ?. let { return it } wmgReflection = try { val klass = Class.forName("android.view.WindowManagerGlobal") val method = klass.getMethod("getInstance") val field = try { klass.getDeclaredField("mRoots").apply { isAccessible = true } } catch (_: NoSuchFieldException) { klass.getDeclaredField("mViews").apply { isAccessible = true } } WmgReflection(klass, method, field) } catch (_: Exception) { WmgReflection(null, null, null) } return wmgReflection }
66. Reflection! 🪞😱 - getting all root surfaces fun getAllRoots(): List<Pair<View, Surface >> { val refl = getWmgReflection() if (refl ?. wmgClass == null || refl.getInstance == null || refl.rootsField == null) { return emptyList() } val result = mutableListOf<Pair<View, Surface >> () val roots = try { val wmg = refl.getInstance.invoke(null) refl.rootsField.get(wmg) as? List <*> ?: emptyList<Pair<View, Surface >> () } catch (_: Exception) { emptyList<Pair<View, Surface >> () } roots.forEach { rootObj -> ... val decor = runCatching { val viewField = rootObj :: class.java.getDeclaredField("mView") .apply { isAccessible = true } viewField.get(rootObj) as? View }.getOrNull() val surface = runCatching { val surfField = rootObj :: class.java.getDeclaredField("mSurface") .apply { isAccessible = true } surfField.get(rootObj) as? Surface }.getOrNull() if (decor != null && surface != null && decor.isAttachedToWindow) { result += decor to surface } } return result }
67. Reflection! 🪞😱 - capturing all window frames suspend fun captureAllWindowsFrame(windowCompositeBitmap: Bitmap) { val canvas = Canvas(windowCompositeBitmap) // For each root, copy its Surface and draw at its on‐screen position for ((index, decorSurface) in getAllRoots().withIndex()) { } }
68. We got it working 🤞
69. We got it working... but we were far from done... 😬
70. • The video encoder codec is super picky about the resolution • The classic "works on my device, why isn ' t it working on yours?" First issue - video encoding and resolution resolution: 2n x 2m
71. • Video was super long (like 5 hours) while both audio & video tracks were short Second issue - syncing video and audio
72. • Video was super long (like 5 hours) while both audio & video tracks were short • Audio speed faster than the video Second issue - syncing video and audio
73. • Video was super long (like 5 hours) while both audio & video tracks were short • Audio speed faster than the video • Audio was working but video was totally black Second issue - syncing video and audio
74. • Video was super long (like 5 hours) while both audio & video tracks were short • Audio speed faster than the video • Audio was working but video was totally black • Different behavior across devices & SDK levels Second issue - syncing video and audio 😵💫
75. • The culprit was the property presentationTimeUs • a timestamp in microseconds Second issue - syncing video and audio val presentationTimeUs = 1_000_000L * (totalBytesSubmitted / bytesPerSample) / 44100 MediaCodec.queueInputBuffer(bufferIndex, 0, bytesToWrite, presentationTimeUs, 0)
76. • The culprit was the property presentationTimeUs • a timestamp in microseconds • Software encoding • manual tracking and usage of presentationTimeUs 😵💫 • Hardware encoding • The system takes care of it automagically ✨ Second issue - syncing video and audio
77. • Setting up video and audio tracks and then starting a MediaMuxer is hard to manage Third issue - lifecycle
78. • Setting up video and audio tracks and then starting a MediaMuxer is hard to manage • Assuming you need both video & audio you must only start muxing AFTER both tracks have been added Third issue - lifecycle
79. • Setting up video and audio tracks and then starting a MediaMuxer is hard to manage • Assuming you need both video & audio you must only start muxing AFTER both tracks have been added • Finishing & cleaning the muxing routines is not straightfoward Third issue - lifecycle
80. • Separate tracks, two separate muxers • Use the MediaExtractor API to extract track data from video & audio files Third issue - lifecycle - solution 🏗 🎤 
 🎦 🛤 🛤 MUX 🎞
81. • Audio • Manually queue end of stream • Video • Hardware encoding • Call MediaCodec.signalEndOfInputStream() • Software encoding • Manually queue the end of the stream Fourth issue - ending the streams
82. MediaCodec.queueInputBuffer( bufferIndex, 0, 0, presentationTimeUs, MediaCodec.BUFFER_FLAG_END_OF_STREAM, ) Fourth issue - ending the streams
83. Bonus issue - reading from AudioRecord without mic permissions val audioRecord = AudioRecord.Builder() .setAudioSource(MediaRecorder.AudioSource.VOICE_COMMUNICATION) .setAudioFormat( AudioFormat.Builder() .setEncoding(AudioFormat.ENCODING_PCM_16BIT) .setSampleRate(44100) .setChannelMask(AudioFormat.CHANNEL_IN_MONO) .build(), ).build()
84. Bonus issue - reading from AudioRecord without mic permissions val audioRecord = AudioRecord.Builder() .setAudioSource(MediaRecorder.AudioSource.VOICE_COMMUNICATION) .setAudioFormat( AudioFormat.Builder() .setEncoding(AudioFormat.ENCODING_PCM_16BIT) .setSampleRate(44100) .setChannelMask(AudioFormat.CHANNEL_IN_MONO) .build(), ).build() val readBytes = audioRecord.read(audioBuffer, 0, minBufferSize)
85. Debugging is hard
86. • File size • Refresh rate • Resolution Final concerns
87. Fun fact Turns out, it’s helpful
88. Demo
89. Thank you! getcircuit.com