Android Makers 2022 - Explaining Scoped Storage

Transcript

1. Android Makers 2022 Explaining Storage on Android Yacine Rezgui (@yrezgui)
2. Agenda Storage Concepts Storage APIs ModernStorage
3. Storage Architecture
4. Storage Architecture
5. Storage Architecture
6. Storage Architecture
7. Storage Architecture
8. Storage Architecture
9. Storage Architecture
10. Storage Architecture
11. Internal • Located on the primary volume (best for performance and availability) • Files are private to other apps • Files are deleted when app is uninstalled • Some devices have limited space on the primary volume ◦ /data/user/0/APP_ID/files External • Files are deleted when app is uninstalled • Files can’t be read or modified by other apps on Android 11+ ◦ /storage/emulated/0/Android/data/APP_ID/files ◦ /storage/17FD-1312/Android/data/APP_ID/files Shared • Accessible to other apps • Files are kept when app is uninstalled ◦ /storage/emulated/0/DCIM Types of Storage Present on all volumes
12. Back in the days Any files and folders (except internal) were accessible Scoped Storage Scoped access only to media files & SAF for document files Pre 10 Android 10+ History of Storage on Android
13. • Unrestricted access to your own app data • Unrestricted media and downloads contribution • Runtime permission only gives read access to media • User confirmation required for modifying media • Location metadata gated by new permission Scoped Storage (Android 10) No permission needed
14. • Enabled file path APIs • Bulk media modifications APIs • All Files Access permission • Private external app storage Scoped Storage Improvements (Android 11+)
15. Back in the days Any files and folders (except internal) were accessible Future Storage usage transparency & privacy centric APIs Scoped Storage Scoped access only to media files & SAF for document files Pre 10 Android 10+ Android 13+ History of Storage on Android
16. • READ_EXTERNAL_STORAGE & WRITE_EXTERNAL_STORAGE are deprecated • New permissions for each media type ◦ READ_MEDIA_IMAGES ◦ READ_MEDIA_VIDEO ◦ READ_MEDIA_AUDIO • No permission dialog shown to the user if existing storage permissions were granted already on a device New storage permissions (Target Android 13+) Only one dialog shown when requested together
17. No permission required & a better UX to access photos & videos Will include cloud providers like Google Photos & backported to Android 11 & 12 Continuously improved through Google Play System Update Photo Picker
18. Storage APIs
19. Add internal file (not visible to other apps) // App internal folder val target = context.filesDir // If you prefer files to not be backed up val target = context.noBackupFilesDir File(target, "config.txt").bufferedWriter().use { it.write("Here's the config content") }
20. Add internal file (not visible to other apps) // App internal folder val target = context.filesDir // If you prefer files to not be backed up val target = context.noBackupFilesDir File(target, "config.txt").bufferedWriter().use { it.write("Here's the config content") }
21. Add internal file (not visible to other apps) // App internal folder val target = context.filesDir // If you prefer files to not be backed up val target = context.noBackupFilesDir File(target, "config.txt").bufferedWriter().use { it.write("Here's the config content") }
22. Add internal file (not visible to other apps) // App internal folder val target = context.filesDir // If you prefer files to not be backed up val target = context.noBackupFilesDir File(target, "config.txt").bufferedWriter().use { it.write("Here's the config content") }
23. Add internal file (not visible to other apps) // App internal folder val target = context.filesDir // If you prefer files to not be backed up val target = context.noBackupFilesDir File(target, "config.txt").bufferedWriter().use { it.write("Here's the config content") }
24. Add big file val target = if (context.filesDir.usableSpace > fileSize) { context.filesDir } else { context.getExternalFilesDirs(null).find { it.usableSpace > fileSize } } ?: throw IOException("Not enough space")
25. Add big file val target = if (context.filesDir.usableSpace > fileSize) { context.filesDir } else { context.getExternalFilesDirs(null).find { it.usableSpace > fileSize } } ?: throw IOException("Not enough space")
26. • System ContentProvider • Indexes all files on shared storage • Indexing running in the background automatically unless triggered manually • Queryable via the ContentResolver MediaStore
27. Query MediaStore - Create a MediaFile class data class MediaFile( val filename: String, val size: Long, val mimeType: String, val path: String, val uri: Uri )
28. Query MediaStore - How to use the ContentResolver ContentResolver.query( uri: Uri, // Collection or item to be queried projection: Array<String>?, // List of properties we want to fetch selection: String?, // Filter query selectionArgs: Array<String>?, // Filter arguments sortOrder: String? // Sort order )
29. Query MediaStore - Query arguments val fileCollection = MediaStore.Files.getContentUri("external") val projection = arrayOf(_ID, DISPLAY_NAME, SIZE, MEDIA_TYPE, MIME_TYPE, DATA) val sortOrder = if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) { "$GENERATION_ADDED DESC" } else { "$DATE_ADDED DESC" }
30. Query MediaStore - Execute query val cursor = contentResolver.query( fileCollection, projection, "$MEDIA_TYPE = ? OR $MEDIA_TYPE = ?", arrayOf(MEDIA_TYPE_IMAGE.toString(), MEDIA_TYPE_VIDEO.toString()), sortOrder )
31. Query MediaStore - Get column indexes val idColumn = cursor.getColumnIndexOrThrow(_ID) val displayNameColumn = cursor.getColumnIndexOrThrow(DISPLAY_NAME) val sizeColumn = cursor.getColumnIndexOrThrow(SIZE) val mediaTypeColumn = cursor.getColumnIndexOrThrow(MEDIA_TYPE) val mimeTypeColumn = cursor.getColumnIndexOrThrow(MIME_TYPE) val dataColumn = cursor.getColumnIndexOrThrow(DATA) val imageCollection = MediaStore.Images.Media.getContentUri("external") val videoCollection = MediaStore.Video.Media.getContentUri("external")
32. Query MediaStore - Loop through the cursor cursor.use { while (cursor.moveToNext()) { val id = cursor.getInt(idColumn) val mediaType = cursor.getInt(mediaTypeColumn) val contentUri = if (mediaType == MEDIA_TYPE_IMAGE) { ContentUris.withAppendedId(imageCollection, id.toLong()) } else { ContentUris.withAppendedId(videoCollection, id.toLong()) } // ...
33. Query MediaStore - Creating a MediaFile instance cursor.use { while (cursor.moveToNext()) { // ... val file = MediaFile( filename = cursor.getString(displayNameColumn), size = cursor.getLong(sizeColumn), mimeType = cursor.getString(mimeTypeColumn), path = cursor.getString(dataColumn), uri = contentUri) println(file)
34. • Allow selection of any type of files • Selection from local files but also from 3rd party providers like Google Drive • Use the ACTION_OPEN_DOCUMENT intent • • Available from Android KitKat (API 19) without any permission Storage Access Framework
35. Storage Access Framework - Usage val picker = rememberLauncherForActivityResult(OpenDocument()) { uri -> if(uri == null) return context.contentResolver.openInputStream(uri).use { input -> File(context.filesDir, "receipt.pdf").outputStream().use { output -> input?.copyTo(output) } } } // Usage picker.launch(arrayOf("application/pdf"))
36. Photo Picker - Usage val picker = rememberLauncherForActivityResult(pickMultipleVisualMedia(3)) { Log.d("Got media files: $it") } // Usage pickVisualMedia.launch( PickVisualMediaRequest(PickVisualMedia.ImageOnly) )
37. Photo Picker - Usage val picker = rememberLauncherForActivityResult(pickMultipleVisualMedia(3)) { Log.d("Got media files: $it") } // Usage pickVisualMedia.launch( PickVisualMediaRequest(PickVisualMedia.SingleMimeType("image/gif")) )
38. Photo Picker - Usage val picker = rememberLauncherForActivityResult(pickMultipleVisualMedia(3)) { Log.d("Got media files: $it") } // Usage pickMultipleVisualMedia.launch( PickVisualMediaRequest(PickVisualMedia.ImageAndVideo) )
39. ModernStorage
40. • Support library for storage, compatible with API 21+ • Set of modules simplifying interactions (permissions, file content reading & writing) • File interactions based on Okio Filesystem API ◦ Single API for java.io, MediaStore & Storage Access Framework ModernStorage
41. State of storage permissions
42. State of storage permissions 😱
43. ModernStorage - Read audio & video val storagePermissions = StoragePermissions(context) // Check if the app can read video & audio files created by all apps storagePermissions.hasAccess( action = Action.READ, types = listOf(FileType.Video, FileType.Audio), createdBy = StoragePermissions.CreatedBy.AllApps )
44. ModernStorage - Request storage permissions // Request permission to read video & audio files created by all apps requestAccess.launch( RequestAccess.Args( action = Action.READ, types = listOf( StoragePermissions.FileType.Video, StoragePermissions.FileType.Audio ), createdBy = StoragePermissions.CreatedBy.AllApps ), )
45. Get file metadata across all storage APIs override fun metadataOrNull(path: Path): FileMetadata? { val uri = path.toUri() return when (uri.authority) { null -> fetchMetadataFromPhysicalFile(path) MediaStore.AUTHORITY -> fetchMetadataFromMediaStore(path, uri) else -> fetchMetadataFromDocumentProvider(path, uri) } } private fun fetchMetadataFromPhysicalFile(path: Path): FileMetadata? { val file = path.toFile() val isRegularFile = file.isFile val isDirectory = file.isDirectory val lastModifiedAtMillis = file.lastModified() val size = file.length() if (!isRegularFile && !isDirectory && lastModifiedAtMillis == 0L && size == 0L && !file.exists() ) { return null } val fileExtension: String = MimeTypeMap.getFileExtensionFromUrl(file.toString()) val mimeType = MimeTypeMap.getSingleton().getMimeTypeFromExtension(fileExtension.lowercase(Locale.getDefault())) val extras = mutableMapOf( Path::class to path, MetadataExtras.DisplayName::class to MetadataExtras.DisplayName(file.name), MetadataExtras.FilePath::class to MetadataExtras.FilePath(file.absolutePath), ) if (mimeType != null) extras[MetadataExtras.MimeType::class] = MetadataExtras.MimeType(mimeType) return FileMetadata( isRegularFile = isRegularFile, isDirectory = isDirectory, symlinkTarget = null, size = size, createdAtMillis = null, lastModifiedAtMillis = lastModifiedAtMillis, lastAccessedAtMillis = null, extras = extras ) } private fun fetchMetadataFromMediaStore(path: Path, uri: Uri): FileMetadata? { if (uri.pathSegments.firstOrNull().isNullOrBlank()) { return null } val isPhotoPickerUri = uri.pathSegments.firstOrNull() == "picker" val projection = if (isPhotoPickerUri) { arrayOf( MediaStore.MediaColumns.DATE_TAKEN, MediaStore.MediaColumns.DISPLAY_NAME, MediaStore.MediaColumns.MIME_TYPE, MediaStore.MediaColumns.SIZE, MediaStore.MediaColumns.DATA, ) } else { arrayOf( MediaStore.MediaColumns.DATE_ADDED, MediaStore.MediaColumns.DATE_MODIFIED, MediaStore.MediaColumns.DISPLAY_NAME, MediaStore.MediaColumns.MIME_TYPE, MediaStore.MediaColumns.SIZE, MediaStore.MediaColumns.DATA, ) } val cursor = contentResolver.query( uri, projection, null, null, null ) ?: return null cursor.use { cursor -> if (!cursor.moveToNext()) { return null } val createdTime: Long var lastModifiedTime: Long? = null if (isPhotoPickerUri) { createdTime = cursor.getLong(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DATE_TAKEN)) } else { createdTime = cursor.getLong(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DATE_ADDED)) lastModifiedTime = cursor.getLong(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DATE_MODIFIED)) } val displayName = cursor.getString(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DISPLAY_NAME)) val mimeType = cursor.getString(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.MIME_TYPE)) val size = cursor.getLong(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.SIZE)) val filePath = cursor.getString(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DATA)) return FileMetadata( isRegularFile = true, isDirectory = false, symlinkTarget = null, size = size, createdAtMillis = createdTime, lastModifiedAtMillis = lastModifiedTime, lastAccessedAtMillis = null, extras = mapOf( Path::class to path, Uri::class to uri, MetadataExtras.DisplayName::class to MetadataExtras.DisplayName(displayName), MetadataExtras.MimeType::class to MetadataExtras.MimeType(mimeType), MetadataExtras.FilePath::class to MetadataExtras.FilePath(filePath), ) ) } } private fun fetchMetadataFromDocumentProvider(path: Path, uri: Uri): FileMetadata? { val cursor = contentResolver.query( uri, arrayOf( DocumentsContract.Document.COLUMN_LAST_MODIFIED, DocumentsContract.Document.COLUMN_DISPLAY_NAME, DocumentsContract.Document.COLUMN_MIME_TYPE, DocumentsContract.Document.COLUMN_SIZE ), null, null, null ) ?: return null cursor.use { cursor -> if (!cursor.moveToNext()) { return null } // DocumentsContract.Document.COLUMN_LAST_MODIFIED val lastModifiedTime = cursor.getLong(0) // DocumentsContract.Document.COLUMN_DISPLAY_NAME val displayName = cursor.getString(1) // DocumentsContract.Document.COLUMN_MIME_TYPE val mimeType = cursor.getString(2) // DocumentsContract.Document.COLUMN_SIZE val size = cursor.getLong(3) val isFolder = mimeType == DocumentsContract.Document.MIME_TYPE_DIR || mimeType == DocumentsContract.Root.MIME_TYPE_ITEM return FileMetadata( isRegularFile = !isFolder, isDirectory = isFolder, symlinkTarget = null, size = size, createdAtMillis = null, lastModifiedAtMillis = lastModifiedTime, lastAccessedAtMillis = null,
46. Get file metadata across all storage APIs override fun metadataOrNull(path: Path): FileMetadata? { val uri = path.toUri() return when (uri.authority) { null -> fetchMetadataFromPhysicalFile(path) MediaStore.AUTHORITY -> fetchMetadataFromMediaStore(path, uri) else -> fetchMetadataFromDocumentProvider(path, uri) } } private fun fetchMetadataFromPhysicalFile(path: Path): FileMetadata? { val file = path.toFile() val isRegularFile = file.isFile val isDirectory = file.isDirectory val lastModifiedAtMillis = file.lastModified() val size = file.length() if (!isRegularFile && !isDirectory && lastModifiedAtMillis == 0L && size == 0L && !file.exists() ) { return null } val fileExtension: String = MimeTypeMap.getFileExtensionFromUrl(file.toString()) val mimeType = MimeTypeMap.getSingleton().getMimeTypeFromExtension(fileExtension.lowercase(Locale.getDefault())) val extras = mutableMapOf( Path::class to path, MetadataExtras.DisplayName::class to MetadataExtras.DisplayName(file.name), MetadataExtras.FilePath::class to MetadataExtras.FilePath(file.absolutePath), ) if (mimeType != null) extras[MetadataExtras.MimeType::class] = MetadataExtras.MimeType(mimeType) return FileMetadata( isRegularFile = isRegularFile, isDirectory = isDirectory, symlinkTarget = null, size = size, createdAtMillis = null, lastModifiedAtMillis = lastModifiedAtMillis, lastAccessedAtMillis = null, extras = extras ) } private fun fetchMetadataFromMediaStore(path: Path, uri: Uri): FileMetadata? { if (uri.pathSegments.firstOrNull().isNullOrBlank()) { return null } val isPhotoPickerUri = uri.pathSegments.firstOrNull() == "picker" val projection = if (isPhotoPickerUri) { arrayOf( MediaStore.MediaColumns.DATE_TAKEN, MediaStore.MediaColumns.DISPLAY_NAME, MediaStore.MediaColumns.MIME_TYPE, MediaStore.MediaColumns.SIZE, MediaStore.MediaColumns.DATA, ) } else { arrayOf( MediaStore.MediaColumns.DATE_ADDED, MediaStore.MediaColumns.DATE_MODIFIED, MediaStore.MediaColumns.DISPLAY_NAME, MediaStore.MediaColumns.MIME_TYPE, MediaStore.MediaColumns.SIZE, MediaStore.MediaColumns.DATA, ) } val cursor = contentResolver.query( uri, projection, null, null, null ) ?: return null cursor.use { cursor -> if (!cursor.moveToNext()) { return null } val createdTime: Long var lastModifiedTime: Long? = null if (isPhotoPickerUri) { createdTime = cursor.getLong(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DATE_TAKEN)) } else { createdTime = cursor.getLong(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DATE_ADDED)) lastModifiedTime = cursor.getLong(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DATE_MODIFIED)) } val displayName = cursor.getString(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DISPLAY_NAME)) val mimeType = cursor.getString(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.MIME_TYPE)) val size = cursor.getLong(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.SIZE)) val filePath = cursor.getString(cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DATA)) return FileMetadata( isRegularFile = true, isDirectory = false, symlinkTarget = null, size = size, createdAtMillis = createdTime, lastModifiedAtMillis = lastModifiedTime, lastAccessedAtMillis = null, extras = mapOf( Path::class to path, Uri::class to uri, MetadataExtras.DisplayName::class to MetadataExtras.DisplayName(displayName), MetadataExtras.MimeType::class to MetadataExtras.MimeType(mimeType), MetadataExtras.FilePath::class to MetadataExtras.FilePath(filePath), ) ) } } private fun fetchMetadataFromDocumentProvider(path: Path, uri: Uri): FileMetadata? { val cursor = contentResolver.query( uri, arrayOf( DocumentsContract.Document.COLUMN_LAST_MODIFIED, DocumentsContract.Document.COLUMN_DISPLAY_NAME, DocumentsContract.Document.COLUMN_MIME_TYPE, DocumentsContract.Document.COLUMN_SIZE ), null, null, null ) ?: return null cursor.use { cursor -> if (!cursor.moveToNext()) { return null } // DocumentsContract.Document.COLUMN_LAST_MODIFIED val lastModifiedTime = cursor.getLong(0) // DocumentsContract.Document.COLUMN_DISPLAY_NAME val displayName = cursor.getString(1) // DocumentsContract.Document.COLUMN_MIME_TYPE val mimeType = cursor.getString(2) // DocumentsContract.Document.COLUMN_SIZE val size = cursor.getLong(3) val isFolder = mimeType == DocumentsContract.Document.MIME_TYPE_DIR || mimeType == DocumentsContract.Root.MIME_TYPE_ITEM return FileMetadata( isRegularFile = !isFolder, isDirectory = isFolder, symlinkTarget = null, size = size, createdAtMillis = null, lastModifiedAtMillis = lastModifiedTime, lastAccessedAtMillis = null, 😱
47. ModernStorage - Get file metadata val filesystem = AndroidFileSystem(context) val uri = Uri.parse("content://media/external/images/media/47") val metadata = fileSystem.metadataOrNull(uri.toOkioPath())
48. ModernStorage - Get file metadata val filesystem = AndroidFileSystem(context) val uri = Uri.parse("content://media/external/images/media/47") val metadata = fileSystem.metadataOrNull(uri.toOkioPath()) println("displayName: ${metadata.extra(DisplayName::class)}") println("size: ${metadata.size}") println("mimeType: ${metadata.extra(MimeType::class)}") println("lastModifiedAtMillis: ${metadata.lastModifiedAtMillis}")
49. ModernStorage - Copy image to internal storage val filesystem = AndroidFileSystem(context) // Image taken from MediaStore val imageUri = Uri.parse("content://media/external/images/media/47") // Target file inside the internal storage val target = File(appContext.filesDir, "cool-image.jpg")
50. ModernStorage - Copy image to internal storage val filesystem = AndroidFileSystem(context) // Image taken from MediaStore val imageUri = Uri.parse("content://media/external/images/media/47") // Target file inside the internal storage val target = File(appContext.filesDir, "cool-image.jpg") // Copying a file is now done in one line! fileSystem.copy(imageUri.toOkioPath(), target.toOkioPath())
51. goo.gle/modernstorage Where can I grab ModernStorage?
52. Please do 🙏 goo.gle/modernstorage Can I add a feature request?
53. Any questions? Ping me on @yrezgui Thank you for listening to my talk 🙌