Getting Started with CameraX

Transcript

1. Getting Started with CameraX @owahltinez @calren24
2. Camera Development can be difficult and frustrating
3. Camera Development can be difficult and frustrating • Supporting different API versions
4. Camera Development can be difficult and frustrating • Supporting different API versions • Providing consistency across devices
5. Camera Development can be difficult and frustrating • Supporting different API versions • Providing consistency across devices • Understanding the complex camera API
6. Camera Development can be difficult and frustrating • Supporting different API versions • Providing consistency across devices • Understanding the complex camera API CameraX support library aims to help solve these problems
7. CameraX • Backwards compatible to L (90% of devices)
8. CameraX • Backwards compatible to L (90% of devices) • Consistent behavior across devices
9. CameraX • Backwards compatible to L (90% of devices) • Consistent behavior across devices • Easy to use APIs
10. Using CameraX results in 70% fewer lines of code vs camera2
11. Preview Image Analysis Capture
12. Preview Image Analysis Capture
13. // configure preview val previewConfig = PreviewConfig.Builder().build()t .setTargetRotation(Surface.ROTATION_180) his as LifecycleOwner, preview)
14. // create preview val previewConfig = PreviewConfig.Builder().build() .setTargetRotation(Surface.ROTATION_180) val preview = Preview(previewConfig)wner, preview)
15. // attach output to view val previewConfig = PreviewConfig.Builder().build() .setTargetRotation(Surface.ROTATION_180) val preview = Preview(previewConfig) preview.setOnPreviewOutputUpdateListener { previewOutput: Preview.PreviewOutput? -> //your code here. e.g. use previewOutput?.surfaceTexture //and post to a GL renderer. } CameraX.bindToLifecycle(this as LifecycleOwner, preview)
16. // attach preview to lifecycle val previewConfig = PreviewConfig.Builder().build() .setTargetRotation(Surface.ROTATION_180) val preview = Preview(previewConfig) preview.setOnPreviewOutputUpdateListener { previewOutput: Preview.PreviewOutput? -> //your code here. e.g. use previewOutput?.surfaceTexture //and post to a GL renderer. } CameraX.bindToLifecycle(this as LifecycleOwner, preview)
17. // display preview on screen val previewConfig = PreviewConfig.Builder().build() .setTargetRotation(Surface.ROTATION_180) val preview = Preview(previewConfig) preview.setOnPreviewOutputUpdateListener { previewOutput: Preview.PreviewOutput? -> //your code here. e.g. use previewOutput?.surfaceTexture //and post to a GL renderer. } CameraX.bindToLifecycle(this as LifecycleOwner, preview)
18. Preview Image Analysis Capture
19. // configure image analysis val imageAnalysisConfig = ImageAnalysisConfig.Builder()
20. // set resolution val imageAnalysisConfig = ImageAnalysisConfig.Builder() .setTargetResolution(Size(1280, 720)) .build()
21. // create image analysis val imageAnalysisConfig = ImageAnalysisConfig.Builder() .setTargetResolution(Size(1280, 720)) .build() val imageAnalysis = ImageAnalysis(imageAnalysisConfig)
22. // attach output val imageAnalysisConfig = ImageAnalysisConfig.Builder() .setTargetResolution(Size(1280, 720)) .build() val imageAnalysis = ImageAnalysis(imageAnalysisConfig) imageAnalysis.setAnalyzer({ image: ImageProxy, rotationDegrees: Int -> val cropRect = image.cropRect // do some analysis })
23. // attach image analysis & preview to lifecycle val imageAnalysisConfig = ImageAnalysisConfig.Builder() .setTargetResolution(Size(1280, 720)) .build() val imageAnalysis = ImageAnalysis(imageAnalysisConfig) imageAnalysis.setAnalyzer({ image: ImageProxy, rotationDegrees: Int -> val cropRect = image.cropRect // do some analysis }) CameraX.bindToLifecycle(this as LifecycleOwner, imageAnalysis, preview)
24. // full setup to process images val imageAnalysisConfig = ImageAnalysisConfig.Builder() .setTargetResolution(Size(1280, 720)) .build() val imageAnalysis = ImageAnalysis(imageAnalysisConfig) imageAnalysis.setAnalyzer({ image: ImageProxy, rotationDegrees: Int -> val cropRect = image.cropRect // do some analysis }) CameraX.bindToLifecycle(this as LifecycleOwner, imageAnalysis, preview)
25. Preview Image Analysis Capture
26. // configure image capture val imageCaptureConfig = ImageCaptureConfig.Builder() .setTargetResolution(Size(1280, 720)) .build()
27. // create image capture val imageCaptureConfig = ImageCaptureConfig.Builder() .setTargetResolution(Size(1280, 720)) .build() val imageCapture = ImageCapture(imageCaptureConfig)
28. // bind all use cases val imageCaptureConfig = ImageCaptureConfig.Builder() .setTargetResolution(Size(1280, 720)) .build() val imageCapture = ImageCapture(imageCaptureConfig) CameraX.bindToLifecycle(this as LifecycleOwner, imageCapture, imageAnalysis, preview)
29. // full setup to capture images val imageCaptureConfig = ImageCaptureConfig.Builder() .setTargetResolution(Size(1280, 720)) .build() val imageCapture = ImageCapture(imageCaptureConfig) CameraX.bindToLifecycle(this as LifecycleOwner, imageCapture, imageAnalysis, preview)
30. // on user action
31. // on user action fun onClick() { val file = File(..) imageCaptureUseCase.takePicture(file, object : ImageCaptureUseCase.OnImageSavedListener { override fun onError(error: ImageCaptureUseCase.UseCaseError, message: String, exc: Throwable?) { //insert your code here } override fun onImageSaved(file: File) { //insert your code here } }) }
32. // on user action save a picture fun onClick() { val file = File(..) imageCapture.takePicture(file, object : ImageCaptureUseCase.OnImageSavedListener { override fun onError(error: ImageCaptureUseCase.UseCaseError, message: String, exc: Throwable?) { //insert your code here } override fun onImageSaved(file: File) { //insert your code here } }) }
33. // save a picture on user action fun onClick() { val file = File(..) imageCapture.takePicture(file, object : ImageCapture.OnImageSavedListener { override fun onError(error: ImageCapture.UseCaseError, message: String, exc: Throwable?) { //insert your code here } override fun onImageSaved(file: File) { //insert your code here } }) }
34. CameraX sample app g.co/camerax/sample CameraX dev mailing list g.co/camerax/developers CameraX StackOverflow tag stackoverflow.com/tags/android-camerax
35. CameraX sample app
36. CameraX sample app • Under Android camera samples GitHub repo
37. CameraX sample app • Under Android camera samples GitHub repo • Monitoring all repo issues and pull requests
38. CameraX sample app • Under Android camera samples GitHub repo • Monitoring all repo issues and pull requests • Sample issues != CameraX library issues
39. CameraX sample app (cont)
40. CameraX sample app (cont) • Very basic indeed
41. CameraX sample app (cont) • Very basic indeed • Preview, image capture and image analysis
42. CameraX sample app (cont) • Very basic indeed • Preview, image capture and image analysis • Future work: vendor extensions and video record
43. CameraX sample app • Very basic indeed • Preview, image capture and image analysis • Future work: vendor extensions and video record
44. Handling state changes beyond Lifecycle
45. Handling state changes beyond Lifecycle • Preview output update (CameraX internals)
46. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle)
47. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle) • Display (180 rotations, display removed / attached)
48. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle) • Display (180 rotations, display removed / attached) • Camera hardware (USB, retractable)
49. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle) • Display (180 rotations, display removed / attached) • Camera hardware (USB, retractable)
50. // Preview output update listener val preview: Preview = ... val viewFinder: TextureView = ... preview.setOnPreviewOutputUpdateListener { previewOutput: Preview.PreviewOutput? -> // Remove it and re-add view (important!) val parent = viewFinder.parent as ViewGroup parent.removeView(viewFinder) parent.addView(viewFinder, 0) // Update internal Surface object viewFinder.surfaceTexture = previewOutput.surfaceTexture // Apply preview output transformations // TODO: Use PreviewOutput textureSize and rotationDegrees here }
51. // Preview output update listener val preview: Preview = ... val viewFinder: TextureView = ... preview.setOnPreviewOutputUpdateListener { previewOutput: Preview.PreviewOutput? -> // Remove it and re-add view (important!) val parent = viewFinder.parent as ViewGroup parent.removeView(viewFinder) parent.addView(viewFinder, 0) // Update internal Surface object viewFinder.surfaceTexture = previewOutput.surfaceTexture // Apply preview output transformations // TODO: Use PreviewOutput textureSize and rotationDegrees here }
52. // Preview output update listener val preview: Preview = ... val viewFinder: TextureView = ... preview.setOnPreviewOutputUpdateListener { previewOutput: Preview.PreviewOutput? -> // Remove it and re-add view (important!) val parent = viewFinder.parent as ViewGroup parent.removeView(viewFinder) parent.addView(viewFinder, 0) // Update internal Surface object viewFinder.surfaceTexture = previewOutput.surfaceTexture // Apply preview output transformations // TODO: Use PreviewOutput textureSize and rotationDegrees here }
53. // Preview output update listener val preview: Preview = ... val viewFinder: TextureView = ... preview.setOnPreviewOutputUpdateListener { previewOutput: Preview.PreviewOutput? -> // Remove it and re-add view (important!) val parent = viewFinder.parent as ViewGroup parent.removeView(viewFinder) parent.addView(viewFinder, 0) // Update internal Surface object viewFinder.surfaceTexture = previewOutput.surfaceTexture // Apply preview output transformations // TODO: Use PreviewOutput textureSize and rotationDegrees here }
54. // Preview output update listener val preview: Preview = ... val viewFinder: TextureView = ... preview.setOnPreviewOutputUpdateListener { previewOutput: Preview.PreviewOutput? -> // Remove it and re-add view (important!) val parent = viewFinder.parent as ViewGroup parent.removeView(viewFinder) parent.addView(viewFinder, 0) // Update internal Surface object viewFinder.surfaceTexture = previewOutput.surfaceTexture // Apply preview output transformations // TODO: Use PreviewOutput textureSize and rotationDegrees here }
55. // Preview output transformation View Finder
56. // Preview output transformation + View Finder Preview Output
57. // Preview output transformation = + View Finder Preview Output Result
58. // Preview output transformation = + View Finder Preview Output Fit
59. // Preview output transformation = + View Finder Preview Output Letterbox
60. // Preview output transformation = + View Finder Preview Output Center Crop
61. // Preview output transformation Center Crop Letterbox Fit
62. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle) • Display (180 rotations, display removed / attached) • Camera hardware (USB, retractable)
63. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle) • Display (180 rotations, display removed / attached) • Camera hardware (USB, retractable)
64. // Layout change listener val viewFinder: TextureView = ... viewFinder.addOnLayoutChangeListener { view, left, top, right, bottom, _, _, _, _ -> val newDimens = Size(right - left, bottom - top) // Apply preview output transformations // TODO: Use `newDimens` here }
65. // Layout change listener val viewFinder: TextureView = ... viewFinder.addOnLayoutChangeListener { view, left, top, right, bottom, _, _, _, _ -> val newDimens = Size(right - left, bottom - top) // Apply preview output transformations // TODO: Use `newDimens` here }
66. // Layout change listener val viewFinder: TextureView = ... viewFinder.addOnLayoutChangeListener { view, left, top, right, bottom, _, _, _, _ -> val newDimens = Size(right - left, bottom - top) // Apply preview output transformations // TODO: Use `newDimens` here }
67. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle) • Display (180 rotations, display removed / attached) • Camera hardware (USB, retractable)
68. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle) • Display (180 rotations, display removed / attached) • Camera hardware (USB, retractable)
69. // Display listener val viewFinder: TextureView = ... val viewFinderDisplayId: Int = viewFinder.display.displayId
70. // Display listener val viewFinder: TextureView = ... val viewFinderDisplayId: Int = viewFinder.display.displayId val displayListener = object : DisplayManager.DisplayListener { override fun onDisplayAdded(displayId: Int) = Unit override fun onDisplayRemoved(displayId: Int) = Unit override fun onDisplayChanged(displayId: Int) { if (displayId == viewFinderDisplayId) { val display = viewFinder.display // Apply preview output transformations // TODO: Use `display.rotation` here } } }
71. // Display listener val viewFinder: TextureView = ... val viewFinderDisplayId: Int = viewFinder.display.displayId val displayListener = object : DisplayManager.DisplayListener { override fun onDisplayAdded(displayId: Int) = Unit override fun onDisplayRemoved(displayId: Int) = Unit override fun onDisplayChanged(displayId: Int) { if (displayId == viewFinderDisplayId) { val display = viewFinder.display // Apply preview output transformations // TODO: Use `display.rotation` here } } }
72. // Display listener val viewFinder: TextureView = ... val viewFinderDisplayId: Int = viewFinder.display.displayId val displayListener = object : DisplayManager.DisplayListener { override fun onDisplayAdded(displayId: Int) = Unit override fun onDisplayRemoved(displayId: Int) = Unit override fun onDisplayChanged(displayId: Int) { if (displayId == viewFinderDisplayId) { val display = viewFinder.display // Apply preview output transformations // TODO: Use `display.rotation` here } } }
73. // Preview output transformation
74. // Preview output transformation // Option 1: apply rotation to view finder (on top of previous transformations) val matrix: Matrix = ... matrix.postRotate(-rotationDegrees, centerX, centerY)
75. // Preview output transformation // Option 1: apply rotation to view finder (on top of previous transformations) val matrix: Matrix = ... matrix.postRotate(-rotationDegrees, centerX, centerY) // Option 2: apply rotation to preview use case, which will trigger // Preview.OnPreviewOutputUpdateListener val preview: Preview = ... preview.setTargetRotation(display.rotation)
76. // Preview output transformation // Option 1: apply rotation to view finder (on top of previous transformations) val matrix: Matrix = ... matrix.postRotate(-rotationDegrees, centerX, centerY) // Option 2: apply rotation to preview use case, which will trigger // Preview.OnPreviewOutputUpdateListener val preview: Preview = ... preview.setTargetRotation(display.rotation) // Use PreviewOutput.rotationDegrees in Preview.OnPreviewOutputUpdateListener // Alternatively, get the view's display and always use display.rotation
77. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle) • Display (180 rotations, display removed / attached) • Camera hardware (USB, retractable)
78. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle) • Display (180 rotations, display removed / attached) • Camera hardware (USB, retractable)
79. Handling state changes beyond Lifecycle • Preview output update (CameraX internals) • Layout (TextureView lifecycle) • Display (180 rotations, display removed / attached) • Camera hardware (USB, retractable)
80. Handling state changes beyond Lifecycle
81. Handling state changes beyond Lifecycle • Use AutoFitPreviewBuilder from the sample
82. @calren24 Start exploring CameraX today! @owahltinez