Getting Started with CameraX

Transcript

1. Getting Started with
   CameraX
   @owahltinez
   @calren24
   

   View Slide

2. Camera Development can be difficult and frustrating
   

   View Slide

3. Camera Development can be difficult and frustrating
   ● Supporting different API versions
   

   View Slide

4. Camera Development can be difficult and frustrating
   ● Supporting different API versions
   ● Providing consistency across devices
   

   View Slide

5. Camera Development can be difficult and frustrating
   ● Supporting different API versions
   ● Providing consistency across devices
   ● Understanding the complex camera API
   

   View Slide

6. Camera Development can be difficult and frustrating
   ● Supporting different API versions
   ● Providing consistency across devices
   ● Understanding the complex camera API
   CameraX support library aims to help solve these problems
   

   View Slide

7. CameraX
   ● Backwards compatible to L (90% of devices)
   

   View Slide

8. CameraX
   ● Backwards compatible to L (90% of devices)
   ● Consistent behavior across devices
   

   View Slide

9. CameraX
   ● Backwards compatible to L (90% of devices)
   ● Consistent behavior across devices
   ● Easy to use APIs
   

   View Slide

10. Using CameraX results in 70% fewer
    lines of code vs camera2
    

    View Slide

11. Preview
    Image Analysis
    Capture
    

    View Slide

12. Preview
    Image Analysis
    Capture
    

    View Slide

13. // configure preview
    val previewConfig = PreviewConfig.Builder().build()t
    .setTargetRotation(Surface.ROTATION_180)
    his as LifecycleOwner, preview)
    

    View Slide

14. // create preview
    val previewConfig = PreviewConfig.Builder().build()
    .setTargetRotation(Surface.ROTATION_180)
    val preview = Preview(previewConfig)wner, preview)
    

    View Slide

15. // attach output to view
    val previewConfig = PreviewConfig.Builder().build()
    .setTargetRotation(Surface.ROTATION_180)
    val preview = Preview(previewConfig)
    preview.setOnPreviewOutputUpdateListener {
    previewOutput: Preview.PreviewOutput? ->
    //your code here. e.g. use previewOutput?.surfaceTexture
    //and post to a GL renderer.
    }
    CameraX.bindToLifecycle(this as LifecycleOwner, preview)
    

    View Slide

16. // attach preview to lifecycle
    val previewConfig = PreviewConfig.Builder().build()
    .setTargetRotation(Surface.ROTATION_180)
    val preview = Preview(previewConfig)
    preview.setOnPreviewOutputUpdateListener {
    previewOutput: Preview.PreviewOutput? ->
    //your code here. e.g. use previewOutput?.surfaceTexture
    //and post to a GL renderer.
    }
    CameraX.bindToLifecycle(this as LifecycleOwner, preview)
    

    View Slide

17. // display preview on screen
    val previewConfig = PreviewConfig.Builder().build()
    .setTargetRotation(Surface.ROTATION_180)
    val preview = Preview(previewConfig)
    preview.setOnPreviewOutputUpdateListener {
    previewOutput: Preview.PreviewOutput? ->
    //your code here. e.g. use previewOutput?.surfaceTexture
    //and post to a GL renderer.
    }
    CameraX.bindToLifecycle(this as LifecycleOwner, preview)
    

    View Slide

18. Preview
    Image Analysis
    Capture
    

    View Slide

19. // configure image analysis
    val imageAnalysisConfig = ImageAnalysisConfig.Builder()
    

    View Slide

20. // set resolution
    val imageAnalysisConfig = ImageAnalysisConfig.Builder()
    .setTargetResolution(Size(1280, 720))
    .build()
    

    View Slide

21. // create image analysis
    val imageAnalysisConfig = ImageAnalysisConfig.Builder()
    .setTargetResolution(Size(1280, 720))
    .build()
    val imageAnalysis = ImageAnalysis(imageAnalysisConfig)
    

    View Slide

22. // attach output
    val imageAnalysisConfig = ImageAnalysisConfig.Builder()
    .setTargetResolution(Size(1280, 720))
    .build()
    val imageAnalysis = ImageAnalysis(imageAnalysisConfig)
    imageAnalysis.setAnalyzer({ image: ImageProxy, rotationDegrees: Int ->
    val cropRect = image.cropRect
    // do some analysis
    })
    

    View Slide

23. // attach image analysis & preview to lifecycle
    val imageAnalysisConfig = ImageAnalysisConfig.Builder()
    .setTargetResolution(Size(1280, 720))
    .build()
    val imageAnalysis = ImageAnalysis(imageAnalysisConfig)
    imageAnalysis.setAnalyzer({ image: ImageProxy, rotationDegrees: Int ->
    val cropRect = image.cropRect
    // do some analysis
    })
    CameraX.bindToLifecycle(this as LifecycleOwner, imageAnalysis, preview)
    

    View Slide

24. // full setup to process images
    val imageAnalysisConfig = ImageAnalysisConfig.Builder()
    .setTargetResolution(Size(1280, 720))
    .build()
    val imageAnalysis = ImageAnalysis(imageAnalysisConfig)
    imageAnalysis.setAnalyzer({ image: ImageProxy, rotationDegrees: Int ->
    val cropRect = image.cropRect
    // do some analysis
    })
    CameraX.bindToLifecycle(this as LifecycleOwner, imageAnalysis, preview)
    

    View Slide

25. Preview
    Image Analysis
    Capture
    

    View Slide

26. // configure image capture
    val imageCaptureConfig = ImageCaptureConfig.Builder()
    .setTargetResolution(Size(1280, 720))
    .build()
    

    View Slide

27. // create image capture
    val imageCaptureConfig = ImageCaptureConfig.Builder()
    .setTargetResolution(Size(1280, 720))
    .build()
    val imageCapture = ImageCapture(imageCaptureConfig)
    

    View Slide

28. // bind all use cases
    val imageCaptureConfig = ImageCaptureConfig.Builder()
    .setTargetResolution(Size(1280, 720))
    .build()
    val imageCapture = ImageCapture(imageCaptureConfig)
    CameraX.bindToLifecycle(this as LifecycleOwner,
    imageCapture, imageAnalysis, preview)
    

    View Slide

29. // full setup to capture images
    val imageCaptureConfig = ImageCaptureConfig.Builder()
    .setTargetResolution(Size(1280, 720))
    .build()
    val imageCapture = ImageCapture(imageCaptureConfig)
    CameraX.bindToLifecycle(this as LifecycleOwner,
    imageCapture, imageAnalysis, preview)
    

    View Slide

30. // on user action
    

    View Slide

31. // on user action
    fun onClick() {
    val file = File(..)
    imageCaptureUseCase.takePicture(file,
    object : ImageCaptureUseCase.OnImageSavedListener {
    override fun onError(error: ImageCaptureUseCase.UseCaseError,
    message: String, exc: Throwable?) {
    //insert your code here
    }
    override fun onImageSaved(file: File) {
    //insert your code here
    }
    })
    }
    

    View Slide

32. // on user action save a picture
    fun onClick() {
    val file = File(..)
    imageCapture.takePicture(file,
    object : ImageCaptureUseCase.OnImageSavedListener {
    override fun onError(error: ImageCaptureUseCase.UseCaseError,
    message: String, exc: Throwable?) {
    //insert your code here
    }
    override fun onImageSaved(file: File) {
    //insert your code here
    }
    })
    }
    

    View Slide

33. // save a picture on user action
    fun onClick() {
    val file = File(..)
    imageCapture.takePicture(file,
    object : ImageCapture.OnImageSavedListener {
    override fun onError(error: ImageCapture.UseCaseError,
    message: String, exc: Throwable?) {
    //insert your code here
    }
    override fun onImageSaved(file: File) {
    //insert your code here
    }
    })
    }
    

    View Slide

34. CameraX sample app
    g.co/camerax/sample
    CameraX dev mailing list
    g.co/camerax/developers
    CameraX StackOverflow tag
    stackoverflow.com/tags/android-camerax
    

    View Slide

35. CameraX sample app
    

    View Slide

36. CameraX sample app
    ● Under Android camera samples GitHub repo
    

    View Slide

37. CameraX sample app
    ● Under Android camera samples GitHub repo
    ● Monitoring all repo issues and pull requests
    

    View Slide

38. CameraX sample app
    ● Under Android camera samples GitHub repo
    ● Monitoring all repo issues and pull requests
    ● Sample issues != CameraX library issues
    

    View Slide

39. CameraX sample app (cont)
    

    View Slide

40. CameraX sample app (cont)
    ● Very basic indeed
    

    View Slide

41. CameraX sample app (cont)
    ● Very basic indeed
    ● Preview, image capture and image analysis
    

    View Slide

42. CameraX sample app (cont)
    ● Very basic indeed
    ● Preview, image capture and image analysis
    ● Future work: vendor extensions and video record
    

    View Slide

43. CameraX sample app
    ● Very basic indeed
    ● Preview, image capture and image analysis
    ● Future work: vendor extensions and video record
    

    View Slide

44. Handling state changes beyond Lifecycle
    

    View Slide

45. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    

    View Slide

46. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    

    View Slide

47. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    ● Display (180 rotations, display removed / attached)
    

    View Slide

48. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    ● Display (180 rotations, display removed / attached)
    ● Camera hardware (USB, retractable)
    

    View Slide

49. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    ● Display (180 rotations, display removed / attached)
    ● Camera hardware (USB, retractable)
    

    View Slide

50. // Preview output update listener
    val preview: Preview = ...
    val viewFinder: TextureView = ...
    preview.setOnPreviewOutputUpdateListener {
    previewOutput: Preview.PreviewOutput? ->
    // Remove it and re-add view (important!)
    val parent = viewFinder.parent as ViewGroup
    parent.removeView(viewFinder)
    parent.addView(viewFinder, 0)
    // Update internal Surface object
    viewFinder.surfaceTexture = previewOutput.surfaceTexture
    // Apply preview output transformations
    // TODO: Use PreviewOutput textureSize and rotationDegrees here
    }
    

    View Slide

51. // Preview output update listener
    val preview: Preview = ...
    val viewFinder: TextureView = ...
    preview.setOnPreviewOutputUpdateListener {
    previewOutput: Preview.PreviewOutput? ->
    // Remove it and re-add view (important!)
    val parent = viewFinder.parent as ViewGroup
    parent.removeView(viewFinder)
    parent.addView(viewFinder, 0)
    // Update internal Surface object
    viewFinder.surfaceTexture = previewOutput.surfaceTexture
    // Apply preview output transformations
    // TODO: Use PreviewOutput textureSize and rotationDegrees here
    }
    

    View Slide

52. // Preview output update listener
    val preview: Preview = ...
    val viewFinder: TextureView = ...
    preview.setOnPreviewOutputUpdateListener {
    previewOutput: Preview.PreviewOutput? ->
    // Remove it and re-add view (important!)
    val parent = viewFinder.parent as ViewGroup
    parent.removeView(viewFinder)
    parent.addView(viewFinder, 0)
    // Update internal Surface object
    viewFinder.surfaceTexture = previewOutput.surfaceTexture
    // Apply preview output transformations
    // TODO: Use PreviewOutput textureSize and rotationDegrees here
    }
    

    View Slide

53. // Preview output update listener
    val preview: Preview = ...
    val viewFinder: TextureView = ...
    preview.setOnPreviewOutputUpdateListener {
    previewOutput: Preview.PreviewOutput? ->
    // Remove it and re-add view (important!)
    val parent = viewFinder.parent as ViewGroup
    parent.removeView(viewFinder)
    parent.addView(viewFinder, 0)
    // Update internal Surface object
    viewFinder.surfaceTexture = previewOutput.surfaceTexture
    // Apply preview output transformations
    // TODO: Use PreviewOutput textureSize and rotationDegrees here
    }
    

    View Slide

54. // Preview output update listener
    val preview: Preview = ...
    val viewFinder: TextureView = ...
    preview.setOnPreviewOutputUpdateListener {
    previewOutput: Preview.PreviewOutput? ->
    // Remove it and re-add view (important!)
    val parent = viewFinder.parent as ViewGroup
    parent.removeView(viewFinder)
    parent.addView(viewFinder, 0)
    // Update internal Surface object
    viewFinder.surfaceTexture = previewOutput.surfaceTexture
    // Apply preview output transformations
    // TODO: Use PreviewOutput textureSize and rotationDegrees here
    }
    

    View Slide

55. // Preview output transformation
    View Finder
    

    View Slide

56. // Preview output transformation
    +
    View Finder Preview Output
    

    View Slide

57. // Preview output transformation
    =
    +
    View Finder Preview Output Result
    

    View Slide

58. // Preview output transformation
    =
    +
    View Finder Preview Output Fit
    

    View Slide

59. // Preview output transformation
    =
    +
    View Finder Preview Output Letterbox
    

    View Slide

60. // Preview output transformation
    =
    +
    View Finder Preview Output Center Crop
    

    View Slide

61. // Preview output transformation
    Center Crop
    Letterbox
    Fit
    

    View Slide

62. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    ● Display (180 rotations, display removed / attached)
    ● Camera hardware (USB, retractable)
    

    View Slide

63. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    ● Display (180 rotations, display removed / attached)
    ● Camera hardware (USB, retractable)
    

    View Slide

64. // Layout change listener
    val viewFinder: TextureView = ...
    viewFinder.addOnLayoutChangeListener { view, left, top, right, bottom, _, _, _, _ ->
    val newDimens = Size(right - left, bottom - top)
    // Apply preview output transformations
    // TODO: Use `newDimens` here
    }
    

    View Slide

65. // Layout change listener
    val viewFinder: TextureView = ...
    viewFinder.addOnLayoutChangeListener { view, left, top, right, bottom, _, _, _, _ ->
    val newDimens = Size(right - left, bottom - top)
    // Apply preview output transformations
    // TODO: Use `newDimens` here
    }
    

    View Slide

66. // Layout change listener
    val viewFinder: TextureView = ...
    viewFinder.addOnLayoutChangeListener { view, left, top, right, bottom, _, _, _, _ ->
    val newDimens = Size(right - left, bottom - top)
    // Apply preview output transformations
    // TODO: Use `newDimens` here
    }
    

    View Slide

67. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    ● Display (180 rotations, display removed / attached)
    ● Camera hardware (USB, retractable)
    

    View Slide

68. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    ● Display (180 rotations, display removed / attached)
    ● Camera hardware (USB, retractable)
    

    View Slide

69. // Display listener
    val viewFinder: TextureView = ...
    val viewFinderDisplayId: Int = viewFinder.display.displayId
    

    View Slide

70. // Display listener
    val viewFinder: TextureView = ...
    val viewFinderDisplayId: Int = viewFinder.display.displayId
    val displayListener = object : DisplayManager.DisplayListener {
    override fun onDisplayAdded(displayId: Int) = Unit
    override fun onDisplayRemoved(displayId: Int) = Unit
    override fun onDisplayChanged(displayId: Int) {
    if (displayId == viewFinderDisplayId) {
    val display = viewFinder.display
    // Apply preview output transformations
    // TODO: Use `display.rotation` here
    }
    }
    }
    

    View Slide

71. // Display listener
    val viewFinder: TextureView = ...
    val viewFinderDisplayId: Int = viewFinder.display.displayId
    val displayListener = object : DisplayManager.DisplayListener {
    override fun onDisplayAdded(displayId: Int) = Unit
    override fun onDisplayRemoved(displayId: Int) = Unit
    override fun onDisplayChanged(displayId: Int) {
    if (displayId == viewFinderDisplayId) {
    val display = viewFinder.display
    // Apply preview output transformations
    // TODO: Use `display.rotation` here
    }
    }
    }
    

    View Slide

72. // Display listener
    val viewFinder: TextureView = ...
    val viewFinderDisplayId: Int = viewFinder.display.displayId
    val displayListener = object : DisplayManager.DisplayListener {
    override fun onDisplayAdded(displayId: Int) = Unit
    override fun onDisplayRemoved(displayId: Int) = Unit
    override fun onDisplayChanged(displayId: Int) {
    if (displayId == viewFinderDisplayId) {
    val display = viewFinder.display
    // Apply preview output transformations
    // TODO: Use `display.rotation` here
    }
    }
    }
    

    View Slide

73. // Preview output transformation
    

    View Slide

74. // Preview output transformation
    // Option 1: apply rotation to view finder (on top of previous transformations)
    val matrix: Matrix = ...
    matrix.postRotate(-rotationDegrees, centerX, centerY)
    

    View Slide

75. // Preview output transformation
    // Option 1: apply rotation to view finder (on top of previous transformations)
    val matrix: Matrix = ...
    matrix.postRotate(-rotationDegrees, centerX, centerY)
    // Option 2: apply rotation to preview use case, which will trigger
    // Preview.OnPreviewOutputUpdateListener
    val preview: Preview = ...
    preview.setTargetRotation(display.rotation)
    

    View Slide

76. // Preview output transformation
    // Option 1: apply rotation to view finder (on top of previous transformations)
    val matrix: Matrix = ...
    matrix.postRotate(-rotationDegrees, centerX, centerY)
    // Option 2: apply rotation to preview use case, which will trigger
    // Preview.OnPreviewOutputUpdateListener
    val preview: Preview = ...
    preview.setTargetRotation(display.rotation)
    // Use PreviewOutput.rotationDegrees in Preview.OnPreviewOutputUpdateListener
    // Alternatively, get the view's display and always use display.rotation
    

    View Slide

77. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    ● Display (180 rotations, display removed / attached)
    ● Camera hardware (USB, retractable)
    

    View Slide

78. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    ● Display (180 rotations, display removed / attached)
    ● Camera hardware (USB, retractable)
    

    View Slide

79. Handling state changes beyond Lifecycle
    ● Preview output update (CameraX internals)
    ● Layout (TextureView lifecycle)
    ● Display (180 rotations, display removed / attached)
    ● Camera hardware (USB, retractable)
    

    View Slide

80. Handling state changes beyond Lifecycle
    

    View Slide

81. Handling state changes beyond Lifecycle
    ● Use AutoFitPreviewBuilder from the sample
    

    View Slide

82. @calren24
    Start exploring CameraX today!
    @owahltinez
    

    View Slide