Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Droidcon NYC 2014: Building Custom Camera Applications

Huyen Tue Dao
September 21, 2014

Droidcon NYC 2014: Building Custom Camera Applications

Presentations slides from Droidcon NYC 2014: Building Custom Camera Applications

Huyen Tue Dao

September 21, 2014

More Decks by Huyen Tue Dao

Other Decks in Programming


  1. BUILDING CUSTOM CAMERA APPLICATIONS Droidcon NYC 2014 Presented by Huyen

    Tue Dao September 21, 2014 376dp wrap_content @+id/randomly_typing
  2. Where to start Camera setup Image capture Camera information Camera

  3. Mobile developer: native Android and native iOS Android developer 4

    years (personally use one) Computer Engineering, University of MD, College Park Gamer (video, board, card, anything): mostly Dota 2, QuizUp ABOUT ME 3 © 2011 Adam C Beamish
  4. WHERE TO START What you want vs what you need

    Balance having critical features with supporting target user base What features the API supports deciding minimum SDK using Build.VERSION What features the device camera has using use-feature and PackageManager querying support level from the Camera being aware of manufacturer/device quirks WANT CAN You can’t always get what you want. But if you try sometimes well you just mind find You get what you need.
  5. <uses-permission android:name="android.permission.CAMERA" /> Without any <uses-feature> specifications, to install a

    device would require: a back-facing camera a front-facing camera auto-focus flash From docs: “This will automatically enforce the <uses-feature> manifest element for all camera features.” If any of the above are not requirements, specify <uses-feature>: <uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/> CAMERA SETUP: THE MANIFEST 5
  6. CAMERA SETUP: CAMERA + PREVIEW The order of things… Add

    a SurfaceView to your layout for the camera preview Implement a SurfaceHolder.Callback: surfaceCreated(), surfaceChanged(), surfaceDestroyed() Pass the callback to the SurfaceHolder instance of the SurfaceView Get android.hardware.Camera instance via Camera.open() Call Camera.getParameters() and perform any initial setup Call Camera.startPreview() wait for surface to be created: #surfaceCreated Call Camera.setPreviewDisplay() with the SurfaceHolder
  7. “Did you speak the exact words?” Must call Camera.setPreviewDisplay() after

    the surface is created Otherwise, no error, no preview Preview size changes must be made between Camera.stopPreview()/Camera.startPreview() all Camera.release() in onPause() to free up the camera The surface is destroyed when SurfaceView is View.INVISIBLE. CAMERA SETUP: THINGS TO NOTE 9
  8. Call Camera.takePicture(ShutterCallback, PictureCallback, PictureCallback, PictureCallback) The 3 PictureCallback parameters =

    3 image formats raw: uncompressed postview: scaled, fully processed JPEG: compressed Raw and postview support depends on device ShutterCallback (more or less) invoked the moment that camera sensor captures an image Camera.takePicture() stops the camera preview; call Camera.startPreview() in/after picture taking callbacks IMAGE CAPTURE 10
  9. CameraInfo: per device camera orientation: angle of rotation when facing

    the camera for the image to match the natural orientation of the device facing: front or back whether the shutter sound can be disabled Camera.getCameraInfo(): camera IDs are 0-based indices Use CameraInfo to swap between front and back Use PackageManager to check if a front camera exists if front camera is not required by <use-feature> Close the current camera before swapping CAMERA INFO: SWITCHING IT UP 12
  10. 14

  11. 15

  12. DEVICE ORIENTATION VS CAMERA ORIENTATION i.e. your camera is sideways

    Natural camera orientation Natural device orientation Camera.Info.orientation = 90 Camera.Info.orientation = 270 90° 270° (looking at the device from the front)
  13. 17

  14. Camera/display rotation + configuration changes -> sucks Complicated Device and

    camera orientation changes don’t play nice Option #1: change activity orientation animation (API 18+): WindowManager.LayoutParams.html#rotationAnimation Option #2: lock the activity orientation CAMERA ROTATION 18
  15. Camera.Parameters information about and access to features Device support: PackageManager

    to check for auto-focus and flash Otherwise check getter return values in Camera.Parameters !isAutoWhiteBalanceLockSupported() Or returns 0 or null API level also a factor (of course) CAMERA PARAMETERS 19
  16. Camera.Parameters changes happen immediately (mostly) Saving/restoring parameters Camera.Parameters.flatten() Camera.Parameters.unflatten() Change

    values on a Camera.Parameters instance and pass it to Camera.setParameters() Always call Camera.getParameters() Do not hold onto Camera.Parameters instances Only flattened string makes it past Camera service CAMERA SETTINGS: UPDATING 21
  17. 22

  18. Camera.Area rectangular bounds within viewfinder metering areas: measuring light to

    calculate exposure + WB focus areas: prioritizing focus Camera.Parameters.setMeteringAreas(List<Camera.Area>) Camera.Parameters.setFocusAreas(List<Camera.Area>) Relative to the current zoom: cannot be outside FOV Camera viewfinder/sensor has its own coordinate system CAMERA FEATURES: AREAS 23
  19. VIEWFINDER VS VIEW COORDINATES Translating between what the camera sees

    and what the user sees Camera (0,0) H W View 1000 -1000 1000 -1000 Viewfinder Full FOV Viewfinder zoom = 1.5x preview surface in layout
  20. 25

  21. Camera.Parameters.getMaxNumDetectedFaces() Camera FaceDetectionListener, setFaceDetectionListener() startFaceDetection(), stopFaceDetection() Camera.Face: data object ID:

    unique per face face bounds score: how confident are we that this is a face? eye and face coordinates FACE DETECTION 26
  22. FaceDetectionListener.onFaceDetection() is called frequently Start only when actually needed Avoid

    intensive work inside the callback No event when no faces are detected Modifying the UI in response to face detection: timer or postDelayed to reset UI after some time without detection event FACE DETECTION 27
  23. Driven by a new Hardware Abstract Layer (HAL3) HAL Version

    1: black box: 3 modes (preview, still, video) HAL3: all requests handled same way, “unified view” More user control for both capture and post-processing Maintainability and efficiency CAMERA API 2 IN L: THE FUTURE IS AWESOME 28
  24. More metadata Camera information Capture configuration part of request and

    result Dynamic metadata: timestamps, exposure time More output options Multiple Surface instances receive results from a single request App-visible JPEG, YUV, RAW Bayer buffers CAMERA API 2 IN L: THE FUTURE IS AWESOME 31
  25. Camera (static bits) -> CameraManager AvailabilityListener: removable cameras Asynchronous opening

    + CameraDevice.StateListener Camera (function access) -> CameraDevice Actually get the ID of the camera! CameraCaptureSession/CaptureRequest Parameters -> CaptureRequest.Builder.set() to configure CameraInfo -> CameraCharacteristics CameraMetadata parent of CameraCharacteristics, CaptureRequest, CaptureResult, TotalCaptureResult CAMERA API 2 IN L: THE FUTURE IS AWESOME 32
  26. Lessons from API 1 do not go away (remember the

    words) More complex… but more is still better More control and performance More consistency in the API More for developers to leverage It’s going to be… CAMERA API 2 IN L: THE FUTURE IS AWESOME 33
  27. THANK YOU + QUESTIONS? Huyen Tue Dao Lead developer |

    Owner Randomly Typing ! @queencodemonkey ! [email protected] randomlytyping.com ! speakerdeck.com/randomlytyping slideshare.net/randomlytyping 34 Code will coming to github ! Things To check out ! Building Apps with Multimedia: Capturing Photos developer.android.com/training/camera/ index.html ! Android Design in Action: Camera Apps youtube.com/watch?v=OLSa7fErTAM ! Standford Digital Image Processing Class stanford.edu/class/ee368/Android/index.html ! DevBytes: Android L - Camera2 API youtube.com/watch?v=Xtp3tH27OFs ! Sample Camera 2 API app: github.com/googlesamples/android- Camera2Basic/blob/master/Camera2BasicSample ! Digital Camera Sensor Sizes gizmag.com/camera-sensor-size-guide/26684/