Droidcon NYC 2014: Building Custom Camera Applications

Presentations slides from Droidcon NYC 2014: Building Custom Camera Applications

Transcript

1. BUILDING CUSTOM CAMERA APPLICATIONS Droidcon NYC 2014 Presented by Huyen Tue Dao September 21, 2014 376dp wrap_content @+id/randomly_typing
2. Where to start Camera setup Image capture Camera information Camera parameters Camera API in L BUILDING A CUSTOM CAMERA APPLICATION 2
3. Mobile developer: native Android and native iOS Android developer 4 years (personally use one) Computer Engineering, University of MD, College Park Gamer (video, board, card, anything): mostly Dota 2, QuizUp ABOUT ME 3 © 2011 Adam C Beamish
4. WHERE TO START What you want vs what you need Balance having critical features with supporting target user base What features the API supports deciding minimum SDK using Build.VERSION What features the device camera has using use-feature and PackageManager querying support level from the Camera being aware of manufacturer/device quirks WANT CAN You can’t always get what you want. But if you try sometimes well you just mind find You get what you need.
5. <uses-permission android:name="android.permission.CAMERA" /> Without any <uses-feature> specifications, to install a device would require: a back-facing camera a front-facing camera auto-focus flash From docs: “This will automatically enforce the <uses-feature> manifest element for all camera features.” If any of the above are not requirements, specify <uses-feature>: <uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/> CAMERA SETUP: THE MANIFEST 5
6. CAMERA SETUP: CAMERA + PREVIEW The order of things… Add a SurfaceView to your layout for the camera preview Implement a SurfaceHolder.Callback: surfaceCreated(), surfaceChanged(), surfaceDestroyed() Pass the callback to the SurfaceHolder instance of the SurfaceView Get android.hardware.Camera instance via Camera.open() Call Camera.getParameters() and perform any initial setup Call Camera.startPreview() wait for surface to be created: #surfaceCreated Call Camera.setPreviewDisplay() with the SurfaceHolder
7. CAMERA SETUP: CAMERA + PREVIEW Open/release camera
8. CAMERA SETUP: CAMERA + PREVIEW Implementing SurfaceHolder.Callback
9. “Did you speak the exact words?” Must call Camera.setPreviewDisplay() after the surface is created Otherwise, no error, no preview Preview size changes must be made between Camera.stopPreview()/Camera.startPreview() all Camera.release() in onPause() to free up the camera The surface is destroyed when SurfaceView is View.INVISIBLE. CAMERA SETUP: THINGS TO NOTE 9
10. Call Camera.takePicture(ShutterCallback, PictureCallback, PictureCallback, PictureCallback) The 3 PictureCallback parameters = 3 image formats raw: uncompressed postview: scaled, fully processed JPEG: compressed Raw and postview support depends on device ShutterCallback (more or less) invoked the moment that camera sensor captures an image Camera.takePicture() stops the camera preview; call Camera.startPreview() in/after picture taking callbacks IMAGE CAPTURE 10
11. IMAGE CAPTURE with a JPEG callback
12. CameraInfo: per device camera orientation: angle of rotation when facing the camera for the image to match the natural orientation of the device facing: front or back whether the shutter sound can be disabled Camera.getCameraInfo(): camera IDs are 0-based indices Use CameraInfo to swap between front and back Use PackageManager to check if a front camera exists if front camera is not required by <use-feature> Close the current camera before swapping CAMERA INFO: SWITCHING IT UP 12
13. CAMERA INFO Finding the right camera
16. DEVICE ORIENTATION VS CAMERA ORIENTATION i.e. your camera is sideways Natural camera orientation Natural device orientation Camera.Info.orientation = 90 Camera.Info.orientation = 270 90° 270° (looking at the device from the front)
18. Camera/display rotation + configuration changes -> sucks Complicated Device and camera orientation changes don’t play nice Option #1: change activity orientation animation (API 18+): WindowManager.LayoutParams.html#rotationAnimation Option #2: lock the activity orientation CAMERA ROTATION 18
19. Camera.Parameters information about and access to features Device support: PackageManager to check for auto-focus and flash Otherwise check getter return values in Camera.Parameters !isAutoWhiteBalanceLockSupported() Or returns 0 or null API level also a factor (of course) CAMERA PARAMETERS 19
20. CAMERA.PARAMETERS SETTING
21. Camera.Parameters changes happen immediately (mostly) Saving/restoring parameters Camera.Parameters.flatten() Camera.Parameters.unflatten() Change values on a Camera.Parameters instance and pass it to Camera.setParameters() Always call Camera.getParameters() Do not hold onto Camera.Parameters instances Only flattened string makes it past Camera service CAMERA SETTINGS: UPDATING 21
23. Camera.Area rectangular bounds within viewfinder metering areas: measuring light to calculate exposure + WB focus areas: prioritizing focus Camera.Parameters.setMeteringAreas(List<Camera.Area>) Camera.Parameters.setFocusAreas(List<Camera.Area>) Relative to the current zoom: cannot be outside FOV Camera viewfinder/sensor has its own coordinate system CAMERA FEATURES: AREAS 23
24. VIEWFINDER VS VIEW COORDINATES Translating between what the camera sees and what the user sees Camera (0,0) H W View 1000 -1000 1000 -1000 Viewfinder Full FOV Viewfinder zoom = 1.5x preview surface in layout
26. Camera.Parameters.getMaxNumDetectedFaces() Camera FaceDetectionListener, setFaceDetectionListener() startFaceDetection(), stopFaceDetection() Camera.Face: data object ID: unique per face face bounds score: how confident are we that this is a face? eye and face coordinates FACE DETECTION 26
27. FaceDetectionListener.onFaceDetection() is called frequently Start only when actually needed Avoid intensive work inside the callback No event when no faces are detected Modifying the UI in response to face detection: timer or postDelayed to reset UI after some time without detection event FACE DETECTION 27
28. Driven by a new Hardware Abstract Layer (HAL3) HAL Version 1: black box: 3 modes (preview, still, video) HAL3: all requests handled same way, “unified view” More user control for both capture and post-processing Maintainability and efficiency CAMERA API 2 IN L: THE FUTURE IS AWESOME 28
29. HAL V1 VS HAL V3 https://source.android.com/devices/camera/camera3.html
30. HAL3 (EXPANDED) https://source.android.com/devices/camera/camera3_requests_hal.html
31. More metadata Camera information Capture configuration part of request and result Dynamic metadata: timestamps, exposure time More output options Multiple Surface instances receive results from a single request App-visible JPEG, YUV, RAW Bayer buffers CAMERA API 2 IN L: THE FUTURE IS AWESOME 31
32. Camera (static bits) -> CameraManager AvailabilityListener: removable cameras Asynchronous opening + CameraDevice.StateListener Camera (function access) -> CameraDevice Actually get the ID of the camera! CameraCaptureSession/CaptureRequest Parameters -> CaptureRequest.Builder.set() to configure CameraInfo -> CameraCharacteristics CameraMetadata parent of CameraCharacteristics, CaptureRequest, CaptureResult, TotalCaptureResult CAMERA API 2 IN L: THE FUTURE IS AWESOME 32
33. Lessons from API 1 do not go away (remember the words) More complex… but more is still better More control and performance More consistency in the API More for developers to leverage It’s going to be… CAMERA API 2 IN L: THE FUTURE IS AWESOME 33
34. THANK YOU + QUESTIONS? Huyen Tue Dao Lead developer | Owner Randomly Typing ! @queencodemonkey ! huyen@randomlytyping.com randomlytyping.com ! speakerdeck.com/randomlytyping slideshare.net/randomlytyping 34 Code will coming to github ! Things To check out ! Building Apps with Multimedia: Capturing Photos developer.android.com/training/camera/ index.html ! Android Design in Action: Camera Apps youtube.com/watch?v=OLSa7fErTAM ! Standford Digital Image Processing Class stanford.edu/class/ee368/Android/index.html ! DevBytes: Android L - Camera2 API youtube.com/watch?v=Xtp3tH27OFs ! Sample Camera 2 API app: github.com/googlesamples/android- Camera2Basic/blob/master/Camera2BasicSample ! Digital Camera Sensor Sizes gizmag.com/camera-sensor-size-guide/26684/