Android Camera APIs Jeff Gilfelt

Camera Integration

Camera Integration

Camera Integration

Camera Integration

Camera Integration

Camera Integration

Camera Integration

New Device Categories

Camera Integration Approaches ● Intents ○ Quick and simple ○ No customisation ● android.hardware.Camera API ○ Not so quick or simple ○ Customisable user experience ○ Full control

Manifest Permissions Camera Permission (not required for Intents) Storage Permission Audio Recording Permission (video only) Location Permission (if tagging media)

Manifest uses-feature Declarations ● If your app is still useful without a camera: ● Other camera feature descriptors: android.hardware.camera.any android.hardware.camera.front android.hardware.camera.autofocus android.hardware.camera.flash

Using Intents ● Intents are handled by the existing camera application that is installed on the device ● Intents are guaranteed to be honoured on any device with a rear facing camera that is certified by CTS and has the Google Play Store

Intent - Taking a Picture // create Intent to take a picture and return control to the calling application Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE); // create a file to save the image fileUri = getOutputMediaFileUri(MEDIA_TYPE_IMAGE); // set the image file name intent.putExtra(MediaStore.EXTRA_OUTPUT, fileUri); // start the image capture Intent startActivityForResult(intent, CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE); ● MediaStore.EXTRA_OUTPUT ○ Uri object specifying target path and filename

Intent - Taking a Picture

Intent - Taking a Picture @Override protected void onActivityResult(int requestCode, int resultCode, Intent data) { if (requestCode == CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE) { if (resultCode == RESULT_OK) { // Image captured and saved to fileUri specified in the Intent Toast.makeText(this, "Image saved to: " + data.getData(), Toast.LENGTH_LONG).show(); } else if (resultCode == RESULT_CANCELED) { // User cancelled the image capture } else { // Image capture failed, advise user } } }

Intent - Recording a Video //create new Intent Intent intent = new Intent(MediaStore.ACTION_VIDEO_CAPTURE); // create a file to save the video fileUri = getOutputMediaFileUri(MEDIA_TYPE_VIDEO); // set the video file name intent.putExtra(MediaStore.EXTRA_OUTPUT, fileUri); // set the video image quality to high intent.putExtra(MediaStore.EXTRA_VIDEO_QUALITY, 1); // start the Video Capture Intent startActivityForResult(intent, CAPTURE_VIDEO_ACTIVITY_REQUEST_CODE);

Intent - Recording a Video ● MediaStore.EXTRA_OUTPUT ○ Uri object specifying target path and filename ● MediaStore.EXTRA_VIDEO_QUALITY ○ 0 = low, 1 = high ● MediaStore.EXTRA_DURATION_LIMIT ○ Limit the video length, in seconds ● MediaStore.EXTRA_SIZE_LIMIT ○ Limit the video file size, in bytes

Intent - Recording a Video @Override protected void onActivityResult(int requestCode, int resultCode, Intent data) { if (requestCode == CAPTURE_VIDEO_ACTIVITY_REQUEST_CODE) { if (resultCode == RESULT_OK) { // Video captured and saved to fileUri specified in the Intent Toast.makeText(this, "Video saved to:\n" + data.getData(), Toast.LENGTH_LONG).show(); } else if (resultCode == RESULT_CANCELED) { // User cancelled the video capture } else { // Video capture failed, advise user } } }

Using Intents ● Advantages: ○ Very simple to implement ○ Only a few lines of code ○ Takes advantage of features present in the device Camera app ○ Perfect if the camera is not core to your application and you just need to capture a quick picture or video clip

Using Intents ● Disadvantages: ○ Cannot create a custom camera interface or user experience ○ Interface and experience will differ slightly across different devices ○ No fine grained control of recording settings (e.g. MediaStore.EXTRA_VIDEO_QUALITY)

Camera API ● Building blocks: ○ Code to detect and safely open the camera ○ Code to test and enable required camera features ○ A custom camera preview view extending SurfaceView that implements the SurfaceHolder. Callbacks interface ○ An Activity or Fragment layout comprising the preview view and your custom controls (shutter button etc) ○ Code to capture user interface events ○ Code to capture and save media files ○ Code to safely release the camera

Camera API - Accessing the Camera If camera is declared as optional in manifest: /** Check if this device has a camera */ private boolean checkCameraHardware(Context context) { if (context.getPackageManager() .hasSystemFeature(PackageManager.FEATURE_CAMERA)){ // this device has a camera return true; } else { // no camera on this device return false; } }

Camera API - Accessing the Camera Open the default rear-facing camera: /** A safe way to get an instance of the Camera object. */ public static Camera getCameraInstance(){ Camera c = null; try { c = Camera.open(); // attempt to get a Camera instance } catch (Exception e){ // Camera is not available (in use or does not exist) } return c; // returns null if camera is unavailable }

Camera API - Accessing the Camera Use Camera.open(int cameraId) to access a specific hardware camera (API 9+) // Find the total number of cameras available int numberOfCameras = Camera.getNumberOfCameras(); // Find the ID of the default camera CameraInfo cameraInfo = new CameraInfo(); for (int i = 0; i < numberOfCameras; i++) { Camera.getCameraInfo(i, cameraInfo); if (cameraInfo.facing == CameraInfo.CAMERA_FACING_BACK) { defaultCameraId = i; } }

Camera API - Create CameraPreview We need a SurfaceView or TextureView (API 14+) to render the camera preview public class CameraPreview extends SurfaceView implements SurfaceHolder.Callback { private Camera mCamera; public CameraPreview(Context context, Camera camera) { super(context); mCamera = camera; // register a SurfaceHolder.Callback so we get notified when the // underlying surface is created and destroyed. getHolder().addCallback(this); } //... }

Camera API - Create CameraPreview public class CameraPreview extends SurfaceView implements SurfaceHolder.Callback { // ... public void surfaceCreated(SurfaceHolder holder) { // The Surface has been created, now tell // the camera to start the preview. } public void surfaceDestroyed(SurfaceHolder holder) { // Take care of releasing the Camera preview. } public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) { // If your preview can change or rotate, take // care of those events here. Make sure to stop // the preview before resizing or reformatting it. } // ... }

Camera API - Activity Lifecycle ● onCreate: ○ Create a new CameraPreview passing it the camera instance, then add it to the content view ● onResume: ○ Open the camera instance ● onPause: ○ Release the camera instance

Camera API - Taking a Picture mCamera.takePicture(null, null, new PictureCallback() { @Override public void onPictureTaken(byte[] data, Camera camera) { File pictureFile = getOutputMediaFile(MEDIA_TYPE_IMAGE); try { FileOutputStream fos = new FileOutputStream(pictureFile); fos.write(data); fos.close(); } catch (IOException e) { Log.d(TAG, "Error accessing file: " + e.getMessage()); } } });

Camera API - Recording a Video ● To start recording: ○ Unlock camera for use by MediaRecorder: ■ Camera.unlock() ○ Configure MediaRecorder: ■ setCamera() ■ setAudioSource() ■ setVideoSource() ■ setProfile() - audio/video format and encoding ■ setOutputFile() ■ setPreviewDisplay() ○ Prepare and start MediaRecorder: ■ prepare() ■ start()

Camera API - Recording a Video ● To stop recording: ○ Stop, reset and release the MediaRecorder: ■ stop() ■ reset() ■ release() ○ Lock the camera so future MediaRecorder sessions can use it: ■ Camera.lock()

Camera API - Camera Features ● API 14+ ○ Face Detection ○ Metering Areas ○ Focus Areas ○ White Balance Lock ○ Exposure Lock ○ Video Snapshot ● API 11+ ○ Time Lapse Video ● API 9+ ○ Multiple Cameras ○ Focus Distance ● API 8+ ○ Zoom ○ Exposure Compensation ● API 5+ ○ GPS Data ○ White Balance ○ Focus Mode ○ Scene Mode ○ JPEG Quality ○ Flash Mode ○ Color Effects ○ Anti-Banding

Camera API - Feature Detection Camera features are controlled through Camera.Parameters: // get Camera parameters Camera.Parameters params = mCamera.getParameters(); List focusModes = params.getSupportedFocusModes(); if (focusModes.contains(Camera.Parameters.FOCUS_MODE_AUTO)) { // Autofocus mode is supported }

Camera API - Enabling Features // get Camera parameters Camera.Parameters params = mCamera.getParameters(); // set the focus mode params.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO); // set Camera parameters mCamera.setParameters(params); ● Some features cannot be changed while a camera preview has started ● Some features require additional code to implement (Face detection etc)

Camera API - Video Size & Codec Use CamcorderProfile to query and use predefined settings supported by the device CamcorderProfile cp = CamcorderProfile.get(CamcorderProfile.QUALITY_HIGH); mMediaRecorder.setProfile(cp); ● API 11+: ○ QUALITY_1080P ○ QUALITY_720P ○ QUALITY_480P ● API 8+ ○ QUALITY_HIGH ○ QUALITY_LOW

Camera API - The Good ● Full control to customise the camera interface and user experience within our app

Camera API - The Bad ● It’s difficult to get started: ○ The camera API guide and training is overly simplistic and lacks important detail ○ No fully fledged SDK sample application ○ AOSP Camera app is quite complex, difficult to pull features from and is not necessarily broadly compatible

Camera API - The Bad ● The state machine and lifecycle event balancing act: ■ Camera ■ MediaRecorder ■ SurfaceView ■ Fragment ■ Activity ● Requires a lot of code

Camera API - The Bad ● Setting an optimal preview size for the view/screen dimensions can be tricky

Camera API - The Bad ● Device specific quirks: ○ Real fragmentation, probably caused by underlying camera hardware differences across manufacturers and device types ○ Camera.Parameters and CamcorderProfile do not always tell the truth

Camera API - The Ugly ● Recorded media is unplayable

Camera API - The Ugly ● Recorded media is distorted

Camera API - The Ugly ● Orientation not honoured

Camera API - The Ugly ● Example code hacks: // TODO hack for GS2 if (Build.MODEL.equals("GT-I9100")) { // green mess in video file without this params.set( "cam_mode", 1 ); } // TODO hack for 720p on GS4 if (Build.MODEL.equals("GT-I9500") || Build.MODEL.equals("GT-I9505")) { candidate = CamcorderProfile.get(CamcorderProfile.QUALITY_HIGH); candidate.videoFrameWidth = 1280; candidate.videoFrameHeight = 720; }

FUCKING SAMSUNG

No content

https://github.com/commonsguy/cwac-camera CWAC-Camera

CWAC-Camera - What Is It? ● Taking Pictures. Made Sensible. ● The simplicity of the Intent approach combined with the flexibility and control provided by the Camera API ● Encapsulates the details behind a scalable API ● Easy to do simple things ● Complex things not impossible ● Open source (Apache 2.0)

CWAC-Camera - Who Wrote It? ● Mark Murphy - Author: The Busy Coder’s Guide to Android Development ○ http://commonsware.com/books ● CWAC = CommonsWare Android Components ○ https://github.com/commonsguy ● Droidcon London - "Android Security: Defending Your Users" ○ http://uk.droidcon.com/2013/buyticket/ ● One day workshop, Oct 22 - “Pushing the UI Envelope” ○ http://skillsmatter.com/event/os-mobile-server/mark- murphys-pushing-the-ui-envelope

CWAC-Camera - Overview ● CameraFragment: ○ Renders the preview using the appropriate surface format, optimally sized for the device ○ Deals with configuration changes and rotation to support both portrait and landscape shooting ○ Provides a simple API for taking pictures and recording video ○ Opens and releases the camera automatically and safely ● Native (API 11+) and support Fragment implementations ● Underlying CameraView also available

CWAC-Camera - Simple Setup ● Add the JAR file to your project ● Include the ActionBarSherlock library project if supporting < API 11 ● Add the appropriate camera manifest permissions ● Create an XML layout for your camera UI with a container (e.g. FrameLayout) in which to attach CameraFragment ● Create and attach an instance of CameraFragment in your Activity: public class MainActivity extends Activity { CameraFragment mCameraFragment; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); mCameraFragment = new CameraFragment(); getFragmentManager().beginTransaction() .replace(R.id.preview_container, mCameraFragment).commit(); } }

CWAC-Camera - Taking a Picture ● Call takePicture() Button snap = (Button) findViewById(R.id.snap); snap.setOnClickListener(new OnClickListener() { @Override public void onClick(View v) { mCameraFragment.takePicture(); } });

CWAC-Camera - Recording a Video ● Call record() and stopRecording() Button record = (Button) findViewById(R.id.record); record.setOnClickListener(new OnClickListener() { @Override public void onClick(View v) { try { if (!mCameraFragment.isRecording()) { mCameraFragment.record(); } else { mCameraFragment.stopRecording(); } } catch (Exception e) { e.printStackTrace(); } } });

CWAC-Camera - Configuration ● Create a custom CameraHost: class MyCameraHost extends SimpleCameraHost { public MyCameraHost(Context context) { super(context); } @Override public boolean useFrontFacingCamera() { return true; } @Override protected File getPhotoDirectory() { return getExternalFilesDir(Environment.DIRECTORY_PICTURES); } } ● Supply CameraHost to CameraFragment: mCameraFragment.setHost(new MyCameraHost(this));

CWAC-Camera - Configuration ● Use a custom CameraHost to configure: ○ Output file names and locations ○ Camera selection ○ Exception handling ○ Auto focus callbacks ○ Shutter callbacks ○ Preview sizes ○ Picture and video recording parameters ○ EXIF rotation behaviour ○ DeviceProfile - a class that wraps any device specific hacks that might be needed

CWAC-Camera - Compatibility ● Tested Devices: ○ Acer Iconia Tab A700 ○ Amazon Kindle Fire HD ○ ASUS Transformer Infinity (1st generation) ○ Galaxy Nexus ○ HTC Droid Incredible 2 ○ HTC One S ○ Lenovo ThinkPad Tablet ○ Nexus 4 ○ Nexus 7 (1st generation, 2012) ○ Nexus 7 (2nd generation, 2013) ○ Nexus 10 ○ Nexus One ○ Nexus S ○ Motorola RAZR i ○ Samsung Galaxy Note 2 ○ Samsung Galaxy S3 ○ Samsung Galaxy S4 (GT- I9500) ○ Samsung Galaxy Tab 2 ○ SONY Ericsson Xperia Play ○ SONY Xperia E ○ Sony Xperia S LT26i ○ SONY Xperia Z

CWAC-Camera - Current Limitations ● No video recording support below API 11 ● No “full bleed” camera preview ● Cannot use CameraFragment directly in XML layouts ● Portrait mode orientation issues on certain devices

Resources ● Android Camera API Guide: ○ https://developer.android.com/guide/topics/media/camera.html ● “Controlling the Camera” Training Lesson: ○ http://developer.android.com/training/camera/cameradirect.html ● CameraPreview SDK sample: ○ samples/ApiDemos/src/com/example/android/apis/graphics/CameraPr eview.java ● AOSP Camera App Package: ○ https://android.googlesource.com/platform/packages/apps/Camera.git ● CWAC-Camera: ○ https://github.com/commonsguy/cwac-camera