Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Android Camera APIs

Jeff Gilfelt
September 25, 2013

Android Camera APIs

Londroid - Wednesday 25th September 2013

Jeff Gilfelt

September 25, 2013
Tweet

More Decks by Jeff Gilfelt

Other Decks in Programming

Transcript

  1. Android Camera APIs
    Jeff Gilfelt

    View full-size slide

  2. Camera Integration

    View full-size slide

  3. Camera Integration

    View full-size slide

  4. Camera Integration

    View full-size slide

  5. Camera Integration

    View full-size slide

  6. Camera Integration

    View full-size slide

  7. Camera Integration

    View full-size slide

  8. Camera Integration

    View full-size slide

  9. New Device Categories

    View full-size slide

  10. Camera Integration Approaches
    ● Intents
    ○ Quick and simple
    ○ No customisation
    ● android.hardware.Camera API
    ○ Not so quick or simple
    ○ Customisable user experience
    ○ Full control

    View full-size slide

  11. Manifest Permissions
    Camera Permission (not required for Intents)

    Storage Permission

    Audio Recording Permission (video only)

    Location Permission (if tagging media)

    View full-size slide

  12. Manifest uses-feature Declarations

    ● If your app is still useful without a camera:
    required="false" />
    ● Other camera feature descriptors:
    android.hardware.camera.any
    android.hardware.camera.front
    android.hardware.camera.autofocus
    android.hardware.camera.flash

    View full-size slide

  13. Using Intents
    ● Intents are handled by the existing camera
    application that is installed on the device
    ● Intents are guaranteed to be honoured on
    any device with a rear facing camera that is
    certified by CTS and has the Google Play
    Store

    View full-size slide

  14. Intent - Taking a Picture
    // create Intent to take a picture and return control to the calling
    application
    Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
    // create a file to save the image
    fileUri = getOutputMediaFileUri(MEDIA_TYPE_IMAGE);
    // set the image file name
    intent.putExtra(MediaStore.EXTRA_OUTPUT, fileUri);
    // start the image capture Intent
    startActivityForResult(intent, CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE);
    ● MediaStore.EXTRA_OUTPUT
    ○ Uri object specifying target path and filename

    View full-size slide

  15. Intent - Taking a Picture

    View full-size slide

  16. Intent - Taking a Picture
    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent
    data) {
    if (requestCode == CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE) {
    if (resultCode == RESULT_OK) {
    // Image captured and saved to fileUri specified in the Intent
    Toast.makeText(this, "Image saved to: " +
    data.getData(), Toast.LENGTH_LONG).show();
    } else if (resultCode == RESULT_CANCELED) {
    // User cancelled the image capture
    } else {
    // Image capture failed, advise user
    }
    }
    }

    View full-size slide

  17. Intent - Recording a Video
    //create new Intent
    Intent intent = new Intent(MediaStore.ACTION_VIDEO_CAPTURE);
    // create a file to save the video
    fileUri = getOutputMediaFileUri(MEDIA_TYPE_VIDEO);
    // set the video file name
    intent.putExtra(MediaStore.EXTRA_OUTPUT, fileUri);
    // set the video image quality to high
    intent.putExtra(MediaStore.EXTRA_VIDEO_QUALITY, 1);
    // start the Video Capture Intent
    startActivityForResult(intent, CAPTURE_VIDEO_ACTIVITY_REQUEST_CODE);

    View full-size slide

  18. Intent - Recording a Video
    ● MediaStore.EXTRA_OUTPUT
    ○ Uri object specifying target path and filename
    ● MediaStore.EXTRA_VIDEO_QUALITY
    ○ 0 = low, 1 = high
    ● MediaStore.EXTRA_DURATION_LIMIT
    ○ Limit the video length, in seconds
    ● MediaStore.EXTRA_SIZE_LIMIT
    ○ Limit the video file size, in bytes

    View full-size slide

  19. Intent - Recording a Video
    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent
    data) {
    if (requestCode == CAPTURE_VIDEO_ACTIVITY_REQUEST_CODE) {
    if (resultCode == RESULT_OK) {
    // Video captured and saved to fileUri specified in the Intent
    Toast.makeText(this, "Video saved to:\n" +
    data.getData(), Toast.LENGTH_LONG).show();
    } else if (resultCode == RESULT_CANCELED) {
    // User cancelled the video capture
    } else {
    // Video capture failed, advise user
    }
    }
    }

    View full-size slide

  20. Using Intents
    ● Advantages:
    ○ Very simple to implement
    ○ Only a few lines of code
    ○ Takes advantage of features present in the device
    Camera app
    ○ Perfect if the camera is not core to your application
    and you just need to capture a quick picture or video
    clip

    View full-size slide

  21. Using Intents
    ● Disadvantages:
    ○ Cannot create a custom camera interface or user
    experience
    ○ Interface and experience will differ slightly across
    different devices
    ○ No fine grained control of recording settings (e.g.
    MediaStore.EXTRA_VIDEO_QUALITY)

    View full-size slide

  22. Camera API
    ● Building blocks:
    ○ Code to detect and safely open the camera
    ○ Code to test and enable required camera features
    ○ A custom camera preview view extending
    SurfaceView that implements the SurfaceHolder.
    Callbacks interface
    ○ An Activity or Fragment layout comprising the
    preview view and your custom controls (shutter
    button etc)
    ○ Code to capture user interface events
    ○ Code to capture and save media files
    ○ Code to safely release the camera

    View full-size slide

  23. Camera API - Accessing the Camera
    If camera is declared as optional in manifest:
    /** Check if this device has a camera */
    private boolean checkCameraHardware(Context context) {
    if (context.getPackageManager()
    .hasSystemFeature(PackageManager.FEATURE_CAMERA)){
    // this device has a camera
    return true;
    } else {
    // no camera on this device
    return false;
    }
    }

    View full-size slide

  24. Camera API - Accessing the Camera
    Open the default rear-facing camera:
    /** A safe way to get an instance of the Camera object. */
    public static Camera getCameraInstance(){
    Camera c = null;
    try {
    c = Camera.open(); // attempt to get a Camera instance
    }
    catch (Exception e){
    // Camera is not available (in use or does not exist)
    }
    return c; // returns null if camera is unavailable
    }

    View full-size slide

  25. Camera API - Accessing the Camera
    Use Camera.open(int cameraId) to access a
    specific hardware camera (API 9+)
    // Find the total number of cameras available
    int numberOfCameras = Camera.getNumberOfCameras();
    // Find the ID of the default camera
    CameraInfo cameraInfo = new CameraInfo();
    for (int i = 0; i < numberOfCameras; i++) {
    Camera.getCameraInfo(i, cameraInfo);
    if (cameraInfo.facing == CameraInfo.CAMERA_FACING_BACK) {
    defaultCameraId = i;
    }
    }

    View full-size slide

  26. Camera API - Create CameraPreview
    We need a SurfaceView or TextureView (API 14+) to
    render the camera preview
    public class CameraPreview extends SurfaceView
    implements SurfaceHolder.Callback {
    private Camera mCamera;
    public CameraPreview(Context context, Camera camera) {
    super(context);
    mCamera = camera;
    // register a SurfaceHolder.Callback so we get notified when the
    // underlying surface is created and destroyed.
    getHolder().addCallback(this);
    }
    //...
    }

    View full-size slide

  27. Camera API - Create CameraPreview
    public class CameraPreview extends SurfaceView
    implements SurfaceHolder.Callback {
    // ...
    public void surfaceCreated(SurfaceHolder holder) {
    // The Surface has been created, now tell
    // the camera to start the preview.
    }
    public void surfaceDestroyed(SurfaceHolder holder) {
    // Take care of releasing the Camera preview.
    }
    public void surfaceChanged(SurfaceHolder holder,
    int format, int w, int h) {
    // If your preview can change or rotate, take
    // care of those events here. Make sure to stop
    // the preview before resizing or reformatting it.
    }
    // ...
    }

    View full-size slide

  28. Camera API - Activity Lifecycle
    ● onCreate:
    ○ Create a new CameraPreview passing it the camera
    instance, then add it to the content view
    ● onResume:
    ○ Open the camera instance
    ● onPause:
    ○ Release the camera instance

    View full-size slide

  29. Camera API - Taking a Picture
    mCamera.takePicture(null, null, new PictureCallback() {
    @Override
    public void onPictureTaken(byte[] data, Camera camera) {
    File pictureFile = getOutputMediaFile(MEDIA_TYPE_IMAGE);
    try {
    FileOutputStream fos = new FileOutputStream(pictureFile);
    fos.write(data);
    fos.close();
    } catch (IOException e) {
    Log.d(TAG, "Error accessing file: " + e.getMessage());
    }
    }
    });

    View full-size slide

  30. Camera API - Recording a Video
    ● To start recording:
    ○ Unlock camera for use by MediaRecorder:
    ■ Camera.unlock()
    ○ Configure MediaRecorder:
    ■ setCamera()
    ■ setAudioSource()
    ■ setVideoSource()
    ■ setProfile() - audio/video format and encoding
    ■ setOutputFile()
    ■ setPreviewDisplay()
    ○ Prepare and start MediaRecorder:
    ■ prepare()
    ■ start()

    View full-size slide

  31. Camera API - Recording a Video
    ● To stop recording:
    ○ Stop, reset and release the MediaRecorder:
    ■ stop()
    ■ reset()
    ■ release()
    ○ Lock the camera so future MediaRecorder sessions
    can use it:
    ■ Camera.lock()

    View full-size slide

  32. Camera API - Camera Features
    ● API 14+
    ○ Face Detection
    ○ Metering Areas
    ○ Focus Areas
    ○ White Balance Lock
    ○ Exposure Lock
    ○ Video Snapshot
    ● API 11+
    ○ Time Lapse Video
    ● API 9+
    ○ Multiple Cameras
    ○ Focus Distance
    ● API 8+
    ○ Zoom
    ○ Exposure Compensation
    ● API 5+
    ○ GPS Data
    ○ White Balance
    ○ Focus Mode
    ○ Scene Mode
    ○ JPEG Quality
    ○ Flash Mode
    ○ Color Effects
    ○ Anti-Banding

    View full-size slide

  33. Camera API - Feature Detection
    Camera features are controlled through
    Camera.Parameters:
    // get Camera parameters
    Camera.Parameters params = mCamera.getParameters();
    List focusModes = params.getSupportedFocusModes();
    if (focusModes.contains(Camera.Parameters.FOCUS_MODE_AUTO)) {
    // Autofocus mode is supported
    }

    View full-size slide

  34. Camera API - Enabling Features
    // get Camera parameters
    Camera.Parameters params = mCamera.getParameters();
    // set the focus mode
    params.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO);
    // set Camera parameters
    mCamera.setParameters(params);
    ● Some features cannot be changed while a
    camera preview has started
    ● Some features require additional code to
    implement (Face detection etc)

    View full-size slide

  35. Camera API - Video Size & Codec
    Use CamcorderProfile to query and use
    predefined settings supported by the device
    CamcorderProfile cp = CamcorderProfile.get(CamcorderProfile.QUALITY_HIGH);
    mMediaRecorder.setProfile(cp);
    ● API 11+:
    ○ QUALITY_1080P
    ○ QUALITY_720P
    ○ QUALITY_480P
    ● API 8+
    ○ QUALITY_HIGH
    ○ QUALITY_LOW

    View full-size slide

  36. Camera API - The Good
    ● Full control to customise the camera
    interface and user experience within our app

    View full-size slide

  37. Camera API - The Bad
    ● It’s difficult to get started:
    ○ The camera API guide and training is overly
    simplistic and lacks important detail
    ○ No fully fledged SDK sample application
    ○ AOSP Camera app is quite complex, difficult to pull
    features from and is not necessarily broadly
    compatible

    View full-size slide

  38. Camera API - The Bad
    ● The state machine and lifecycle event
    balancing act:
    ■ Camera
    ■ MediaRecorder
    ■ SurfaceView
    ■ Fragment
    ■ Activity
    ● Requires a lot of code

    View full-size slide

  39. Camera API - The Bad
    ● Setting an optimal preview size for the
    view/screen dimensions can be tricky

    View full-size slide

  40. Camera API - The Bad
    ● Device specific quirks:
    ○ Real fragmentation, probably caused by underlying
    camera hardware differences across manufacturers
    and device types
    ○ Camera.Parameters and CamcorderProfile do not
    always tell the truth

    View full-size slide

  41. Camera API - The Ugly
    ● Recorded media is unplayable

    View full-size slide

  42. Camera API - The Ugly
    ● Recorded media is distorted

    View full-size slide

  43. Camera API - The Ugly
    ● Orientation not honoured

    View full-size slide

  44. Camera API - The Ugly
    ● Example code hacks:
    // TODO hack for GS2
    if (Build.MODEL.equals("GT-I9100")) {
    // green mess in video file without this
    params.set( "cam_mode", 1 );
    }
    // TODO hack for 720p on GS4
    if (Build.MODEL.equals("GT-I9500") || Build.MODEL.equals("GT-I9505")) {
    candidate = CamcorderProfile.get(CamcorderProfile.QUALITY_HIGH);
    candidate.videoFrameWidth = 1280;
    candidate.videoFrameHeight = 720;
    }

    View full-size slide

  45. FUCKING SAMSUNG

    View full-size slide

  46. https://github.com/commonsguy/cwac-camera
    CWAC-Camera

    View full-size slide

  47. CWAC-Camera - What Is It?
    ● Taking Pictures. Made Sensible.
    ● The simplicity of the Intent approach
    combined with the flexibility and control
    provided by the Camera API
    ● Encapsulates the details behind a scalable
    API
    ● Easy to do simple things
    ● Complex things not impossible
    ● Open source (Apache 2.0)

    View full-size slide

  48. CWAC-Camera - Who Wrote It?
    ● Mark Murphy - Author: The Busy Coder’s Guide to
    Android Development
    ○ http://commonsware.com/books
    ● CWAC = CommonsWare Android Components
    ○ https://github.com/commonsguy
    ● Droidcon London - "Android Security: Defending Your
    Users"
    ○ http://uk.droidcon.com/2013/buyticket/
    ● One day workshop, Oct 22 - “Pushing the UI Envelope”
    ○ http://skillsmatter.com/event/os-mobile-server/mark-
    murphys-pushing-the-ui-envelope

    View full-size slide

  49. CWAC-Camera - Overview
    ● CameraFragment:
    ○ Renders the preview using the appropriate surface
    format, optimally sized for the device
    ○ Deals with configuration changes and rotation to
    support both portrait and landscape shooting
    ○ Provides a simple API for taking pictures and
    recording video
    ○ Opens and releases the camera automatically and
    safely
    ● Native (API 11+) and support Fragment
    implementations
    ● Underlying CameraView also available

    View full-size slide

  50. CWAC-Camera - Simple Setup
    ● Add the JAR file to your project
    ● Include the ActionBarSherlock library project if supporting < API 11
    ● Add the appropriate camera manifest permissions
    ● Create an XML layout for your camera UI with a container (e.g.
    FrameLayout) in which to attach CameraFragment
    ● Create and attach an instance of CameraFragment in your Activity:
    public class MainActivity extends Activity {
    CameraFragment mCameraFragment;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    mCameraFragment = new CameraFragment();
    getFragmentManager().beginTransaction()
    .replace(R.id.preview_container, mCameraFragment).commit();
    }
    }

    View full-size slide

  51. CWAC-Camera - Taking a Picture
    ● Call takePicture()
    Button snap = (Button) findViewById(R.id.snap);
    snap.setOnClickListener(new OnClickListener() {
    @Override
    public void onClick(View v) {
    mCameraFragment.takePicture();
    }
    });

    View full-size slide

  52. CWAC-Camera - Recording a Video
    ● Call record() and stopRecording()
    Button record = (Button) findViewById(R.id.record);
    record.setOnClickListener(new OnClickListener() {
    @Override
    public void onClick(View v) {
    try {
    if (!mCameraFragment.isRecording()) {
    mCameraFragment.record();
    } else {
    mCameraFragment.stopRecording();
    }
    } catch (Exception e) {
    e.printStackTrace();
    }
    }
    });

    View full-size slide

  53. CWAC-Camera - Configuration
    ● Create a custom CameraHost:
    class MyCameraHost extends SimpleCameraHost {
    public MyCameraHost(Context context) {
    super(context);
    }
    @Override
    public boolean useFrontFacingCamera() {
    return true;
    }
    @Override
    protected File getPhotoDirectory() {
    return getExternalFilesDir(Environment.DIRECTORY_PICTURES);
    }
    }
    ● Supply CameraHost to CameraFragment:
    mCameraFragment.setHost(new MyCameraHost(this));

    View full-size slide

  54. CWAC-Camera - Configuration
    ● Use a custom CameraHost to configure:
    ○ Output file names and locations
    ○ Camera selection
    ○ Exception handling
    ○ Auto focus callbacks
    ○ Shutter callbacks
    ○ Preview sizes
    ○ Picture and video recording parameters
    ○ EXIF rotation behaviour
    ○ DeviceProfile - a class that wraps any device
    specific hacks that might be needed

    View full-size slide

  55. CWAC-Camera - Compatibility
    ● Tested Devices:
    ○ Acer Iconia Tab A700
    ○ Amazon Kindle Fire HD
    ○ ASUS Transformer Infinity
    (1st generation)
    ○ Galaxy Nexus
    ○ HTC Droid Incredible 2
    ○ HTC One S
    ○ Lenovo ThinkPad Tablet
    ○ Nexus 4
    ○ Nexus 7 (1st generation,
    2012)
    ○ Nexus 7 (2nd generation,
    2013)
    ○ Nexus 10
    ○ Nexus One
    ○ Nexus S
    ○ Motorola RAZR i
    ○ Samsung Galaxy Note 2
    ○ Samsung Galaxy S3
    ○ Samsung Galaxy S4 (GT-
    I9500)
    ○ Samsung Galaxy Tab 2
    ○ SONY Ericsson Xperia Play
    ○ SONY Xperia E
    ○ Sony Xperia S LT26i
    ○ SONY Xperia Z

    View full-size slide

  56. CWAC-Camera - Current Limitations
    ● No video recording support below API 11
    ● No “full bleed” camera preview
    ● Cannot use CameraFragment directly in
    XML layouts
    ● Portrait mode orientation issues on certain
    devices

    View full-size slide

  57. Resources
    ● Android Camera API Guide:
    ○ https://developer.android.com/guide/topics/media/camera.html
    ● “Controlling the Camera” Training Lesson:
    ○ http://developer.android.com/training/camera/cameradirect.html
    ● CameraPreview SDK sample:
    ○ samples/ApiDemos/src/com/example/android/apis/graphics/CameraPr
    eview.java
    ● AOSP Camera App Package:
    ○ https://android.googlesource.com/platform/packages/apps/Camera.git
    ● CWAC-Camera:
    ○ https://github.com/commonsguy/cwac-camera

    View full-size slide