Slide 1
Slide 1 text
Camera2 API and
beyond
Chanoma, inc.
Tomoaki Imai
Slide 2
Slide 2 text
https://chomp.app/
Slide 3
Slide 3 text
One day, an intern and a
mentor…
Slide 4
Slide 4 text
One day, an intern and a
mentor…
Slide 5
Slide 5 text
Our source of truth
Slide 6
Slide 6 text
Our source of truth
Slide 7
Slide 7 text
Our source of truth
Slide 8
Slide 8 text
Understand the basic
Break the basic
Beyond the basic
- Flows, Classes , Callbacks
- Preview calculation
- Adjusting previews size
- Controlling Auto Exposure/Focus
- Using filters with OpenGL
Slide 9
Slide 9 text
Sample Codes
https://github.com/tomoima525/android-Camera2Basic
- Made Camera Class independent
- Callbacks are (slightly) organized
- Some features (manual focus, wb adjust)
Slide 10
Slide 10 text
Understand the basic
Slide 11
Slide 11 text
Understand the basic
• Understand Camera2 from Callbacks
• Know how the preview screen is calculated
Slide 12
Slide 12 text
Revisiting Architecture of
Camera2
$BNFSB%FWJDF )BSEXBSF
)BSEXBSF"CTUSBDUJPO-BZFS
$BNFSB4FSWJDF $
#JOEFS*OUFSGBDF +BWB
$BNFSB"1* +BWB
:PVSDPEF
Slide 13
Slide 13 text
Revisiting Architecture of
Camera2
$BNFSB%FWJDF )BSEXBSF
)BSEXBSF"CTUSBDUJPO-BZFS
$BNFSB4FSWJDF $
#JOEFS*OUFSGBDF +BWB
$BNFSB"1* +BWB
:PVSDPEF
$BNFSB%FWJDF
4UBUF$BMMCBDL
$BQUVSF4FTTJPO
4UBUF$BMMCBDL
$BQUVSF4FTTJPO
$BQUVSF$BMMCBDL
*NBHF3FBEFS
0O*NBHF"WBJMBCMF
$BNFSB$POTUSBJO
FE)JHI4QFFE$BMM
CBDL
$BNFSB%FWJDF
"WBJMBCJMJUZ$BMMCBDL
Slide 14
Slide 14 text
Revisiting Architecture of
Camera2
$BNFSB%FWJDF )BSEXBSF
)BSEXBSF"CTUSBDUJPO-BZFS
$BNFSB4FSWJDF $
#JOEFS*OUFSGBDF +BWB
$BNFSB"1* +BWB
:PVSDPEF
$BNFSB%FWJDF
4UBUF$BMMCBDL
$BQUVSF4FTTJPO
4UBUF$BMMCBDL
$BQUVSF4FTTJPO
$BQUVSF$BMMCBDL
*NBHF3FBEFS
0O*NBHF"WBJMBCMF
$BNFSB$POTUSBJO
FE)JHI4QFFE$BMM
CBDL
$BNFSB%FWJDF
"WBJMBCJMJUZ$BMMCBDL
$BMMCBDL
Slide 15
Slide 15 text
Launching Camera
Slide 16
Slide 16 text
5FYUVSF7JFX $BNFSB.BOBHFS
Slide 17
Slide 17 text
5FYUVSF7JFX $BNFSB.BOBHFS
val surfaceTextureListener = object : TextureView.SurfaceTextureListener {
override fun onSurfaceTextureAvailable(
texture: SurfaceTexture, width: Int, height: Int) {
openCamera(width, height)
}
4VSGBDF5FYUVSF
SurfaceTextureListener
Slide 18
Slide 18 text
5FYUVSF7JFX $BNFSB.BOBHFS
4VSGBDF5FYUVSF
SurfaceTextureListener
Slide 19
Slide 19 text
5FYUVSF7JFX $BNFSB.BOBHFS
4VSGBDF5FYUVSF
SurfaceTextureListener
Slide 20
Slide 20 text
5FYUVSF7JFX $BNFSB.BOBHFS
4VSGBDF5FYUVSF
SurfaceTextureListener
Slide 21
Slide 21 text
5FYUVSF7JFX $BNFSB.BOBHFS
4VSGBDF5FYUVSF
SurfaceTextureListener
Slide 22
Slide 22 text
5FYUVSF7JFX $BNFSB.BOBHFS
4VSGBDF5FYUVSF
SurfaceTextureListener
#BDL
HSPVOE
5ISFBE
Slide 23
Slide 23 text
5FYUVSF7JFX $BNFSB.BOBHFS
$BNFSB%FWJDF
CameraDevice.StateCallback
4VSGBDF5FYUVSF
SurfaceTextureListener
#BDL
HSPVOE
5ISFBE
$BNFSB
'SBNF
8PSL
openCamera
Slide 24
Slide 24 text
5FYUVSF7JFX $BNFSB.BOBHFS
$BNFSB%FWJDF
CameraDevice.StateCallback
4VSGBDF5FYUVSF
SurfaceTextureListener
#BDL
HSPVOE
5ISFBE
$BNFSB
'SBNF
8PSL
openCamera
Slide 25
Slide 25 text
5FYUVSF7JFX $BNFSB.BOBHFS
$BNFSB%FWJDF
CameraDevice.StateCallback
4VSGBDF5FYUVSF
SurfaceTextureListener
#BDL
HSPVOE
5ISFBE
$BNFSB
'SBNF
8PSL
openCamera
Slide 26
Slide 26 text
5FYUVSF7JFX $BNFSB.BOBHFS
$BNFSB%FWJDF
CameraDevice.StateCallback
4VSGBDF5FYUVSF
SurfaceTextureListener
#BDL
HSPVOE
5ISFBE
$BNFSB
'SBNF
8PSL
openCamera
createCaptureSession
Slide 27
Slide 27 text
5FYUVSF7JFX $BNFSB.BOBHFS
$BNFSB%FWJDF
$BQUVSF4FTTJPO
CameraDevice.StateCallback
CaptureSession.StateCallback
4VSGBDF5FYUVSF
SurfaceTextureListener
#BDL
HSPVOE
5ISFBE
$BNFSB
'SBNF
8PSL
openCamera
createCaptureSession
Slide 28
Slide 28 text
5FYUVSF7JFX $BNFSB.BOBHFS
$BNFSB%FWJDF
$BQUVSF4FTTJPO
CameraDevice.StateCallback
4VSGBDF5FYUVSF
CaptureSession.CaptureCallback
SurfaceTextureListener
#BDL
HSPVOE
5ISFBE
$BNFSB
'SBNF
8PSL
openCamera
CaptureSession.StateCallback
createCaptureSession
Slide 29
Slide 29 text
Taking Photo
Slide 30
Slide 30 text
One Session, One Callback
Slide 31
Slide 31 text
One Session, One Callback
Slide 32
Slide 32 text
override fun onCaptureCompleted(session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult) {
// Check for Auto Focus
val afState = result.get(CaptureResult.CONTROL_AF_STATE)
if(afState == CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED) { }
}
Slide 33
Slide 33 text
override fun onCaptureCompleted(session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult) {
// Check for Auto Focus
val afState = result.get(CaptureResult.CONTROL_AF_STATE)
if(afState == CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED) {
// Check for Auto Exposure Status
val aeState = result.get(CaptureResult.CONTROL_AE_STATE)
if(aeState == CaptureResult.CONTROL_AE_STATE_CONVERGED) {
}
}
Slide 34
Slide 34 text
override fun onCaptureCompleted(session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult) {
// Check for Auto Focus
val afState = result.get(CaptureResult.CONTROL_AF_STATE)
if(afState == CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED) {
// Check for Auto Exposure Status
val aeState = result.get(CaptureResult.CONTROL_AE_STATE)
if(aeState == CaptureResult.CONTROL_AE_STATE_CONVERGED) {
// Auto Exposure is ready
captureStillPicture()
} else if(aeState != CaptureResult.CONTROL_AE_STATE_INACTIVE)) {
runPreCapture()
}
}
}
Slide 35
Slide 35 text
override fun onCaptureCompleted(session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult) {
// Check for Auto Focus
val afState = result.get(CaptureResult.CONTROL_AF_STATE)
if(afState == CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED) {
// Check for Auto Exposure Status
val aeState = result.get(CaptureResult.CONTROL_AE_STATE)
if(aeState == CaptureResult.CONTROL_AE_STATE_CONVERGED) {
// Auto Exposure is ready
captureStillPicture()
} else if(aeState != CaptureResult.CONTROL_AE_STATE_INACTIVE)) {
runPreCapture()
}
} else if(afState == null) {
// Auto Focus is null when you first request, in that case
// request Auto Exposure until Auto Focus returns
runPreCapture()
}
}
Slide 36
Slide 36 text
override fun onCaptureCompleted(session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult) {
// Check for Auto Focus
val afState = result.get(CaptureResult.CONTROL_AF_STATE)
if(afState == CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED) {
// Check for Auto Exposure Status
val aeState = result.get(CaptureResult.CONTROL_AE_STATE)
if(aeState == CaptureResult.CONTROL_AE_STATE_CONVERGED) {
// Auto Exposure is ready
captureStillPicture()
} else if(aeState != CaptureResult.CONTROL_AE_STATE_INACTIVE)) {
runPreCapture()
} else if(aeState == null) {
// Some device do not support AutoExposure. in that case
// skip Auto Exposure and request Camera Capture
captureStillPicture()
}
} else if(afState == null) {
// Auto Focus is null when you first request, in that case
// request Auto Exposure until Auto Focus returns
runPreCapture()
}
}
Slide 37
Slide 37 text
override fun onCaptureCompleted(session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult) {
// Check for Auto Focus
val afState = result.get(CaptureResult.CONTROL_AF_STATE)
if(afState == CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED) {
// Check for Auto Exposure Status
val aeState = result.get(CaptureResult.CONTROL_AE_STATE)
if(aeState == CaptureResult.CONTROL_AE_STATE_CONVERGED
// Auto Exposure might request Flash
// but we can't wait for that...
|| aeState == CaptureRequest.CONTROL_AE_STATE_FLASH_REQUIRED) {
// Auto Exposure is ready
captureStillPicture()
} else if(aeState != CaptureResult.CONTROL_AE_STATE_INACTIVE)) {
runPreCapture()
} else if(aeState == null) {
// Some device do not support AutoExposure. in that case
// skip Auto Exposure and request Camera Capture
captureStillPicture()
}
} else if(afState == null) {
// Auto Focus is null when you first request, in that case
// request Auto Exposure until Auto Focus returns
runPreCapture()
}
}
Slide 38
Slide 38 text
override fun onCaptureCompleted(session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult) {
// Check for Auto Focus
val afState = result.get(CaptureResult.CONTROL_AF_STATE)
if(afState == CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED) {
// Check for Auto Exposure Status
val aeState = result.get(CaptureResult.CONTROL_AE_STATE)
if(aeState == CaptureResult.CONTROL_AE_STATE_CONVERGED
// Auto Exposure might request Flash
// but we can't wait for that...
|| aeState == CaptureRequest.CONTROL_AE_STATE_FLASH_REQUIRED) {
// Auto Exposure is ready
captureStillPicture()
} else if(aeState != CaptureResult.CONTROL_AE_STATE_INACTIVE)) {
runPreCapture()
} else if(aeState == null) {
// Some device do not support AutoExposure. in that case
// skip Auto Exposure and request Camera Capture
captureStillPicture()
}
} else if(afState == null) {
// Auto Focus is null when you first request, in that case
// request Auto Exposure until Auto Focus returns
runPreCapture()
}
}
Slide 39
Slide 39 text
private enum class State {
PREVIEW,
WAITING_LOCK,
WAITING_PRECAPTURE,
WAITING_NON_PRECAPTURE,
TAKEN
}
override fun onCaptureCompleted(session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult) {
when (state) {
State.WAITING_LOCK -> {
// Check for Auto Focus state
val afState = result.get(CaptureResult.CONTROL_AF_STATE)
}
State.WAITING_PRECAPTURE -> {
// Check for Auto Exposure State
val aeState = result.get(CaptureResult.CONTROL_AE_STATE)
}
…
}
Keep track of Request State
Slide 40
Slide 40 text
focus
private fun lockFocus() {
state = State.WAITING_LOCK
val builder = createPreviewRequestBuilder()
captureSession?.capture(
builder?.build(), captureCallback, backgroundHandler)
}
private fun createPreviewRequestBuilder(): CaptureRequest.Builder? {
…
if (characteristics.isContinuousAutoFocusSupported()) {
builder.set(
CaptureRequest.CONTROL_AF_MODE,
CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE)
} else {
builder.set(
CaptureRequest.CONTROL_AF_MODE,
CaptureRequest.CONTROL_AF_MODE_AUTO
)
}
}
Slide 41
Slide 41 text
focus
private fun lockFocus() {
state = State.WAITING_LOCK
val builder = createPreviewRequestBuilder()
captureSession?.capture(
builder?.build(), captureCallback, backgroundHandler)
}
State.WAITING_LOCK -> {
val afState = result.get(CaptureResult.CONTROL_AF_STATE)
if (CaptureResult.CONTROL_AF_STATE_INACTIVE == afState ||
CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED == afState ||
CaptureResult.CONTROL_AF_STATE_NOT_FOCUSED_LOCKED == afState) {
// Request Auto Exposure
runPreCapture()
} else {
captureStillPicture()
}
}
Slide 42
Slide 42 text
focus precapture
fun runPreCapture() {
state = State.WAITING_PRECAPTURE
val builder = createPreviewRequestBuilder()
builder?.set(
CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER,
CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER_START
)
captureSession?.capture(
builder?.build(), captureCallback, backgroundHandler)
}
Slide 43
Slide 43 text
focus precapture
fun runPreCapture() {
state = State.WAITING_PRECAPTURE
val builder = createPreviewRequestBuilder()
builder?.set(
CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER,
CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER_START
)
captureSession?.capture(
builder?.build(), captureCallback, backgroundHandler)
}
Slide 44
Slide 44 text
focus precapture capture
private fun captureStillPicture() {
state = State.TAKEN
val builder = cameraDevice?.createCaptureRequest(
CameraDevice.TEMPLATE_STILL_CAPTURE)
// bind imageReader so that we can pass the result
builder?.addTarget(imageReader?.surface)
builder?.addTarget(surface)
captureSession?.stopRepeating()
captureSession?.capture(
builder?.build(),{…},backgroundHandler)
}
Slide 45
Slide 45 text
focus precapture capture take picture
fun takePicture(handler : ImageHandler) {
imageReader?.setOnImageAvailableListener(
object: ImageReader.OnImageAvailableListener{
override fun onImageAvailable(reader: ImageReader) {
val image = reader.acquireNextImage()
backgroundHandler?.post(
handler.handleImage(image = image))
}
}, backgroundHandler)
lockFocus()
}
object : ImageHandler {
override fun handleImage(image: Image): Runnable {
// Save your image
}
Slide 46
Slide 46 text
Anatomy of calculating
Preview Size
Slide 47
Slide 47 text
Adjusting Preview size
aspect ratio : 1.33
width : 1080
height : 1436
aspect ratio : 1.22
width : 1080
height : 1436
aspect ratio : 1.77
width : 1000
height : 1778
Slide 48
Slide 48 text
Steps to adjust preview size
• Calculate view rotation
• Pick the supported preview size from Camera
• Adjust TextureView based on the preview
size
Slide 49
Slide 49 text
Calculate view rotation
characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION)
Sensor Orientation
- Angle that needs to be rotated to get upright image on device
- Based on how the sensor is implemented on device
h
w
what Camera Sensor
Captures
SENSOR_ORIENTATION = 90
Image output
Slide 50
Slide 50 text
Calculate view rotation
Display Orientation
DisplayOrientation = ROTATION_0 (Portrait)
activity?.windowManager?.defaultDisplay?.rotation
screen_w
screen_h
Slide 51
Slide 51 text
height
width
CAMERA DEVICE WORLD
width(displaySize.x)
height
(displaySize.y)
DISPLAY WORLD
previewSize =
cameraCharacteristics.chooseOptimalSize(
textureViewWidth = textureHeight,
textureViewHeight = textureWidth,
maxWidth = displaySize.y,
maxHeight = displaySize.x,
aspectRatio = largest)
Calculate view rotation
Slide 52
Slide 52 text
val map: StreamConfigurationMap =
cameraCharacteristic.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
val choices = map.getOutputSizes(SurfaceTexture::class.java)
Fetch supported Sizes
Slide 53
Slide 53 text
[4032x3024, 4000x3000, 3840x2160,
4000x2000, 3264x2448, 3200x2400,
2688x1512, 2592x1944, 2048x1536,
1920x1440, 1920x1080, 1600x1200,
1920x960, 1280x960, 1280x768,
1280x720, 1024x768, 800x400,
800x600, 800x480, 720x480,
640x400, 640x480, 640x360,
352x288, 320x240, 176x144,
160x120]
Fetch supported Sizes
val map: StreamConfigurationMap =
cameraCharacteristic.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
val choices = map.getOutputSizes(SurfaceTexture::class.java)
Slide 54
Slide 54 text
Look for supported Sizes
1) Smaller than defined max size
[4032x3024, 4000x3000, 3840x2160,
4000x2000, 3264x2448, 3200x2400,
2688x1512, 2592x1944, 2048x1536,
1920x1440, 1920x1080, 1600x1200,
1920x960, 1280x960, 1280x768,
1280x720, 1024x768, 800x400,
800x600, 800x480, 720x480,
640x400, 640x480, 640x360,
352x288, 320x240, 176x144,
160x120]
val choices = map.getOutputSizes(SurfaceTexture::class.java)
for (op in choices) {
if (op.width <= MAX_PREVIEW_WIDTH &&
op.height <= MAX_PREVIEW_HEIGHT) {
}
}
Slide 55
Slide 55 text
Look for supported Sizes
2) Pick ones aspect ratio matches
val w = aspectRatio.width // 4032
val h = aspectRatio.height // 3024
for (op in choices) {
if (op.width <= _maxWidth &&
op.height <= _maxHeight &&
op.height == op.width * h / w) {
}
[4032x3024, 4000x3000, 3840x2160,
4000x2000, 3264x2448, 3200x2400,
2688x1512, 2592x1944, 2048x1536,
1920x1440, 1920x1080, 1600x1200,
1920x960, 1280x960, 1280x768,
1280x720, 1024x768, 800x400,
800x600, 800x480, 720x480,
640x400, 640x480, 640x360,
352x288, 320x240, 176x144,
160x120]
How to decide Aspect Ratio of Preview?
- Pick the largest from supported ex) 4032 / 3024 = 1.33
Slide 56
Slide 56 text
Look for supported Sizes
2) Pick ones aspect ratio matches
[4032x3024, 4000x3000, 3840x2160,
4000x2000, 3264x2448, 3200x2400,
2688x1512, 2592x1944, 2048x1536,
1920x1440, 1920x1080, 1600x1200,
1920x960, 1280x960, 1280x768,
1280x720, 1024x768, 800x400,
800x600, 800x480, 720x480,
640x400, 640x480, 640x360,
352x288, 320x240, 176x144,
160x120]
val w = aspectRatio.width // 4032
val h = aspectRatio.height // 3024
for (op in choices) {
if (op.width <= _maxWidth &&
op.height <= _maxHeight &&
op.height == op.width * h / w) {
}
How to decide Aspect Ratio of Preview?
- Pick the largest from supported ex) 4032 / 3024 = 1.33
Slide 57
Slide 57 text
Look for supported Sizes
3) Pick the closest one you can get
[4032x3024, 4000x3000, 3840x2160,
4000x2000, 3264x2448, 3200x2400,
2688x1512, 2592x1944, 2048x1536,
1920x1440, 1920x1080, 1600x1200,
1920x960, 1280x960, 1280x768,
1280x720, 1024x768, 800x400,
800x600, 800x480, 720x480,
640x400, 640x480, 640x360,
352x288, 320x240, 176x144,
160x120]
// textureWidth = 1080
// textureHeight = 2080
if (op.width >= textureWidth &&
op.height >= textureHeight) {
bigEnough.add(op)
} else {
notBigEnough.add(op)
}
- If candidates are larger than texture size, pick the smallest
return when {
bigEnough.size > 0 -> bigEnough.sortedWith(area()).first()
notBigEnough.size > 0 -> notBigEnough.sortedWith(area()).last()
}
- If candidates are smaller than texture size, pick the largest
Slide 58
Slide 58 text
Adjust TextView Size with Preview
ratio
5FYUVSF7JFX
1080
1436
ratioWidth = 1280
ratioHeight = 960
// aspect = ratioHeight/ratioWidth = 1.33
override fun onMeasure(
widthMeasureSpec: Int,
heightMeasureSpec: Int) {
val width =
MeasureSpec.getSize(widthMeasureSpec)
val height =
MeasureSpec.getSize(heightMeasureSpec)
setMeasuredDimension(
width,
width * ratioHeight / ratioWidth)
}
TextureView
Slide 59
Slide 59 text
Break the basic
Slide 60
Slide 60 text
Break the basic
• Implement Full screen Preview
• Control Camera Exposure mode
• Manual focus
Slide 61
Slide 61 text
Full Screen Preview
Slide 62
Slide 62 text
Prerequiste
- <item name="android:windowFullscreen">true</item>
+ <item name="android:windowTranslucentStatus">true</item>
Slide 63
Slide 63 text
Full Screen Preview
5FYUVSF7JFX
1080
2160
[4032x3024, 4000x3000, 3840x2160,
4000x2000, 3264x2448, 3200x2400,
2688x1512, 2592x1944, 2048x1536,
1920x1440, 1920x1080, 1600x1200,
1920x960, 1280x960, 1280x768,
1280x720, 1024x768, 800x400,
800x600, 800x480, 720x480,
640x400, 640x480, 640x360,
352x288, 320x240, 176x144,
160x120]
val realSize = Point()
activity?.windowManager?.defaultDisplay?.getRealSize(realSize)
val aspectRatio = realSize.x.toFloat()/ realSize.y.toFloat() // 2.0
Define Aspect Ratio from windowSize
Slide 64
Slide 64 text
Full Screen Preview
[4032x3024, 4000x3000, 3840x2160,
4000x2000, 3264x2448, 3200x2400,
2688x1512, 2592x1944, 2048x1536,
1920x1440, 1920x1080, 1600x1200,
1920x960, 1280x960, 1280x768,
1280x720, 1024x768, 800x400,
800x600, 800x480, 720x480,
640x400, 640x480, 640x360,
352x288, 320x240, 176x144,
160x120]
val realSize = Point()
activity?.windowManager?.defaultDisplay?.getRealSize(realSize)
val aspectRatio = realSize.x.toFloat()/ realSize.y.toFloat() // 2.0
Closest Aspect Ratio
Define Aspect Ratio from windowSize
Slide 65
Slide 65 text
Full Screen Preview
val realSize = Point()
activity?.windowManager?.defaultDisplay?.getRealSize(realSize)
val aspectRatio = realSize.x.toFloat()/ realSize.y.toFloat() // 2.0
Aspect Ratio = 2.0
&& height < 1960 && width <1080
Define Aspect Ratio from windowSize
[4032x3024, 4000x3000, 3840x2160,
4000x2000, 3264x2448, 3200x2400,
2688x1512, 2592x1944, 2048x1536,
1920x1440, 1920x1080, 1600x1200,
1920x960, 1280x960, 1280x768,
1280x720, 1024x768, 800x400,
800x600, 800x480, 720x480,
640x400, 640x480, 640x360,
352x288, 320x240, 176x144,
160x120]
val w = aspectRatio.width // 4000
val h = aspectRatio.height // 2000
for (op in choices) {
if (op.width <= _maxWidth &&
op.height <= _maxHeight &&
op.height == op.width * h / w) {
}
Slide 66
Slide 66 text
Controlling Camera Modes
Slide 67
Slide 67 text
Controlling Camera Modes
• Know what kind of mode is supported on
your device
• Add an parameters
CameraRequest.requestBuilder and call
CaptureSession.setRepeatingRequest
Slide 68
Slide 68 text
Know what kind of mode is
supported
CameraCharasteristics.get()
- Returns CameraMetaData, Range, Region, Int, etc
Slide 69
Slide 69 text
Know what kind of mode is
supported
val list = characteristics.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_MODES)
CameraCharasteristics.get(Key modes)
Slide 70
Slide 70 text
Know what kind of mode is
supported
val list = characteristics.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_MODES)
// list: {0, 1, 2, 3, 4}
CameraCharasteristics.get(Key modes)
Slide 71
Slide 71 text
Know what kind of mode is
supported
val list = characteristics.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_MODES)
// list: {0, 1, 2, 3, 4}
CameraCharasteristics.get(Key modes)
https://developer.android.com/reference/android/hardware/camera2/
CameraCharacteristics#CONTROL_AE_AVAILABLE_MODES
Slide 72
Slide 72 text
Know what kind of mode is
supported
val list = characteristics.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_MODES)
// list: {0, 1, 2, 3, 4}
CameraCharasteristics.get(Key modes)
https://developer.android.com/reference/android/hardware/camera2/CaptureRequest.html#CONTROL_AE_MODE
Slide 73
Slide 73 text
Know what kind of mode is
supported
val list = characteristics.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_MODES)
// list: {0, 1, 2, 3, 4}
CameraCharasteristics.get(Key modes)
https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#CONTROL_AE_MODE_ON
Slide 74
Slide 74 text
Know what kind of mode is
supported
val list = characteristics.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_MODES)
// list: {0, 1, 2, 3, 4} = {OFF, ON, AUTO_FLASH, ALWAYS_FLASH AUTO_FLASH_REDEYE}
CameraCharasteristics.get(Key modes)
https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#CONTROL_AE_MODE_ON
Slide 75
Slide 75 text
Know what kind of mode is
supported
CameraCharasteristics.get(Key modes)
- Returns CameraMetaData
- Use CaptureRequest(Subclass of CameraMetaData) for
Requesting capture session
// You can write like this, but don’t
builder.set(
CaptureRequest.CONTROL_AF_TRIGGER,
CameraMetadata.CONTROL_AF_TRIGGER_START)
// Do
builder.set(
CaptureRequest.CONTROL_AF_TRIGGER,
CaptureRequest.CONTROL_AF_TRIGGER_START)
Slide 76
Slide 76 text
Requesting Camera Modes
override fun onClick(view: View) {
when (view.id) {
R.id.sun -> {
camera?.wbMode = WBMode.SUNNY
openCamera(textureView.width, textureView.height) // Re-request
}
CameraFragment
Slide 77
Slide 77 text
Requesting Camera Modes
override fun onClick(view: View) {
when (view.id) {
R.id.sun -> {
camera?.wbMode = WBMode.SUNNY
openCamera(textureView.width, textureView.height) // Re-request
}
val builder = cameraDevice?.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)
when(wbMode) {
WBMode.SUNNY -> {
builder.set(
CaptureRequest.CONTROL_AWB_MODE, CaptureRequest.CONTROL_AWB_MODE_DAYLIGHT)
}
}
captureSession?.setRepeatingRequest(
builder?.build(), captureCallback, backgroundHandler)
CameraFragment
Camera
Slide 78
Slide 78 text
Enabling Tap to Focus/Exposure
Slide 79
Slide 79 text
Understand camera metering
val rect = Rect(
focusLeft, focusBottom, focusLeft + areaSize, focusBottom + areaSize)
val meteringRectangle = MeteringRectangle(rect, weight = 500)
builder.set(CaptureRequest.CONTROL_AF_REGIONS, arrayOf(meteringRectangle))
builder.set(CaptureRequest.CONTROL_AE_REGIONS, arrayOf(meteringRectangle))
val numOfMaxregionsAF =
cameraCharacteristic.get(
CameraCharacteristics.CONTROL_MAX_REGIONS_AF)
#
XFJHIUSBOHF
"
Slide 80
Slide 80 text
Understand camera metering
3024
4032
Image Sensor
1080
2160
3024 * 500/ 1080 = 1400
4032 * 1000 / 2160 = 1866
Screen
val rect = characteristics.get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE)
val right = rect.right
val bottom = rect.bottom
val censorX= right * screenX / screenWidth
val censorY = bottom * screenY / screenHeight
Slide 81
Slide 81 text
Beyond the basic
Using filters with OpenGL
Slide 82
Slide 82 text
How OpenGL and Camera
works
5FYUVSF7JFX
4VSGBDF5FYUVSF DBNFSB$BNFSB%FWJDF
Camera Stream
Slide 83
Slide 83 text
4VSGBDF5FYUVSF
How OpenGL and Camera
works
5FYUVSF7JFX
8JOEPX4VSGBDF
&(-#BTF4VSGBDF
4VSGBDF5FYUVSF
DBNFSB$BNFSB%FWJDF
5FYUVSF<>
4QFDJBM4VSGBDFUIBU
JOUFSBDUXJUI0QFO(-
Camera Stream
https://github.com/google/grafika/blob/master/app/src/main/java/com/android/grafika/gles/WindowSurface.java
Slide 84
Slide 84 text
Activity(Fragment)
SurfaceTextureListener
.onSurfaceTexture
Available
TextureView
openCamera
onResume
startBackground
Thread
startPreview
setCameraOutput
configureTransform startCamera
Camera
Slide 85
Slide 85 text
Activity(Fragment)
SurfaceTextureListener
.onSurfaceTexture
Available
TextureView
openCamera
onResume
startBackground
Thread
startPreview
setCameraOutput
configureTransform startCamera
Camera
setup EGL
and WindowSurface
OpenGL layer
EGL
- Interface between OpenGL and Native Window system
8JOEPX4VSGBDF
&(-#BTF4VSGBDF
Slide 86
Slide 86 text
Activity(Fragment)
SurfaceTextureListener
.onSurfaceTexture
Available
TextureView
openCamera
onResume
startBackground
Thread
startPreview
setCameraOutput
configureTransform startCamera
Camera
setup EGL
and WindowSurface
OpenGL layer
setup Textures
Textures
- Render graphics up on WindowSurface
8JOEPX4VSGBDF
&(-#BTF4VSGBDF
5FYUVSF<>
5FYUVSF<>
5FYUVSF<>
5FYUVSF<>
Slide 87
Slide 87 text
Activity(Fragment)
SurfaceTextureListener
.onSurfaceTexture
Available
TextureView
openCamera
onResume
startBackground
Thread
startPreview
setCameraOutput
configureTransform startCamera
Camera
setup EGL
and WindowSurface
OpenGL layer
setup Textures
setup
SurfaceTexture
SurfaceTexture
//set texture[0] to camera texture
GLES20.glActiveTexture(GLES20.GL_TEXTURE0)
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, texturesIds[0])
previewSurfaceTexture = SurfaceTexture(texturesIds[0])
Slide 88
Slide 88 text
Activity(Fragment)
SurfaceTextureListener
.onSurfaceTexture
Available
TextureView
openCamera
onResume
startBackground
Thread
startPreview
setCameraOutput
configureTransform startCamera
Camera
setup EGL
and WindowSurface
OpenGL layer
setup Textures
setup
SurfaceTexture
load Shader
Shaders (written with glsl)
- VertexShader
Renders graphics with given Texture Coordinates and Transform
Matrix
- FragmentShader
Renders Colors for each pixels
Slide 89
Slide 89 text
Activity(Fragment)
SurfaceTextureListener
.onSurfaceTexture
Available
TextureView
openCamera
onResume
startBackground
Thread
startPreview
setCameraOutput
configureTransform startCamera
Camera
setup EGL
and WindowSurface
OpenGL layer
setup Textures
setup
SurfaceTexture
load Shader
onRendererReady
Slide 90
Slide 90 text
Demo
• Randomly updates background color with
filter
Slide 91
Slide 91 text
Wrap up
• Learn Callbacks!
• UnderStand Camera Sensor and Screen are
different worlds
• Write sample camera app by yourself
Slide 92
Slide 92 text
Reference
https://github.com/googlecreativelab/shadercam
OpenGL
Camera
https://photographylife.com/category/photography-
tutorials
https://developer.android.com/reference/android/
hardware/camera2/package-summary
https://learnopengl.com/Getting-started/Shaders
Slide 93
Slide 93 text
Thank you