How to use multi-camera API in Android

2ddbd112dbc8c6572fe79821584c1465?s=47 Kurun
January 17, 2020

How to use multi-camera API in Android

2ddbd112dbc8c6572fe79821584c1465?s=128

Kurun

January 17, 2020
Tweet

Transcript

  1. How to use multi-camera API in Android shibuya.apk #38 -

    report from KotlinConf 2019 (2020/01/17) @kurun_pan
  2. Self-introduction @kurun_pan @Kurun-pan Software Engineer @Sony Linux / Android /εϓϥτΡʔϯΛѪ͢ΔΤϯδχΞ

    Kurun / ͘ΔΜ
  3. Support multi-camera API from Android 9 https://source.android.com/devices/camera/multi-camera

  4. Multi-camera use case • Ұ൪ଟ͍Ϣʔεέʔε͸ɺϢʔβͷζʔϜૢ࡞ʹ࿈ಈͨ͠ΧϝϥϨϯζʢྫ͑͹ɺ޿֯/๬ԕϨ ϯζ㱻ඪ४Ϩϯζʣͷ੾Γସ͑ • ͦͷଞɺεςϨΦΧϝϥͰDepth (Ԟߦ͖) ΋ࢉग़Ͱ͖ͨΓ΋

    ޿֯Χϝϥ ඪ४Χϝϥ
  5. Logical and physical camera from Android 9 • CameraManagerܦ༝ͰऔಘՄೳͳΧϝϥ(ID)ʹlogical cameraͱphysical

    cameraͷ2छྨ͕ଘࡏ • ྫ͑͹ɺlogical-cameraΛར༻ͯ͠ɺಉ࣌ʹෳ਺ΧϝϥͷετϦʔϜऔಘͳͲ͕Մೳ ʙ"OESPJE "OESPJEʙ %FWJDFWFOEPST "QQ%FWFMPQFST ͋͘·Ͱ΋આ໌༻ͷΠϝʔδߏ੒Ͱ͢ MPHJDBMDBNFSB #BDLDBNFSB #BDLDBNFSB #BDLDBNFSB MPHJDBMDBNFSB 'SPOUDBNFSB 'SPOUDBNFSB 'SPOUDBNFSB #BDLDBNFSB #BDLDBNFSB #BDLDBNFSB MPHJDBMDBNFSB 'SPOUDBNFSB 'SPOUDBNFSB %FWJDFWFOEPST "QQ%FWFMPQFST MPHJDBMDBNFSB #BDLDBNFSB #BDLDBNFSB $BNFSB*% $BNFSB*% QIZTJDBMDBNFSB MPHJDBMDBNFSB 'SPOUDBNFSB 'SPOUDBNFSB 'SPOUDBNFSB $BNFSB*% $BNFSB*% $BNFSB*% $BNFSB*% $BNFSB*% $BNFSB*%
  6. Multi-camera API (Camera2) Ͱ ෳ਺Χϝϥͷಉ੍࣌ޚ͢Δํ๏ʹ͍ͭͯ

  7. Step 1. Get camera IDs CameraManager#getCameraIdListͰcamera idͷҰཡΛऔಘ ͨͩ͠ɺlogical-cameraʹଐ͢Δphysical-cameraͷID͸ؚ·Εͳ͍ val manager:

    CameraManager = getSystemService(Context.CAMERA_SERVICE) as CameraManager for (cameraId in manager.cameraIdList) { … } #BDLDBNFSB #BDLDBNFSB #BDLDBNFSB MPHJDBMDBNFSB 'SPOUDBNFSB 'SPOUDBNFSB %FWJDFWFOEPST "QQ%FWFMPQFST MPHJDBMDBNFSB #BDLDBNFSB #BDLDBNFSB 'SPOUDBNFSB 'SPOUDBNFSB 'SPOUDBNFSB $BNFSB*% $BNFSB*% $BNFSB*% $BNFSB*% $BNFSB*% $BNFSB*% ӈਤͷ৔߹ɺDBNFSB*E-JTU͸ ͷΈ ˞௨ৗɺ*%͕σϑΥϧτͷഎ໘Χϝϥɺ *%͕લ໘Χϝϥ
  8. Step 2. Check multi-camera is supported or not for (cameraId

    in manager.cameraIdList) { val characteristics = manager.getCameraCharacteristics(cameraId) // get supported capabilities val capabilities = characteristics.get( CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES ) // logical-camera or not? val isLogicalCamera = capabilities!!.contains( CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA ) if (isLogicalCamera) { val physicalCameraIds = characteristics.physicalCameraIds … } … } CameraManager#getCameraCharacteristicsͰlogical cameraͷαϙʔτ༗ແνΣοΫ ✓ REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERAͷνΣοΫ ✓ CameraCharacteristics#getPhysicalCameraIdsͰphysical camera idΛऔಘ #BDLDBNFSB #BDLDBNFSB #BDLDBNFSB %FWJDFWFOEPST "QQ%FWFMPQFST MPHJDBMDBNFSB #BDLDBNFSB #BDLDBNFSB $BNFSB*% $BNFSB*% $BNFSB*% ӈਤͷ৔߹ɺ QIZTJDBM$BNFSB*ET͸ 
  9. Step 3. Open logical camera CameraManager#openCameraͰɺΧϝϥIDʹର৅ͷlogical camera idΛࢦఆ ※جຊతͳCamera2Ͱͷ΍Γํͱಉ͘͡ɺstateCallbackͰCameraDeviceΛऔಘ cameraManager.openCamera(logicalCameraId,

    stateCallback, backgroundHander) #BDLDBNFSB #BDLDBNFSB #BDLDBNFSB %FWJDFWFOEPST "QQ%FWFMPQFST MPHJDBMDBNFSB #BDLDBNFSB #BDLDBNFSB $BNFSB*% $BNFSB*% $BNFSB*%
  10. Step 4. Create CaptureSession CaptureSessionͷ࡞੒ɹ※SessionConfiguration͸Multi-camera APIͱͯ͠API 28͔Β௥Ճ ✓ CameraDevice#createCaptureSessionͷҾ਺SessionConfigurationʹର৅ͷphysical camera

    id/TextureViewΛࢦఆ ✓ ͦͷޙ͸Camera2ͷ͓࡞๏ͱಉ༷ ✓ (ੜ੒ͨ͠CaptureSessionʹରͯ͠CaptureSession#setRepeatingRequestΛ࣮ߦͯ͠Ωϟϓνϟ։࢝) private fun createDualCamera(surfaces: List<Surface>) { val outputConfigs = surfaces.mapIndexed { index, surface -> val config = OutputConfiguration(surface) val physicalCameraId = physicalCameraIds.toList()[index] config.setPhysicalCameraId(physicalCameraId) config ^mapIndexed } val executor = Executors.newCachedThreadPool() val sessionConfig = SessionConfiguration( SessionConfiguration.SESSION_REGULAR, outputConfigs, executor, captureStateCallback ) cameraDevice?.createCaptureSession(sessionConfig) } #BDLDBNFSB #BDLDBNFSB #BDLDBNFSB %FWJDFWFOEPST "QQ%FWFMPQFST MPHJDBMDBNFSB #BDLDBNFSB #BDLDBNFSB $BNFSB*% $BNFSB*% $BNFSB*%
  11. ࢖ͬͯΈͨײ૝ (·ͱΊ)

  12. Points I noticed after using multi-camera API • ͍͔ͭ͘ͷAndroid 10ػछͰࢼͯ͠Έ͕ͨɺlogical-cameraඇαϙʔτػछ͕ଟͦ͏ͳؾ഑

    ✓ Pixel 3͸αϙʔτ ✓ Galaxy S10 Plus͸ඇαϙʔτ ✓ physical-camera΋શΧϝϥ෼ͷIDΛऔಘͰ͖ͳ͔ͬͨ • logical-cameraͷΈΛࢦఆͯ͠ར༻ͨ͠৔߹ͷڍಈ͕ෆ໌ ✓ Pixel3ͷ৔߹ɺ಺෦తʹ͸Ұͭͷݻఆphysical-cameraΛར༻͍ͯ͠Δ༷ࢠ ➡ ζʔϜʹԠͯ͡޿֯Ϩϯζͱඪ४Ϩϯζͷ੾Γସ͑ͳͲ΋Ұ੾ͳ͠ ➡ CaptureRequestͰઃఆՄೳͳԿ͔ͷઃఆʹ࿈ಈ͍ͯ͠Δʁ • logical-cameraʹؚ·ΕΔphysical-camera͸୯ҰͰͷར༻͕Մೳͩͬͨ
  13. Camera Architecture in Android $BNFSB)BSEXBSF -JOVY,FSOFM $BNFSB%FWJDF%SJWFS $BNFSB)"- "OESPJE'SBNFXPSL $BNFSB$BNFSB9

     "QQMJDBUJPO εϚʔτϑΥϯ 4P$  ϕϯμʔʹґଘ "OESPJE04WFSTJPO "1*-FWFM ʹґଘ ࠓ࣌఺Ͱ͸$BNFSB9͸.VMUJDBNFSB"1*ඇαϙʔτ Ұൠతʹϋʔυ΢ΣΞܥ(Bluetooth BLEͳͲ)ͷػೳ͸ػछʹڧ͘ґଘͯ͠͠·͏ͨΊɺ ΞϓϦͰͷϚϧνΧϝϥػೳαϙʔτ͕೉͍͠
  14. Thank you!☺