作ってわかる！LiDARによるカメラの暗所オートフォーカス機能

by tamaUdon

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

© ZOZO Technologies, Inc. 28 AVFoundation  AVCaptureSession  通常のカメラ  ARKit  ARSession  LiDAR  * fPOI … focusPointOfInterest  fPOI 座標  LiDARによるAF機能の実装　 ④  再度ボタンタップで  LiDAR 停止  AVFoundation 側に  fPOI* の座標を渡す 

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

© ZOZO Technologies, Inc. 32 let arConfiguration = ARWorldTrackingConfiguration() let arSession: ARSession = ARSession() ... func setupLiDAR() { if type(of: arConfiguration).supportsFrameSemantics(.sceneDepth){ arConfiguration.frameSemantics = .sceneDepth arConfiguration.isAutoFocusEnabled = false } arSession.delegate = self } LiDAR セットアップ  LiDARによるAF機能の実装　

Slide 33

Slide 33 text

© ZOZO Technologies, Inc. 33 ボタンタップの処理 - AVCaptureSession  LiDARによるAF機能の実装　 var handler: ((UIImage) -> Void)? ... func runAvCapture(_ handler: @escaping (UIImage) -> Void) { arSession.pause() // ARSessionを停止 self.handler = handler // handlerを設定 self.setMode(focusMode: .locked, exposureMode: .locked) captureSession.startRunning() if (isfPOIExist) { // fPOI設定 self.setMode(focusMode: .autoFocus, exposureMode: .autoExpose, point: CGPoint(x: x, y: y)) } ... }

Slide 34

Slide 34 text

© ZOZO Technologies, Inc. 34 ボタンタップの処理 - ARSession  LiDARによるAF機能の実装　 var handler: ((UIImage) -> Void)? ... func runAR(_ handler: @escaping (UIImage) -> Void) { captureSession.stopRunning() // AVCaptureSessionを停止 self.handler = handler // handler設定 arSession.run(arConfiguration) }

Slide 35

Slide 35 text

© ZOZO Technologies, Inc. 35 ボタンタップの処理 - UI  LiDARによるAF機能の実装　 VStack (alignment: .center) { ... Button(action: { if (isAvCaptureRunning) { captureWithDepth.runAR { uiImage in DispatchQueue.main.async { self.image = uiImage } } ... } else { captureWithDepth.runAvCapture { uiImage in DispatchQueue.main.async { self.image = uiImage } } ...

Slide 36

Slide 36 text

© ZOZO Technologies, Inc. 36 ARSessionDelegateの処理  LiDARによるAF機能の実装　 extension MyClass: ARSessionDelegate { func session(_ session: ARSession, didUpdate frame: ARFrame) { if let handler = handler { guard let imageBuffer = frame.sceneDepth?.depthMap else ... ... if (isfPOIExist){ if let imageWithfPOI = drawfPOIOnImage(fPOI) { image = imageWithfPOI } } handler(image) DispatchQueue.global(qos: .userInitiated).async { self.computefPOI(imageBuffer: imageBuffer, image: image) } } } }

Slide 37

Slide 37 text

Slide 38

Slide 38 text

© ZOZO Technologies, Inc. 38 computefPOIの処理 ①  LiDARによるAF機能の実装　 var depthArray: [Float32]? ... func computefPOI(imageBuffer: CVPixelBuffer, image: UIImage) { frameCounter += 1 self.buildDepthInfo(depthMap: imageBuffer) // depthArrayを取得 if (frameCounter >= 60) { ... } else { // 最もカメラに近い点の座標 x, y を計算 if let minDepthVal = depthArray.min(), let fidx = depthArray.firstIndex(of: minDepthVal) { x_sum += (fidx % Int(image.size.width)) y_sum += (fidx / Int(image.size.height)) } } }

Slide 39

Slide 39 text

© ZOZO Technologies, Inc. 39 computefPOIの処理 ②  LiDARによるAF機能の実装　 func computefPOI(imageBuffer: CVPixelBuffer, image: UIImage) { frameCounter += 1 ... if (frameCounter >= 60) { // 平均 centroid_x = x_sum / frameCounter centroid_y = y_sum / frameCounter x_sum = 0 y_sum = 0 frameCounter = 0 } else { ...

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

© ZOZO Technologies, Inc. 45 参考  ● Autofocus - Wikipedia  ● オートフォーカスを活用しましょう！ | 富士フイルム  ● iPhone 12 Pro - 技術仕様  ● LiDARとは？ | レーザーダイオードとは？ | エレクトロニクス豆知識 | ローム株式会社 - ROHM Semiconductor ● iPhone、iPad、iPod touch で計測 App を使う - Apple サポート (日本)  