Come learn about what’s new in iOS 11 with AVFoundation, as David Okun walks us through new ways to capture still images, live photos, and depth maps via his open source framework, Lumina.
What We’ll Cover • A history of AVFoundation’s input and output • What changed in iOS 10 • New methods in iOS 11 • Capturing Depth Data • Lumina @dokun24
What We’ll Cover • A history of AVFoundation’s input and output • What changed in iOS 10 • New methods in iOS 11 • Capturing Depth Data • Lumina • Synchronizing Asynchronously Streaming Data @dokun24
Since iOS 10 AVCapturePhotoSettings var uniqueID: Int64 // generated on init var flashMode: AVCaptureDevice.FlashMode var isAutoStillImageStabilizationEnabled: Bool @dokun24
Since iOS 10 AVCapturePhotoSettings var uniqueID: Int64 // generated on init var flashMode: AVCaptureDevice.FlashMode var isAutoStillImageStabilizationEnabled: Bool var isHighResolutionPhotoEnabled: Bool @dokun24
Since iOS 10 AVCapturePhotoSettings var uniqueID: Int64 // generated on init var flashMode: AVCaptureDevice.FlashMode var isAutoStillImageStabilizationEnabled: Bool var isHighResolutionPhotoEnabled: Bool var isAutoDualCameraFusionEnabled: Bool // iOS 10.2+ @dokun24
Since iOS 10 AVCaptureResolvedPhotoSettings var uniqueID: Int64 var photoDimensions: CMVideoDimensions var livePhotoMovieDimensions: CMVideoDimensions @dokun24
Since iOS 10 AVCaptureResolvedPhotoSettings var uniqueID: Int64 var photoDimensions: CMVideoDimensions var livePhotoMovieDimensions: CMVideoDimensions var isFlashEnabled: Bool @dokun24
Since iOS 10 AVCaptureResolvedPhotoSettings var uniqueID: Int64 var photoDimensions: CMVideoDimensions var livePhotoMovieDimensions: CMVideoDimensions var isFlashEnabled: Bool var isStillImageStabilizationEnabled: Bool @dokun24
Since iOS 10 AVCaptureResolvedPhotoSettings var uniqueID: Int64 var photoDimensions: CMVideoDimensions var livePhotoMovieDimensions: CMVideoDimensions var isFlashEnabled: Bool var isStillImageStabilizationEnabled: Bool var isDualCameraFusionEnabled: Bool // iOS 10.2+ @dokun24
New in iOS 11 AVCapturePhoto var pixelBuffer: CVPixelBuffer? var previewPixelBuffer: CVPixelBuffer? var metadata: [String : Any] var resolvedSettings: AVCaptureResolvedPhotoSettings @dokun24
New in iOS 11 AVCapturePhoto var pixelBuffer: CVPixelBuffer? var previewPixelBuffer: CVPixelBuffer? var metadata: [String : Any] var resolvedSettings: AVCaptureResolvedPhotoSettings var photoCount: Int @dokun24
New in iOS 11 AVCapturePhoto var pixelBuffer: CVPixelBuffer? var previewPixelBuffer: CVPixelBuffer? var metadata: [String : Any] var resolvedSettings: AVCaptureResolvedPhotoSettings var photoCount: Int var depthData: AVDepthData? @dokun24
New in iOS 11 AVCapturePhoto @dokun24 var pixelBuffer: CVPixelBuffer? var previewPixelBuffer: CVPixelBuffer? var metadata: [String : Any] var resolvedSettings: AVCaptureResolvedPhotoSettings var photoCount: Int var depthData: AVDepthData?
New in iOS 11 AVCapturePhoto @dokun24 var pixelBuffer: CVPixelBuffer? var previewPixelBuffer: CVPixelBuffer? var metadata: [String : Any] var resolvedSettings: AVCaptureResolvedPhotoSettings var photoCount: Int var depthData: AVDepthData?
Capturing Depth Data AVDepthData var depthDataAccuracy: AVDepthData.Accuracy var isDepthDataFiltered: Bool var depthDataQuality: AVDepthData.Quality @dokun24
Capturing Depth Data AVDepthData var depthDataAccuracy: AVDepthData.Accuracy var isDepthDataFiltered: Bool var depthDataQuality: AVDepthData.Quality var cameraCalibrationData: AVCameraCalibrationData? @dokun24
Capturing Depth Data AVDepthData var depthDataAccuracy: AVDepthData.Accuracy var isDepthDataFiltered: Bool var depthDataQuality: AVDepthData.Quality var cameraCalibrationData: AVCameraCalibrationData? var depthDataMap: CVPixelBuffer @dokun24
Capturing Depth Data // in your capture session configuration if self.photoOutput.isDepthDataDeliverySupported { self.photoOutput.isDepthDataDeliveryEnabled = true } @dokun24
Capturing Depth Data // in your capture session configuration if self.photoOutput.isDepthDataDeliverySupported { self.photoOutput.isDepthDataDeliveryEnabled = true } // in your preparation for capturing a still image with AVCapturePhotoSettings if self.photoOutput.isDepthDataDeliverySupported { settings.isDepthDataDeliveryEnabled = true } @dokun24
Lumina • Gratuitous plugging! • A full service camera framework • QR codes, bar codes, live photos, the list goes on… • …and the easiest CoreML integration you’ll see @dokun24
Lumina • Gratuitous plugging! • A full service camera framework • QR codes, bar codes, live photos, the list goes on… • …and the easiest CoreML integration you’ll see @dokun24
Synchronizing Asynchronously Streaming Data • AVCaptureVideoDataOutput -> video frames • AVCaptureMetadataOutput -> QR codes, barcodes, faces • AVCapturePhotoOutput -> still photos • AVCaptureDepthDataOutput -> depth data maps • You see the problem here, right? @dokun24