HEVC Video with Alpha Channel

Transcript

1. HEVC Video with Alpha Channel @TachibanaKaoru 2019/7/24 #potatotips

2. About Me @TachibanaKaoru Freelance iOS Engineer झຯɿϘϧμϦϯάɺBotWͰཾͷྡྷङ ࠷ۙͷςʔϚɿཱྀΛ͠ͳ͕Β࢓ࣄΛ͢Δ όϦౡɺαϯτϦʔχౡɺόϯίΫɺόϧηϩφɺΞΠεϥϯυ

3. Alpha Channel alpha channel each pixel include transparency information popular

   in images

4. Alpha Channel alpha channel each pixel include transparency information popular

   in images

5. video codec support alpha channel Apple ProRes 4444 WebM VP8

6. media with alpha Image Video High bitrate PNG, TIFF Apple

   ProRes 4444 Flexible bitrate HEIF, HEIFS HEVC with alpha

7. HEVC support os iOS 13~ tvOS13 ~ macOS Catalina ~

8. Playback AVPlayerͰͷ࠶ੜ

9. Demo Sample Code https://github.com/TachibanaKaoru/ HEVCAlphaSample

10. Encoding HEVC video with API Color Chroma key from Apple

    ProRes 4444 to HEVC with alpha

11. Color Chroma key

12. let asset = AVAsset(url: sourceURL) guard let exportSession = AVAssetExportSession(asset:

    asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { print("Failed to create export session to HEVC with alpha.") handleExportCompletion(.failed) return } // Make CIFilter let filter = makeChromaKeyFilter(usingHueFrom: 0.3, to: 0.4, brightnessFrom: 0.05, to: 1.0 ) let chromaKeyComposition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in let source = request.sourceImage.clampedToExtent() filter.setValue(source, forKey: kCIInputImageKey) let output = filter.outputImage! // Provide the filter output to the composition request.finish(with: output, context: nil) }) // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.videoComposition = chromaKeyComposition exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) }

13. let asset = AVAsset(url: sourceURL) guard let exportSession = AVAssetExportSession(asset:

    asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { print("Failed to create export session to HEVC with alpha.") handleExportCompletion(.failed) return } // Make CIFilter let filter = makeChromaKeyFilter(usingHueFrom: 0.3, to: 0.4, brightnessFrom: 0.05, to: 1.0 ) let chromaKeyComposition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in let source = request.sourceImage.clampedToExtent() filter.setValue(source, forKey: kCIInputImageKey) let output = filter.outputImage! // Provide the filter output to the composition request.finish(with: output, context: nil) }) // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.videoComposition = chromaKeyComposition exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) }

14. let asset = AVAsset(url: sourceURL) guard let exportSession = AVAssetExportSession(asset:

    asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { print("Failed to create export session to HEVC with alpha.") handleExportCompletion(.failed) return } // Make CIFilter let filter = makeChromaKeyFilter(usingHueFrom: 0.3, to: 0.4, brightnessFrom: 0.05, to: 1.0 ) let chromaKeyComposition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in let source = request.sourceImage.clampedToExtent() filter.setValue(source, forKey: kCIInputImageKey) let output = filter.outputImage! // Provide the filter output to the composition request.finish(with: output, context: nil) }) // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.videoComposition = chromaKeyComposition exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) }

15. from Apple ProRes 4444 to HEVC with alpha

16. let asset = AVAsset(url: sourceURL) AVAssetExportSession.determineCompatibility(ofExportPreset: AVAssetExportPresetHEVCHighestQualityWithAlpha, with: asset, outputFileType:

    .mov) { compatible in if compatible { guard let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { handleExportCompletion(.failed) return } // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) } } else { handleExportCompletion(.failed) } }

17. let asset = AVAsset(url: sourceURL) AVAssetExportSession.determineCompatibility(ofExportPreset: AVAssetExportPresetHEVCHighestQualityWithAlpha, with: asset, outputFileType:

    .mov) { compatible in if compatible { guard let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { handleExportCompletion(.failed) return } // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) } } else { handleExportCompletion(.failed) } }

18. let asset = AVAsset(url: sourceURL) AVAssetExportSession.determineCompatibility(ofExportPreset: AVAssetExportPresetHEVCHighestQualityWithAlpha, with: asset, outputFileType:

    .mov) { compatible in if compatible { guard let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { handleExportCompletion(.failed) return } // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) } } else { handleExportCompletion(.failed) } }

19. ·ͱΊ HEVC͸iOS13͔Β࢖͑ΔAlpha͖ͭಈըϑΥʔϚοτͰ͢ɻ ࠶ੜ͸AVPlayer΍WKWebViewͰ؆୯ʹͰ͖·͢ɻ SpriteKitɺSceneKitɺMetalͰ΋࠶ੜՄೳɻARͰ΋࢖͑·͢ɻ

20. Links HEVC Video with Alpha https://developer.apple.com/videos/play/wwdc2019/506/ Using HEVC Video with

    Alpha https://developer.apple.com/documentation/avfoundation/ media_assets_playback_and_editing/using_hevc_video_with_alpha HEVC Video with Alpha https://developer.apple.com/av-foundation/HEVC-Video-with-Alpha-Interoperability- Profile.pdf