HEVC Video with Alpha Channel

HEVC Video with Alpha Channel

61a68b2172503ecdf7a7f7757df56071?s=128

Tachibana Kaoru

July 24, 2019
Tweet

Transcript

  1. HEVC Video with Alpha Channel @TachibanaKaoru 2019/7/24 #potatotips

  2. About Me @TachibanaKaoru Freelance iOS Engineer झຯɿϘϧμϦϯάɺBotWͰཾͷྡྷङ ࠷ۙͷςʔϚɿཱྀΛ͠ͳ͕Β࢓ࣄΛ͢Δ όϦౡɺαϯτϦʔχౡɺόϯίΫɺόϧηϩφɺΞΠεϥϯυ

  3. Alpha Channel alpha channel each pixel include transparency information popular

    in images
  4. Alpha Channel alpha channel each pixel include transparency information popular

    in images
  5. video codec support alpha channel Apple ProRes 4444 WebM VP8

  6. media with alpha Image Video High bitrate PNG, TIFF Apple

    ProRes 4444 Flexible bitrate HEIF, HEIFS HEVC with alpha
  7. HEVC support os iOS 13~ tvOS13 ~ macOS Catalina ~

  8. Playback AVPlayerͰͷ࠶ੜ

  9. Demo Sample Code https://github.com/TachibanaKaoru/ HEVCAlphaSample

  10. Encoding HEVC video with API Color Chroma key from Apple

    ProRes 4444 to HEVC with alpha
  11. Color Chroma key

  12. let asset = AVAsset(url: sourceURL) guard let exportSession = AVAssetExportSession(asset:

    asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { print("Failed to create export session to HEVC with alpha.") handleExportCompletion(.failed) return } // Make CIFilter let filter = makeChromaKeyFilter(usingHueFrom: 0.3, to: 0.4, brightnessFrom: 0.05, to: 1.0 ) let chromaKeyComposition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in let source = request.sourceImage.clampedToExtent() filter.setValue(source, forKey: kCIInputImageKey) let output = filter.outputImage! // Provide the filter output to the composition request.finish(with: output, context: nil) }) // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.videoComposition = chromaKeyComposition exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) }
  13. let asset = AVAsset(url: sourceURL) guard let exportSession = AVAssetExportSession(asset:

    asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { print("Failed to create export session to HEVC with alpha.") handleExportCompletion(.failed) return } // Make CIFilter let filter = makeChromaKeyFilter(usingHueFrom: 0.3, to: 0.4, brightnessFrom: 0.05, to: 1.0 ) let chromaKeyComposition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in let source = request.sourceImage.clampedToExtent() filter.setValue(source, forKey: kCIInputImageKey) let output = filter.outputImage! // Provide the filter output to the composition request.finish(with: output, context: nil) }) // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.videoComposition = chromaKeyComposition exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) }
  14. let asset = AVAsset(url: sourceURL) guard let exportSession = AVAssetExportSession(asset:

    asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { print("Failed to create export session to HEVC with alpha.") handleExportCompletion(.failed) return } // Make CIFilter let filter = makeChromaKeyFilter(usingHueFrom: 0.3, to: 0.4, brightnessFrom: 0.05, to: 1.0 ) let chromaKeyComposition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in let source = request.sourceImage.clampedToExtent() filter.setValue(source, forKey: kCIInputImageKey) let output = filter.outputImage! // Provide the filter output to the composition request.finish(with: output, context: nil) }) // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.videoComposition = chromaKeyComposition exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) }
  15. from Apple ProRes 4444 to HEVC with alpha

  16. let asset = AVAsset(url: sourceURL) AVAssetExportSession.determineCompatibility(ofExportPreset: AVAssetExportPresetHEVCHighestQualityWithAlpha, with: asset, outputFileType:

    .mov) { compatible in if compatible { guard let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { handleExportCompletion(.failed) return } // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) } } else { handleExportCompletion(.failed) } }
  17. let asset = AVAsset(url: sourceURL) AVAssetExportSession.determineCompatibility(ofExportPreset: AVAssetExportPresetHEVCHighestQualityWithAlpha, with: asset, outputFileType:

    .mov) { compatible in if compatible { guard let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { handleExportCompletion(.failed) return } // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) } } else { handleExportCompletion(.failed) } }
  18. let asset = AVAsset(url: sourceURL) AVAssetExportSession.determineCompatibility(ofExportPreset: AVAssetExportPresetHEVCHighestQualityWithAlpha, with: asset, outputFileType:

    .mov) { compatible in if compatible { guard let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetHEVCHighestQualityWithAlpha) else { handleExportCompletion(.failed) return } // Export exportSession.outputURL = destinationURL exportSession.outputFileType = .mov exportSession.exportAsynchronously { handleExportCompletion(exportSession.status) } } else { handleExportCompletion(.failed) } }
  19. ·ͱΊ HEVC͸iOS13͔Β࢖͑ΔAlpha͖ͭಈըϑΥʔϚοτͰ͢ɻ ࠶ੜ͸AVPlayer΍WKWebViewͰ؆୯ʹͰ͖·͢ɻ SpriteKitɺSceneKitɺMetalͰ΋࠶ੜՄೳɻARͰ΋࢖͑·͢ɻ

  20. Links HEVC Video with Alpha https://developer.apple.com/videos/play/wwdc2019/506/ Using HEVC Video with

    Alpha https://developer.apple.com/documentation/avfoundation/ media_assets_playback_and_editing/using_hevc_video_with_alpha HEVC Video with Alpha https://developer.apple.com/av-foundation/HEVC-Video-with-Alpha-Interoperability- Profile.pdf