HEVC Video with Alpha Channel

Transcript

1. HEVC Video with Alpha Channel
   @TachibanaKaoru
   2019/7/24 #potatotips
   

   View Slide

2. About Me
   @TachibanaKaoru
   Freelance iOS Engineer
   झຯɿϘϧμϦϯάɺBotWͰཾͷྡྷङ
   ࠷ۙͷςʔϚɿཱྀΛ͠ͳ͕Β࢓ࣄΛ͢Δ
   όϦౡɺαϯτϦʔχౡɺόϯίΫɺόϧηϩφɺΞΠεϥϯυ
   

   View Slide

3. Alpha Channel
   alpha channel
   each pixel include transparency
   information
   popular in images
   

   View Slide

4. Alpha Channel
   alpha channel
   each pixel include transparency
   information
   popular in images
   

   View Slide

5. video codec support alpha channel
   Apple ProRes 4444
   WebM VP8
   

   View Slide

6. media with alpha
   Image Video
   High bitrate PNG, TIFF Apple ProRes 4444
   Flexible bitrate HEIF, HEIFS HEVC with alpha
   

   View Slide

7. HEVC support os
   iOS 13~
   tvOS13 ~
   macOS Catalina ~
   

   View Slide

8. Playback
   AVPlayerͰͷ࠶ੜ
   

   View Slide

9. Demo
   Sample Code
   https://github.com/TachibanaKaoru/
   HEVCAlphaSample
   

   View Slide

10. Encoding HEVC video with API
    Color Chroma key
    from Apple ProRes 4444 to HEVC with alpha
    

    View Slide

11. Color Chroma key
    

    View Slide

12. let asset = AVAsset(url: sourceURL)
    guard let exportSession = AVAssetExportSession(asset: asset, presetName:
    AVAssetExportPresetHEVCHighestQualityWithAlpha) else {
    print("Failed to create export session to HEVC with alpha.")
    handleExportCompletion(.failed)
    return
    }
    // Make CIFilter
    let filter = makeChromaKeyFilter(usingHueFrom: 0.3, to: 0.4, brightnessFrom: 0.05, to: 1.0 )
    let chromaKeyComposition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in
    let source = request.sourceImage.clampedToExtent()
    filter.setValue(source, forKey: kCIInputImageKey)
    let output = filter.outputImage!
    // Provide the filter output to the composition
    request.finish(with: output, context: nil)
    })
    // Export
    exportSession.outputURL = destinationURL
    exportSession.outputFileType = .mov
    exportSession.videoComposition = chromaKeyComposition
    exportSession.exportAsynchronously {
    handleExportCompletion(exportSession.status)
    }
    

    View Slide

13. let asset = AVAsset(url: sourceURL)
    guard let exportSession = AVAssetExportSession(asset: asset, presetName:
    AVAssetExportPresetHEVCHighestQualityWithAlpha) else {
    print("Failed to create export session to HEVC with alpha.")
    handleExportCompletion(.failed)
    return
    }
    // Make CIFilter
    let filter = makeChromaKeyFilter(usingHueFrom: 0.3, to: 0.4, brightnessFrom: 0.05, to: 1.0 )
    let chromaKeyComposition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in
    let source = request.sourceImage.clampedToExtent()
    filter.setValue(source, forKey: kCIInputImageKey)
    let output = filter.outputImage!
    // Provide the filter output to the composition
    request.finish(with: output, context: nil)
    })
    // Export
    exportSession.outputURL = destinationURL
    exportSession.outputFileType = .mov
    exportSession.videoComposition = chromaKeyComposition
    exportSession.exportAsynchronously {
    handleExportCompletion(exportSession.status)
    }
    

    View Slide

14. let asset = AVAsset(url: sourceURL)
    guard let exportSession = AVAssetExportSession(asset: asset, presetName:
    AVAssetExportPresetHEVCHighestQualityWithAlpha) else {
    print("Failed to create export session to HEVC with alpha.")
    handleExportCompletion(.failed)
    return
    }
    // Make CIFilter
    let filter = makeChromaKeyFilter(usingHueFrom: 0.3, to: 0.4, brightnessFrom: 0.05, to: 1.0 )
    let chromaKeyComposition = AVVideoComposition(asset: asset, applyingCIFiltersWithHandler: { request in
    let source = request.sourceImage.clampedToExtent()
    filter.setValue(source, forKey: kCIInputImageKey)
    let output = filter.outputImage!
    // Provide the filter output to the composition
    request.finish(with: output, context: nil)
    })
    // Export
    exportSession.outputURL = destinationURL
    exportSession.outputFileType = .mov
    exportSession.videoComposition = chromaKeyComposition
    exportSession.exportAsynchronously {
    handleExportCompletion(exportSession.status)
    }
    

    View Slide

15. from Apple ProRes 4444 to HEVC with
    alpha
    

    View Slide

16. let asset = AVAsset(url: sourceURL)
    AVAssetExportSession.determineCompatibility(ofExportPreset:
    AVAssetExportPresetHEVCHighestQualityWithAlpha, with: asset, outputFileType: .mov) {
    compatible in if compatible {
    guard let exportSession = AVAssetExportSession(asset: asset, presetName:
    AVAssetExportPresetHEVCHighestQualityWithAlpha) else {
    handleExportCompletion(.failed)
    return
    }
    // Export
    exportSession.outputURL = destinationURL
    exportSession.outputFileType = .mov
    exportSession.exportAsynchronously {
    handleExportCompletion(exportSession.status)
    }
    } else {
    handleExportCompletion(.failed)
    }
    }
    

    View Slide

17. let asset = AVAsset(url: sourceURL)
    AVAssetExportSession.determineCompatibility(ofExportPreset:
    AVAssetExportPresetHEVCHighestQualityWithAlpha, with: asset, outputFileType: .mov) {
    compatible in if compatible {
    guard let exportSession = AVAssetExportSession(asset: asset, presetName:
    AVAssetExportPresetHEVCHighestQualityWithAlpha) else {
    handleExportCompletion(.failed)
    return
    }
    // Export
    exportSession.outputURL = destinationURL
    exportSession.outputFileType = .mov
    exportSession.exportAsynchronously {
    handleExportCompletion(exportSession.status)
    }
    } else {
    handleExportCompletion(.failed)
    }
    }
    

    View Slide

18. let asset = AVAsset(url: sourceURL)
    AVAssetExportSession.determineCompatibility(ofExportPreset:
    AVAssetExportPresetHEVCHighestQualityWithAlpha, with: asset, outputFileType: .mov) {
    compatible in if compatible {
    guard let exportSession = AVAssetExportSession(asset: asset, presetName:
    AVAssetExportPresetHEVCHighestQualityWithAlpha) else {
    handleExportCompletion(.failed)
    return
    }
    // Export
    exportSession.outputURL = destinationURL
    exportSession.outputFileType = .mov
    exportSession.exportAsynchronously {
    handleExportCompletion(exportSession.status)
    }
    } else {
    handleExportCompletion(.failed)
    }
    }
    

    View Slide

19. ·ͱΊ
    HEVC͸iOS13͔Β࢖͑ΔAlpha͖ͭಈըϑΥʔϚοτͰ͢ɻ
    ࠶ੜ͸AVPlayer΍WKWebViewͰ؆୯ʹͰ͖·͢ɻ
    SpriteKitɺSceneKitɺMetalͰ΋࠶ੜՄೳɻARͰ΋࢖͑·͢ɻ
    

    View Slide

20. Links
    HEVC Video with Alpha
    https://developer.apple.com/videos/play/wwdc2019/506/
    Using HEVC Video with Alpha
    https://developer.apple.com/documentation/avfoundation/
    media_assets_playback_and_editing/using_hevc_video_with_alpha
    HEVC Video with Alpha
    https://developer.apple.com/av-foundation/HEVC-Video-with-Alpha-Interoperability-
    Profile.pdf
    

    View Slide