Core Image: Great when it works

Ddd6d3bac7772fa67fc5e312a18bdaec?s=47 sammyd
April 12, 2019

Core Image: Great when it works

Core Image is an oft-overlooked framework available on iOS, tvOS and macOS. It offers a very easy way to perform highly-efficient image processing functionality, via both built-in and bespoke filters.

In this talk we'll take a look at how Core Image fits together, how to chain filters together to produce great-looking portrait effects, before building our own custom filters with Metal kernels.

This was presented at the iOS Conf/Meetup in Budapest.

Ddd6d3bac7772fa67fc5e312a18bdaec?s=128

sammyd

April 12, 2019
Tweet

Transcript

  1. Core Image great when it works

  2. None
  3. “a framework for the efficient processing of image data”

  4. None
  5. None
  6. None
  7. None
  8. pipeline processing input output

  9. pipeline processing input output

  10. pipeline processing input output

  11. pipeline processing input output

  12. objects CIContext CIImage CIFilter CIKernel

  13. objects CIContext CIImage CIFilter CIKernel

  14. objects CIContext CIImage CIFilter CIKernel

  15. objects CIContext CIImage CIFilter CIKernel

  16. objects CIContext CIImage CIFilter CIKernel

  17. efficient CIFilter CIImage

  18. efficient CIFilter CIImage

  19. efficient CIFilter CIImage

  20. filters $R0: [String] = 207 values { [0] = "CIAccordionFoldTransition"

    [1] = "CIAdditionCompositing" [2] = "CIAffineClamp" [3] = "CIAffineTile" [4] = "CIAffineTransform" [5] = "CIAreaAverage" [6] = "CIAreaHistogram" [7] = "CIAreaMaximum" [8] = "CIAreaMaximumAlpha" [9] = "CIAreaMinimum" [10] = "CIAreaMinimumAlpha"
  21. filters $R0: [String] = 207 values { [0] = "CIAccordionFoldTransition"

    [1] = "CIAdditionCompositing" [2] = "CIAffineClamp" [3] = "CIAffineTile" [4] = "CIAffineTransform" [5] = "CIAreaAverage" [6] = "CIAreaHistogram" [7] = "CIAreaMaximum" [8] = "CIAreaMaximumAlpha" [9] = "CIAreaMinimum" [10] = "CIAreaMinimumAlpha" [11] = "CIAreaMinMax" [12] = "CIAreaMinMaxRed" [13] = "CIAttributedTextImageGenerator" [14] = "CIAztecCodeGenerator" [15] = "CIBarcodeGenerator" [16] = "CIBarsSwipeTransition" [17] = "CIBicubicScaleTransform" [18] = "CIBlendWithAlphaMask" [19] = "CIBlendWithBlueMask" [20] = "CIBlendWithMask" [21] = "CIBlendWithRedMask" [22] = "CIBloom" [23] = "CIBokehBlur" [24] = "CIBoxBlur" [25] = "CIBumpDistortion" [26] = "CIBumpDistortionLinear" [27] = "CICameraCalibrationLensCorrection" [28] = "CICheckerboardGenerator" [29] = "CICircleSplashDistortion" [30] = "CICircularScreen" [31] = "CICircularWrap" [32] = "CIClamp" [33] = "CICMYKHalftone" [34] = "CICode128BarcodeGenerator" [35] = "CIColorBlendMode" [36] = "CIColorBurnBlendMode" [37] = "CIColorClamp" [38] = "CIColorControls" [39] = "CIColorCrossPolynomial" [40] = "CIColorCube"
  22. $R0: [String] = 207 values { [0] = "CIAccordionFoldTransition" [1]

    = "CIAdditionCompositing" [2] = "CIAffineClamp" [3] = "CIAffineTile" [4] = "CIAffineTransform" [5] = "CIAreaAverage" [6] = "CIAreaHistogram" [7] = "CIAreaMaximum" [8] = "CIAreaMaximumAlpha" [9] = "CIAreaMinimum" [10] = "CIAreaMinimumAlpha" [11] = "CIAreaMinMax" [12] = "CIAreaMinMaxRed" [13] = "CIAttributedTextImageGenerator" [14] = "CIAztecCodeGenerator" [15] = "CIBarcodeGenerator" [16] = "CIBarsSwipeTransition" [17] = "CIBicubicScaleTransform" [18] = "CIBlendWithAlphaMask" [19] = "CIBlendWithBlueMask" [20] = "CIBlendWithMask" [21] = "CIBlendWithRedMask" [22] = "CIBloom" [23] = "CIBokehBlur" [24] = "CIBoxBlur" [25] = "CIBumpDistortion" [26] = "CIBumpDistortionLinear" [27] = "CICameraCalibrationLensCorrection" [28] = "CICheckerboardGenerator" [29] = "CICircleSplashDistortion" [30] = "CICircularScreen" [31] = "CICircularWrap" [32] = "CIClamp" [33] = "CICMYKHalftone" [34] = "CICode128BarcodeGenerator" [35] = "CIColorBlendMode" [36] = "CIColorBurnBlendMode" [37] = "CIColorClamp" [38] = "CIColorControls" [39] = "CIColorCrossPolynomial" [40] = "CIColorCube" [41] = "CIColorCubesMixedWithMask" [42] = "CIColorCubeWithColorSpace" [43] = "CIColorCurves" [44] = "CIColorDodgeBlendMode" [45] = "CIColorInvert" [46] = "CIColorMap" [47] = "CIColorMatrix" [48] = "CIColorMonochrome" [49] = "CIColorPolynomial" [50] = "CIColorPosterize" [51] = "CIColumnAverage" [52] = "CIComicEffect" [53] = "CIConstantColorGenerator" [54] = "CIConvolution3X3" [55] = "CIConvolution5X5" [56] = "CIConvolution7X7" [57] = "CIConvolution9Horizontal" [58] = "CIConvolution9Vertical" [59] = "CICopyMachineTransition" [60] = "CICoreMLModelFilter" [61] = "CICrop" [62] = "CICrystallize" [63] = "CIDarkenBlendMode" [64] = "CIDepthBlurEffect" [65] = "CIDepthOfField" [66] = "CIDepthToDisparity" [67] = "CIDifferenceBlendMode" [68] = "CIDiscBlur" [69] = "CIDisintegrateWithMaskTransition" [70] = "CIDisparityToDepth" [71] = "CIDisplacementDistortion" [72] = "CIDissolveTransition" [73] = "CIDither" [74] = "CIDivideBlendMode" [75] = "CIDotScreen" [76] = "CIDroste" [77] = "CIEdgePreserveUpsampleFilter" [78] = "CIEdges" [79] = "CIEdgeWork" [80] = "CIEightfoldReflectedTile" [81] = "CIExclusionBlendMode" [82] = "CIExposureAdjust" [83] = "CIFalseColor" [84] = "CIFlashTransition" [85] = "CIFourfoldReflectedTile" [86] = "CIFourfoldRotatedTile" [87] = "CIFourfoldTranslatedTile" [88] = "CIGammaAdjust" [89] = "CIGaussianBlur" [90] = "CIGaussianGradient" [91] = "CIGlassDistortion" [92] = "CIGlassLozenge" [93] = "CIGlideReflectedTile" [94] = "CIGloom" [95] = "CIGuidedFilter" [96] = "CIHardLightBlendMode" [97] = "CIHatchedScreen" [98] = "CIHeightFieldFromMask" [99] = "CIHexagonalPixellate" [100] = "CIHighlightShadowAdjust" [101] = "CIHistogramDisplayFilter" [102] = "CIHoleDistortion" [103] = "CIHueAdjust" [104] = "CIHueBlendMode" [105] = "CIHueSaturationValueGradient" [106] = "CIKaleidoscope" [107] = "CILabDeltaE" [108] = "CILanczosScaleTransform" [109] = "CILenticularHaloGenerator" [110] = "CILightenBlendMode" [111] = "CILightTunnel" [112] = "CILinearBurnBlendMode" [113] = "CILinearDodgeBlendMode" [114] = "CILinearGradient" [115] = "CILinearToSRGBToneCurve" [116] = "CILineOverlay" [117] = "CILineScreen" [118] = "CILuminosityBlendMode" [119] = "CIMaskedVariableBlur" [120] = "CIMaskToAlpha" [121] = "CIMaximumComponent" [122] = "CIMaximumCompositing" [123] = "CIMedianFilter" [124] = "CIMeshGenerator" [125] = "CIMinimumComponent" [126] = "CIMinimumCompositing" [127] = "CIMix" [128] = "CIModTransition" [129] = "CIMorphologyGradient" [130] = "CIMorphologyMaximum" filters
  23. filters $R0: [String] = 207 values { [0] = "CIAccordionFoldTransition"

    [1] = "CIAdditionCompositing" [2] = "CIAffineClamp" [3] = "CIAffineTile" [4] = "CIAffineTransform" [5] = "CIAreaAverage" [6] = "CIAreaHistogram" [7] = "CIAreaMaximum" [8] = "CIAreaMaximumAlpha" [9] = "CIAreaMinimum" [10] = "CIAreaMinimumAlpha" [11] = "CIAreaMinMax" [12] = "CIAreaMinMaxRed" [13] = "CIAttributedTextImageGenerator" [14] = "CIAztecCodeGenerator" [15] = "CIBarcodeGenerator" [16] = "CIBarsSwipeTransition" [17] = "CIBicubicScaleTransform" [18] = "CIBlendWithAlphaMask" [19] = "CIBlendWithBlueMask" [20] = "CIBlendWithMask" [21] = "CIBlendWithRedMask" [22] = "CIBloom" [23] = "CIBokehBlur" [24] = "CIBoxBlur" [25] = "CIBumpDistortion" [26] = "CIBumpDistortionLinear" [27] = "CICameraCalibrationLensCorrection" [28] = "CICheckerboardGenerator" [29] = "CICircleSplashDistortion" [30] = "CICircularScreen" [31] = "CICircularWrap" [32] = "CIClamp" [33] = "CICMYKHalftone" [34] = "CICode128BarcodeGenerator" [35] = "CIColorBlendMode" [36] = "CIColorBurnBlendMode" [37] = "CIColorClamp" [38] = "CIColorControls" [39] = "CIColorCrossPolynomial" [40] = "CIColorCube" [41] = "CIColorCubesMixedWithMask" [42] = "CIColorCubeWithColorSpace" [43] = "CIColorCurves" [44] = "CIColorDodgeBlendMode" [45] = "CIColorInvert" [46] = "CIColorMap" [47] = "CIColorMatrix" [48] = "CIColorMonochrome" [49] = "CIColorPolynomial" [50] = "CIColorPosterize" [51] = "CIColumnAverage" [52] = "CIComicEffect" [53] = "CIConstantColorGenerator" [54] = "CIConvolution3X3" [55] = "CIConvolution5X5" [56] = "CIConvolution7X7" [57] = "CIConvolution9Horizontal" [58] = "CIConvolution9Vertical" [59] = "CICopyMachineTransition" [60] = "CICoreMLModelFilter" [61] = "CICrop" [62] = "CICrystallize" [63] = "CIDarkenBlendMode" [64] = "CIDepthBlurEffect" [65] = "CIDepthOfField" [66] = "CIDepthToDisparity" [67] = "CIDifferenceBlendMode" [68] = "CIDiscBlur" [69] = "CIDisintegrateWithMaskTransition" [70] = "CIDisparityToDepth" [71] = "CIDisplacementDistortion" [72] = "CIDissolveTransition" [73] = "CIDither" [74] = "CIDivideBlendMode" [75] = "CIDotScreen" [76] = "CIDroste" [77] = "CIEdgePreserveUpsampleFilter" [78] = "CIEdges" [79] = "CIEdgeWork" [80] = "CIEightfoldReflectedTile" [81] = "CIExclusionBlendMode" [82] = "CIExposureAdjust" [83] = "CIFalseColor" [84] = "CIFlashTransition" [85] = "CIFourfoldReflectedTile" [86] = "CIFourfoldRotatedTile" [87] = "CIFourfoldTranslatedTile" [88] = "CIGammaAdjust" [89] = "CIGaussianBlur" [90] = "CIGaussianGradient" [91] = "CIGlassDistortion" [92] = "CIGlassLozenge" [93] = "CIGlideReflectedTile" [94] = "CIGloom" [95] = "CIGuidedFilter" [96] = "CIHardLightBlendMode" [97] = "CIHatchedScreen" [98] = "CIHeightFieldFromMask" [99] = "CIHexagonalPixellate" [100] = "CIHighlightShadowAdjust" [101] = "CIHistogramDisplayFilter" [102] = "CIHoleDistortion" [103] = "CIHueAdjust" [104] = "CIHueBlendMode" [105] = "CIHueSaturationValueGradient" [106] = "CIKaleidoscope" [107] = "CILabDeltaE" [108] = "CILanczosScaleTransform" [109] = "CILenticularHaloGenerator" [110] = "CILightenBlendMode" [111] = "CILightTunnel" [112] = "CILinearBurnBlendMode" [113] = "CILinearDodgeBlendMode" [114] = "CILinearGradient" [115] = "CILinearToSRGBToneCurve" [116] = "CILineOverlay" [117] = "CILineScreen" [118] = "CILuminosityBlendMode" [119] = "CIMaskedVariableBlur" [120] = "CIMaskToAlpha" [121] = "CIMaximumComponent" [122] = "CIMaximumCompositing" [123] = "CIMedianFilter" [124] = "CIMeshGenerator" [125] = "CIMinimumComponent" [126] = "CIMinimumCompositing" [127] = "CIMix" [128] = "CIModTransition" [129] = "CIMorphologyGradient" [130] = "CIMorphologyMaximum" [131] = "CIMorphologyMinimum" [132] = "CIMotionBlur" [133] = "CIMultiplyBlendMode" [134] = "CIMultiplyCompositing" [135] = "CINinePartStretched" [136] = "CINinePartTiled" [137] = "CINoiseReduction" [138] = "CIOpTile" [139] = "CIOverlayBlendMode" [140] = "CIPageCurlTransition" [141] = "CIPageCurlWithShadowTransition" [142] = "CIParallelogramTile" [143] = "CIPDF417BarcodeGenerator" [144] = "CIPerspectiveCorrection" [145] = "CIPerspectiveTile" [146] = "CIPerspectiveTransform" [147] = "CIPerspectiveTransformWithExtent" [148] = "CIPhotoEffectChrome" [149] = "CIPhotoEffectFade" [150] = "CIPhotoEffectInstant" [151] = "CIPhotoEffectMono" [152] = "CIPhotoEffectNoir" [153] = "CIPhotoEffectProcess" [154] = "CIPhotoEffectTonal" [155] = "CIPhotoEffectTransfer" [156] = "CIPinchDistortion" [157] = "CIPinLightBlendMode" [158] = "CIPixellate" [159] = "CIPointillize" [160] = "CIQRCodeGenerator" [161] = "CIRadialGradient" [162] = "CIRandomGenerator" [163] = "CIRippleTransition" [164] = "CIRowAverage" [165] = "CISaliencyMapFilter" [166] = "CISampleNearest" [167] = "CISaturationBlendMode" [168] = "CIScreenBlendMode" [169] = "CISepiaTone" [170] = "CIShadedMaterial" [171] = "CISharpenLuminance" [172] = "CISixfoldReflectedTile" [173] = "CISixfoldRotatedTile" [174] = "CISmoothLinearGradient" [175] = "CISoftLightBlendMode" [176] = "CISourceAtopCompositing" [177] = "CISourceInCompositing" [178] = "CISourceOutCompositing" [179] = "CISourceOverCompositing" [180] = "CISpotColor" [181] = "CISpotLight" [182] = "CISRGBToneCurveToLinear" [183] = "CIStarShineGenerator" [184] = "CIStraightenFilter" [185] = "CIStretchCrop" [186] = "CIStripesGenerator" [187] = "CISubtractBlendMode" [188] = "CISunbeamsGenerator" [189] = "CISwipeTransition" [190] = "CITemperatureAndTint" [191] = "CITextImageGenerator" [192] = "CIThermal" [193] = "CIToneCurve" [194] = "CITorusLensDistortion" [195] = "CITriangleKaleidoscope" [196] = "CITriangleTile" [197] = "CITwelvefoldReflectedTile" [198] = "CITwirlDistortion" [199] = "CIUnsharpMask" [200] = "CIVibrance" [201] = "CIVignette" [202] = "CIVignetteEffect" [203] = "CIVortexDistortion" [204] = "CIWhitePointAdjust" [205] = "CIXRay" [206] = "CIZoomBlur"
  24. filters

  25. filters $ swift import CoreImage // List all the filters

    CIFilter.filterNames(inCategories: nil) // Learn about a filter CIFilter.localizedDescription(forFilterName: "CIBokehBlur") // Instantiate it let f = CIFilter(name: "CIBokehBlur")! // Interrogate it f.inputKeys f.attributes
  26. filters

  27. example

  28. example

  29. example

  30. example

  31. custom filter Colour General Warp

  32. custom filter Colour General Warp

  33. custom filter Colour General Warp

  34. custom filter Warp

  35. custom filter Colour General Warp

  36. YCbCr filter https://commons.wikimedia.org/wiki/File:Barns_grand_tetons_YCbCr_separation.jpg

  37. YCbCr filter https://commons.wikimedia.org/wiki/File:Barns_grand_tetons_YCbCr_separation.jpg

  38. YCbCr filter https://commons.wikimedia.org/wiki/File:Barns_grand_tetons_YCbCr_separation.jpg

  39. bilateral filter https://en.wikipedia.org/wiki/Bilateral_filter

  40. bilateral filter https://en.wikipedia.org/wiki/Bilateral_filter

  41. bilateral filter

  42. bilateral filter

  43. bilateral filter

  44. bilateral filter

  45. summary

  46. however...

  47. Core Image

  48. Core Image

  49. Core Image

  50. resources itunes.apple.com/gb/ book/core-image-for- swift/id1073029980 cifilter.io @iwantmyrealname speakerdeck.com/ sammyd github.com/sammyd/ CoreImage_iOSConf2019