Core Image: Great when it works

Core Image is an oft-overlooked framework available on iOS, tvOS and macOS. It offers a very easy way to perform highly-efficient image processing functionality, via both built-in and bespoke filters.

In this talk we'll take a look at how Core Image fits together, how to chain filters together to produce great-looking portrait effects, before building our own custom filters with Metal kernels.

This was presented at the iOS Conf/Meetup in Budapest.

Transcript

1. Core Image great when it works
3. “a framework for the efficient processing of image data”
8. pipeline processing input output
9. pipeline processing input output
10. pipeline processing input output
11. pipeline processing input output
12. objects CIContext CIImage CIFilter CIKernel
13. objects CIContext CIImage CIFilter CIKernel
14. objects CIContext CIImage CIFilter CIKernel
15. objects CIContext CIImage CIFilter CIKernel
16. objects CIContext CIImage CIFilter CIKernel
17. efficient CIFilter CIImage
18. efficient CIFilter CIImage
19. efficient CIFilter CIImage
20. filters $R0: [String] = 207 values { [0] = "CIAccordionFoldTransition" [1] = "CIAdditionCompositing" [2] = "CIAffineClamp" [3] = "CIAffineTile" [4] = "CIAffineTransform" [5] = "CIAreaAverage" [6] = "CIAreaHistogram" [7] = "CIAreaMaximum" [8] = "CIAreaMaximumAlpha" [9] = "CIAreaMinimum" [10] = "CIAreaMinimumAlpha"
21. filters $R0: [String] = 207 values { [0] = "CIAccordionFoldTransition" [1] = "CIAdditionCompositing" [2] = "CIAffineClamp" [3] = "CIAffineTile" [4] = "CIAffineTransform" [5] = "CIAreaAverage" [6] = "CIAreaHistogram" [7] = "CIAreaMaximum" [8] = "CIAreaMaximumAlpha" [9] = "CIAreaMinimum" [10] = "CIAreaMinimumAlpha" [11] = "CIAreaMinMax" [12] = "CIAreaMinMaxRed" [13] = "CIAttributedTextImageGenerator" [14] = "CIAztecCodeGenerator" [15] = "CIBarcodeGenerator" [16] = "CIBarsSwipeTransition" [17] = "CIBicubicScaleTransform" [18] = "CIBlendWithAlphaMask" [19] = "CIBlendWithBlueMask" [20] = "CIBlendWithMask" [21] = "CIBlendWithRedMask" [22] = "CIBloom" [23] = "CIBokehBlur" [24] = "CIBoxBlur" [25] = "CIBumpDistortion" [26] = "CIBumpDistortionLinear" [27] = "CICameraCalibrationLensCorrection" [28] = "CICheckerboardGenerator" [29] = "CICircleSplashDistortion" [30] = "CICircularScreen" [31] = "CICircularWrap" [32] = "CIClamp" [33] = "CICMYKHalftone" [34] = "CICode128BarcodeGenerator" [35] = "CIColorBlendMode" [36] = "CIColorBurnBlendMode" [37] = "CIColorClamp" [38] = "CIColorControls" [39] = "CIColorCrossPolynomial" [40] = "CIColorCube"
22. $R0: [String] = 207 values { [0] = "CIAccordionFoldTransition" [1] = "CIAdditionCompositing" [2] = "CIAffineClamp" [3] = "CIAffineTile" [4] = "CIAffineTransform" [5] = "CIAreaAverage" [6] = "CIAreaHistogram" [7] = "CIAreaMaximum" [8] = "CIAreaMaximumAlpha" [9] = "CIAreaMinimum" [10] = "CIAreaMinimumAlpha" [11] = "CIAreaMinMax" [12] = "CIAreaMinMaxRed" [13] = "CIAttributedTextImageGenerator" [14] = "CIAztecCodeGenerator" [15] = "CIBarcodeGenerator" [16] = "CIBarsSwipeTransition" [17] = "CIBicubicScaleTransform" [18] = "CIBlendWithAlphaMask" [19] = "CIBlendWithBlueMask" [20] = "CIBlendWithMask" [21] = "CIBlendWithRedMask" [22] = "CIBloom" [23] = "CIBokehBlur" [24] = "CIBoxBlur" [25] = "CIBumpDistortion" [26] = "CIBumpDistortionLinear" [27] = "CICameraCalibrationLensCorrection" [28] = "CICheckerboardGenerator" [29] = "CICircleSplashDistortion" [30] = "CICircularScreen" [31] = "CICircularWrap" [32] = "CIClamp" [33] = "CICMYKHalftone" [34] = "CICode128BarcodeGenerator" [35] = "CIColorBlendMode" [36] = "CIColorBurnBlendMode" [37] = "CIColorClamp" [38] = "CIColorControls" [39] = "CIColorCrossPolynomial" [40] = "CIColorCube" [41] = "CIColorCubesMixedWithMask" [42] = "CIColorCubeWithColorSpace" [43] = "CIColorCurves" [44] = "CIColorDodgeBlendMode" [45] = "CIColorInvert" [46] = "CIColorMap" [47] = "CIColorMatrix" [48] = "CIColorMonochrome" [49] = "CIColorPolynomial" [50] = "CIColorPosterize" [51] = "CIColumnAverage" [52] = "CIComicEffect" [53] = "CIConstantColorGenerator" [54] = "CIConvolution3X3" [55] = "CIConvolution5X5" [56] = "CIConvolution7X7" [57] = "CIConvolution9Horizontal" [58] = "CIConvolution9Vertical" [59] = "CICopyMachineTransition" [60] = "CICoreMLModelFilter" [61] = "CICrop" [62] = "CICrystallize" [63] = "CIDarkenBlendMode" [64] = "CIDepthBlurEffect" [65] = "CIDepthOfField" [66] = "CIDepthToDisparity" [67] = "CIDifferenceBlendMode" [68] = "CIDiscBlur" [69] = "CIDisintegrateWithMaskTransition" [70] = "CIDisparityToDepth" [71] = "CIDisplacementDistortion" [72] = "CIDissolveTransition" [73] = "CIDither" [74] = "CIDivideBlendMode" [75] = "CIDotScreen" [76] = "CIDroste" [77] = "CIEdgePreserveUpsampleFilter" [78] = "CIEdges" [79] = "CIEdgeWork" [80] = "CIEightfoldReflectedTile" [81] = "CIExclusionBlendMode" [82] = "CIExposureAdjust" [83] = "CIFalseColor" [84] = "CIFlashTransition" [85] = "CIFourfoldReflectedTile" [86] = "CIFourfoldRotatedTile" [87] = "CIFourfoldTranslatedTile" [88] = "CIGammaAdjust" [89] = "CIGaussianBlur" [90] = "CIGaussianGradient" [91] = "CIGlassDistortion" [92] = "CIGlassLozenge" [93] = "CIGlideReflectedTile" [94] = "CIGloom" [95] = "CIGuidedFilter" [96] = "CIHardLightBlendMode" [97] = "CIHatchedScreen" [98] = "CIHeightFieldFromMask" [99] = "CIHexagonalPixellate" [100] = "CIHighlightShadowAdjust" [101] = "CIHistogramDisplayFilter" [102] = "CIHoleDistortion" [103] = "CIHueAdjust" [104] = "CIHueBlendMode" [105] = "CIHueSaturationValueGradient" [106] = "CIKaleidoscope" [107] = "CILabDeltaE" [108] = "CILanczosScaleTransform" [109] = "CILenticularHaloGenerator" [110] = "CILightenBlendMode" [111] = "CILightTunnel" [112] = "CILinearBurnBlendMode" [113] = "CILinearDodgeBlendMode" [114] = "CILinearGradient" [115] = "CILinearToSRGBToneCurve" [116] = "CILineOverlay" [117] = "CILineScreen" [118] = "CILuminosityBlendMode" [119] = "CIMaskedVariableBlur" [120] = "CIMaskToAlpha" [121] = "CIMaximumComponent" [122] = "CIMaximumCompositing" [123] = "CIMedianFilter" [124] = "CIMeshGenerator" [125] = "CIMinimumComponent" [126] = "CIMinimumCompositing" [127] = "CIMix" [128] = "CIModTransition" [129] = "CIMorphologyGradient" [130] = "CIMorphologyMaximum" filters
23. filters $R0: [String] = 207 values { [0] = "CIAccordionFoldTransition" [1] = "CIAdditionCompositing" [2] = "CIAffineClamp" [3] = "CIAffineTile" [4] = "CIAffineTransform" [5] = "CIAreaAverage" [6] = "CIAreaHistogram" [7] = "CIAreaMaximum" [8] = "CIAreaMaximumAlpha" [9] = "CIAreaMinimum" [10] = "CIAreaMinimumAlpha" [11] = "CIAreaMinMax" [12] = "CIAreaMinMaxRed" [13] = "CIAttributedTextImageGenerator" [14] = "CIAztecCodeGenerator" [15] = "CIBarcodeGenerator" [16] = "CIBarsSwipeTransition" [17] = "CIBicubicScaleTransform" [18] = "CIBlendWithAlphaMask" [19] = "CIBlendWithBlueMask" [20] = "CIBlendWithMask" [21] = "CIBlendWithRedMask" [22] = "CIBloom" [23] = "CIBokehBlur" [24] = "CIBoxBlur" [25] = "CIBumpDistortion" [26] = "CIBumpDistortionLinear" [27] = "CICameraCalibrationLensCorrection" [28] = "CICheckerboardGenerator" [29] = "CICircleSplashDistortion" [30] = "CICircularScreen" [31] = "CICircularWrap" [32] = "CIClamp" [33] = "CICMYKHalftone" [34] = "CICode128BarcodeGenerator" [35] = "CIColorBlendMode" [36] = "CIColorBurnBlendMode" [37] = "CIColorClamp" [38] = "CIColorControls" [39] = "CIColorCrossPolynomial" [40] = "CIColorCube" [41] = "CIColorCubesMixedWithMask" [42] = "CIColorCubeWithColorSpace" [43] = "CIColorCurves" [44] = "CIColorDodgeBlendMode" [45] = "CIColorInvert" [46] = "CIColorMap" [47] = "CIColorMatrix" [48] = "CIColorMonochrome" [49] = "CIColorPolynomial" [50] = "CIColorPosterize" [51] = "CIColumnAverage" [52] = "CIComicEffect" [53] = "CIConstantColorGenerator" [54] = "CIConvolution3X3" [55] = "CIConvolution5X5" [56] = "CIConvolution7X7" [57] = "CIConvolution9Horizontal" [58] = "CIConvolution9Vertical" [59] = "CICopyMachineTransition" [60] = "CICoreMLModelFilter" [61] = "CICrop" [62] = "CICrystallize" [63] = "CIDarkenBlendMode" [64] = "CIDepthBlurEffect" [65] = "CIDepthOfField" [66] = "CIDepthToDisparity" [67] = "CIDifferenceBlendMode" [68] = "CIDiscBlur" [69] = "CIDisintegrateWithMaskTransition" [70] = "CIDisparityToDepth" [71] = "CIDisplacementDistortion" [72] = "CIDissolveTransition" [73] = "CIDither" [74] = "CIDivideBlendMode" [75] = "CIDotScreen" [76] = "CIDroste" [77] = "CIEdgePreserveUpsampleFilter" [78] = "CIEdges" [79] = "CIEdgeWork" [80] = "CIEightfoldReflectedTile" [81] = "CIExclusionBlendMode" [82] = "CIExposureAdjust" [83] = "CIFalseColor" [84] = "CIFlashTransition" [85] = "CIFourfoldReflectedTile" [86] = "CIFourfoldRotatedTile" [87] = "CIFourfoldTranslatedTile" [88] = "CIGammaAdjust" [89] = "CIGaussianBlur" [90] = "CIGaussianGradient" [91] = "CIGlassDistortion" [92] = "CIGlassLozenge" [93] = "CIGlideReflectedTile" [94] = "CIGloom" [95] = "CIGuidedFilter" [96] = "CIHardLightBlendMode" [97] = "CIHatchedScreen" [98] = "CIHeightFieldFromMask" [99] = "CIHexagonalPixellate" [100] = "CIHighlightShadowAdjust" [101] = "CIHistogramDisplayFilter" [102] = "CIHoleDistortion" [103] = "CIHueAdjust" [104] = "CIHueBlendMode" [105] = "CIHueSaturationValueGradient" [106] = "CIKaleidoscope" [107] = "CILabDeltaE" [108] = "CILanczosScaleTransform" [109] = "CILenticularHaloGenerator" [110] = "CILightenBlendMode" [111] = "CILightTunnel" [112] = "CILinearBurnBlendMode" [113] = "CILinearDodgeBlendMode" [114] = "CILinearGradient" [115] = "CILinearToSRGBToneCurve" [116] = "CILineOverlay" [117] = "CILineScreen" [118] = "CILuminosityBlendMode" [119] = "CIMaskedVariableBlur" [120] = "CIMaskToAlpha" [121] = "CIMaximumComponent" [122] = "CIMaximumCompositing" [123] = "CIMedianFilter" [124] = "CIMeshGenerator" [125] = "CIMinimumComponent" [126] = "CIMinimumCompositing" [127] = "CIMix" [128] = "CIModTransition" [129] = "CIMorphologyGradient" [130] = "CIMorphologyMaximum" [131] = "CIMorphologyMinimum" [132] = "CIMotionBlur" [133] = "CIMultiplyBlendMode" [134] = "CIMultiplyCompositing" [135] = "CINinePartStretched" [136] = "CINinePartTiled" [137] = "CINoiseReduction" [138] = "CIOpTile" [139] = "CIOverlayBlendMode" [140] = "CIPageCurlTransition" [141] = "CIPageCurlWithShadowTransition" [142] = "CIParallelogramTile" [143] = "CIPDF417BarcodeGenerator" [144] = "CIPerspectiveCorrection" [145] = "CIPerspectiveTile" [146] = "CIPerspectiveTransform" [147] = "CIPerspectiveTransformWithExtent" [148] = "CIPhotoEffectChrome" [149] = "CIPhotoEffectFade" [150] = "CIPhotoEffectInstant" [151] = "CIPhotoEffectMono" [152] = "CIPhotoEffectNoir" [153] = "CIPhotoEffectProcess" [154] = "CIPhotoEffectTonal" [155] = "CIPhotoEffectTransfer" [156] = "CIPinchDistortion" [157] = "CIPinLightBlendMode" [158] = "CIPixellate" [159] = "CIPointillize" [160] = "CIQRCodeGenerator" [161] = "CIRadialGradient" [162] = "CIRandomGenerator" [163] = "CIRippleTransition" [164] = "CIRowAverage" [165] = "CISaliencyMapFilter" [166] = "CISampleNearest" [167] = "CISaturationBlendMode" [168] = "CIScreenBlendMode" [169] = "CISepiaTone" [170] = "CIShadedMaterial" [171] = "CISharpenLuminance" [172] = "CISixfoldReflectedTile" [173] = "CISixfoldRotatedTile" [174] = "CISmoothLinearGradient" [175] = "CISoftLightBlendMode" [176] = "CISourceAtopCompositing" [177] = "CISourceInCompositing" [178] = "CISourceOutCompositing" [179] = "CISourceOverCompositing" [180] = "CISpotColor" [181] = "CISpotLight" [182] = "CISRGBToneCurveToLinear" [183] = "CIStarShineGenerator" [184] = "CIStraightenFilter" [185] = "CIStretchCrop" [186] = "CIStripesGenerator" [187] = "CISubtractBlendMode" [188] = "CISunbeamsGenerator" [189] = "CISwipeTransition" [190] = "CITemperatureAndTint" [191] = "CITextImageGenerator" [192] = "CIThermal" [193] = "CIToneCurve" [194] = "CITorusLensDistortion" [195] = "CITriangleKaleidoscope" [196] = "CITriangleTile" [197] = "CITwelvefoldReflectedTile" [198] = "CITwirlDistortion" [199] = "CIUnsharpMask" [200] = "CIVibrance" [201] = "CIVignette" [202] = "CIVignetteEffect" [203] = "CIVortexDistortion" [204] = "CIWhitePointAdjust" [205] = "CIXRay" [206] = "CIZoomBlur"
24. filters
25. filters $ swift import CoreImage // List all the filters CIFilter.filterNames(inCategories: nil) // Learn about a filter CIFilter.localizedDescription(forFilterName: "CIBokehBlur") // Instantiate it let f = CIFilter(name: "CIBokehBlur")! // Interrogate it f.inputKeys f.attributes
26. filters
27. example
28. example
29. example
30. example
31. custom filter Colour General Warp
32. custom filter Colour General Warp
33. custom filter Colour General Warp
34. custom filter Warp
35. custom filter Colour General Warp
36. YCbCr filter https://commons.wikimedia.org/wiki/File:Barns_grand_tetons_YCbCr_separation.jpg
37. YCbCr filter https://commons.wikimedia.org/wiki/File:Barns_grand_tetons_YCbCr_separation.jpg
38. YCbCr filter https://commons.wikimedia.org/wiki/File:Barns_grand_tetons_YCbCr_separation.jpg
39. bilateral filter https://en.wikipedia.org/wiki/Bilateral_filter
40. bilateral filter https://en.wikipedia.org/wiki/Bilateral_filter
41. bilateral filter
42. bilateral filter
43. bilateral filter
44. bilateral filter
45. summary
46. however...
47. Core Image
48. Core Image
49. Core Image
50. resources itunes.apple.com/gb/ book/core-image-for- swift/id1073029980 cifilter.io @iwantmyrealname speakerdeck.com/ sammyd github.com/sammyd/ CoreImage_iOSConf2019