UIImage and CIImage for Image Processing

UIImage and CIImage for Image Processing

画像処理におけるUIImageとCGImageとCIImageの効果的な使い分け
iOSDC 2019 発表資料
https://fortee.jp/iosdc-japan-2019/proposal/3c30c4b4-a647-4198-8e8c-e8100293ee93

Ee6254efe6c4bc4c08967c11d4939245?s=128

kotetu (kotetuco)

September 06, 2019
Tweet

Transcript

  1. ը૾ॲཧʹ͓͚Δ 6**NBHFͱ$(*NBHFͱ$**NBHF ͷޮՌతͳ࢖͍෼͚ ܀ࢁప !LPUFUV J04%$ 

  2. ܀ࢁప  !LPUFUV LPUFUVDP w J04"QQ%FWFMPQFSBU w ઃܭपΓ"OESPJEࣗ࡞04 3VTU $5'

    J04%$ 
  3. J04%$-5 άϥϑΟοΫεҎ֎ͷ෼໺ Ͱͷ.FUBMͷՄೳੑ J04%$ 

  4. w ๏ਓ޲໊͚ࢗ؅ཧαʔϏε w ໿ ͕ࣾܖ໿ w ແྉτϥΠΞϧͰ͖·͢ ຕ·Ͱొ࿥Մ J04%$ 

  5. 2JJUB+PCT ࢖༻ٕज़ɺ։ൃମ੍ͳͲΛ ঺հ͍ͯ͠·͢ J04%$ 

  6. ߨԋͷςʔϚ 4BOTBOΞϓϦͷ໊ࢗొ࿥ػೳΛ୊ࡐʹ ϦΞϧλΠϜը૾ ೝࣝ ॲཧʹ͓͚Δ ը૾Ϋϥεͷ࢖͍෼͚ʹ͍ͭͯൃද͠·͢ɻ J04%$ 

  7. ओʹϦαΠζ ॖখ ॲཧʹ͓͚Δ$**NBHFͱ 6**NBHFͷ࢖͍෼͚ͷ࿩͕ϝΠϯʹͳΓ·͢ɻ J04%$ 

  8. 4BOTBOΞϓϦͷ ໊ࢗొ࿥ػೳ J04%$ 

  9. ໊ࢗొ࿥ػೳ w ը໘্ͷ໊ࢗΛࣗಈݕग़ w ࡱӨޙ͙͢ػցతʹσʔλ Խ͞Ε·͢ w ද໘ཪ໘ࡱӨͰ͖·͢ J04%$ 

  10. ࣗಈݕग़ػೳͷ όʔδϣϯΞοϓ ೥݄ϦϦʔε J04%$ 

  11. ࣗಈݕग़ػೳͷόʔδϣϯΞοϓ w ໊ࢗͷݕग़ਫ਼౓޲্ ! w ಺෦ॲཧͷஔ͖׵͕͑ඞཁ w ໊ࢗొ࿥ॲཧΛݟ௚ͦ͏ʂ J04%$ 

  12. ݟ௚͍ͯ͠Δ͏ͪʹ J04%$ 

  13. ͜Ε͸J04%$ͷ ωλʹͳΔͷͰ͸ J04%$ 

  14. ৽ͨͳΔ$G1஀ੜͷ ॠؒͰ͋Δ J04%$ 

  15. $POUFOUT  ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ  4BOTBOJ04ΞϓϦͷ࣮૷  ϦαΠζॲཧͷվળ 

    ·ͱΊ J04%$ 
  16. $POUFOUT  ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ  4BOTBOJ04ΞϓϦͷ࣮૷  ϦαΠζॲཧͷվળ 

    ·ͱΊ J04%$ 
  17. ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ J04%$ 

  18. ϦΞϧλΠϜը૾ ೝࣝ ॲཧ ΧϝϥͳͲ͔ΒϦΞϧλΠϜʹ౉ͬͯ͘Δը૾Λ ࢖ͬͯը૾ॲཧ ·ͨ͸ը૾ೝࣝ Λߦ͍݁ՌΛ ग़ྗ͢Δ J04%$ 

  19. ॲཧͷྲྀΕ Ұൠతͳέʔε w Χϝϥ͔Βը૾Λऔಘ w ը૾ ೝࣝ ॲཧ΁ͷೖྗͷͨΊͷલॲཧ w ը૾

    ೝࣝ ॲཧ w ॲཧ݁Ռͷදࣔ J04%$ 
  20. ॲཧͷྲྀΕ Ұൠతͳέʔε w Χϝϥ͔Βը૾Λऔಘ w ը૾ ೝࣝ ॲཧ΁ͷೖྗͷͨΊͷલॲཧ w ը૾

    ೝࣝ ॲཧ w ॲཧ݁Ռͷදࣔ Ωϟϓνϟͨ͠Χϝϥө૾ΛϑϨʔϜ͝ͱʹऔಘ J04%$ 
  21. ॲཧͷྲྀΕ Ұൠతͳέʔε w Χϝϥ͔Βը૾Λऔಘ w ը૾ ೝࣝ ॲཧ΁ͷೖྗͷͨΊͷલॲཧ w ը૾

    ೝࣝ ॲཧ w ॲཧ݁Ռͷදࣔ ը૾ܗࣜͷม׵΍ϦαΠζɾάϨʔεέʔϧԽ ը૾ೝࣝͷ৔߹ ͳͲ J04%$ 
  22. ॲཧͷྲྀΕ Ұൠతͳέʔε w Χϝϥ͔Βը૾Λऔಘ w ը૾ ೝࣝ ॲཧ΁ͷೖྗͷͨΊͷલॲཧ w ը૾

    ೝࣝ ॲཧ w ॲཧ݁Ռͷදࣔ ΅͔͠΍৭ຯͷมߋɺಛ௃఺ݕग़ͳͲ J04%$ 
  23. ॲཧͷྲྀΕ Ұൠతͳέʔε w Χϝϥ͔Βը૾Λऔಘ w ը૾ ೝࣝ ॲཧ΁ͷೖྗͷͨΊͷલॲཧ w ը૾

    ೝࣝ ॲཧ w ॲཧ݁Ռͷදࣔ ॲཧͨ͠ޙͷը૾΍ೝࣝ݁ՌΛը໘ʹදࣔ J04%$ 
  24. $POUFOUT  ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ  4BOTBOJ04ΞϓϦͷ࣮૷  ϦαΠζॲཧͷվળ 

    ·ͱΊ J04%$ 
  25. 4BOTBOJ04ΞϓϦͷ࣮૷ J04%$ 

  26. J04ʹ͓͚Δ ϦΞϧλΠϜը૾ ೝࣝ ॲཧ J04%$ 

  27. J04%$ 

  28. αϯϓϧΞϓϦ 0QFO$7Λ࢖ͬͨԁݕग़ J04%$ 

  29. J04%$ 

  30. Χϝϥ͔Βը૾Λऔಘ J04%$ 

  31. "7'PVOEBUJPO w ΧϝϥॲཧΛ࣮૷͢Δͱ͖ʹ࢖͏'SBNFXPSL w ϦΞϧλΠϜॲཧΛߦ͏৔߹͸ "7$BQUVSF7JEFP%BUB0VUQVUΛ࢖͏ J04%$ 

  32. ը૾Λड͚औΔ class Capture: AVCaptureVideoDataOutputSampleBufferDelegate { func captureOutput(_ output: AVCaptureOutput, didOutput

    sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) { ... } } J04%$ 
  33. ը૾Λड͚औΔ class Capture: AVCaptureVideoDataOutputSampleBufferDelegate { func captureOutput(_ output: AVCaptureOutput, didOutput

    sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) { ... } } ը૾σʔλ͸$.4BNQMF#VGGFSͰड͚औΔͷ Ͱผͳը૾Ϋϥε΁ม׵ͯ͋͛͠ͳ͍ͱ͍͚ͳ͍ J04%$ 
  34. J04%$ 

  35. $**NBHF΁ม׵ w $.4BNQMF#V⒎FS͸$**NBHF΁ม׵͢Δɹ ͜ͱ͕ଟ͍ w $7*NBHF#VGGFSΛܦ༝͢Δ J04%$ 

  36. $**NBHF΁ม׵ func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection:

    AVCaptureConnection) { let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) let ciImage = CIImage(cvImageBuffer: imageBuffer!) } J04%$ 
  37. $**NBHF΁ม׵ func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection:

    AVCaptureConnection) { let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) let ciImage = CIImage(cvImageBuffer: imageBuffer!) } $.4BNQMF#V⒎FS͔Β$7*NBHF#VGGFS΁ɹ ม׵ J04%$ 
  38. $**NBHF΁ม׵ func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection:

    AVCaptureConnection) { let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) let ciImage = CIImage(cvImageBuffer: imageBuffer!) } $7*NBHF#V⒎FS͔Β$**NBHF΁ม׵ J04%$ 
  39. 4BOTBOΞϓϦʹ͓͚Δ "7'PVOEBUJPO w ΑΓଟ͘ͷ໊ࢗΛೝ͍ࣝͨ͠ w ,αΠζ ,ඇରԠػछ͸'VMM)%αΠζ w ೥ʑΧϝϥͷੑೳ্͕͕͍ͬͯΔ J04%$

    
  40. J04%$ 

  41. ը૾ ೝࣝ ॲཧ΁ͷೖྗͷ ͨΊͷલॲཧ J04%$ 

  42. 4BOTBOΞϓϦʹ͓͚Δલॲཧ w ը૾ॲཧ΁ͷೖྗ͸6**NBHF w ೖྗը૾Λॖখ͠ͳ͍ͱ͍͚ͳ͍ ۃ୺ʹେ͖ ͳը૾ͷ৔߹ʹॲཧ͕͔͔࣌ؒͬͯ͠·͏ J04%$ 

  43. 6**NBHF΁ม׵ let context = CIContext() let cgImage = context.createCGImage(ciImage, from:

    ciImage.extent) let uiImage = UIImage(cgImage: cgImage!) J04%$ 
  44. 6**NBHF΁ม׵ let context = CIContext() let cgImage = context.createCGImage(ciImage, from:

    ciImage.extent) let uiImage = UIImage(cgImage: cgImage!) έʔεʹ΋ΑΔ͕ $(*NBHFΛܦ༝͢Δέʔε ͕ଟ͍ J04%$ 
  45. J04%$ 

  46. ը૾ ೝࣝ ॲཧ J04%$ 

  47. 4BOTBOΞϓϦʹ͓͚Δ໊ࢗೝࣝॲཧ w ෦ॺؒ &JHIUࣄۀ෦ ͰίʔυΛڞ༻ w $BSUIBHFରԠ w Ұ෦"OESPJEΞϓϦͱίʔυΛڞ༻ w

    ೖྗ͸6**NBHF J04%$ 
  48. J04%$ 

  49. ॲཧ݁Ռͷදࣔ J04%$ 

  50. ॲཧ݁Ռͷදࣔ w ը૾ॲཧޙͷը૾ w 6**NBHF7JFX΍.5,7JFX .FUBM,JU w ೝࣝ݁Ռ w 6*#F[JFS1BUI΍$"4IBQF-BZFS

    J04%$ 
  51. J04%$ 

  52. J04%$ 

  53. $POUFOUT  ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ  4BOTBOJ04ΞϓϦͷ࣮૷  ϦαΠζॲཧͷվળ 

    ·ͱΊ J04%$ 
  54. ϦαΠζॲཧͷվળ J04%$ 

  55. 4BOTBOΞϓϦͷલॲཧ w ը૾ೝࣝॲཧ΁ೖྗ͢Δલॲཧ w 6**NBHF΁ม׵ w ಛఆͷαΠζ΁ॖখ J04%$ 

  56. ը૾Ϋϥεͷ࢖༻ํ਑ w جຊతʹ$**NBHFΛ࢖͍ճ͢ w ඞཁͳͱ͜ΖͰ6**NBHF΁ม׵ ϦαΠζΛ6**NBHFͰߦ͏͔$**NBHFͰߦ͏͔ ͷํ਑͕ᐆດ J04%$ 

  57. ϦαΠζॲཧΛ6**NBHF͔ $**NBHFʹد͍ͤͨ J04%$ 

  58. ͲͪΒʹدͤΔ΂͖͔ J04%$ 

  59. $**NBHFͱ6**NBHFʹ͍ͭͯ ௐ΂ͯΈͨ J04%$ 

  60. $**NBHF $PSF*NBHF J04%$ 

  61. $**NBHF $PSF*NBHFʹؚ·ΕΔը૾Ϋϥε J04%$ 

  62. $PSF*NBHF w ಈըɾ੩ࢭըͷը૾ॲཧΛߦ͏ͨΊͷ 'SBNFXPSL w .FUBMͷ஌ࣝͳ͠ʹ(16Λ࢖ͬͨը૾ॲཧΛ ࣮૷Ͱ͖Δ J04%$ 

  63. $**NBHFͷಛ௃ w Πϛϡʔλϒϧ w Ұ౓࡞੒͢Δͱը૾৘ใͷมߋ͸Ͱ͖ͳ͍ w ผεϨου͔Β҆શʹࢀরՄೳ w ஗Ԇ࣮ߦ w

    ࣮ࡍʹ࢖͏ͱ͖·Ͱ͸ॲཧ͕ߦΘΕͳ͍ J04%$ 
  64. $**NBHFͷ஗Ԇ࣮ߦ let filter = CIFilter(name: "CIGaussianBlur")! filter.setValue(ciImage, forKey: kCIInputImageKey) filter.setValue(10.0,

    forKey: kCIInputRadiusKey) let filteredImage = filter.outputImage! CIContext().createCGImage(filteredImage, from: filteredImage.extent) J04%$ 
  65. $**NBHFͷ஗Ԇ࣮ߦ let filter = CIFilter(name: "CIGaussianBlur")! filter.setValue(ciImage, forKey: kCIInputImageKey) filter.setValue(10.0,

    forKey: kCIInputRadiusKey) let filteredImage = filter.outputImage! CIContext().createCGImage(filteredImage, from: filteredImage.extent) ͜͜Ͱ͸ը૾ॲཧ͸ߦΘΕͳ͍ J04%$ 
  66. $**NBHFͷ஗Ԇ࣮ߦ let filter = CIFilter(name: "CIGaussianBlur")! filter.setValue(ciImage, forKey: kCIInputImageKey) filter.setValue(10.0,

    forKey: kCIInputRadiusKey) let filteredImage = filter.outputImage! CIContext().createCGImage(filteredImage, from: filteredImage.extent) ͜͜Ͱը૾ॲཧ͕ߦΘΕΔ J04%$ 
  67. $**NBHFΛ࢖ͬͨ ϦαΠζॲཧ J04%$ 

  68. $**NBHFΛ࢖ͬͨϦαΠζॲཧ w $*'JMUFSΛ࢖͏ w $*-BOD[PT4DBMF5SBOTGPSN w $*#JDVCJD4DBMF5SBOTGPSN w $*'JMUFSΛ࢖Θͳ͍ w

    ΞϑΟϯม׵ $("⒏OF5SBOTGPSN J04%$ 
  69. $*-BOD[PT4DBMF5SBOTGPSN extension CIImage { func resize(to scale: CGFloat) -> CIImage?

    { guard let filter = CIFilter(name: "CILanczosScaleTransform") else { return nil } filter.setDefaults() filter.setValue(self, forKey: kCIInputImageKey) filter.setValue(scale, forKey: kCIInputScaleKey) return filter.outputImage } } J04%$ 
  70. ΞϑΟϯม׵ extension CIImage { func resize(to scale: CGFloat) -> CIImage?

    { let matrix = CGAffineTransform(scaleX: scale, y: scale) return self.transformed(by: matrix) } } J04%$ 
  71. 6**NBHF 6*,JU J04%$ 

  72. 6**NBHF w J04ΞϓϦͰը૾σʔλΛѻ͏ͨΊͷΫϥε w Πϛϡʔλϒϧ w J04Ͱαϙʔτ͞Ε͍ͯΔը૾ܗࣜʹରԠ w 1/(ͱ+1&(ʹ࠷దԽ͞Ε͍ͯΔ J04%$

    
  73. 6**NBHFΛ࢖ͬͨ ϦαΠζॲཧ J04%$ 

  74. ϦαΠζॲཧ ʙJ04 extension UIImage { func resize(to size: CGSize) ->

    UIImage? { UIGraphicsBeginImageContext(size) defer { UIGraphicsEndImageContext() } let newRect = CGRect(origin: .zero, size: size) draw(in: newRect) return UIGraphicsGetImageFromCurrentImageContext() } } J04%$ 
  75. ϦαΠζॲཧ J04ʙ extension UIImage { func resize(to size: CGSize) ->

    UIImage { let format = UIGraphicsImageRendererFormat() format.scale = 1 let renderer = UIGraphicsImageRenderer(size: size, format: format) return renderer.image(actions: { context in let newRect = CGRect(origin: .zero, size: size) self.draw(in: newRect) }) } } J04%$ 
  76. ϦαΠζॲཧ J04ʙ extension UIImage { func resize(to size: CGSize) ->

    UIImage { let format = UIGraphicsImageRendererFormat() format.scale = 1 let renderer = UIGraphicsImageRenderer(size: size, format: format) return renderer.image(actions: { context in let newRect = CGRect(origin: .zero, size: size) self.draw(in: newRect) }) } } /PO0QUJPOBMʹͳΔʂ J04%$ 
  77. ϦαΠζॲཧͷ࣮૷ํ๏͕ Θ͔ͬͨͱ͜ΖͰ J04%$ 

  78. ॲཧ࣌ؒΛܭଌͯ͠Έͨ J04%$ 

  79. ϦαΠζॲཧͷॲཧ࣌ؒ αϯϓϧΞϓϦͰߦ͍ͬͯΔԁݕग़ॲཧʮճ͋ ͨΓͷ૯ॲཧ࣌ؒʯ͔Βʮԁݕग़ॲཧʹ͔͔ͬͨ ࣌ؒʯΛࠩ͠Ҿ͍ͨ࣌ؒΛܭଌ J04%$ 

  80. ࣮ݧʹ࢖ͬͨJ1IPOF  J1IPOF94.BY J04  J1IPOFT J04 ೖྗը૾͸'VMM)% ʷ J04%$

    
  81. ᶃJ1IPOF94.BY J04 J04%$ 

  82. ܭଌ݁Ռ 6**NBHF ํࣜ ฏۉॲཧ࣌ؒ 6*(SBQIJDT#FHJO*NBHF$POUFYU ʙJ04  6*(SBQIJDT*NBHF3FOEFSFS J04ʙ 

    શମͷॲཧ͔࣌ؒΒೝࣝॲཧΛࠩ͠Ҿ͍ͨ࣌ؒ ୯Ґ͸NT J04%$ 
  83. ܭଌ݁Ռ $**NBHF ํࣜ ฏۉॲཧ࣌ؒ ΞϑΟϯม׵  $*-BOD[PT4DBMF5SBOTGPSN  $*#JDVCJD4DBMF5SBOTGPSN 

    શମͷॲཧ͔࣌ؒΒೝࣝॲཧΛࠩ͠Ҿ͍ͨ࣌ؒ ୯Ґ͸NT J04%$ 
  84. ᶄJ1IPOFT J04 J04%$ 

  85. ܭଌ݁Ռ 6**NBHF ํࣜ ฏۉॲཧ࣌ؒ 6*(SBQIJDT#FHJO*NBHF$POUFYU ʙJ04  6*(SBQIJDT*NBHF3FOEFSFS J04ʙ 

    શମͷॲཧ͔࣌ؒΒೝࣝॲཧΛࠩ͠Ҿ͍ͨ࣌ؒ ୯Ґ͸NT J04%$ 
  86. ܭଌ݁Ռ $**NBHF ํࣜ ฏۉॲཧ࣌ؒ ΞϑΟϯม׵  $*-BOD[PT4DBMF5SBOTGPSN  $*#JDVCJD4DBMF5SBOTGPSN 

    શମͷॲཧ͔࣌ؒΒೝࣝॲཧΛࠩ͠Ҿ͍ͨ࣌ؒ ୯Ґ͸NT J04%$ 
  87. ܭଌ݁Ռ͔ΒΘ͔ͬͨ͜ͱ w $**NBHFΛ࢖ͬͨ΄͏͕ѹ౗తʹ଎͍ w ΞϑΟϯม׵ɺ$*#JDVCJD4DBMF5SBOTGPSNɺ $*-BOD[PT4DBMF5SBOTGPSNͷॱʹ଎͍ w 6*(SBQIJDT*NBHF3FOEFSFS͸ࢥͬͨΑΓ࣌ ͕͔͔ؒͬͨ ࣮૷ͷ໰୊ʁ

    J04%$ 
  88. $POUFOUT  ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ  4BOTBOJ04ΞϓϦͷ࣮૷  ϦαΠζॲཧͷվળ 

    ·ͱΊ J04%$ 
  89. ·ͱΊ J04%$ 

  90. ϦΞϧλΠϜը૾ ೝࣝ ॲཧͷύϑΥʔϚϯε ΞοϓͷͨΊͷը૾Ϋϥεͷ࢖͍෼͚ ᶃ(16 $**NBHF Λ༗ޮ׆༻͢Δ ᶄద੾ͳॲཧํࣜΛબͿ J04%$ 

  91. ᶃ(16 $**NBHF Λ༗ޮ׆༻͢Δ w ൺֱతݹ͍୺຤Ͱ΋ߴ଎ʹը૾ॲཧͰ͖Δ w J04ͷόʔδϣϯ͕৽͍͠ํ͕ύϑΥʔϚϯ ε͕ྑ͍ w ๛෋ͳը૾ॲཧ

    $*'JMUFS w J04ͷόʔδϣϯʹΑͬͯ͸࢖͑ͳ͍΋ͷ΋ ͋Δ J04%$ 
  92. ᶄద੾ͳॲཧํࣜΛબͿ w ಉ͡ϦαΠζํࣜͰ΋ॲཧ଎౓ʹࠩ w ॲཧޙͷը૾ͷ඼࣭ʹ΋ࠩ w ॖখ࣌ͷิؒํࣜͷࠩ w ը૾ ೝࣝ

    ॲཧͷಛੑʹ߹Θͤͯద੾ͳํࣜΛ બఆ͢Δ J04%$ 
  93. ྫ ԁݕग़΍ۣܗݕग़ w ઢ͕औΕΔΑ͏ʹ͍ͨ͠ w ྠֲ͕ڧௐ͞ΕΔͷ͕ྑ͍ w όΠΩϡʔϏοΫ $*#JDVCJD4DBMF5SBOTGPSN J04%$

    
  94. ओͳࢀߟࢿྉ w "QQMF%FWFMPQFS $PSF*NBHF 6*,JUͳͲ w *NBHF3FTJ[JOH5FDIOJRVFT w ը૾ϦαϯϓϦϯάͷൺֱݕূ ଞଟ਺

    J04%$ 
  95. J04%$ 

  96. ͝੩ௌ ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ J04%$