画像処理における、UIImageとCGImageとCIImageの効果的な使い分け / Effective use of UIImage, CGImage, and CIImage in image processing

■イベント
iOSDC Japan 2019
https://iosdc.jp/2019/

■登壇概要
タイトル：
画像処理における、UIImageとCGImageとCIImageの効果的な使い分け

登壇者：
Sansan事業部 プロダクト開発部 栗山徹

https://fortee.jp/iosdc-japan-2019/proposal/3c30c4b4-a647-4198-8e8c-e8100293ee93

▼Sansan Builders Box
https://buildersbox.corp-sansan.com/

Transcript

1. ը૾ॲཧʹ͓͚Δ 6**NBHFͱ$(*NBHFͱ$**NBHF ͷޮՌతͳ࢖͍෼͚ ܀ࢁప
   !LPUFUV J04%$
    
2. ܀ࢁ
   ప
   
   
   !LPUFUV
   
   LPUFUVDP w J04
   "QQ
   %FWFMPQFS
   BU
   w ઃܭपΓ
   
   "OESPJE
   
   ࣗ࡞04
   3VTU
   
   $5' J04%$
    
3. J04%$
   
   -5 άϥϑΟοΫεҎ֎ͷ෼໺ Ͱͷ.FUBMͷՄೳੑ J04%$
    
4. w ๏ਓ޲໊͚ࢗ؅ཧαʔϏε w ໿ ͕ࣾܖ໿ w ແྉτϥΠΞϧͰ͖·͢
   ຕ·Ͱొ࿥Մ J04%$
    
5. 2JJUB
   +PCT ࢖༻ٕज़ɺ։ൃମ੍ͳͲΛ ঺հ͍ͯ͠·͢ J04%$
    
6. ߨԋͷςʔϚ 4BOTBOΞϓϦͷ໊ࢗొ࿥ػೳΛ୊ࡐʹ ϦΞϧλΠϜը૾ ೝࣝ ॲཧʹ͓͚Δ ը૾Ϋϥεͷ࢖͍෼͚ʹ͍ͭͯൃද͠·͢ɻ J04%$
    
7. ओʹϦαΠζ ॖখ ॲཧʹ͓͚Δ$**NBHFͱ 6**NBHFͷ࢖͍෼͚ͷ࿩͕ϝΠϯʹͳΓ·͢ɻ J04%$
    
8. 4BOTBOΞϓϦͷ ໊ࢗొ࿥ػೳ J04%$
    
9. ໊ࢗొ࿥ػೳ w ը໘্ͷ໊ࢗΛࣗಈݕग़ w ࡱӨޙ͙͢ػցతʹσʔλ Խ͞Ε·͢ w ද໘
   
   ཪ໘
   ࡱӨͰ͖·͢ J04%$
    
10. ࣗಈݕग़ػೳͷ όʔδϣϯΞοϓ ೥݄ϦϦʔε J04%$
     
11. ࣗಈݕग़ػೳͷόʔδϣϯΞοϓ w ໊ࢗͷݕग़ਫ਼౓޲্
    ! w ಺෦ॲཧͷஔ͖׵͕͑ඞཁ w ໊ࢗొ࿥ॲཧΛݟ௚ͦ͏ʂ J04%$
     
12. ݟ௚͍ͯ͠Δ͏ͪʹ J04%$
     
13. ͜Ε͸J04%$ͷ ωλʹͳΔͷͰ͸ J04%$
     
14. ৽ͨͳΔ$G1஀ੜͷ ॠؒͰ͋Δ J04%$
     
15. $POUFOUT  ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ  4BOTBO
    J04ΞϓϦͷ࣮૷  ϦαΠζॲཧͷվળ  ·ͱΊ J04%$
     
16. $POUFOUT  ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ  4BOTBO
    J04ΞϓϦͷ࣮૷  ϦαΠζॲཧͷվળ  ·ͱΊ J04%$
     
17. 
    ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ J04%$
     
18. ϦΞϧλΠϜը૾ ೝࣝ ॲཧ ΧϝϥͳͲ͔ΒϦΞϧλΠϜʹ౉ͬͯ͘Δը૾Λ ࢖ͬͯը૾ॲཧ ·ͨ͸ը૾ೝࣝ Λߦ͍݁ՌΛ ग़ྗ͢Δ J04%$
     
19. ॲཧͷྲྀΕ Ұൠతͳέʔε w Χϝϥ͔Βը૾Λऔಘ w ը૾ ೝࣝ ॲཧ΁ͷೖྗͷͨΊͷલॲཧ w ը૾ ೝࣝ ॲཧ w ॲཧ݁Ռͷදࣔ J04%$
     
20. ॲཧͷྲྀΕ Ұൠతͳέʔε w Χϝϥ͔Βը૾Λऔಘ w ը૾ ೝࣝ ॲཧ΁ͷೖྗͷͨΊͷલॲཧ w ը૾ ೝࣝ ॲཧ w ॲཧ݁Ռͷදࣔ Ωϟϓνϟͨ͠Χϝϥө૾ΛϑϨʔϜ͝ͱʹऔಘ J04%$
     
21. ॲཧͷྲྀΕ Ұൠతͳέʔε w Χϝϥ͔Βը૾Λऔಘ w ը૾ ೝࣝ ॲཧ΁ͷೖྗͷͨΊͷલॲཧ w ը૾ ೝࣝ ॲཧ w ॲཧ݁Ռͷදࣔ ը૾ܗࣜͷม׵΍ϦαΠζɾάϨʔεέʔϧԽ ը૾ೝࣝͷ৔߹ ͳͲ J04%$
     
22. ॲཧͷྲྀΕ Ұൠతͳέʔε w Χϝϥ͔Βը૾Λऔಘ w ը૾ ೝࣝ ॲཧ΁ͷೖྗͷͨΊͷલॲཧ w ը૾ ೝࣝ ॲཧ w ॲཧ݁Ռͷදࣔ ΅͔͠΍৭ຯͷมߋɺಛ௃఺ݕग़ͳͲ J04%$
     
23. ॲཧͷྲྀΕ Ұൠతͳέʔε w Χϝϥ͔Βը૾Λऔಘ w ը૾ ೝࣝ ॲཧ΁ͷೖྗͷͨΊͷલॲཧ w ը૾ ೝࣝ ॲཧ w ॲཧ݁Ռͷදࣔ ॲཧͨ͠ޙͷը૾΍ೝࣝ݁ՌΛը໘ʹදࣔ J04%$
     
24. $POUFOUT  ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ  4BOTBO
    J04ΞϓϦͷ࣮૷  ϦαΠζॲཧͷվળ  ·ͱΊ J04%$
     
25. 
    4BOTBO
    J04
    ΞϓϦͷ࣮૷ J04%$
     
26. J04ʹ͓͚Δ ϦΞϧλΠϜը૾ ೝࣝ ॲཧ J04%$
     
27. J04%$
     
28. 
    αϯϓϧΞϓϦ 0QFO$7Λ࢖ͬͨԁݕग़ J04%$
     
29. J04%$
     
30. Χϝϥ͔Βը૾Λऔಘ J04%$
     
31. "7'PVOEBUJPO w ΧϝϥॲཧΛ࣮૷͢Δͱ͖ʹ࢖͏'SBNFXPSL w ϦΞϧλΠϜॲཧΛߦ͏৔߹͸ "7$BQUVSF7JEFP%BUB0VUQVUΛ࢖͏ J04%$
     
32. ը૾Λड͚औΔ class Capture: AVCaptureVideoDataOutputSampleBufferDelegate { func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) { ... } } J04%$
     
33. ը૾Λड͚औΔ class Capture: AVCaptureVideoDataOutputSampleBufferDelegate { func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) { ... } } ը૾σʔλ͸$.4BNQMF#VGGFSͰड͚औΔͷ Ͱผͳը૾Ϋϥε΁ม׵ͯ͋͛͠ͳ͍ͱ͍͚ͳ͍ J04%$
     
34. J04%$
     
35. $**NBHF΁ม׵ w $.4BNQMF#V⒎FS͸$**NBHF΁ม׵͢Δɹ ͜ͱ͕ଟ͍ w $7*NBHF#VGGFSΛܦ༝͢Δ J04%$
     
36. $**NBHF΁ม׵ func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) { let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) let ciImage = CIImage(cvImageBuffer: imageBuffer!) } J04%$
     
37. $**NBHF΁ม׵ func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) { let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) let ciImage = CIImage(cvImageBuffer: imageBuffer!) } $.4BNQMF#V⒎FS͔Β$7*NBHF#VGGFS΁ɹ ม׵ J04%$
     
38. $**NBHF΁ม׵ func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) { let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) let ciImage = CIImage(cvImageBuffer: imageBuffer!) } $7*NBHF#V⒎FS͔Β$**NBHF΁ม׵ J04%$
     
39. 4BOTBOΞϓϦʹ͓͚Δ "7'PVOEBUJPO w ΑΓଟ͘ͷ໊ࢗΛೝ͍ࣝͨ͠ w ,αΠζ
    ,ඇରԠػछ͸'VMM
    )%αΠζ w ೥ʑΧϝϥͷੑೳ্͕͕͍ͬͯΔ J04%$
     
40. J04%$
     
41. ը૾ ೝࣝ ॲཧ΁ͷೖྗͷ ͨΊͷલॲཧ J04%$
     
42. 4BOTBOΞϓϦʹ͓͚Δલॲཧ w ը૾ॲཧ΁ͷೖྗ͸6**NBHF w ೖྗը૾Λॖখ͠ͳ͍ͱ͍͚ͳ͍ ۃ୺ʹେ͖ ͳը૾ͷ৔߹ʹॲཧ͕͔͔࣌ؒͬͯ͠·͏ J04%$
     
43. 6**NBHF΁ม׵ let context = CIContext() let cgImage = context.createCGImage(ciImage, from: ciImage.extent) let uiImage = UIImage(cgImage: cgImage!) J04%$
     
44. 6**NBHF΁ม׵ let context = CIContext() let cgImage = context.createCGImage(ciImage, from: ciImage.extent) let uiImage = UIImage(cgImage: cgImage!) έʔεʹ΋ΑΔ͕ $(*NBHFΛܦ༝͢Δέʔε ͕ଟ͍ J04%$
     
45. J04%$
     
46. ը૾ ೝࣝ ॲཧ J04%$
     
47. 4BOTBOΞϓϦʹ͓͚Δ໊ࢗೝࣝॲཧ w ෦ॺؒ
    &JHIUࣄۀ෦
    ͰίʔυΛڞ༻ w $BSUIBHFରԠ w Ұ෦"OESPJEΞϓϦͱίʔυΛڞ༻ w ೖྗ͸6**NBHF J04%$
     
48. J04%$
     
49. ॲཧ݁Ռͷදࣔ J04%$
     
50. ॲཧ݁Ռͷදࣔ w ը૾ॲཧޙͷը૾ w 6**NBHF7JFX΍.5,7JFX .FUBM,JU w ೝࣝ݁Ռ w 6*#F[JFS1BUI΍$"4IBQF-BZFS J04%$
     
51. J04%$
     
52. J04%$
     
53. $POUFOUT  ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ  4BOTBO
    J04ΞϓϦͷ࣮૷  ϦαΠζॲཧͷվળ  ·ͱΊ J04%$
     
54. 
    ϦαΠζॲཧͷվળ J04%$
     
55. 4BOTBOΞϓϦͷલॲཧ w ը૾ೝࣝॲཧ΁ೖྗ͢Δલॲཧ w 6**NBHF΁ม׵ w ಛఆͷαΠζ΁ॖখ J04%$
     
56. ը૾Ϋϥεͷ࢖༻ํ਑ w جຊతʹ$**NBHFΛ࢖͍ճ͢ w ඞཁͳͱ͜ΖͰ6**NBHF΁ม׵ ϦαΠζΛ6**NBHFͰߦ͏͔$**NBHFͰߦ͏͔ ͷํ਑͕ᐆດ J04%$
     
57. ϦαΠζॲཧΛ6**NBHF͔ $**NBHFʹد͍ͤͨ J04%$
     
58. ͲͪΒʹدͤΔ΂͖͔ J04%$
     
59. $**NBHFͱ6**NBHFʹ͍ͭͯ ௐ΂ͯΈͨ J04%$
     
60. $**NBHF $PSF
    *NBHF J04%$
     
61. $**NBHF $PSF
    *NBHFʹؚ·ΕΔը૾Ϋϥε J04%$
     
62. $PSF
    *NBHF w ಈըɾ੩ࢭըͷը૾ॲཧΛߦ͏ͨΊͷ 'SBNFXPSL w .FUBMͷ஌ࣝͳ͠ʹ(16Λ࢖ͬͨը૾ॲཧΛ ࣮૷Ͱ͖Δ J04%$
     
63. $**NBHFͷಛ௃ w Πϛϡʔλϒϧ w Ұ౓࡞੒͢Δͱը૾৘ใͷมߋ͸Ͱ͖ͳ͍ w ผεϨου͔Β҆શʹࢀরՄೳ w ஗Ԇ࣮ߦ w ࣮ࡍʹ࢖͏ͱ͖·Ͱ͸ॲཧ͕ߦΘΕͳ͍ J04%$
     
64. $**NBHFͷ஗Ԇ࣮ߦ let filter = CIFilter(name: "CIGaussianBlur")! filter.setValue(ciImage, forKey: kCIInputImageKey) filter.setValue(10.0, forKey: kCIInputRadiusKey) let filteredImage = filter.outputImage! CIContext().createCGImage(filteredImage, from: filteredImage.extent) J04%$
     
65. $**NBHFͷ஗Ԇ࣮ߦ let filter = CIFilter(name: "CIGaussianBlur")! filter.setValue(ciImage, forKey: kCIInputImageKey) filter.setValue(10.0, forKey: kCIInputRadiusKey) let filteredImage = filter.outputImage! CIContext().createCGImage(filteredImage, from: filteredImage.extent) ͜͜Ͱ͸ը૾ॲཧ͸ߦΘΕͳ͍ J04%$
     
66. $**NBHFͷ஗Ԇ࣮ߦ let filter = CIFilter(name: "CIGaussianBlur")! filter.setValue(ciImage, forKey: kCIInputImageKey) filter.setValue(10.0, forKey: kCIInputRadiusKey) let filteredImage = filter.outputImage! CIContext().createCGImage(filteredImage, from: filteredImage.extent) ͜͜Ͱը૾ॲཧ͕ߦΘΕΔ J04%$
     
67. $**NBHFΛ࢖ͬͨ ϦαΠζॲཧ J04%$
     
68. $**NBHFΛ࢖ͬͨϦαΠζॲཧ w $*'JMUFSΛ࢖͏ w $*-BOD[PT4DBMF5SBOTGPSN w $*#JDVCJD4DBMF5SBOTGPSN w $*'JMUFSΛ࢖Θͳ͍ w ΞϑΟϯม׵ $("⒏OF5SBOTGPSN J04%$
     
69. $*-BOD[PT4DBMF5SBOTGPSN extension CIImage { func resize(to scale: CGFloat) -> CIImage? { guard let filter = CIFilter(name: "CILanczosScaleTransform") else { return nil } filter.setDefaults() filter.setValue(self, forKey: kCIInputImageKey) filter.setValue(scale, forKey: kCIInputScaleKey) return filter.outputImage } } J04%$
     
70. ΞϑΟϯม׵ extension CIImage { func resize(to scale: CGFloat) -> CIImage? { let matrix = CGAffineTransform(scaleX: scale, y: scale) return self.transformed(by: matrix) } } J04%$
     
71. 6**NBHF 6*,JU J04%$
     
72. 6**NBHF w J04ΞϓϦͰը૾σʔλΛѻ͏ͨΊͷΫϥε w Πϛϡʔλϒϧ w J04Ͱαϙʔτ͞Ε͍ͯΔը૾ܗࣜʹରԠ w 1/(ͱ+1&(ʹ࠷దԽ͞Ε͍ͯΔ J04%$
     
73. 6**NBHFΛ࢖ͬͨ ϦαΠζॲཧ J04%$
     
74. ϦαΠζॲཧ ʙJ04 extension UIImage { func resize(to size: CGSize) -> UIImage? { UIGraphicsBeginImageContext(size) defer { UIGraphicsEndImageContext() } let newRect = CGRect(origin: .zero, size: size) draw(in: newRect) return UIGraphicsGetImageFromCurrentImageContext() } } J04%$
     
75. ϦαΠζॲཧ J04ʙ extension UIImage { func resize(to size: CGSize) -> UIImage { let format = UIGraphicsImageRendererFormat() format.scale = 1 let renderer = UIGraphicsImageRenderer(size: size, format: format) return renderer.image(actions: { context in let newRect = CGRect(origin: .zero, size: size) self.draw(in: newRect) }) } } J04%$
     
76. ϦαΠζॲཧ J04ʙ extension UIImage { func resize(to size: CGSize) -> UIImage { let format = UIGraphicsImageRendererFormat() format.scale = 1 let renderer = UIGraphicsImageRenderer(size: size, format: format) return renderer.image(actions: { context in let newRect = CGRect(origin: .zero, size: size) self.draw(in: newRect) }) } } /PO0QUJPOBMʹͳΔʂ J04%$
     
77. ϦαΠζॲཧͷ࣮૷ํ๏͕ Θ͔ͬͨͱ͜ΖͰ J04%$
     
78. ॲཧ࣌ؒΛܭଌͯ͠Έͨ J04%$
     
79. ϦαΠζॲཧͷॲཧ࣌ؒ αϯϓϧΞϓϦͰߦ͍ͬͯΔԁݕग़ॲཧʮճ͋ ͨΓͷ૯ॲཧ࣌ؒʯ͔Βʮԁݕग़ॲཧʹ͔͔ͬͨ ࣌ؒʯΛࠩ͠Ҿ͍ͨ࣌ؒΛܭଌ J04%$
     
80. ࣮ݧʹ࢖ͬͨJ1IPOF  J1IPOF
    94
    .BY
    J04  J1IPOF
    T
    J04 ೖྗը૾͸'VMM
    )% 
    ʷ
     J04%$
     
81. ᶃ
    J1IPOF
    94
    .BY
    J04 J04%$
     
82. ܭଌ݁Ռ 6**NBHF ํࣜ ฏۉॲཧ࣌ؒ 6*(SBQIJDT#FHJO*NBHF$POUFYU ʙJ04  6*(SBQIJDT*NBHF3FOEFSFS J04ʙ  શମͷॲཧ͔࣌ؒΒೝࣝॲཧΛࠩ͠Ҿ͍ͨ࣌ؒ ୯Ґ͸NT J04%$
     
83. ܭଌ݁Ռ $**NBHF ํࣜ ฏۉॲཧ࣌ؒ ΞϑΟϯม׵  $*-BOD[PT4DBMF5SBOTGPSN  $*#JDVCJD4DBMF5SBOTGPSN  શମͷॲཧ͔࣌ؒΒೝࣝॲཧΛࠩ͠Ҿ͍ͨ࣌ؒ ୯Ґ͸NT J04%$
     
84. ᶄ
    J1IPOF
    T
    J04 J04%$
     
85. ܭଌ݁Ռ 6**NBHF ํࣜ ฏۉॲཧ࣌ؒ 6*(SBQIJDT#FHJO*NBHF$POUFYU ʙJ04  6*(SBQIJDT*NBHF3FOEFSFS J04ʙ  શମͷॲཧ͔࣌ؒΒೝࣝॲཧΛࠩ͠Ҿ͍ͨ࣌ؒ ୯Ґ͸NT J04%$
     
86. ܭଌ݁Ռ $**NBHF ํࣜ ฏۉॲཧ࣌ؒ ΞϑΟϯม׵  $*-BOD[PT4DBMF5SBOTGPSN  $*#JDVCJD4DBMF5SBOTGPSN  શମͷॲཧ͔࣌ؒΒೝࣝॲཧΛࠩ͠Ҿ͍ͨ࣌ؒ ୯Ґ͸NT J04%$
     
87. ܭଌ݁Ռ͔ΒΘ͔ͬͨ͜ͱ w $**NBHFΛ࢖ͬͨ΄͏͕ѹ౗తʹ଎͍ w ΞϑΟϯม׵ɺ$*#JDVCJD4DBMF5SBOTGPSNɺ $*-BOD[PT4DBMF5SBOTGPSNͷॱʹ଎͍ w 6*(SBQIJDT*NBHF3FOEFSFS͸ࢥͬͨΑΓ࣌ ͕͔͔ؒͬͨ
    ࣮૷ͷ໰୊ʁ J04%$
     
88. $POUFOUT  ϦΞϧλΠϜը૾ ೝࣝ ॲཧͱ͸ʁ  4BOTBO
    J04ΞϓϦͷ࣮૷  ϦαΠζॲཧͷվળ  ·ͱΊ J04%$
     
89. 
    ·ͱΊ J04%$
     
90. ϦΞϧλΠϜը૾ ೝࣝ ॲཧͷύϑΥʔϚϯε ΞοϓͷͨΊͷը૾Ϋϥεͷ࢖͍෼͚ ᶃ
    (16 $**NBHF Λ༗ޮ׆༻͢Δ ᶄ
    ద੾ͳॲཧํࣜΛબͿ J04%$
     
91. ᶃ
    (16 $**NBHF Λ༗ޮ׆༻͢Δ w ൺֱతݹ͍୺຤Ͱ΋ߴ଎ʹը૾ॲཧͰ͖Δ w J04ͷόʔδϣϯ͕৽͍͠ํ͕ύϑΥʔϚϯ ε͕ྑ͍ w ๛෋ͳը૾ॲཧ $*'JMUFS w J04ͷόʔδϣϯʹΑͬͯ͸࢖͑ͳ͍΋ͷ΋ ͋Δ J04%$
     
92. ᶄ
    ద੾ͳॲཧํࣜΛબͿ w ಉ͡ϦαΠζํࣜͰ΋ॲཧ଎౓ʹࠩ w ॲཧޙͷը૾ͷ඼࣭ʹ΋ࠩ w ॖখ࣌ͷิؒํࣜͷࠩ w ը૾ ೝࣝ ॲཧͷಛੑʹ߹Θͤͯద੾ͳํࣜΛ બఆ͢Δ J04%$
     
93. ྫ
    ԁݕग़΍ۣܗݕग़ w ઢ͕औΕΔΑ͏ʹ͍ͨ͠ w ྠֲ͕ڧௐ͞ΕΔͷ͕ྑ͍ w όΠΩϡʔϏοΫ $*#JDVCJD4DBMF5SBOTGPSN J04%$
     
94. ओͳࢀߟࢿྉ w "QQMF
    %FWFMPQFS $PSF
    *NBHF
    6*,JUͳͲ w *NBHF
    3FTJ[JOH
    5FDIOJRVFT w ը૾ϦαϯϓϦϯάͷൺֱݕূ ଞଟ਺ J04%$
     
95. ͝੩ௌ ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ J04%$