Core ML でiOSアプリはどう変わるのか

第二回 合同勉強会 in 福岡での発表資料です

Transcript

1. 2017/08/26 Core ML Ͱ iOSΞϓϦ͸ Ͳ͏มΘΔͷ͔ ୈೋճ ߹ಉษڧձ in ෱Ԭ
2. About Me
3. • ాத ޹໌ (Takaaki Tanaka) • Ϋϥεϝιουגࣜձࣾ • @kongmingtrap • Swift / Objective-C / Scala / JavaScript … • GyazSquare / GitHub
4. Introduction
5. WWDC17
7. Swift 4 ARKit Core ML HomeKit MusicKit Files
 Vision SiriKit Xcode
8. Swift 4 ARKit Core ML HomeKit MusicKit Files
 Vision SiriKit Xcode
9. Core ML
   IUUQTEFWFMPQFSBQQMFDPNNBDIJOFMFBSOJOH
10. Core ML Core ML͸ɺSiriɺCameraɺQuickTypeͳͲɺApple ੡඼શମͰ࢖༻͞Ε͍ͯΔ৽͍͠جૅతͳϚγϯֶ शϑϨʔϜϫʔΫΛ׆༻͍ͯͩ͘͠͞ɻ Core ML ͸ɺػցֶशϞσϧΛ؆୯ʹ౷߹ͯ͠ڻ͘΄Ͳߴ଎ ͳύϑΥʔϚϯεΛ࣮ݱ͠ɺ਺ߦͷίʔυΛ࢖༻͠ ͯΠϯςϦδΣϯτͳ৽ػೳΛඋ͑ͨΞϓϦΛߏ ஙͰ͖·͢ɻ
    
11. Core ML ͷొ৔ʹΑͬͯ • ઐ໳஌ࣝΛඞཁͱͨ͠ɺը૾ೖྗˠʮֶश Ϟσϧʯˠʮ݁Ռʯˠʮ݁Ռग़ྗʯΛ஥հ ͢ΔͨΊɺiOSΞϓϦ୲౰ऀʹͱͬͯɺʮֶ शϞσϧʯΛཧղͤͣͱ΋ػցֶशͷԸܙ ΛಘΒΕΔΑ͏ʹͳͬͨɻ
    
12. Agenda • Core MLͱपลٕज़ • Core MLΛࢼ͢ • ࣗ࡞ͷֶशϞσϧΛΠϯϙʔτͯ͠ΈΔ
13. Caution!! • ػցֶशɾσΟʔϓϥʔχϯάʹ͍ͭͯ͸ ৄղ͠·ͤΜ • Ϟσϧͷੜ੒ϩδοΫʹؔͯ͠ͷઆ໌͸͠ ·ͤΜ
14. Core MLͱपลٕज़
15. Core ML
    
16. Core ML
    
17. Vision
    
18. 
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZ XXED
19. Vision
    IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPOWJTJPO
20. Face Detection and Recognition
    
    
21. Face Detection and Recognition
    
    
22. Face Detection and Recognition
    
    
23. Face Detection and Recognition
    
    
24. Face Detection and Recognition
    
    
25. Machine Learning Image Analysis
    
    
26. Barcode Detection
    
    
27. Barcode Detection
    
    
28. Image Alignment Analysis
    
    
29. Text Detection
    
    
30. Text Detection
    
    
31. Horizon Detection
    
    
32. Object Detection and Tracking
    
    
33. Object Detection and Tracking
    
    
34. Natural language processing
    
35. 
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZ XXED
36. 
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZ XXED
37. 
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZ XXED
38. Core MLΛࢼ͢
39. ͳΔ΂͘ҙࣝͤͣʹͰ͖Δ͜ ͱΛ΍Γ͍ͨ
40. IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPODPSFNM HFUUJOH@B@DPSF@NM@NPEFM
41. IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPODPSFNM JOUFHSBUJOH@B@DPSF@NM@NPEFM@JOUP@ZPVS@BQQ
42. IUUQTEFWFMPQFSBQQMFDPNNBDIJOFMFBSOJOH
43. Core ML
    
44. Import
46. VNCoreMLModel
47. VNCoreMLRequest guard let coreMLModel = try? VNCoreMLModel(for: Inceptionv3().model) else { fatalError("faild create VMCoreMLModel") } guard let ciImage = CIImage(image: image) else { fatalError("faild convert CIImage") } let request = VNCoreMLRequest(model: coreMLModel) { request, error in guard let results = request.results as? [VNClassificationObservation] else { fatalError("Error faild results") } if let classification = results.first { self.identifierLabel.text = classification.identifier self.confidenceLabel.text = "\(classification.confidence)" } else { print("error") } } let handler = VNImageRequestHandler(ciImage: ciImage, options: [:]) guard (try? handler.perform([request])) != nil else { fatalError("faild handler.perform”) }
48. VNCoreMLRequest guard let coreMLModel = try? VNCoreMLModel(for: Inceptionv3().model) else { fatalError("faild create VMCoreMLModel") } guard let ciImage = CIImage(image: image) else { fatalError("faild convert CIImage") } let request = VNCoreMLRequest(model: coreMLModel) { request, error in guard let results = request.results as? [VNClassificationObservation] else { fatalError("Error faild results") } if let classification = results.first { self.identifierLabel.text = classification.identifier self.confidenceLabel.text = "\(classification.confidence)" } else { print("error") } } let handler = VNImageRequestHandler(ciImage: ciImage, options: [:]) guard (try? handler.perform([request])) != nil else { fatalError("faild handler.perform”) } $PSF.-.PEFMΛࢦఆ
49. VNCoreMLRequest guard let coreMLModel = try? VNCoreMLModel(for: Inceptionv3().model) else { fatalError("faild create VMCoreMLModel") } guard let ciImage = CIImage(image: image) else { fatalError("faild convert CIImage") } let request = VNCoreMLRequest(model: coreMLModel) { request, error in guard let results = request.results as? [VNClassificationObservation] else { fatalError("Error faild results") } if let classification = results.first { self.identifierLabel.text = classification.identifier self.confidenceLabel.text = "\(classification.confidence)" } else { print("error") } } let handler = VNImageRequestHandler(ciImage: ciImage, options: [:]) guard (try? handler.perform([request])) != nil else { fatalError("faild handler.perform”) } $**NBHFΛऔಘ
50. VNCoreMLRequest guard let coreMLModel = try? VNCoreMLModel(for: Inceptionv3().model) else { fatalError("faild create VMCoreMLModel") } guard let ciImage = CIImage(image: image) else { fatalError("faild convert CIImage") } let request = VNCoreMLRequest(model: coreMLModel) { request, error in guard let results = request.results as? [VNClassificationObservation] else { fatalError("Error faild results") } if let classification = results.first { self.identifierLabel.text = classification.identifier self.confidenceLabel.text = "\(classification.confidence)" } else { print("error") } } let handler = VNImageRequestHandler(ciImage: ciImage, options: [:]) guard (try? handler.perform([request])) != nil else { fatalError("faild handler.perform”) } 7/$PSF.-3FRVFTUΛ࡞੒
51. VNCoreMLRequest guard let coreMLModel = try? VNCoreMLModel(for: Inceptionv3().model) else { fatalError("faild create VMCoreMLModel") } guard let ciImage = CIImage(image: image) else { fatalError("faild convert CIImage") } let request = VNCoreMLRequest(model: coreMLModel) { request, error in guard let results = request.results as? [VNClassificationObservation] else { fatalError("Error faild results") } if let classification = results.first { self.identifierLabel.text = classification.identifier self.confidenceLabel.text = "\(classification.confidence)" } else { print("error") } } let handler = VNImageRequestHandler(ciImage: ciImage, options: [:]) guard (try? handler.perform([request])) != nil else { fatalError("faild handler.perform”) } ݁Ռ͕ϋϯυϥʔͰฦͬͯ͘ΔͨΊ
    ॲཧ͍ͨ͠಺༰Λهड़
52. VNCoreMLRequest guard let coreMLModel = try? VNCoreMLModel(for: Inceptionv3().model) else { fatalError("faild create VMCoreMLModel") } guard let ciImage = CIImage(image: image) else { fatalError("faild convert CIImage") } let request = VNCoreMLRequest(model: coreMLModel) { request, error in guard let results = request.results as? [VNClassificationObservation] else { fatalError("Error faild results") } if let classification = results.first { self.identifierLabel.text = classification.identifier self.confidenceLabel.text = "\(classification.confidence)" } else { print("error") } } let handler = VNImageRequestHandler(ciImage: ciImage, options: [:]) guard (try? handler.perform([request])) != nil else { fatalError("faild handler.perform”) } ը૾ղੳϋϯυϥʔΛ࡞੒͠
    QFSGPSN͢Δ
53. Demo
54. ࣗ࡞ͷֶशϞσϧΛ Πϯϙʔτͯ͠ΈΔ
55. 
    motivation
56. IUUQTXXXTMJEFTIBSFOFUDIPNBEP YBNBSJONTBJDVTUPNWJTJPOTFSWJDF
57. 
    
58. 
    Custom Vision Service͠Ύ͍͝…
59. Core ML modelͷ࡞੒
    
60. mlmodel Core ML Λར༻͢Δʹ͋ͨͬͯɺ۩ମతʹ͸ɺ Keras ΍ Caffe ͷΑ͏ͳػցֶशϑϨʔϜ ϫʔΫ͕ग़ྗͨ͠Ϟσϧ͔Β .mlmodel ͱ͍͏ ϑΝΠϧΛ࡞੒͠ɺ͜ΕΛ Xcode ʹΠϯϙʔ τ͠·͢ɻ ҎԼͰ͸ɺ.mlmodel ϑΝΠϧͷ࡞ ੒ํ๏Λݟ͍͖ͯ·͢ɻ
61. Converting Trained Models to Core ML IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPODPSFNM DPOWFSUJOH@USBJOFE@NPEFMT@UP@DPSF@NM
62. coremltools 0.5.1 IUUQTQZQJQZUIPOPSHQZQJDPSFNMUPPMT Keras ΍ Caffe ͷΑ͏ͳػցֶशϑϨʔϜ ϫʔΫ͕ग़ྗͨ͠Ϟσϧ͔Β .mlmodel ʹม׵ Ͱ͖ΔϓϩάϥϜΛఏڙɻʢΦʔϓϯιʔεʣ
63. coremltools 0.5.1 IUUQTQZQJQZUIPOPSHQZQJDPSFNMUPPMT
64. Keras IUUQTLFSBTJPKB
65. KerasͰֶशϞσϧΛੜ੒ • ੒ޭը૾ɺࣦഊը૾Λ४උ • ੒ޭɾࣦഊը૾ͷਖ਼౴σʔλΛ४උ • EC2ʹػցֶश༻ͷAMI͕ఏڙ͞Ε͍ͯΔ ͷͰΠϯελϯεΛ֬อ
71. KerasͰֶशϞσϧΛੜ੒ • ܇࿅ϓϩάϥϜͷ࡞੒ • ܇࿅Λ࣮ߦ • ܇࿅ͨ݁͠ՌͷֶशϞσϧΛcoremltoolsΛ ར༻ͯ͠mlmodelʹม׵͢Δ
72. ܇࿅ϓϩάϥϜ • CIFAR10Λࢀߟʹ܇࿅ϓϩάϥϜΛ࡞੒͢ Δ IUUQTHJUIVCDPNGDIPMMFULFSBTCMPCNBTUFS FYBNQMFTDJGBS@DOOQZ
74. 
    QZUIPO
    VEO@NBJOQZ ܇࿅Λ࣮ߦ
76. coremltoolsͰίϯόʔτ • coremeltoolsΛΠϯετʔϧ͢Δ 
    QJQ
    JOTUBMM
    DPSFNMUPPMT
77. coremltoolsͰίϯόʔτ • KerasͷֶशϞσϧΛCore MLͰར༻Ͱ͖Δ ܗࣜʹมߋ͢Δ • ม׵༻ͷϓϩάϥϜΛ࡞੒͢Δ
78. 
    QZUIPO
    DPOWFSUQZ ม׵Λ࣮ߦ
79. coremltools
80. Recap • Vision frameworkͷڧྗͳαϙʔτͰը૾ղ ੳɺจࣈݕग़͸ָʹ࣮૷Մೳ • ֶशϞσϧͷ࡞੒ʹ͸ઐ໳஌͕ࣝඞཁ • ωοτϫʔΫΛհͣ͞ʹػցֶशϞσϧΛ ར༻͢Δ͜ͱ͕Ͱ͖Δ • ओʹը૾ݕग़Ͱར༻͢Δ৔߹͕ଟͦ͏
81. ँࣙ • KerasͰ͸͡ΊΔDeepLearning • http://dev.classmethod.jp/machine-learning/introduction-keras- deeplearning/ • Keras + iOS11 CoreML + Vision Framework ʹΑΔɺ΋΋Ϋϩإࣝ ผΞϓϦͷ։ൃ • http://qiita.com/kenmaz/items/d416b191f79f60e07752 • 20170818_Xamarinجຊ + MSͷAIαʔϏεʮCustom Vision Serviceʯʹ͍ͭͯ • https://www.slideshare.net/chomado/20170818xamarin-msaicustom- vision-service
82. એ఻
83. એ఻
84. Thank you!