Slide 1
Slide 1 text
Core ML / Vision Framework
ΛͬͯͰ͖Δ͜ͱ
ɹ
2017/06/30 WWDC - Developer's Living @ LIFULL
shingt (Shinichi Goto)
Slide 2
Slide 2 text
shingt (Shinichi Goto)
GitHub: @shingt
Twi5er: @_shingt
2
Slide 3
Slide 3 text
Core ML
Vision Framework
3
Slide 4
Slide 4 text
4
Slide 5
Slide 5 text
Outline
• Core MLͷ֓ཁ
• Vision Frameworkͷ֓ཁ
• Ͱ͖Δ͜ͱ / ࣄྫհ
5
Slide 6
Slide 6 text
Core ML
6
Slide 7
Slide 7 text
ML (Machine Learning)
7
Slide 8
Slide 8 text
8
Slide 9
Slide 9 text
9
Slide 10
Slide 10 text
10
Slide 11
Slide 11 text
Core ML
• ֶशࡁͷModelΛར༻ͯ͠ͷਪʹಛԽ
• Core ML model format (**.mlmodel)
• Xcode͕Swi6ͷΠϯλʔϑΣΠεΛࣗಈੜ
• αϯϓϧϞσϧApple͕ެ։
• Accerelate / Metal্ʹࡌ͓ͬͯΓϋΠύϑΥʔϚϯε
• coremltools
11
Slide 12
Slide 12 text
ɹ
let animalModel = AnimalModel()
if let prediction = try? animalModel.prediction(animalImage: image) {
return prediction.animalType
}
12
Slide 13
Slide 13 text
ɹ
let animalModel = AnimalModel()
if let prediction = try? animalModel.prediction(animalImage: image) {
return prediction.animalType
}
13
Slide 14
Slide 14 text
ɹ
let animalModel = AnimalModel()
if let prediction = try? animalModel.prediction(animalImage: image) {
return prediction.animalType
}
14
Slide 15
Slide 15 text
ɹ
let animalModel = AnimalModel()
if let prediction = try? animalModel.prediction(animalImage: image) {
return prediction.animalType
}
15
Slide 16
Slide 16 text
coremltools
• "iOS্Ͱѻ͏ͨΊͷModelΛͲ͏༻ҙ͢Δ͔" ͷղܾࡦ
• ओཁͳػցֶशπʔϧͷֶशࡁModelΛCore ML༻ͷModelม
• Keras, Caffe, scikit-learn, etc.
16
Slide 17
Slide 17 text
17
Slide 18
Slide 18 text
6/28ʹKeras 2.0αϙʔτʢൃද࣌1.2.2ͷΈͩͬͨʣ
h"ps:/
/forums.developer.apple.com/thread/81196
18
Slide 19
Slide 19 text
Vision Framework
19
Slide 20
Slide 20 text
Vision Framework
• Core ML্ʹࡌͬͨը૾ೝࣝɾମݕग़ͳͲͷը૾ղੳ༻ͷϑϨʔϜϫʔΫ
• Detec,on
• Face, Face landmarks, Rectangle, Barcode, Text, Horizon
• طଘͷͷਫ਼্ʢDeep Learningͷ׆༻ʣ
• Tracking
• Image Registra,on
• Core MLͱͷΈ߹Θͤ
20
Slide 21
Slide 21 text
21
Slide 22
Slide 22 text
Tracking
• ը૾ʢಈըʣதͷମͷ
• إͷTrackingCIDetectorͰՄೳͩͬͨ
• ҙͷରʹରͯ͠ͷTracking͕Մೳʹ
• VisionͰͷݕग़݁Ռ
• ҙͷྖҬࢦఆ
22
Slide 23
Slide 23 text
23
Slide 24
Slide 24 text
Demo
(Rectangle Detec,on + Tracking)
h"ps:/
/github.com/shingt/VisionTrackerSample
24
Slide 25
Slide 25 text
զʑCV/MLͷΤΩεύʔτͰ͋Δඞཁͳ͍
ʢͱɺAppleηογϣϯதʹݴ͍ͬͯΔʣ
25
Slide 26
Slide 26 text
Կ͕Ͱ͖Δͷ͔ʁ
ʢΞϓϦέʔγϣϯΤϯδχΞͱͯ͠ͷࢹ͔Βʣ
26
Slide 27
Slide 27 text
27
Slide 28
Slide 28 text
28
Slide 29
Slide 29 text
ࣄྫհ
29
Slide 30
Slide 30 text
ମݕग़
30
Slide 31
Slide 31 text
31
Slide 32
Slide 32 text
YOLO
• YOLO (You only look once)
• ߴͳ͜ͱ͕ಛͷମݕग़༻ͷ
χϡʔϥϧωοτϫʔΫ
• h1ps:/
/www.youtube.com/watch?
v=VOC3huqHrss
• ͜ΕҰൠతͳYOLO
32
Slide 33
Slide 33 text
• iOSࣄྫ
• YOLO: Core ML versus MPSNNGraph
• Core MLΛ༻͍ͯiOS্ͰYOLOΛಈ࡞
• Tiny YOLOʢެ։͞Ε͍ͯΔModelʣΛར༻
33
Slide 34
Slide 34 text
34
Slide 35
Slide 35 text
ը૾ੜ
35
Slide 36
Slide 36 text
Goodfellow, Ian J.; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley, David; Ozair,
Sherjil; Courville, Aaron; Bengio, Yoshua. GeneraIve Adversarial Networks. arXiv:1406.2661, 2014.
36
Slide 37
Slide 37 text
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representa>on learning with deep
convolu>onal genera>ve adversarial networks. arXiv preprint arXiv:1511.06434, 2015.
37
Slide 38
Slide 38 text
• GAN (Genera+ve Adversarial Nets)
• ֶशσʔλͱࣅͨσʔλΛੜ͢ΔϞσϧͷҰछ
• iOSࣄྫ
• Crea+ve AI on the iPhone: Genera+ve Adversarial Networks
(GAN) with Apple's CoreML Tools
• MNISTΛσʔληοτͱͯ͠ɺCore MLΛ༻͍ͯiOS্Ͱࣈ
ʢʹࣅͨʣը૾Λੜ
38
Slide 39
Slide 39 text
39
Slide 40
Slide 40 text
Summary
• Core ML / Vision Framework
• iOS্Ͱͷը૾ղੳٕज़ͷར༻ϋʔυϧ͕Լ
• ͱ͍͑ࣝ͋Δఔඞཁʢͱײͨ͡ʣ
• Ͱ͖Δ͜ͱ
• ը૾ೝࣝ / τϥοΩϯά / ମݕग़ / ը૾ੜ / etc.
• Follow @mhollemans
40
Slide 41
Slide 41 text
ࢀߟηογϣϯ
• Introducing Core ML
• Core ML in depth
• Vision Framework: Building on Core ML
41
Slide 42
Slide 42 text
ࢀߟࢿྉ
• iOS 11: Machine Learning for everyone
• Google’s MobileNets on the iPhone
• YOLO: Core ML versus MPSNNGraph
• CreaAve AI on the iPhone: GeneraAve Adversarial Networks (GAN)
with Apple's CoreML Tools - Zedge
• Why Core ML will not work for your app (most likely)
• θϩ͔Β࡞ΔDeep Learning
42
Slide 43
Slide 43 text
Thanks!
43
Slide 44
Slide 44 text
ʢิʣͰ͖ͳ͍͜ͱ / ੍ͳͲ
• ֶशෆՄ
• αϙʔτ͍ͯ͠ΔػցֶशϑϨʔϜϫʔΫʹ͍ͭͯɺಛఆͷόʔδϣϯʹറΒΕΔʢগͳ͘ͱ
ݱঢ়ʣ
• Kerasͷ2.0αϙʔτೖͬͨ͠ɺࠓޙ͍͛ͯ͘ͷ͔
• ModelͷαΠζ͕େ͖͗͢Δ
• RegressionͱClassifica5onͷΈʢ☓ ΫϥελϦϯάɺϥϯΩϯάֶशɺetc.ʣ
• ϥϯλΠϜͰϢʔβͷೖྗɾߦಈΛModelʹөͤ͞Δ͜ͱͰ͖ͳ͍
• etc.
44