Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Core ML / Vision Frameworkを使ってできること / What can ...
Search
Shinichi Goto
June 30, 2017
Programming
3
4.3k
Core ML / Vision Frameworkを使ってできること / What can we achieve using Core ML and Vision framework
2017/06/30 WWDC - Developer's Living #lifull_wwdc
Shinichi Goto
June 30, 2017
Tweet
Share
More Decks by Shinichi Goto
See All by Shinichi Goto
WWDC18 ML Overview
_shingt
1
1.4k
Core ML 🏃 iOS Engineer
_shingt
1
590
Wantedly Peopleのスキャン画面の裏側 / Wantedly People Scanning Screen
_shingt
6
6.4k
Providing Better Feedback in Real-time Object Detection Apps
_shingt
2
1.5k
Value Types in WWDC16
_shingt
3
2.6k
debug-remote-local-notification-on-watchos
_shingt
0
11k
Server Side Swift question
_shingt
3
1.1k
Other Decks in Programming
See All in Programming
Let's Write a Train Tracking Algorithm
twocentstudios
0
220
uniqueパッケージの内部実装を支えるweak pointerの話
magavel
0
920
ソフトウェア設計の実践的な考え方
masuda220
PRO
3
480
CSC509 Lecture 01
javiergs
PRO
1
430
AI Coding Meetup #3 - 導入セッション / ai-coding-meetup-3
izumin5210
0
590
開発者への寄付をアプリ内課金として実装する時の気の使いどころ
ski
0
350
2分台で1500examples完走!爆速CIを支える環境構築術 - Kaigi on Rails 2025
falcon8823
3
3.1k
iOS 17で追加されたSubscriptionStoreView を利用して5分でサブスク実装チャレンジ
natmark
0
590
Back to the Future: Let me tell you about the ACP protocol
terhechte
0
130
私達はmodernize packageに夢を見るか feat. go/analysis, go/ast / Go Conference 2025
kaorumuta
2
490
CSC509 Lecture 04
javiergs
PRO
0
290
Pull-Requestの内容を1クリックで動作確認可能にするワークフロー
natmark
2
450
Featured
See All Featured
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Rebuilding a faster, lazier Slack
samanthasiow
84
9.2k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
850
Making the Leap to Tech Lead
cromwellryan
135
9.5k
Automating Front-end Workflow
addyosmani
1371
200k
Done Done
chrislema
185
16k
Principles of Awesome APIs and How to Build Them.
keavy
127
17k
Faster Mobile Websites
deanohume
310
31k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
It's Worth the Effort
3n
187
28k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
162
15k
The Straight Up "How To Draw Better" Workshop
denniskardys
237
140k
Transcript
Core ML / Vision Framework ΛͬͯͰ͖Δ͜ͱ ɹ 2017/06/30 WWDC -
Developer's Living @ LIFULL shingt (Shinichi Goto)
shingt (Shinichi Goto) GitHub: @shingt Twi5er: @_shingt 2
Core ML Vision Framework 3
4
Outline • Core MLͷ֓ཁ • Vision Frameworkͷ֓ཁ • Ͱ͖Δ͜ͱ /
ࣄྫհ 5
Core ML 6
ML (Machine Learning) 7
8
9
10
Core ML • ֶशࡁͷModelΛར༻ͯ͠ͷਪʹಛԽ • Core ML model format (**.mlmodel)
• Xcode͕Swi6ͷΠϯλʔϑΣΠεΛࣗಈੜ • αϯϓϧϞσϧApple͕ެ։ • Accerelate / Metal্ʹࡌ͓ͬͯΓϋΠύϑΥʔϚϯε • coremltools 11
ɹ let animalModel = AnimalModel() if let prediction = try?
animalModel.prediction(animalImage: image) { return prediction.animalType } 12
ɹ let animalModel = AnimalModel() if let prediction = try?
animalModel.prediction(animalImage: image) { return prediction.animalType } 13
ɹ let animalModel = AnimalModel() if let prediction = try?
animalModel.prediction(animalImage: image) { return prediction.animalType } 14
ɹ let animalModel = AnimalModel() if let prediction = try?
animalModel.prediction(animalImage: image) { return prediction.animalType } 15
coremltools • "iOS্Ͱѻ͏ͨΊͷModelΛͲ͏༻ҙ͢Δ͔" ͷղܾࡦ • ओཁͳػցֶशπʔϧͷֶशࡁModelΛCore ML༻ͷModelม • Keras,
Caffe, scikit-learn, etc. 16
17
6/28ʹKeras 2.0αϙʔτʢൃද࣌1.2.2ͷΈͩͬͨʣ h"ps:/ /forums.developer.apple.com/thread/81196 18
Vision Framework 19
Vision Framework • Core ML্ʹࡌͬͨը૾ೝࣝɾମݕग़ͳͲͷը૾ղੳ༻ͷϑϨʔϜϫʔΫ • Detec,on • Face, Face
landmarks, Rectangle, Barcode, Text, Horizon • طଘͷͷਫ਼্ʢDeep Learningͷ׆༻ʣ • Tracking • Image Registra,on • Core MLͱͷΈ߹Θͤ 20
21
Tracking • ը૾ʢಈըʣதͷମͷ • إͷTrackingCIDetectorͰՄೳͩͬͨ • ҙͷରʹରͯ͠ͷTracking͕Մೳʹ • VisionͰͷݕग़݁Ռ •
ҙͷྖҬࢦఆ 22
23
Demo (Rectangle Detec,on + Tracking) h"ps:/ /github.com/shingt/VisionTrackerSample 24
զʑCV/MLͷΤΩεύʔτͰ͋Δඞཁͳ͍ ʢͱɺAppleηογϣϯதʹݴ͍ͬͯΔʣ 25
Կ͕Ͱ͖Δͷ͔ʁ ʢΞϓϦέʔγϣϯΤϯδχΞͱͯ͠ͷࢹ͔Βʣ 26
27
28
ࣄྫհ 29
ମݕग़ 30
31
YOLO • YOLO (You only look once) • ߴͳ͜ͱ͕ಛͷମݕग़༻ͷ χϡʔϥϧωοτϫʔΫ
• h1ps:/ /www.youtube.com/watch? v=VOC3huqHrss • ͜ΕҰൠతͳYOLO 32
• iOSࣄྫ • YOLO: Core ML versus MPSNNGraph • Core
MLΛ༻͍ͯiOS্ͰYOLOΛಈ࡞ • Tiny YOLOʢެ։͞Ε͍ͯΔModelʣΛར༻ 33
34
ը૾ੜ 35
Goodfellow, Ian J.; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley,
David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua. GeneraIve Adversarial Networks. arXiv:1406.2661, 2014. 36
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representa>on learning
with deep convolu>onal genera>ve adversarial networks. arXiv preprint arXiv:1511.06434, 2015. 37
• GAN (Genera+ve Adversarial Nets) • ֶशσʔλͱࣅͨσʔλΛੜ͢ΔϞσϧͷҰछ • iOSࣄྫ •
Crea+ve AI on the iPhone: Genera+ve Adversarial Networks (GAN) with Apple's CoreML Tools • MNISTΛσʔληοτͱͯ͠ɺCore MLΛ༻͍ͯiOS্Ͱࣈ ʢʹࣅͨʣը૾Λੜ 38
39
Summary • Core ML / Vision Framework • iOS্Ͱͷը૾ղੳٕज़ͷར༻ϋʔυϧ͕Լ •
ͱ͍͑ࣝ͋Δఔඞཁʢͱײͨ͡ʣ • Ͱ͖Δ͜ͱ • ը૾ೝࣝ / τϥοΩϯά / ମݕग़ / ը૾ੜ / etc. • Follow @mhollemans 40
ࢀߟηογϣϯ • Introducing Core ML • Core ML in depth
• Vision Framework: Building on Core ML 41
ࢀߟࢿྉ • iOS 11: Machine Learning for everyone • Google’s
MobileNets on the iPhone • YOLO: Core ML versus MPSNNGraph • CreaAve AI on the iPhone: GeneraAve Adversarial Networks (GAN) with Apple's CoreML Tools - Zedge • Why Core ML will not work for your app (most likely) • θϩ͔Β࡞ΔDeep Learning 42
Thanks! 43
ʢิʣͰ͖ͳ͍͜ͱ / ੍ͳͲ • ֶशෆՄ • αϙʔτ͍ͯ͠ΔػցֶशϑϨʔϜϫʔΫʹ͍ͭͯɺಛఆͷόʔδϣϯʹറΒΕΔʢগͳ͘ͱ ݱঢ়ʣ • Kerasͷ2.0αϙʔτೖͬͨ͠ɺࠓޙ͍͛ͯ͘ͷ͔
• ModelͷαΠζ͕େ͖͗͢Δ • RegressionͱClassifica5onͷΈʢ☓ ΫϥελϦϯάɺϥϯΩϯάֶशɺetc.ʣ • ϥϯλΠϜͰϢʔβͷೖྗɾߦಈΛModelʹөͤ͞Δ͜ͱͰ͖ͳ͍ • etc. 44