Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Core ML / Vision Frameworkを使ってできること / What can ...
Search
Shinichi Goto
June 30, 2017
Programming
3
4.3k
Core ML / Vision Frameworkを使ってできること / What can we achieve using Core ML and Vision framework
2017/06/30 WWDC - Developer's Living #lifull_wwdc
Shinichi Goto
June 30, 2017
Tweet
Share
More Decks by Shinichi Goto
See All by Shinichi Goto
WWDC18 ML Overview
_shingt
1
1.4k
Core ML 🏃 iOS Engineer
_shingt
1
600
Wantedly Peopleのスキャン画面の裏側 / Wantedly People Scanning Screen
_shingt
6
6.5k
Providing Better Feedback in Real-time Object Detection Apps
_shingt
2
1.5k
Value Types in WWDC16
_shingt
3
2.6k
debug-remote-local-notification-on-watchos
_shingt
0
11k
Server Side Swift question
_shingt
3
1.1k
Other Decks in Programming
See All in Programming
理論と実務のギャップを超える
eycjur
0
180
Go言語の特性を活かした公式MCP SDKの設計
hond0413
2
530
Developer Joy - The New Paradigm
hollycummins
1
370
Introduce Hono CLI
yusukebe
6
3.1k
オープンソースソフトウェアへの解像度🔬
utam0k
17
3.1k
AIと人間の共創開発!OSSで試行錯誤した開発スタイル
mae616
2
810
CSC509 Lecture 08
javiergs
PRO
0
250
AI 駆動開発におけるコミュニティと AWS CDK の価値
konokenj
5
250
Six and a half ridiculous things to do with Quarkus
hollycummins
0
210
Goで実践するドメイン駆動開発 AIと歩み始めた新規プロダクト開発の現在地
imkaoru
4
900
SwiftDataを使って10万件のデータを読み書きする
akidon0000
0
240
Devoxx BE - Local Development in the AI Era
kdubois
0
140
Featured
See All Featured
Bash Introduction
62gerente
615
210k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
46
2.5k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
253
22k
Raft: Consensus for Rubyists
vanstee
140
7.2k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
32
1.7k
Principles of Awesome APIs and How to Build Them.
keavy
127
17k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.5k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
10
620
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
9
930
Build your cross-platform service in a week with App Engine
jlugia
233
18k
Agile that works and the tools we love
rasmusluckow
331
21k
Transcript
Core ML / Vision Framework ΛͬͯͰ͖Δ͜ͱ ɹ 2017/06/30 WWDC -
Developer's Living @ LIFULL shingt (Shinichi Goto)
shingt (Shinichi Goto) GitHub: @shingt Twi5er: @_shingt 2
Core ML Vision Framework 3
4
Outline • Core MLͷ֓ཁ • Vision Frameworkͷ֓ཁ • Ͱ͖Δ͜ͱ /
ࣄྫհ 5
Core ML 6
ML (Machine Learning) 7
8
9
10
Core ML • ֶशࡁͷModelΛར༻ͯ͠ͷਪʹಛԽ • Core ML model format (**.mlmodel)
• Xcode͕Swi6ͷΠϯλʔϑΣΠεΛࣗಈੜ • αϯϓϧϞσϧApple͕ެ։ • Accerelate / Metal্ʹࡌ͓ͬͯΓϋΠύϑΥʔϚϯε • coremltools 11
ɹ let animalModel = AnimalModel() if let prediction = try?
animalModel.prediction(animalImage: image) { return prediction.animalType } 12
ɹ let animalModel = AnimalModel() if let prediction = try?
animalModel.prediction(animalImage: image) { return prediction.animalType } 13
ɹ let animalModel = AnimalModel() if let prediction = try?
animalModel.prediction(animalImage: image) { return prediction.animalType } 14
ɹ let animalModel = AnimalModel() if let prediction = try?
animalModel.prediction(animalImage: image) { return prediction.animalType } 15
coremltools • "iOS্Ͱѻ͏ͨΊͷModelΛͲ͏༻ҙ͢Δ͔" ͷղܾࡦ • ओཁͳػցֶशπʔϧͷֶशࡁModelΛCore ML༻ͷModelม • Keras,
Caffe, scikit-learn, etc. 16
17
6/28ʹKeras 2.0αϙʔτʢൃද࣌1.2.2ͷΈͩͬͨʣ h"ps:/ /forums.developer.apple.com/thread/81196 18
Vision Framework 19
Vision Framework • Core ML্ʹࡌͬͨը૾ೝࣝɾମݕग़ͳͲͷը૾ղੳ༻ͷϑϨʔϜϫʔΫ • Detec,on • Face, Face
landmarks, Rectangle, Barcode, Text, Horizon • طଘͷͷਫ਼্ʢDeep Learningͷ׆༻ʣ • Tracking • Image Registra,on • Core MLͱͷΈ߹Θͤ 20
21
Tracking • ը૾ʢಈըʣதͷମͷ • إͷTrackingCIDetectorͰՄೳͩͬͨ • ҙͷରʹରͯ͠ͷTracking͕Մೳʹ • VisionͰͷݕग़݁Ռ •
ҙͷྖҬࢦఆ 22
23
Demo (Rectangle Detec,on + Tracking) h"ps:/ /github.com/shingt/VisionTrackerSample 24
զʑCV/MLͷΤΩεύʔτͰ͋Δඞཁͳ͍ ʢͱɺAppleηογϣϯதʹݴ͍ͬͯΔʣ 25
Կ͕Ͱ͖Δͷ͔ʁ ʢΞϓϦέʔγϣϯΤϯδχΞͱͯ͠ͷࢹ͔Βʣ 26
27
28
ࣄྫհ 29
ମݕग़ 30
31
YOLO • YOLO (You only look once) • ߴͳ͜ͱ͕ಛͷମݕग़༻ͷ χϡʔϥϧωοτϫʔΫ
• h1ps:/ /www.youtube.com/watch? v=VOC3huqHrss • ͜ΕҰൠతͳYOLO 32
• iOSࣄྫ • YOLO: Core ML versus MPSNNGraph • Core
MLΛ༻͍ͯiOS্ͰYOLOΛಈ࡞ • Tiny YOLOʢެ։͞Ε͍ͯΔModelʣΛར༻ 33
34
ը૾ੜ 35
Goodfellow, Ian J.; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley,
David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua. GeneraIve Adversarial Networks. arXiv:1406.2661, 2014. 36
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representa>on learning
with deep convolu>onal genera>ve adversarial networks. arXiv preprint arXiv:1511.06434, 2015. 37
• GAN (Genera+ve Adversarial Nets) • ֶशσʔλͱࣅͨσʔλΛੜ͢ΔϞσϧͷҰछ • iOSࣄྫ •
Crea+ve AI on the iPhone: Genera+ve Adversarial Networks (GAN) with Apple's CoreML Tools • MNISTΛσʔληοτͱͯ͠ɺCore MLΛ༻͍ͯiOS্Ͱࣈ ʢʹࣅͨʣը૾Λੜ 38
39
Summary • Core ML / Vision Framework • iOS্Ͱͷը૾ղੳٕज़ͷར༻ϋʔυϧ͕Լ •
ͱ͍͑ࣝ͋Δఔඞཁʢͱײͨ͡ʣ • Ͱ͖Δ͜ͱ • ը૾ೝࣝ / τϥοΩϯά / ମݕग़ / ը૾ੜ / etc. • Follow @mhollemans 40
ࢀߟηογϣϯ • Introducing Core ML • Core ML in depth
• Vision Framework: Building on Core ML 41
ࢀߟࢿྉ • iOS 11: Machine Learning for everyone • Google’s
MobileNets on the iPhone • YOLO: Core ML versus MPSNNGraph • CreaAve AI on the iPhone: GeneraAve Adversarial Networks (GAN) with Apple's CoreML Tools - Zedge • Why Core ML will not work for your app (most likely) • θϩ͔Β࡞ΔDeep Learning 42
Thanks! 43
ʢิʣͰ͖ͳ͍͜ͱ / ੍ͳͲ • ֶशෆՄ • αϙʔτ͍ͯ͠ΔػցֶशϑϨʔϜϫʔΫʹ͍ͭͯɺಛఆͷόʔδϣϯʹറΒΕΔʢগͳ͘ͱ ݱঢ়ʣ • Kerasͷ2.0αϙʔτೖͬͨ͠ɺࠓޙ͍͛ͯ͘ͷ͔
• ModelͷαΠζ͕େ͖͗͢Δ • RegressionͱClassifica5onͷΈʢ☓ ΫϥελϦϯάɺϥϯΩϯάֶशɺetc.ʣ • ϥϯλΠϜͰϢʔβͷೖྗɾߦಈΛModelʹөͤ͞Δ͜ͱͰ͖ͳ͍ • etc. 44