Core ML / Vision Frameworkを使ってできること / What can we achieve using Core ML and Vision framework

2017/06/30 WWDC - Developer's Living #lifull_wwdc

Transcript

1. Core ML / Vision Framework Λ࢖ͬͯͰ͖Δ͜ͱ ɹ 2017/06/30 WWDC - Developer's Living @ LIFULL shingt (Shinichi Goto)
2. shingt (Shinichi Goto) GitHub: @shingt Twi5er: @_shingt 2
3. Core ML Vision Framework 3
5. Outline • Core MLͷ֓ཁ • Vision Frameworkͷ֓ཁ • Ͱ͖Δ͜ͱ / ࣄྫ঺հ 5
6. Core ML 6
7. ML (Machine Learning) 7
11. Core ML • ֶशࡁͷModelΛར༻ͯ͠ͷਪ࿦ʹಛԽ • Core ML model format (**.mlmodel) • Xcode͕Swi6ͷΠϯλʔϑΣΠεΛࣗಈੜ੒ • αϯϓϧϞσϧ΋Apple͕ެ։ • Accerelate / Metal্ʹࡌ͓ͬͯΓϋΠύϑΥʔϚϯε • coremltools 11
12. ɹ let animalModel = AnimalModel() if let prediction = try? animalModel.prediction(animalImage: image) { return prediction.animalType } 12
13. ɹ let animalModel = AnimalModel() if let prediction = try? animalModel.prediction(animalImage: image) { return prediction.animalType } 13
14. ɹ let animalModel = AnimalModel() if let prediction = try? animalModel.prediction(animalImage: image) { return prediction.animalType } 14
15. ɹ let animalModel = AnimalModel() if let prediction = try? animalModel.prediction(animalImage: image) { return prediction.animalType } 15
16. coremltools • "iOS্Ͱѻ͏ͨΊͷModelΛͲ͏༻ҙ͢Δ͔" ΁ͷղܾࡦ • ओཁͳػցֶशπʔϧͷֶशࡁModelΛCore ML༻ͷModel΁ม ׵ • Keras, Caffe, scikit-learn, etc. 16
18. 6/28ʹKeras 2.0΋αϙʔτʢൃද࣌͸1.2.2ͷΈͩͬͨʣ h"ps:/ /forums.developer.apple.com/thread/81196 18
19. Vision Framework 19
20. Vision Framework • Core ML্ʹࡌͬͨը૾ೝࣝɾ෺ମݕग़ͳͲͷը૾ղੳ༻ͷϑϨʔϜϫʔΫ • Detec,on • Face, Face landmarks, Rectangle, Barcode, Text, Horizon • طଘͷ΋ͷ΋ਫ਼౓޲্ʢDeep Learningͷ׆༻ʣ • Tracking • Image Registra,on • Core MLͱͷ૊Έ߹Θͤ 20
22. Tracking • ը૾ʢಈըʣதͷ෺ମͷ௥੻ • إͷTracking͸CIDetectorͰ΋Մೳͩͬͨ • ೚ҙͷର৅ʹରͯ͠ͷTracking͕Մೳʹ • VisionͰͷݕग़݁Ռ • ೚ҙͷྖҬࢦఆ 22
24. Demo (Rectangle Detec,on + Tracking) h"ps:/ /github.com/shingt/VisionTrackerSample 24
25. զʑ͸CV/MLͷΤΩεύʔτͰ͋Δඞཁ͸ͳ͍ ʢͱɺApple͸ηογϣϯதʹݴ͍ͬͯΔʣ 25
26. Կ͕Ͱ͖Δͷ͔ʁ ʢΞϓϦέʔγϣϯΤϯδχΞͱͯ͠ͷࢹ఺͔Βʣ 26
29. ࣄྫ঺հ 29
30. ෺ମݕग़ 30
32. YOLO • YOLO (You only look once) • ߴ଎ͳ͜ͱ͕ಛ௃ͷ෺ମݕग़༻ͷ χϡʔϥϧωοτϫʔΫ • h1ps:/ /www.youtube.com/watch? v=VOC3huqHrss • ͜Ε͸ҰൠతͳYOLO 32
33. • iOSࣄྫ • YOLO: Core ML versus MPSNNGraph • Core MLΛ༻͍ͯiOS্ͰYOLOΛಈ࡞ • Tiny YOLOʢެ։͞Ε͍ͯΔModelʣΛར༻ 33
35. ը૾ੜ੒ 35
36. Goodfellow, Ian J.; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley, David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua. GeneraIve Adversarial Networks. arXiv:1406.2661, 2014. 36
37. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representa>on learning with deep convolu>onal genera>ve adversarial networks. arXiv preprint arXiv:1511.06434, 2015. 37
38. • GAN (Genera+ve Adversarial Nets) • ֶशσʔλͱࣅͨσʔλΛੜ੒͢ΔϞσϧͷҰछ • iOSࣄྫ • Crea+ve AI on the iPhone: Genera+ve Adversarial Networks (GAN) with Apple's CoreML Tools • MNISTΛσʔληοτͱͯ͠ɺCore MLΛ༻͍ͯiOS্Ͱ਺ࣈ ʢʹࣅͨʣը૾Λੜ੒ 38
40. Summary • Core ML / Vision Framework • iOS্Ͱͷը૾ղੳٕज़ͷར༻ϋʔυϧ͕௿Լ • ͱ͸͍͑஌ࣝ͸͋Δఔ౓ඞཁʢͱײͨ͡ʣ • Ͱ͖Δ͜ͱ • ը૾ೝࣝ / τϥοΩϯά / ෺ମݕग़ / ը૾ੜ੒ / etc. • Follow @mhollemans 40
41. ࢀߟηογϣϯ • Introducing Core ML • Core ML in depth • Vision Framework: Building on Core ML 41
42. ࢀߟࢿྉ • iOS 11: Machine Learning for everyone • Google’s MobileNets on the iPhone • YOLO: Core ML versus MPSNNGraph • CreaAve AI on the iPhone: GeneraAve Adversarial Networks (GAN) with Apple's CoreML Tools - Zedge • Why Core ML will not work for your app (most likely) • θϩ͔Β࡞ΔDeep Learning 42
43. Thanks! 43
44. ʢิ଍ʣͰ͖ͳ͍͜ͱ / ੍໿ͳͲ • ֶश͸ෆՄ • αϙʔτ͍ͯ͠ΔػցֶशϑϨʔϜϫʔΫʹ͍ͭͯɺಛఆͷόʔδϣϯʹറΒΕΔʢগͳ͘ͱ ΋ݱঢ়͸ʣ • Kerasͷ2.0αϙʔτೖͬͨ͠ɺࠓޙ޿͍͛ͯ͘ͷ͔΋ • ModelͷαΠζ͕େ͖͗͢Δ໰୊ • RegressionͱClassifica5onͷΈʢ☓ ΫϥελϦϯάɺϥϯΩϯάֶशɺetc.ʣ • ϥϯλΠϜͰϢʔβͷೖྗɾߦಈΛModelʹ൓өͤ͞Δ͜ͱ͸Ͱ͖ͳ͍ • etc. 44