Core ML / Vision Frameworkを使ってできること / What can we achieve using Core ML and Vision framework

Transcript

1. Core ML / Vision Framework Λ࢖ͬͯͰ͖Δ͜ͱ ɹ 2017/06/30 WWDC -

   Developer's Living @ LIFULL shingt (Shinichi Goto)

2. shingt (Shinichi Goto) GitHub: @shingt Twi5er: @_shingt 2

3. Core ML Vision Framework 3

4. 4

5. Outline • Core MLͷ֓ཁ • Vision Frameworkͷ֓ཁ • Ͱ͖Δ͜ͱ /

   ࣄྫ঺հ 5

6. Core ML 6

7. ML (Machine Learning) 7

8. 8

9. 9

10. 10

11. Core ML • ֶशࡁͷModelΛར༻ͯ͠ͷਪ࿦ʹಛԽ • Core ML model format (**.mlmodel)

    • Xcode͕Swi6ͷΠϯλʔϑΣΠεΛࣗಈੜ੒ • αϯϓϧϞσϧ΋Apple͕ެ։ • Accerelate / Metal্ʹࡌ͓ͬͯΓϋΠύϑΥʔϚϯε • coremltools 11

12. ɹ let animalModel = AnimalModel() if let prediction = try?

    animalModel.prediction(animalImage: image) { return prediction.animalType } 12

13. ɹ let animalModel = AnimalModel() if let prediction = try?

    animalModel.prediction(animalImage: image) { return prediction.animalType } 13

14. ɹ let animalModel = AnimalModel() if let prediction = try?

    animalModel.prediction(animalImage: image) { return prediction.animalType } 14

15. ɹ let animalModel = AnimalModel() if let prediction = try?

    animalModel.prediction(animalImage: image) { return prediction.animalType } 15

16. coremltools • "iOS্Ͱѻ͏ͨΊͷModelΛͲ͏༻ҙ͢Δ͔" ΁ͷղܾࡦ • ओཁͳػցֶशπʔϧͷֶशࡁModelΛCore ML༻ͷModel΁ม ׵ • Keras,

    Caffe, scikit-learn, etc. 16

17. 17

18. 6/28ʹKeras 2.0΋αϙʔτʢൃද࣌͸1.2.2ͷΈͩͬͨʣ h"ps:/ /forums.developer.apple.com/thread/81196 18

19. Vision Framework 19

20. Vision Framework • Core ML্ʹࡌͬͨը૾ೝࣝɾ෺ମݕग़ͳͲͷը૾ղੳ༻ͷϑϨʔϜϫʔΫ • Detec,on • Face, Face

    landmarks, Rectangle, Barcode, Text, Horizon • طଘͷ΋ͷ΋ਫ਼౓޲্ʢDeep Learningͷ׆༻ʣ • Tracking • Image Registra,on • Core MLͱͷ૊Έ߹Θͤ 20

21. 21

22. Tracking • ը૾ʢಈըʣதͷ෺ମͷ௥੻ • إͷTracking͸CIDetectorͰ΋Մೳͩͬͨ • ೚ҙͷର৅ʹରͯ͠ͷTracking͕Մೳʹ • VisionͰͷݕग़݁Ռ •

    ೚ҙͷྖҬࢦఆ 22

23. 23

24. Demo (Rectangle Detec,on + Tracking) h"ps:/ /github.com/shingt/VisionTrackerSample 24

25. զʑ͸CV/MLͷΤΩεύʔτͰ͋Δඞཁ͸ͳ͍ ʢͱɺApple͸ηογϣϯதʹݴ͍ͬͯΔʣ 25

26. Կ͕Ͱ͖Δͷ͔ʁ ʢΞϓϦέʔγϣϯΤϯδχΞͱͯ͠ͷࢹ఺͔Βʣ 26

27. 27

28. 28

29. ࣄྫ঺հ 29

30. ෺ମݕग़ 30

31. 31

32. YOLO • YOLO (You only look once) • ߴ଎ͳ͜ͱ͕ಛ௃ͷ෺ମݕग़༻ͷ χϡʔϥϧωοτϫʔΫ

    • h1ps:/ /www.youtube.com/watch? v=VOC3huqHrss • ͜Ε͸ҰൠతͳYOLO 32

33. • iOSࣄྫ • YOLO: Core ML versus MPSNNGraph • Core

    MLΛ༻͍ͯiOS্ͰYOLOΛಈ࡞ • Tiny YOLOʢެ։͞Ε͍ͯΔModelʣΛར༻ 33

34. 34

35. ը૾ੜ੒ 35

36. Goodfellow, Ian J.; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley,

    David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua. GeneraIve Adversarial Networks. arXiv:1406.2661, 2014. 36

37. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representa>on learning

    with deep convolu>onal genera>ve adversarial networks. arXiv preprint arXiv:1511.06434, 2015. 37

38. • GAN (Genera+ve Adversarial Nets) • ֶशσʔλͱࣅͨσʔλΛੜ੒͢ΔϞσϧͷҰछ • iOSࣄྫ •

    Crea+ve AI on the iPhone: Genera+ve Adversarial Networks (GAN) with Apple's CoreML Tools • MNISTΛσʔληοτͱͯ͠ɺCore MLΛ༻͍ͯiOS্Ͱ਺ࣈ ʢʹࣅͨʣը૾Λੜ੒ 38

39. 39

40. Summary • Core ML / Vision Framework • iOS্Ͱͷը૾ղੳٕज़ͷར༻ϋʔυϧ͕௿Լ •

    ͱ͸͍͑஌ࣝ͸͋Δఔ౓ඞཁʢͱײͨ͡ʣ • Ͱ͖Δ͜ͱ • ը૾ೝࣝ / τϥοΩϯά / ෺ମݕग़ / ը૾ੜ੒ / etc. • Follow @mhollemans 40

41. ࢀߟηογϣϯ • Introducing Core ML • Core ML in depth

    • Vision Framework: Building on Core ML 41

42. ࢀߟࢿྉ • iOS 11: Machine Learning for everyone • Google’s

    MobileNets on the iPhone • YOLO: Core ML versus MPSNNGraph • CreaAve AI on the iPhone: GeneraAve Adversarial Networks (GAN) with Apple's CoreML Tools - Zedge • Why Core ML will not work for your app (most likely) • θϩ͔Β࡞ΔDeep Learning 42

43. Thanks! 43

44. ʢิ଍ʣͰ͖ͳ͍͜ͱ / ੍໿ͳͲ • ֶश͸ෆՄ • αϙʔτ͍ͯ͠ΔػցֶशϑϨʔϜϫʔΫʹ͍ͭͯɺಛఆͷόʔδϣϯʹറΒΕΔʢগͳ͘ͱ ΋ݱঢ়͸ʣ • Kerasͷ2.0αϙʔτೖͬͨ͠ɺࠓޙ޿͍͛ͯ͘ͷ͔΋

    • ModelͷαΠζ͕େ͖͗͢Δ໰୊ • RegressionͱClassifica5onͷΈʢ☓ ΫϥελϦϯάɺϥϯΩϯάֶशɺetc.ʣ • ϥϯλΠϜͰϢʔβͷೖྗɾߦಈΛModelʹ൓өͤ͞Δ͜ͱ͸Ͱ͖ͳ͍ • etc. 44