Core ML / Vision Frameworkを使ってできること / What can we achieve using Core ML and Vision framework

Core ML / Vision Frameworkを使ってできること / What can we achieve using Core ML and Vision framework

2017/06/30 WWDC - Developer's Living #lifull_wwdc

3652e140db258288623a0adf248baec0?s=128

Shinichi Goto

June 30, 2017
Tweet

Transcript

  1. Core ML / Vision Framework Λ࢖ͬͯͰ͖Δ͜ͱ ɹ 2017/06/30 WWDC -

    Developer's Living @ LIFULL shingt (Shinichi Goto)
  2. shingt (Shinichi Goto) GitHub: @shingt Twi5er: @_shingt 2

  3. Core ML Vision Framework 3

  4. 4

  5. Outline • Core MLͷ֓ཁ • Vision Frameworkͷ֓ཁ • Ͱ͖Δ͜ͱ /

    ࣄྫ঺հ 5
  6. Core ML 6

  7. ML (Machine Learning) 7

  8. 8

  9. 9

  10. 10

  11. Core ML • ֶशࡁͷModelΛར༻ͯ͠ͷਪ࿦ʹಛԽ • Core ML model format (**.mlmodel)

    • Xcode͕Swi6ͷΠϯλʔϑΣΠεΛࣗಈੜ੒ • αϯϓϧϞσϧ΋Apple͕ެ։ • Accerelate / Metal্ʹࡌ͓ͬͯΓϋΠύϑΥʔϚϯε • coremltools 11
  12. ɹ let animalModel = AnimalModel() if let prediction = try?

    animalModel.prediction(animalImage: image) { return prediction.animalType } 12
  13. ɹ let animalModel = AnimalModel() if let prediction = try?

    animalModel.prediction(animalImage: image) { return prediction.animalType } 13
  14. ɹ let animalModel = AnimalModel() if let prediction = try?

    animalModel.prediction(animalImage: image) { return prediction.animalType } 14
  15. ɹ let animalModel = AnimalModel() if let prediction = try?

    animalModel.prediction(animalImage: image) { return prediction.animalType } 15
  16. coremltools • "iOS্Ͱѻ͏ͨΊͷModelΛͲ͏༻ҙ͢Δ͔" ΁ͷղܾࡦ • ओཁͳػցֶशπʔϧͷֶशࡁModelΛCore ML༻ͷModel΁ม ׵ • Keras,

    Caffe, scikit-learn, etc. 16
  17. 17

  18. 6/28ʹKeras 2.0΋αϙʔτʢൃද࣌͸1.2.2ͷΈͩͬͨʣ h"ps:/ /forums.developer.apple.com/thread/81196 18

  19. Vision Framework 19

  20. Vision Framework • Core ML্ʹࡌͬͨը૾ೝࣝɾ෺ମݕग़ͳͲͷը૾ղੳ༻ͷϑϨʔϜϫʔΫ • Detec,on • Face, Face

    landmarks, Rectangle, Barcode, Text, Horizon • طଘͷ΋ͷ΋ਫ਼౓޲্ʢDeep Learningͷ׆༻ʣ • Tracking • Image Registra,on • Core MLͱͷ૊Έ߹Θͤ 20
  21. 21

  22. Tracking • ը૾ʢಈըʣதͷ෺ମͷ௥੻ • إͷTracking͸CIDetectorͰ΋Մೳͩͬͨ • ೚ҙͷର৅ʹରͯ͠ͷTracking͕Մೳʹ • VisionͰͷݕग़݁Ռ •

    ೚ҙͷྖҬࢦఆ 22
  23. 23

  24. Demo (Rectangle Detec,on + Tracking) h"ps:/ /github.com/shingt/VisionTrackerSample 24

  25. զʑ͸CV/MLͷΤΩεύʔτͰ͋Δඞཁ͸ͳ͍ ʢͱɺApple͸ηογϣϯதʹݴ͍ͬͯΔʣ 25

  26. Կ͕Ͱ͖Δͷ͔ʁ ʢΞϓϦέʔγϣϯΤϯδχΞͱͯ͠ͷࢹ఺͔Βʣ 26

  27. 27

  28. 28

  29. ࣄྫ঺հ 29

  30. ෺ମݕग़ 30

  31. 31

  32. YOLO • YOLO (You only look once) • ߴ଎ͳ͜ͱ͕ಛ௃ͷ෺ମݕग़༻ͷ χϡʔϥϧωοτϫʔΫ

    • h1ps:/ /www.youtube.com/watch? v=VOC3huqHrss • ͜Ε͸ҰൠతͳYOLO 32
  33. • iOSࣄྫ • YOLO: Core ML versus MPSNNGraph • Core

    MLΛ༻͍ͯiOS্ͰYOLOΛಈ࡞ • Tiny YOLOʢެ։͞Ε͍ͯΔModelʣΛར༻ 33
  34. 34

  35. ը૾ੜ੒ 35

  36. Goodfellow, Ian J.; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley,

    David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua. GeneraIve Adversarial Networks. arXiv:1406.2661, 2014. 36
  37. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representa>on learning

    with deep convolu>onal genera>ve adversarial networks. arXiv preprint arXiv:1511.06434, 2015. 37
  38. • GAN (Genera+ve Adversarial Nets) • ֶशσʔλͱࣅͨσʔλΛੜ੒͢ΔϞσϧͷҰछ • iOSࣄྫ •

    Crea+ve AI on the iPhone: Genera+ve Adversarial Networks (GAN) with Apple's CoreML Tools • MNISTΛσʔληοτͱͯ͠ɺCore MLΛ༻͍ͯiOS্Ͱ਺ࣈ ʢʹࣅͨʣը૾Λੜ੒ 38
  39. 39

  40. Summary • Core ML / Vision Framework • iOS্Ͱͷը૾ղੳٕज़ͷར༻ϋʔυϧ͕௿Լ •

    ͱ͸͍͑஌ࣝ͸͋Δఔ౓ඞཁʢͱײͨ͡ʣ • Ͱ͖Δ͜ͱ • ը૾ೝࣝ / τϥοΩϯά / ෺ମݕग़ / ը૾ੜ੒ / etc. • Follow @mhollemans 40
  41. ࢀߟηογϣϯ • Introducing Core ML • Core ML in depth

    • Vision Framework: Building on Core ML 41
  42. ࢀߟࢿྉ • iOS 11: Machine Learning for everyone • Google’s

    MobileNets on the iPhone • YOLO: Core ML versus MPSNNGraph • CreaAve AI on the iPhone: GeneraAve Adversarial Networks (GAN) with Apple's CoreML Tools - Zedge • Why Core ML will not work for your app (most likely) • θϩ͔Β࡞ΔDeep Learning 42
  43. Thanks! 43

  44. ʢิ଍ʣͰ͖ͳ͍͜ͱ / ੍໿ͳͲ • ֶश͸ෆՄ • αϙʔτ͍ͯ͠ΔػցֶशϑϨʔϜϫʔΫʹ͍ͭͯɺಛఆͷόʔδϣϯʹറΒΕΔʢগͳ͘ͱ ΋ݱঢ়͸ʣ • Kerasͷ2.0αϙʔτೖͬͨ͠ɺࠓޙ޿͍͛ͯ͘ͷ͔΋

    • ModelͷαΠζ͕େ͖͗͢Δ໰୊ • RegressionͱClassifica5onͷΈʢ☓ ΫϥελϦϯάɺϥϯΩϯάֶशɺetc.ʣ • ϥϯλΠϜͰϢʔβͷೖྗɾߦಈΛModelʹ൓өͤ͞Δ͜ͱ͸Ͱ͖ͳ͍ • etc. 44