Core ML / Vision Frameworkを使ってできること / What can we achieve using Core ML and Vision framework

2017/06/30 WWDC - Developer's Living #lifull_wwdc

Transcript

1. Core ML / Vision Framework
   Λ࢖ͬͯͰ͖Δ͜ͱ
   ɹ
   2017/06/30 WWDC - Developer's Living @ LIFULL
   shingt (Shinichi Goto)
   

   View Slide

2. shingt (Shinichi Goto)
   GitHub: @shingt
   Twi5er: @_shingt
   2
   

   View Slide

3. Core ML
   Vision Framework
   3
   

   View Slide

4. View Slide

5. Outline
   • Core MLͷ֓ཁ
   • Vision Frameworkͷ֓ཁ
   • Ͱ͖Δ͜ͱ / ࣄྫ঺հ
   5
   

   View Slide

6. Core ML
   6
   

   View Slide

7. ML (Machine Learning)
   7
   

   View Slide

8. View Slide

9. View Slide

10. View Slide

11. Core ML
    • ֶशࡁͷModelΛར༻ͯ͠ͷਪ࿦ʹಛԽ
    • Core ML model format (**.mlmodel)
    • Xcode͕Swi6ͷΠϯλʔϑΣΠεΛࣗಈੜ੒
    • αϯϓϧϞσϧ΋Apple͕ެ։
    • Accerelate / Metal্ʹࡌ͓ͬͯΓϋΠύϑΥʔϚϯε
    • coremltools
    11
    

    View Slide

12. ɹ
    let animalModel = AnimalModel()
    if let prediction = try? animalModel.prediction(animalImage: image) {
    return prediction.animalType
    }
    12
    

    View Slide

13. ɹ
    let animalModel = AnimalModel()
    if let prediction = try? animalModel.prediction(animalImage: image) {
    return prediction.animalType
    }
    13
    

    View Slide

14. ɹ
    let animalModel = AnimalModel()
    if let prediction = try? animalModel.prediction(animalImage: image) {
    return prediction.animalType
    }
    14
    

    View Slide

15. ɹ
    let animalModel = AnimalModel()
    if let prediction = try? animalModel.prediction(animalImage: image) {
    return prediction.animalType
    }
    15
    

    View Slide

16. coremltools
    • "iOS্Ͱѻ͏ͨΊͷModelΛͲ͏༻ҙ͢Δ͔" ΁ͷղܾࡦ
    • ओཁͳػցֶशπʔϧͷֶशࡁModelΛCore ML༻ͷModel΁ม
    ׵
    • Keras, Caffe, scikit-learn, etc.
    16
    

    View Slide

17. View Slide

18. 6/28ʹKeras 2.0΋αϙʔτʢൃද࣌͸1.2.2ͷΈͩͬͨʣ
    h"ps:/
    /forums.developer.apple.com/thread/81196
    18
    

    View Slide

19. Vision Framework
    19
    

    View Slide

20. Vision Framework
    • Core ML্ʹࡌͬͨը૾ೝࣝɾ෺ମݕग़ͳͲͷը૾ղੳ༻ͷϑϨʔϜϫʔΫ
    • Detec,on
    • Face, Face landmarks, Rectangle, Barcode, Text, Horizon
    • طଘͷ΋ͷ΋ਫ਼౓޲্ʢDeep Learningͷ׆༻ʣ
    • Tracking
    • Image Registra,on
    • Core MLͱͷ૊Έ߹Θͤ
    20
    

    View Slide

21. View Slide

22. Tracking
    • ը૾ʢಈըʣதͷ෺ମͷ௥੻
    • إͷTracking͸CIDetectorͰ΋Մೳͩͬͨ
    • ೚ҙͷର৅ʹରͯ͠ͷTracking͕Մೳʹ
    • VisionͰͷݕग़݁Ռ
    • ೚ҙͷྖҬࢦఆ
    22
    

    View Slide

23. View Slide

24. Demo
    (Rectangle Detec,on + Tracking)
    h"ps:/
    /github.com/shingt/VisionTrackerSample
    24
    

    View Slide

25. զʑ͸CV/MLͷΤΩεύʔτͰ͋Δඞཁ͸ͳ͍
    ʢͱɺApple͸ηογϣϯதʹݴ͍ͬͯΔʣ
    25
    

    View Slide

26. Կ͕Ͱ͖Δͷ͔ʁ
    ʢΞϓϦέʔγϣϯΤϯδχΞͱͯ͠ͷࢹ఺͔Βʣ
    26
    

    View Slide

27. View Slide

28. View Slide

29. ࣄྫ঺հ
    29
    

    View Slide

30. ෺ମݕग़
    30
    

    View Slide

31. View Slide

32. YOLO
    • YOLO (You only look once)
    • ߴ଎ͳ͜ͱ͕ಛ௃ͷ෺ମݕग़༻ͷ
    χϡʔϥϧωοτϫʔΫ
    • h1ps:/
    /www.youtube.com/watch?
    v=VOC3huqHrss
    • ͜Ε͸ҰൠతͳYOLO
    32
    

    View Slide

33. • iOSࣄྫ
    • YOLO: Core ML versus MPSNNGraph
    • Core MLΛ༻͍ͯiOS্ͰYOLOΛಈ࡞
    • Tiny YOLOʢެ։͞Ε͍ͯΔModelʣΛར༻
    33
    

    View Slide

34. View Slide

35. ը૾ੜ੒
    35
    

    View Slide

36. Goodfellow, Ian J.; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley, David; Ozair,
    Sherjil; Courville, Aaron; Bengio, Yoshua. GeneraIve Adversarial Networks. arXiv:1406.2661, 2014.
    36
    

    View Slide

37. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representa>on learning with deep
    convolu>onal genera>ve adversarial networks. arXiv preprint arXiv:1511.06434, 2015.
    37
    

    View Slide

38. • GAN (Genera+ve Adversarial Nets)
    • ֶशσʔλͱࣅͨσʔλΛੜ੒͢ΔϞσϧͷҰछ
    • iOSࣄྫ
    • Crea+ve AI on the iPhone: Genera+ve Adversarial Networks
    (GAN) with Apple's CoreML Tools
    • MNISTΛσʔληοτͱͯ͠ɺCore MLΛ༻͍ͯiOS্Ͱ਺ࣈ
    ʢʹࣅͨʣը૾Λੜ੒
    38
    

    View Slide

39. View Slide

40. Summary
    • Core ML / Vision Framework
    • iOS্Ͱͷը૾ղੳٕज़ͷར༻ϋʔυϧ͕௿Լ
    • ͱ͸͍͑஌ࣝ͸͋Δఔ౓ඞཁʢͱײͨ͡ʣ
    • Ͱ͖Δ͜ͱ
    • ը૾ೝࣝ / τϥοΩϯά / ෺ମݕग़ / ը૾ੜ੒ / etc.
    • Follow @mhollemans
    40
    

    View Slide

41. ࢀߟηογϣϯ
    • Introducing Core ML
    • Core ML in depth
    • Vision Framework: Building on Core ML
    41
    

    View Slide

42. ࢀߟࢿྉ
    • iOS 11: Machine Learning for everyone
    • Google’s MobileNets on the iPhone
    • YOLO: Core ML versus MPSNNGraph
    • CreaAve AI on the iPhone: GeneraAve Adversarial Networks (GAN)
    with Apple's CoreML Tools - Zedge
    • Why Core ML will not work for your app (most likely)
    • θϩ͔Β࡞ΔDeep Learning
    42
    

    View Slide

43. Thanks!
    43
    

    View Slide

44. ʢิ଍ʣͰ͖ͳ͍͜ͱ / ੍໿ͳͲ
    • ֶश͸ෆՄ
    • αϙʔτ͍ͯ͠ΔػցֶशϑϨʔϜϫʔΫʹ͍ͭͯɺಛఆͷόʔδϣϯʹറΒΕΔʢগͳ͘ͱ
    ΋ݱঢ়͸ʣ
    • Kerasͷ2.0αϙʔτೖͬͨ͠ɺࠓޙ޿͍͛ͯ͘ͷ͔΋
    • ModelͷαΠζ͕େ͖͗͢Δ໰୊
    • RegressionͱClassifica5onͷΈʢ☓ ΫϥελϦϯάɺϥϯΩϯάֶशɺetc.ʣ
    • ϥϯλΠϜͰϢʔβͷೖྗɾߦಈΛModelʹ൓өͤ͞Δ͜ͱ͸Ͱ͖ͳ͍
    • etc.
    44
    

    View Slide