Core ML First Impressions

CA.swift #3 WWDC17報告会

Transcript

1. Core ML First Impressions
   CA.swift #3 WWDC17 ใࠂձ
   2017/6/19
   ߦ໦ ઍय़
   

   View Slide

2. ࣗݾ঺հ
   • ߦ໦ ઍय़ʢͳΊ͖ ͪ͸Δʣ
   
   • AmebaΞϓϦ iOSΤϯδχΞ
   
   • 2015೥3݄த్ೖࣾ

   macOS޲͚ΞϓϦ΍ϛυϧ΢ΣΞΛ࡞ͬͯ·ͨ͠
   
   • WWDC΁ͷࢀՃ͸2015೥Ҏདྷ̎౓໨
   ←!?
   

   View Slide

3. Core ML Framework
   

   View Slide

4. WWDC17 Keynote
   

   View Slide

5. • ΫϥΠΞϯταΠυͰػցֶशͷ݁ՌͷϞσϧΛར༻͢ΔͨΊͷI/F

   ͜Ε·ͰެࣜͷࣸਅΞϓϦͳͲͰར༻͞Ε͍ͯͨػೳ͕3rd Partyʹղ์͞Εͨɻ
   
   • ϋʔυ΢ΣΞʹΑΔߴ଎ͳॲཧɻCPU, GPUΛࣗಈతʹબ୒ɻ
   
   • ୺຤্Ͱ࣮ߦ͢ΔͨΊҎԼͷϝϦοτ͕͋Δɻ
   

   View Slide

6. Underlying Function
   Introducing Core ML
   

   View Slide

7. Accelerate & BNNS Metal Performance Shaders
   CPU GPU
   Core ML
   Vision Natural Language Processing
   APP
   NEW
   NEW
   Architecture
   

   View Slide

8. Accelerate & BNNS Metal Performance Shaders
   CPU GPU
   Core ML
   Vision NLP NSLinguisticTagger
   APP
   iOS 4.0+, macOS 10.3+, tv0S 9.0+, watchOS 4.0+
   iOS 11.0+, macOS 10.3+, tv0S 11.0+, watchOS 4.0+
   iOS 9.0+, macOS 10.13+, tv0S 9.0+
   iOS 11.0+, macOS 10.13+, tv0S 11.0+ iOS 5.0+, macOS 10.7+, tv0S 9.0+, watchOS 2.0+
   OS version & Hardware limitations
   Metal supported A7ʢ= iPhone 5sʣand later
   NEW
   NEW
   

   View Slide

9. Platforms State of the Union
   ։ൃϑϩʔ
   

   View Slide

10. Core ML in Depth
    ։ൃϑϩʔ
    

    View Slide

11. https://developer.apple.com/documentation/
    coreml/converting_trained_models_to_core_ml
    Converting Trained Models
    to Core ML ΑΓ
    • CaffeͳͲͰੜ੒ͨ͠Ϟσϧ
    Λ .mlmodel ϑΝΠϧʹม׵ͯ͠ར༻
    
    • ม׵πʔϧ coremltool

    ʢOpen Sourceʣ
    
    • coremltoolΛࢀߟʹม׵πʔϧͷ࡞੒
    ΋Մೳ
    αϙʔτπʔϧ

    & Ϟσϧ
    

    View Slide

12. API
    Core ML Framework
    MLModel
    MLFeatureProvider MLFeatureProvider
    INPUT OUTPUT
    func prediction(from input: MLFeatureProvider) throws -> MLFeatureProvider
    

    View Slide

13. API
    Core ML Framework
    MLFeatureProvider
    featureNames
    featureValue(for: String)
    MLFeatureValue
    type
    
    
    
    int64Value
    
    doubleValue
    
    stringValue
    
    dictionaryValue: [AnyHashable: NSNumber]
    
    imageBufferValue: CVPixelBuffer?
    
    multiArrayValue: MLMultiArray?
    ಛ௃ྔͷϥούʔ
    ಛ௃ྔʹΞΫηε͢ΔͨΊͷI/F
    ਺஋ͷଟ࣍ݩ഑ྻ
    

    View Slide

14. .mlmodel͔Βࣗಈੜ੒͞ΕΔίʔυ
    MLModel
    MLFeatureProvider MLFeatureProvider
    ModelInput ModelOutput
    func prediction(input: ModelInput) throws -> ModelOutput
    Model
    ※ Model͸࣮ࡍʹ͸Ϟσϧ໊ʹͳΔ
    

    View Slide

15. Vision Framework
    

    View Slide

16. Vision Framework
    • Core ML frameworkͷ্ʹ੒ΓཱͭɺComputer Visionͷ
    ྖҬʹಛԽͨ͠API
    
    • ඇৗʹଟ͘ͷೖྗϑΥʔϚοτΛαϙʔτ
    
    • CVPixelBuffer, CGImage, CIImage, NSURL, NSData
    
    • CVPixelBufferʹม׵ͯ͠Core MLʹ౉ͯ͘͠ΕΔ
    
    • ը૾ͷΦϦΤϯςʔγϣϯ΍ղੳʹదͨ͠αΠζ΁ͷεέʔ
    Ϧϯά΋αϙʔτ

    iOS 11.0+, macOS 10.13+, tv0S 11.0+
    

    View Slide

17. Vision Framework
    • إݕग़
    
    • إύʔπݕग़
    
    • ը૾߹੒ʢύϊϥϚͳͲʣ
    
    • ۣܗݕग़
    
    • όʔίʔυݕग़
    
    • ςΩετݕग़
    
    • τϥοΩϯά
    
    • ਫฏݕग़
    
    • MLModelΛར༻ͨ͠ղੳ
    

    View Slide

18. Vision Framework
    Deep LearningΛར༻͍ͯ͠ΔͷͰ
    ԣإɾإͷҰ෦͕ӅΕ͍ͯΔɾ๧ࢠɺαϯάϥεண༻࣌ͳͲ΋OK
    

    View Slide

19. API
    Vision Framework
    VNRequest
    VNRequestHandler VNObservation
    completionHandler
    handler.perform([request])
    ೖྗը૾ & ϦΫΤετͷॲཧ
    ϦΫΤετ ݁Ռ
    VNDetectFaceRectanglesRequest
    VNDetectFaceLandmarksRequest
    VNDetectRectanglesRequest
    VNCoreMLRequest
    VNDetectBarcodesRequest
    VNTrackingRequest
    VNDetectHorizonRequest
    …
    VNImageRequestHandler
    VNSequenceRequestHandler
    VNFaceObservation
    VNTextObservation
    VNHorizonObservation
    VNRectangleObservation
    VNBarcodeObservation
    VNTrackObjectRequest
    VNClassificationObservation
    

    View Slide

20. Demo
    

    View Slide

21. Core ML First Impressions
    • ϞσϧͷऔΓࠐΈ͸ඇৗʹ؆୯
    
    • ϓϩμΫτʹऔΓࠐΉͷ͸ਫ਼౓΍ݹ͍σόΠεͰͷ

    ଎౓Λݟͳ͕Β
    
    • ೖྗͷิॿͳͲʹྑͦ͞͏
    
    • Ϟσϧͷ՝୊Λղܾ͢Δඞཁ͕͋Δ
    
    • ༰ྔΛখ͘͞
    
    • ਫ਼౓޲্
    

    View Slide

22. ࢀߟηογϣϯ
    • WWDC17 Keynote
    
    • Platforms State of the Union
    
    • Introducing Core ML
    
    • Core ML in Depth
    
    • Vision Framework: Building on Core ML
    
    • Natural Language Processing and your Apps
    

    View Slide

23. Thank you.
    Apple Park June 4th 2017. Still Constructing …
    

    View Slide