Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Core ML First Impressions

Core ML First Impressions

CA.swift #3 WWDC17報告会

Chiharu Nameki

June 19, 2017
Tweet

More Decks by Chiharu Nameki

Other Decks in Technology

Transcript

  1. Core ML First Impressions
    CA.swift #3 WWDC17 ใࠂձ
    2017/6/19
    ߦ໦ ઍय़

    View full-size slide

  2. ࣗݾ঺հ
    • ߦ໦ ઍय़ʢͳΊ͖ ͪ͸Δʣ

    • AmebaΞϓϦ iOSΤϯδχΞ

    • 2015೥3݄த్ೖࣾ

    macOS޲͚ΞϓϦ΍ϛυϧ΢ΣΞΛ࡞ͬͯ·ͨ͠

    • WWDC΁ͷࢀՃ͸2015೥Ҏདྷ̎౓໨
    ←!?

    View full-size slide

  3. Core ML Framework

    View full-size slide

  4. WWDC17 Keynote

    View full-size slide

  5. • ΫϥΠΞϯταΠυͰػցֶशͷ݁ՌͷϞσϧΛར༻͢ΔͨΊͷI/F

    ͜Ε·ͰެࣜͷࣸਅΞϓϦͳͲͰར༻͞Ε͍ͯͨػೳ͕3rd Partyʹղ์͞Εͨɻ

    • ϋʔυ΢ΣΞʹΑΔߴ଎ͳॲཧɻCPU, GPUΛࣗಈతʹબ୒ɻ

    • ୺຤্Ͱ࣮ߦ͢ΔͨΊҎԼͷϝϦοτ͕͋Δɻ

    View full-size slide

  6. Underlying Function
    Introducing Core ML

    View full-size slide

  7. Accelerate & BNNS Metal Performance Shaders
    CPU GPU
    Core ML
    Vision Natural Language Processing
    APP
    NEW
    NEW
    Architecture

    View full-size slide

  8. Accelerate & BNNS Metal Performance Shaders
    CPU GPU
    Core ML
    Vision NLP NSLinguisticTagger
    APP
    iOS 4.0+, macOS 10.3+, tv0S 9.0+, watchOS 4.0+
    iOS 11.0+, macOS 10.3+, tv0S 11.0+, watchOS 4.0+
    iOS 9.0+, macOS 10.13+, tv0S 9.0+
    iOS 11.0+, macOS 10.13+, tv0S 11.0+ iOS 5.0+, macOS 10.7+, tv0S 9.0+, watchOS 2.0+
    OS version & Hardware limitations
    Metal supported A7ʢ= iPhone 5sʣand later
    NEW
    NEW

    View full-size slide

  9. Platforms State of the Union
    ։ൃϑϩʔ

    View full-size slide

  10. Core ML in Depth
    ։ൃϑϩʔ

    View full-size slide

  11. https://developer.apple.com/documentation/
    coreml/converting_trained_models_to_core_ml
    Converting Trained Models
    to Core ML ΑΓ
    • CaffeͳͲͰੜ੒ͨ͠Ϟσϧ
    Λ .mlmodel ϑΝΠϧʹม׵ͯ͠ར༻

    • ม׵πʔϧ coremltool

    ʢOpen Sourceʣ

    • coremltoolΛࢀߟʹม׵πʔϧͷ࡞੒
    ΋Մೳ
    αϙʔτπʔϧ

    & Ϟσϧ

    View full-size slide

  12. API
    Core ML Framework
    MLModel
    MLFeatureProvider MLFeatureProvider
    INPUT OUTPUT
    func prediction(from input: MLFeatureProvider) throws -> MLFeatureProvider

    View full-size slide

  13. API
    Core ML Framework
    MLFeatureProvider
    featureNames
    featureValue(for: String)
    MLFeatureValue
    type



    int64Value

    doubleValue

    stringValue

    dictionaryValue: [AnyHashable: NSNumber]

    imageBufferValue: CVPixelBuffer?

    multiArrayValue: MLMultiArray?
    ಛ௃ྔͷϥούʔ
    ಛ௃ྔʹΞΫηε͢ΔͨΊͷI/F
    ਺஋ͷଟ࣍ݩ഑ྻ

    View full-size slide

  14. .mlmodel͔Βࣗಈੜ੒͞ΕΔίʔυ
    MLModel
    MLFeatureProvider MLFeatureProvider
    ModelInput ModelOutput
    func prediction(input: ModelInput) throws -> ModelOutput
    Model
    ※ Model͸࣮ࡍʹ͸Ϟσϧ໊ʹͳΔ

    View full-size slide

  15. Vision Framework

    View full-size slide

  16. Vision Framework
    • Core ML frameworkͷ্ʹ੒ΓཱͭɺComputer Visionͷ
    ྖҬʹಛԽͨ͠API

    • ඇৗʹଟ͘ͷೖྗϑΥʔϚοτΛαϙʔτ

    • CVPixelBuffer, CGImage, CIImage, NSURL, NSData

    • CVPixelBufferʹม׵ͯ͠Core MLʹ౉ͯ͘͠ΕΔ

    • ը૾ͷΦϦΤϯςʔγϣϯ΍ղੳʹదͨ͠αΠζ΁ͷεέʔ
    Ϧϯά΋αϙʔτ

    iOS 11.0+, macOS 10.13+, tv0S 11.0+

    View full-size slide

  17. Vision Framework
    • إݕग़

    • إύʔπݕग़

    • ը૾߹੒ʢύϊϥϚͳͲʣ

    • ۣܗݕग़

    • όʔίʔυݕग़

    • ςΩετݕग़

    • τϥοΩϯά

    • ਫฏݕग़

    • MLModelΛར༻ͨ͠ղੳ

    View full-size slide

  18. Vision Framework
    Deep LearningΛར༻͍ͯ͠ΔͷͰ
    ԣإɾإͷҰ෦͕ӅΕ͍ͯΔɾ๧ࢠɺαϯάϥεண༻࣌ͳͲ΋OK

    View full-size slide

  19. API
    Vision Framework
    VNRequest
    VNRequestHandler VNObservation
    completionHandler
    handler.perform([request])
    ೖྗը૾ & ϦΫΤετͷॲཧ
    ϦΫΤετ ݁Ռ
    VNDetectFaceRectanglesRequest
    VNDetectFaceLandmarksRequest
    VNDetectRectanglesRequest
    VNCoreMLRequest
    VNDetectBarcodesRequest
    VNTrackingRequest
    VNDetectHorizonRequest

    VNImageRequestHandler
    VNSequenceRequestHandler
    VNFaceObservation
    VNTextObservation
    VNHorizonObservation
    VNRectangleObservation
    VNBarcodeObservation
    VNTrackObjectRequest
    VNClassificationObservation

    View full-size slide

  20. Core ML First Impressions
    • ϞσϧͷऔΓࠐΈ͸ඇৗʹ؆୯

    • ϓϩμΫτʹऔΓࠐΉͷ͸ਫ਼౓΍ݹ͍σόΠεͰͷ

    ଎౓Λݟͳ͕Β

    • ೖྗͷิॿͳͲʹྑͦ͞͏

    • Ϟσϧͷ՝୊Λղܾ͢Δඞཁ͕͋Δ

    • ༰ྔΛখ͘͞

    • ਫ਼౓޲্

    View full-size slide

  21. ࢀߟηογϣϯ
    • WWDC17 Keynote

    • Platforms State of the Union

    • Introducing Core ML

    • Core ML in Depth

    • Vision Framework: Building on Core ML

    • Natural Language Processing and your Apps

    View full-size slide

  22. Thank you.
    Apple Park June 4th 2017. Still Constructing …

    View full-size slide