Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Core ML First Impressions

Core ML First Impressions

CA.swift #3 WWDC17報告会

4ef794bdea63fc6153845bb26bac38a2?s=128

Chiharu Nameki

June 19, 2017
Tweet

Transcript

  1. Core ML First Impressions CA.swift #3 WWDC17 ใࠂձ 2017/6/19 ߦ໦

    ઍय़
  2. ࣗݾ঺հ • ߦ໦ ઍय़ʢͳΊ͖ ͪ͸Δʣ • AmebaΞϓϦ iOSΤϯδχΞ • 2015೥3݄த్ೖࣾ


    macOS޲͚ΞϓϦ΍ϛυϧ΢ΣΞΛ࡞ͬͯ·ͨ͠ • WWDC΁ͷࢀՃ͸2015೥Ҏདྷ̎౓໨ ←!?
  3. Core ML Framework

  4. WWDC17 Keynote

  5. • ΫϥΠΞϯταΠυͰػցֶशͷ݁ՌͷϞσϧΛར༻͢ΔͨΊͷI/F
 ͜Ε·ͰެࣜͷࣸਅΞϓϦͳͲͰར༻͞Ε͍ͯͨػೳ͕3rd Partyʹղ์͞Εͨɻ • ϋʔυ΢ΣΞʹΑΔߴ଎ͳॲཧɻCPU, GPUΛࣗಈతʹબ୒ɻ • ୺຤্Ͱ࣮ߦ͢ΔͨΊҎԼͷϝϦοτ͕͋Δɻ

  6. Underlying Function Introducing Core ML

  7. Accelerate & BNNS Metal Performance Shaders CPU GPU Core ML

    Vision Natural Language Processing APP NEW NEW Architecture
  8. Accelerate & BNNS Metal Performance Shaders CPU GPU Core ML

    Vision NLP NSLinguisticTagger APP iOS 4.0+, macOS 10.3+, tv0S 9.0+, watchOS 4.0+ iOS 11.0+, macOS 10.3+, tv0S 11.0+, watchOS 4.0+ iOS 9.0+, macOS 10.13+, tv0S 9.0+ iOS 11.0+, macOS 10.13+, tv0S 11.0+ iOS 5.0+, macOS 10.7+, tv0S 9.0+, watchOS 2.0+ OS version & Hardware limitations Metal supported A7ʢ= iPhone 5sʣand later NEW NEW
  9. Platforms State of the Union ։ൃϑϩʔ

  10. Core ML in Depth ։ൃϑϩʔ

  11. https://developer.apple.com/documentation/ coreml/converting_trained_models_to_core_ml Converting Trained Models to Core ML ΑΓ •

    CaffeͳͲͰੜ੒ͨ͠Ϟσϧ Λ .mlmodel ϑΝΠϧʹม׵ͯ͠ར༻ • ม׵πʔϧ coremltool
 ʢOpen Sourceʣ • coremltoolΛࢀߟʹม׵πʔϧͷ࡞੒ ΋Մೳ αϙʔτπʔϧ
 & Ϟσϧ
  12. API Core ML Framework MLModel MLFeatureProvider MLFeatureProvider INPUT OUTPUT func

    prediction(from input: MLFeatureProvider) throws -> MLFeatureProvider
  13. API Core ML Framework MLFeatureProvider featureNames featureValue(for: String) MLFeatureValue type

    int64Value doubleValue stringValue dictionaryValue: [AnyHashable: NSNumber] imageBufferValue: CVPixelBuffer? multiArrayValue: MLMultiArray? ಛ௃ྔͷϥούʔ ಛ௃ྔʹΞΫηε͢ΔͨΊͷI/F ਺஋ͷଟ࣍ݩ഑ྻ
  14. .mlmodel͔Βࣗಈੜ੒͞ΕΔίʔυ MLModel MLFeatureProvider MLFeatureProvider ModelInput ModelOutput func prediction(input: ModelInput) throws

    -> ModelOutput Model ※ Model͸࣮ࡍʹ͸Ϟσϧ໊ʹͳΔ
  15. Vision Framework

  16. Vision Framework • Core ML frameworkͷ্ʹ੒ΓཱͭɺComputer Visionͷ ྖҬʹಛԽͨ͠API • ඇৗʹଟ͘ͷೖྗϑΥʔϚοτΛαϙʔτ

    • CVPixelBuffer, CGImage, CIImage, NSURL, NSData • CVPixelBufferʹม׵ͯ͠Core MLʹ౉ͯ͘͠ΕΔ • ը૾ͷΦϦΤϯςʔγϣϯ΍ղੳʹదͨ͠αΠζ΁ͷεέʔ Ϧϯά΋αϙʔτ
 iOS 11.0+, macOS 10.13+, tv0S 11.0+
  17. Vision Framework • إݕग़ • إύʔπݕग़ • ը૾߹੒ʢύϊϥϚͳͲʣ • ۣܗݕग़

    • όʔίʔυݕग़ • ςΩετݕग़ • τϥοΩϯά • ਫฏݕग़ • MLModelΛར༻ͨ͠ղੳ
  18. Vision Framework Deep LearningΛར༻͍ͯ͠ΔͷͰ ԣإɾإͷҰ෦͕ӅΕ͍ͯΔɾ๧ࢠɺαϯάϥεண༻࣌ͳͲ΋OK

  19. API Vision Framework VNRequest VNRequestHandler VNObservation completionHandler handler.perform([request]) ೖྗը૾ &

    ϦΫΤετͷॲཧ ϦΫΤετ ݁Ռ VNDetectFaceRectanglesRequest VNDetectFaceLandmarksRequest VNDetectRectanglesRequest VNCoreMLRequest VNDetectBarcodesRequest VNTrackingRequest VNDetectHorizonRequest … VNImageRequestHandler VNSequenceRequestHandler VNFaceObservation VNTextObservation VNHorizonObservation VNRectangleObservation VNBarcodeObservation VNTrackObjectRequest VNClassificationObservation
  20. Demo

  21. Core ML First Impressions • ϞσϧͷऔΓࠐΈ͸ඇৗʹ؆୯ • ϓϩμΫτʹऔΓࠐΉͷ͸ਫ਼౓΍ݹ͍σόΠεͰͷ
 ଎౓Λݟͳ͕Β •

    ೖྗͷิॿͳͲʹྑͦ͞͏ • Ϟσϧͷ՝୊Λղܾ͢Δඞཁ͕͋Δ • ༰ྔΛখ͘͞ • ਫ਼౓޲্
  22. ࢀߟηογϣϯ • WWDC17 Keynote • Platforms State of the Union

    • Introducing Core ML • Core ML in Depth • Vision Framework: Building on Core ML • Natural Language Processing and your Apps
  23. Thank you. Apple Park June 4th 2017. Still Constructing …