Core ML First Impressions

CA.swift #3 WWDC17報告会

Transcript

1. Core ML First Impressions CA.swift #3 WWDC17 ใࠂձ 2017/6/19 ߦ໦ ઍय़
2. ࣗݾ঺հ • ߦ໦ ઍय़ʢͳΊ͖ ͪ͸Δʣ • AmebaΞϓϦ iOSΤϯδχΞ • 2015೥3݄த్ೖࣾ
 macOS޲͚ΞϓϦ΍ϛυϧ΢ΣΞΛ࡞ͬͯ·ͨ͠ • WWDC΁ͷࢀՃ͸2015೥Ҏདྷ̎౓໨ ←!?
3. Core ML Framework
4. WWDC17 Keynote
5. • ΫϥΠΞϯταΠυͰػցֶशͷ݁ՌͷϞσϧΛར༻͢ΔͨΊͷI/F
 ͜Ε·ͰެࣜͷࣸਅΞϓϦͳͲͰར༻͞Ε͍ͯͨػೳ͕3rd Partyʹղ์͞Εͨɻ • ϋʔυ΢ΣΞʹΑΔߴ଎ͳॲཧɻCPU, GPUΛࣗಈతʹબ୒ɻ • ୺຤্Ͱ࣮ߦ͢ΔͨΊҎԼͷϝϦοτ͕͋Δɻ
6. Underlying Function Introducing Core ML
7. Accelerate & BNNS Metal Performance Shaders CPU GPU Core ML Vision Natural Language Processing APP NEW NEW Architecture
8. Accelerate & BNNS Metal Performance Shaders CPU GPU Core ML Vision NLP NSLinguisticTagger APP iOS 4.0+, macOS 10.3+, tv0S 9.0+, watchOS 4.0+ iOS 11.0+, macOS 10.3+, tv0S 11.0+, watchOS 4.0+ iOS 9.0+, macOS 10.13+, tv0S 9.0+ iOS 11.0+, macOS 10.13+, tv0S 11.0+ iOS 5.0+, macOS 10.7+, tv0S 9.0+, watchOS 2.0+ OS version & Hardware limitations Metal supported A7ʢ= iPhone 5sʣand later NEW NEW
9. Platforms State of the Union ։ൃϑϩʔ
10. Core ML in Depth ։ൃϑϩʔ
11. https://developer.apple.com/documentation/ coreml/converting_trained_models_to_core_ml Converting Trained Models to Core ML ΑΓ • CaffeͳͲͰੜ੒ͨ͠Ϟσϧ Λ .mlmodel ϑΝΠϧʹม׵ͯ͠ར༻ • ม׵πʔϧ coremltool
 ʢOpen Sourceʣ • coremltoolΛࢀߟʹม׵πʔϧͷ࡞੒ ΋Մೳ αϙʔτπʔϧ
 & Ϟσϧ
12. API Core ML Framework MLModel MLFeatureProvider MLFeatureProvider INPUT OUTPUT func prediction(from input: MLFeatureProvider) throws -> MLFeatureProvider
13. API Core ML Framework MLFeatureProvider featureNames featureValue(for: String) MLFeatureValue type int64Value doubleValue stringValue dictionaryValue: [AnyHashable: NSNumber] imageBufferValue: CVPixelBuffer? multiArrayValue: MLMultiArray? ಛ௃ྔͷϥούʔ ಛ௃ྔʹΞΫηε͢ΔͨΊͷI/F ਺஋ͷଟ࣍ݩ഑ྻ
14. .mlmodel͔Βࣗಈੜ੒͞ΕΔίʔυ MLModel MLFeatureProvider MLFeatureProvider ModelInput ModelOutput func prediction(input: ModelInput) throws -> ModelOutput Model ※ Model͸࣮ࡍʹ͸Ϟσϧ໊ʹͳΔ
15. Vision Framework
16. Vision Framework • Core ML frameworkͷ্ʹ੒ΓཱͭɺComputer Visionͷ ྖҬʹಛԽͨ͠API • ඇৗʹଟ͘ͷೖྗϑΥʔϚοτΛαϙʔτ • CVPixelBuffer, CGImage, CIImage, NSURL, NSData • CVPixelBufferʹม׵ͯ͠Core MLʹ౉ͯ͘͠ΕΔ • ը૾ͷΦϦΤϯςʔγϣϯ΍ղੳʹదͨ͠αΠζ΁ͷεέʔ Ϧϯά΋αϙʔτ
 iOS 11.0+, macOS 10.13+, tv0S 11.0+
17. Vision Framework • إݕग़ • إύʔπݕग़ • ը૾߹੒ʢύϊϥϚͳͲʣ • ۣܗݕग़ • όʔίʔυݕग़ • ςΩετݕग़ • τϥοΩϯά • ਫฏݕग़ • MLModelΛར༻ͨ͠ղੳ
18. Vision Framework Deep LearningΛར༻͍ͯ͠ΔͷͰ ԣإɾإͷҰ෦͕ӅΕ͍ͯΔɾ๧ࢠɺαϯάϥεண༻࣌ͳͲ΋OK
19. API Vision Framework VNRequest VNRequestHandler VNObservation completionHandler handler.perform([request]) ೖྗը૾ & ϦΫΤετͷॲཧ ϦΫΤετ ݁Ռ VNDetectFaceRectanglesRequest VNDetectFaceLandmarksRequest VNDetectRectanglesRequest VNCoreMLRequest VNDetectBarcodesRequest VNTrackingRequest VNDetectHorizonRequest … VNImageRequestHandler VNSequenceRequestHandler VNFaceObservation VNTextObservation VNHorizonObservation VNRectangleObservation VNBarcodeObservation VNTrackObjectRequest VNClassificationObservation
20. Demo
21. Core ML First Impressions • ϞσϧͷऔΓࠐΈ͸ඇৗʹ؆୯ • ϓϩμΫτʹऔΓࠐΉͷ͸ਫ਼౓΍ݹ͍σόΠεͰͷ
 ଎౓Λݟͳ͕Β • ೖྗͷิॿͳͲʹྑͦ͞͏ • Ϟσϧͷ՝୊Λղܾ͢Δඞཁ͕͋Δ • ༰ྔΛখ͘͞ • ਫ਼౓޲্
22. ࢀߟηογϣϯ • WWDC17 Keynote • Platforms State of the Union • Introducing Core ML • Core ML in Depth • Vision Framework: Building on Core ML • Natural Language Processing and your Apps
23. Thank you. Apple Park June 4th 2017. Still Constructing …