Embracing Core ML

Exploring the possibilities of using machine learning on Apple platforms.

Transcript

1. VADYM MARKOV iOS Developer HYPER
2. Embracing Core ML
3. Core ML is a brand new machine learning framework, announced during WWDC17, that comes along with iOS 11.0+, macOS 10.13+, tvOS 11.0+, watchOS 4.0+
4. Builds on top of Accelerate, Basic Neural Network Subroutines and Metal Performance shaders Supports Vision framework for image analysis Supports Foundation for Natural Language Processing
5. Core ML is optimised for on-device performance It minimises memory footprint and power consumption Your app remains functional and responsive even offline Running on the device ensures the privacy of user data
6. Machine learning, huh?
7. –Tom Mitchell, 1998 “Well posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.”
8. –Arthur Samuel, 1959 “Machine Learning is a field of study that gives computers the ability to learn without being explicitly programmed.”
9. Why would I bring ML to my app?
10. Intelligent features Personalisation Identification Meaning extraction Efficient searching Visual and audio recognition
11. It’s not that easy to find a good use case for ML But it’s relatively easy to build awesome, but impractical demos
12. Object Classification (input -> prediction)
13. Object Classification - Go to https:/ /developer.apple.com/machine- learning/ - Choose a pre-trained model (e.g. SqueezeNet) - Add .mlmodel to your Xcode project - Write some code
14. func classify(image: CIImage) throws { // Create a Vision request let handler = VNImageRequestHandler(ciImage: image) let request = try makeRequest() // Run the Core ML classifier DispatchQueue.global(qos: .userInteractive).async { do { try handler.perform([request]) } catch { print(error) } } }
15. private func makeRequest() throws -> VNCoreMLRequest { // Load the ML model through its generated class let model = try VNCoreMLModel(for: SqueezeNet().model) // Create a Vision request with completion handler return VNCoreMLRequest(model: model) { [weak self] request, error in guard let strongSelf = self else { return } // Print result print(strongSelf.makeResult(from: request)) } }
16. private func makeResult(from request: VNRequest) -> String { // Cast `results` to `VNClassificationObservation` guard let results = request.results as? [VNClassificationObservation] else { return " ¯\\_(ツ)_/¯ " } // Return the top two classifications return Array(results.prefix(2)) // With a confidence score more than 70% .filter({ $0.confidence > 0.7 }) // Make an array of strings with object’s label and confidence in percents .map({ "\($0.identifier): \(Int($0.confidence * 100))%" }) // Join the array to a single, newline-separated string .joined(separator: "\n") } }
17. Vision Face tracking Face, text, rectangle, barcode detection ⚽ Object tracking Image registration
18. Gender Classification
19. Gender Classification -Go to https:/ /developer.apple.com/machine-learning/ - -Find/create third-party ML model -Convert your model to Core ML model using coremltools
20. Converters - Caffe - Keras (1.2.2, 2.0.4+) with Tensorflow (1.0.x, 1.1.x) - libSVM - Scikit-learn (0.15+) - Xgboost (0.6+) - Torch7
21. import coremltools model = coremltools.converters.sklearn.convert(pipeline) model.author = 'ML Guru' model.license = 'MIT' model.short_description = 'Gender Classification' model.input_description['input'] = 'First name features' model.output_description['classLabel'] = 'Gender (F|M)' model.output_description['classProbability'] = 'Probabilities' coreml_model.save('NamesDT.mlmodel')
22. func predictGender(from name: String) throws { let model = NamesDT() let features = makeFeatures(from: name) let output = try model.prediction(input: features) let prob = output.classProbability[output.classLabel] print("\(output.classLabel): \(probability)") }
23. func makeFeatures(from name: String) -> [String: Double] { // ... let name = name.lowercased() var features = [String: Double]() features["first-letter=\(name.prefix(1))"] = 1.0 features["first2-letters=\(name.prefix(2))"] = 1.0 features["first3-letters=\(name.prefix(3))"] = 1.0 features["last-letter=\(name.suffix(1))"] = 1.0 features["last2-letters=\(name.suffix(2))"] = 1.0 features["last3-letters\(name.suffix(3))"] = 1.0 return features }
24. NLP - Language identification - Tokenisation - Lemmatisation - Named entity recognition
25. Is Core ML actually a machine learning framework?
26. Can you train model on data using Core ML? A Can you change model based on user’s input?
27. Core ML is a framework that allows to integrate trained machine learning models into your app
28. Limitations - Core ML supports only two types of ML: regression and classification - coremltools converters are limited - Models are not encrypted - Core ML doesn’t compress models
29. Is Core ML cool? Core ML is another step toward making apps more intelligent But Core ML is not the only way to bring ML into your apps
30. Will robots take over the world and destroy all our jobs?
31. –the Doctor Who “When faced with the inevitable, don't waste precious time resisting it.”
32. Take a chill pill Embrace the AI “Revolution” Get started with CoreML Take small steps to bring ML to your apps
33. Core ML Resources - https:/ /developer.apple.com/machine-learning/ - https:/ /github.com/cocoa-ai/ - https:/ /github.com/onmyway133/fantastic-machine- learning - https:/ /github.com/likedan/Awesome-CoreML-Models
34. Core ML is not enough? Learn Machine Learning
35. While you’re reading this, Google’s ML software learns to write ML software L
36. https:/ /github.com/vadymmarkov @vadymmarkov https:/ /github.com/hyperoslo @hyperoslo