Share
Embed
Embracing Core ML
Exploring the possibilities of using machine learning on Apple platforms.
Vadym Markov
December 05, 2017
Transcript
36 slides
-
Core ML is a brand new machine learning framework, announced
during WWDC17, that comes along with iOS 11.0+, macOS 10.13+, tvOS 11.0+, watchOS 4.0+ -
Builds on top of Accelerate, Basic Neural Network Subroutines and
Metal Performance shaders Supports Vision framework for image analysis Supports Foundation for Natural Language Processing -
Core ML is optimised for on-device performance It minimises memory
footprint and power consumption Your app remains functional and responsive even offline Running on the device ensures the privacy of user data -
–Tom Mitchell, 1998 “Well posed Learning Problem: A computer program
is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.” -
–Arthur Samuel, 1959 “Machine Learning is a field of study
that gives computers the ability to learn without being explicitly programmed.” -
It’s not that easy to find a good use case
for ML But it’s relatively easy to build awesome, but impractical demos -
Object Classification - Go to https:/ /developer.apple.com/machine- learning/ - Choose
a pre-trained model (e.g. SqueezeNet) - Add .mlmodel to your Xcode project - Write some code -
func classify(image: CIImage) throws { // Create a Vision request
let handler = VNImageRequestHandler(ciImage: image) let request = try makeRequest() // Run the Core ML classifier DispatchQueue.global(qos: .userInteractive).async { do { try handler.perform([request]) } catch { print(error) } } } -
private func makeRequest() throws -> VNCoreMLRequest { // Load the
ML model through its generated class let model = try VNCoreMLModel(for: SqueezeNet().model) // Create a Vision request with completion handler return VNCoreMLRequest(model: model) { [weak self] request, error in guard let strongSelf = self else { return } // Print result print(strongSelf.makeResult(from: request)) } } -
private func makeResult(from request: VNRequest) -> String { // Cast
`results` to `VNClassificationObservation` guard let results = request.results as? [VNClassificationObservation] else { return " ¯\\_(ツ)_/¯ " } // Return the top two classifications return Array(results.prefix(2)) // With a confidence score more than 70% .filter({ $0.confidence > 0.7 }) // Make an array of strings with object’s label and confidence in percents .map({ "\($0.identifier): \(Int($0.confidence * 100))%" }) // Join the array to a single, newline-separated string .joined(separator: "\n") } } -
Vision Face tracking Face, text, rectangle, barcode detection ⚽ Object
tracking Image registration -
Gender Classification -Go to https:/ /developer.apple.com/machine-learning/ - -Find/create third-party ML
model -Convert your model to Core ML model using coremltools -
Converters - Caffe - Keras (1.2.2, 2.0.4+) with Tensorflow (1.0.x,
1.1.x) - libSVM - Scikit-learn (0.15+) - Xgboost (0.6+) - Torch7 -
import coremltools model = coremltools.converters.sklearn.convert(pipeline) model.author = 'ML Guru' model.license
= 'MIT' model.short_description = 'Gender Classification' model.input_description['input'] = 'First name features' model.output_description['classLabel'] = 'Gender (F|M)' model.output_description['classProbability'] = 'Probabilities' coreml_model.save('NamesDT.mlmodel') -
func predictGender(from name: String) throws { let model = NamesDT()
let features = makeFeatures(from: name) let output = try model.prediction(input: features) let prob = output.classProbability[output.classLabel] print("\(output.classLabel): \(probability)") } -
func makeFeatures(from name: String) -> [String: Double] { // ...
let name = name.lowercased() var features = [String: Double]() features["first-letter=\(name.prefix(1))"] = 1.0 features["first2-letters=\(name.prefix(2))"] = 1.0 features["first3-letters=\(name.prefix(3))"] = 1.0 features["last-letter=\(name.suffix(1))"] = 1.0 features["last2-letters=\(name.suffix(2))"] = 1.0 features["last3-letters\(name.suffix(3))"] = 1.0 return features } -
NLP - Language identification - Tokenisation - Lemmatisation - Named
entity recognition -
Can you train model on data using Core ML? A
Can you change model based on user’s input? -
Core ML is a framework that allows to integrate trained
machine learning models into your app -
Limitations - Core ML supports only two types of ML:
regression and classification - coremltools converters are limited - Models are not encrypted - Core ML doesn’t compress models -
Is Core ML cool? Core ML is another step toward
making apps more intelligent But Core ML is not the only way to bring ML into your apps -
–the Doctor Who “When faced with the inevitable, don't waste
precious time resisting it.” -
Take a chill pill Embrace the AI “Revolution” Get started
with CoreML Take small steps to bring ML to your apps -
Core ML Resources - https:/ /developer.apple.com/machine-learning/ - https:/ /github.com/cocoa-ai/ -
https:/ /github.com/onmyway133/fantastic-machine- learning - https:/ /github.com/likedan/Awesome-CoreML-Models