Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Embracing Core ML

Embracing Core ML

Exploring the possibilities of using machine learning on Apple platforms.

Vadym Markov

December 05, 2017
Tweet

More Decks by Vadym Markov

Other Decks in Programming

Transcript

  1. Core ML is a brand new machine learning framework, announced

    during WWDC17, that comes along with iOS 11.0+, macOS 10.13+, tvOS 11.0+, watchOS 4.0+
  2. Builds on top of Accelerate, Basic Neural Network Subroutines and

    Metal Performance shaders Supports Vision framework for image analysis Supports Foundation for Natural Language Processing
  3. Core ML is optimised for on-device performance It minimises memory

    footprint and power consumption Your app remains functional and responsive even offline Running on the device ensures the privacy of user data
  4. –Tom Mitchell, 1998 “Well posed Learning Problem: A computer program

    is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.”
  5. –Arthur Samuel, 1959 “Machine Learning is a field of study

    that gives computers the ability to learn without being explicitly programmed.”
  6. It’s not that easy to find a good use case

    for ML But it’s relatively easy to build awesome, but impractical demos
  7. Object Classification - Go to https:/ /developer.apple.com/machine- learning/ - Choose

    a pre-trained model (e.g. SqueezeNet) - Add .mlmodel to your Xcode project - Write some code
  8. func classify(image: CIImage) throws { // Create a Vision request

    let handler = VNImageRequestHandler(ciImage: image) let request = try makeRequest() // Run the Core ML classifier DispatchQueue.global(qos: .userInteractive).async { do { try handler.perform([request]) } catch { print(error) } } }
  9. private func makeRequest() throws -> VNCoreMLRequest { // Load the

    ML model through its generated class let model = try VNCoreMLModel(for: SqueezeNet().model) // Create a Vision request with completion handler return VNCoreMLRequest(model: model) { [weak self] request, error in guard let strongSelf = self else { return } // Print result print(strongSelf.makeResult(from: request)) } }
  10. private func makeResult(from request: VNRequest) -> String { // Cast

    `results` to `VNClassificationObservation` guard let results = request.results as? [VNClassificationObservation] else { return " ¯\\_(ツ)_/¯ " } // Return the top two classifications return Array(results.prefix(2)) // With a confidence score more than 70% .filter({ $0.confidence > 0.7 }) // Make an array of strings with object’s label and confidence in percents .map({ "\($0.identifier): \(Int($0.confidence * 100))%" }) // Join the array to a single, newline-separated string .joined(separator: "\n") } }
  11. Converters - Caffe - Keras (1.2.2, 2.0.4+) with Tensorflow (1.0.x,

    1.1.x) - libSVM - Scikit-learn (0.15+) - Xgboost (0.6+) - Torch7
  12. import coremltools model = coremltools.converters.sklearn.convert(pipeline) model.author = 'ML Guru' model.license

    = 'MIT' model.short_description = 'Gender Classification' model.input_description['input'] = 'First name features' model.output_description['classLabel'] = 'Gender (F|M)' model.output_description['classProbability'] = 'Probabilities' coreml_model.save('NamesDT.mlmodel')
  13. func predictGender(from name: String) throws { let model = NamesDT()

    let features = makeFeatures(from: name) let output = try model.prediction(input: features) let prob = output.classProbability[output.classLabel] print("\(output.classLabel): \(probability)") }
  14. func makeFeatures(from name: String) -> [String: Double] { // ...

    let name = name.lowercased() var features = [String: Double]() features["first-letter=\(name.prefix(1))"] = 1.0 features["first2-letters=\(name.prefix(2))"] = 1.0 features["first3-letters=\(name.prefix(3))"] = 1.0 features["last-letter=\(name.suffix(1))"] = 1.0 features["last2-letters=\(name.suffix(2))"] = 1.0 features["last3-letters\(name.suffix(3))"] = 1.0 return features }
  15. Can you train model on data using Core ML? A

    Can you change model based on user’s input?
  16. Core ML is a framework that allows to integrate trained

    machine learning models into your app
  17. Limitations - Core ML supports only two types of ML:

    regression and classification - coremltools converters are limited - Models are not encrypted - Core ML doesn’t compress models
  18. Is Core ML cool? Core ML is another step toward

    making apps more intelligent But Core ML is not the only way to bring ML into your apps
  19. Take a chill pill Embrace the AI “Revolution” Get started

    with CoreML Take small steps to bring ML to your apps
  20. Core ML Resources - https:/ /developer.apple.com/machine-learning/ - https:/ /github.com/cocoa-ai/ -

    https:/ /github.com/onmyway133/fantastic-machine- learning - https:/ /github.com/likedan/Awesome-CoreML-Models