Slide 1

Slide 1 text

VADYM MARKOV iOS Developer HYPER

Slide 2

Slide 2 text

Embracing Core ML

Slide 3

Slide 3 text

Core ML is a brand new machine learning framework, announced during WWDC17, that comes along with iOS 11.0+, macOS 10.13+, tvOS 11.0+, watchOS 4.0+

Slide 4

Slide 4 text

Builds on top of Accelerate, Basic Neural Network Subroutines and Metal Performance shaders Supports Vision framework for image analysis Supports Foundation for Natural Language Processing

Slide 5

Slide 5 text

Core ML is optimised for on-device performance It minimises memory footprint and power consumption Your app remains functional and responsive even offline Running on the device ensures the privacy of user data

Slide 6

Slide 6 text

Machine learning, huh?

Slide 7

Slide 7 text

–Tom Mitchell, 1998 “Well posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.”

Slide 8

Slide 8 text

–Arthur Samuel, 1959 “Machine Learning is a field of study that gives computers the ability to learn without being explicitly programmed.”

Slide 9

Slide 9 text

Why would I bring ML to my app?

Slide 10

Slide 10 text

Intelligent features Personalisation Identification Meaning extraction Efficient searching Visual and audio recognition

Slide 11

Slide 11 text

It’s not that easy to find a good use case for ML But it’s relatively easy to build awesome, but impractical demos

Slide 12

Slide 12 text

Object Classification (input -> prediction)

Slide 13

Slide 13 text

Object Classification - Go to https:/ /developer.apple.com/machine- learning/ - Choose a pre-trained model (e.g. SqueezeNet) - Add .mlmodel to your Xcode project - Write some code

Slide 14

Slide 14 text

func classify(image: CIImage) throws { // Create a Vision request let handler = VNImageRequestHandler(ciImage: image) let request = try makeRequest() // Run the Core ML classifier DispatchQueue.global(qos: .userInteractive).async { do { try handler.perform([request]) } catch { print(error) } } }

Slide 15

Slide 15 text

private func makeRequest() throws -> VNCoreMLRequest { // Load the ML model through its generated class let model = try VNCoreMLModel(for: SqueezeNet().model) // Create a Vision request with completion handler return VNCoreMLRequest(model: model) { [weak self] request, error in guard let strongSelf = self else { return } // Print result print(strongSelf.makeResult(from: request)) } }

Slide 16

Slide 16 text

private func makeResult(from request: VNRequest) -> String { // Cast `results` to `VNClassificationObservation` guard let results = request.results as? [VNClassificationObservation] else { return " ¯\\_(ツ)_/¯ " } // Return the top two classifications return Array(results.prefix(2)) // With a confidence score more than 70% .filter({ $0.confidence > 0.7 }) // Make an array of strings with object’s label and confidence in percents .map({ "\($0.identifier): \(Int($0.confidence * 100))%" }) // Join the array to a single, newline-separated string .joined(separator: "\n") } }

Slide 17

Slide 17 text

Vision Face tracking Face, text, rectangle, barcode detection ⚽ Object tracking Image registration

Slide 18

Slide 18 text

Gender Classification

Slide 19

Slide 19 text

Gender Classification -Go to https:/ /developer.apple.com/machine-learning/ - -Find/create third-party ML model -Convert your model to Core ML model using coremltools

Slide 20

Slide 20 text

Converters - Caffe - Keras (1.2.2, 2.0.4+) with Tensorflow (1.0.x, 1.1.x) - libSVM - Scikit-learn (0.15+) - Xgboost (0.6+) - Torch7

Slide 21

Slide 21 text

import coremltools model = coremltools.converters.sklearn.convert(pipeline) model.author = 'ML Guru' model.license = 'MIT' model.short_description = 'Gender Classification' model.input_description['input'] = 'First name features' model.output_description['classLabel'] = 'Gender (F|M)' model.output_description['classProbability'] = 'Probabilities' coreml_model.save('NamesDT.mlmodel')

Slide 22

Slide 22 text

func predictGender(from name: String) throws { let model = NamesDT() let features = makeFeatures(from: name) let output = try model.prediction(input: features) let prob = output.classProbability[output.classLabel] print("\(output.classLabel): \(probability)") }

Slide 23

Slide 23 text

func makeFeatures(from name: String) -> [String: Double] { // ... let name = name.lowercased() var features = [String: Double]() features["first-letter=\(name.prefix(1))"] = 1.0 features["first2-letters=\(name.prefix(2))"] = 1.0 features["first3-letters=\(name.prefix(3))"] = 1.0 features["last-letter=\(name.suffix(1))"] = 1.0 features["last2-letters=\(name.suffix(2))"] = 1.0 features["last3-letters\(name.suffix(3))"] = 1.0 return features }

Slide 24

Slide 24 text

NLP - Language identification - Tokenisation - Lemmatisation - Named entity recognition

Slide 25

Slide 25 text

Is Core ML actually a machine learning framework?

Slide 26

Slide 26 text

Can you train model on data using Core ML? A Can you change model based on user’s input?

Slide 27

Slide 27 text

Core ML is a framework that allows to integrate trained machine learning models into your app

Slide 28

Slide 28 text

Limitations - Core ML supports only two types of ML: regression and classification - coremltools converters are limited - Models are not encrypted - Core ML doesn’t compress models

Slide 29

Slide 29 text

Is Core ML cool? Core ML is another step toward making apps more intelligent But Core ML is not the only way to bring ML into your apps

Slide 30

Slide 30 text

Will robots take over the world and destroy all our jobs?

Slide 31

Slide 31 text

–the Doctor Who “When faced with the inevitable, don't waste precious time resisting it.”

Slide 32

Slide 32 text

Take a chill pill Embrace the AI “Revolution” Get started with CoreML Take small steps to bring ML to your apps

Slide 33

Slide 33 text

Core ML Resources - https:/ /developer.apple.com/machine-learning/ - https:/ /github.com/cocoa-ai/ - https:/ /github.com/onmyway133/fantastic-machine- learning - https:/ /github.com/likedan/Awesome-CoreML-Models

Slide 34

Slide 34 text

Core ML is not enough? Learn Machine Learning

Slide 35

Slide 35 text

While you’re reading this, Google’s ML software learns to write ML software L

Slide 36

Slide 36 text

https:/ /github.com/vadymmarkov @vadymmarkov https:/ /github.com/hyperoslo @hyperoslo