Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Pragmatic Machine Learning for mobile apps

Khoa Pham
November 06, 2019

Pragmatic Machine Learning for mobile apps

Khoa Pham

November 06, 2019
Tweet

More Decks by Khoa Pham

Other Decks in Technology

Transcript

  1. Pragmatic Machine
    Learning for iOS apps

    View full-size slide

  2. About
    • Khoa
    • iOS developer, at Shortcut
    • https:/
    /onmyway133.github.io

    View full-size slide

  3. Agenda
    • Deep Learning
    • Layer in Neural Network
    • Activation Functions
    • Training in Neural Network
    • Loss, Learning Rate
    • Train, Test & Validation Sets
    • Predicting
    • Overfitting

    View full-size slide

  4. Agenda (cont)
    • Convolutional Neural networks & visualizing
    • Zero padding, max pooling
    • Explain back propagation
    • Vanishing & Exploding Gradient
    • Weight Initialization & Bias
    • Learnable parameters
    • Regularizationm, batch size, fine tuning and batch normalization

    View full-size slide

  5. Pragmatic
    How I replicated an $86 million project in 57 lines of code

    View full-size slide

  6. CoreML
    • Computer Vision: image classification
    • Natural Language: language idenification, tokenization
    • Speech: speech recognition

    View full-size slide

  7. Image classification
    Preprocess photos using the Vision framework and classify them with
    a Core ML model.
    iOS app Avengers https:/
    /github.com/onmyway133/avengers

    View full-size slide

  8. Avengers
    • SwiftUI
    • CoreML
    • Vision

    View full-size slide

  9. Tools
    • IBM Watson
    • Azure Custom Vision
    • Google AutoML
    • Firebase Vision Edge
    • CreateML
    • Turi Create

    View full-size slide

  10. Data set
    • Google images download https:/
    /github.com/hardikvasa/google-
    images-download
    • Data augmentation

    View full-size slide

  11. IBM Watson Visual Recognition
    • https:/
    /www.ibm.com/watson/services/visual-recognition/
    • https:/
    /cloud.ibm.com/developer/watson/services
    • https:/
    /www.ibm.com/cloud/watson-studio
    • https:/
    /dataplatform.cloud.ibm.com/home

    View full-size slide

  12. IBM Watson Visual Recognition -
    Service

    View full-size slide

  13. IBM Watson Visual Recognition -
    Assets

    View full-size slide

  14. IBM Watson Visual Recognition -
    Train

    View full-size slide

  15. IBM Watson Visual Recognition -
    Watson SDK
    let classifierID = "your-classifier-id"
    let failure = { (error: Error) in print(error) }
    visualRecognition.updateLocalModel(classifierID: classifierID, failure: failure) {
    print("model updated")
    }

    View full-size slide

  16. CoreML + Vision
    let model = try VNCoreMLModel(for: IBMWatson().model)
    let request = VNCoreMLRequest(model: model, completionHandler: { request, error in
    let results = request.results as? [VNClassificationObservation],
    let handler = VNImageRequestHandler(cgImage: image.cgImage!, options: [:])
    try handler.perform([request])

    View full-size slide

  17. Azure Custom Vision
    • https:/
    /www.customvision.ai/

    View full-size slide

  18. Azure Custom Vision - Project

    View full-size slide

  19. Azure Custom Vision - Assets

    View full-size slide

  20. Azure Custom Vision - Train

    View full-size slide

  21. Google Cloud AutoML Vision
    • https:/
    /cloud.google.com/automl/
    • https:/
    /console.cloud.google.com/vision

    View full-size slide

  22. Google Cloud AutoML Vision

    View full-size slide

  23. Google Cloud AutoML Vision - Dataset

    View full-size slide

  24. Google Cloud AutoML Vision - Google
    Cloud Storage
    https:/
    /console.cloud.google.com/storage

    View full-size slide

  25. Firebase AutoML Vision Edge

    View full-size slide

  26. Firebase AutoML Vision Edge -
    Dataset

    View full-size slide

  27. Firebase AutoML Vision Edge -
    Dataset

    View full-size slide

  28. Firebase AutoML Vision Edge - Train
    pod 'Firebase/MLModelInterpreter'

    View full-size slide

  29. CreateML
    • Activity, Sound, Image, Text, Tabular Classification
    • Word tagger
    • Recommendor
    • Object detection

    View full-size slide

  30. CreateML - Create Document

    View full-size slide

  31. CreateML - Data

    View full-size slide

  32. CreateML - Train

    View full-size slide

  33. CreateMLUI Playground
    • macOS Playground

    View full-size slide

  34. Turi Create
    • https:/
    /github.com/apple/turicreate
    • Open source Python framework
    • Latest version 5.8

    View full-size slide

  35. Turi Create - SFrame
    import turicreate as tc
    import os
    # 1. Load images
    data = tc.image_analysis.load_images('dataset', with_path=True)
    # 2. Create label column based on folder name
    data['hero_name'] = data['path'].apply(lambda path: os.path.basename(os.path.dirname(path)))
    # 3. Save as .sframe
    data.save('turi.sframe')
    # 4. Explore
    data.explore()

    View full-size slide

  36. Turi Create - Visualization

    View full-size slide

  37. Turi Create - Training
    import turicreate as tc
    # 1. Load the data
    data = tc.SFrame('turi.sframe')
    # 2. Split to train and test data
    train_data, test_data = data.random_split(0.8)
    # 3. Create model
    model = tc.image_classifier.create(train_data, target='hero_name')
    # 4. Predictions
    predictions = model.predict(test_data)
    # 5. Evaluate the model and show metrics
    metrics = model.evaluate(test_data)
    print(metrics['accuracy'])
    # 6. Save the model
    model.save('turi.model')
    # 7. Export to CoreML format
    model.export_coreml('model/TuriCreate.mlmodel')

    View full-size slide

  38. Turi Create - Transfer Learning
    resnet-50
    model = tc.image_classifier.create(train_data, target='hero_name', model='squeezenet_v1.1')

    View full-size slide

  39. Writing
    • Machine Learning in iOS: IBM Watson and CoreML
    • Machine Learning in iOS: Azure Custom Vision and CoreML
    • Machine Learning in iOS: Turi Create and CoreML
    • Vision in iOS: Text detection and Tesseract recognition

    View full-size slide

  40. Thank you
    May your code continue to compile

    View full-size slide