Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Pragmatic Machine Learning for mobile apps

Khoa Pham
November 06, 2019

Pragmatic Machine Learning for mobile apps

Khoa Pham

November 06, 2019
Tweet

More Decks by Khoa Pham

Other Decks in Technology

Transcript

  1. Pragmatic Machine
    Learning for iOS apps

    View Slide

  2. About
    • Khoa
    • iOS developer, at Shortcut
    • https:/
    /onmyway133.github.io

    View Slide

  3. Agenda
    • Deep Learning
    • Layer in Neural Network
    • Activation Functions
    • Training in Neural Network
    • Loss, Learning Rate
    • Train, Test & Validation Sets
    • Predicting
    • Overfitting

    View Slide

  4. Agenda (cont)
    • Convolutional Neural networks & visualizing
    • Zero padding, max pooling
    • Explain back propagation
    • Vanishing & Exploding Gradient
    • Weight Initialization & Bias
    • Learnable parameters
    • Regularizationm, batch size, fine tuning and batch normalization

    View Slide

  5. Pragmatic

    View Slide

  6. My Sentosa

    View Slide

  7. Pragmatic

    View Slide

  8. Pragmatic

    View Slide

  9. Pragmatic
    How I replicated an $86 million project in 57 lines of code

    View Slide

  10. Farming

    View Slide

  11. CoreML

    View Slide

  12. CoreML
    • Computer Vision: image classification
    • Natural Language: language idenification, tokenization
    • Speech: speech recognition

    View Slide

  13. Image classification
    Preprocess photos using the Vision framework and classify them with
    a Core ML model.
    iOS app Avengers https:/
    /github.com/onmyway133/avengers

    View Slide

  14. Avengers
    • SwiftUI
    • CoreML
    • Vision

    View Slide

  15. Tools
    • IBM Watson
    • Azure Custom Vision
    • Google AutoML
    • Firebase Vision Edge
    • CreateML
    • Turi Create

    View Slide

  16. Data set
    • Google images download https:/
    /github.com/hardikvasa/google-
    images-download
    • Data augmentation

    View Slide

  17. IBM Watson Visual Recognition
    • https:/
    /www.ibm.com/watson/services/visual-recognition/
    • https:/
    /cloud.ibm.com/developer/watson/services
    • https:/
    /www.ibm.com/cloud/watson-studio
    • https:/
    /dataplatform.cloud.ibm.com/home

    View Slide

  18. IBM Watson Visual Recognition -
    Service

    View Slide

  19. IBM Watson Visual Recognition -
    Assets

    View Slide

  20. IBM Watson Visual Recognition -
    Train

    View Slide

  21. IBM Watson Visual Recognition -
    Watson SDK
    let classifierID = "your-classifier-id"
    let failure = { (error: Error) in print(error) }
    visualRecognition.updateLocalModel(classifierID: classifierID, failure: failure) {
    print("model updated")
    }

    View Slide

  22. Vision

    View Slide

  23. Vision

    View Slide

  24. CoreML + Vision
    let model = try VNCoreMLModel(for: IBMWatson().model)
    let request = VNCoreMLRequest(model: model, completionHandler: { request, error in
    let results = request.results as? [VNClassificationObservation],
    let handler = VNImageRequestHandler(cgImage: image.cgImage!, options: [:])
    try handler.perform([request])

    View Slide

  25. Azure Custom Vision
    • https:/
    /www.customvision.ai/

    View Slide

  26. Azure Custom Vision - Project

    View Slide

  27. Azure Custom Vision - Assets

    View Slide

  28. Azure Custom Vision - Train

    View Slide

  29. Google Cloud AutoML Vision
    • https:/
    /cloud.google.com/automl/
    • https:/
    /console.cloud.google.com/vision

    View Slide

  30. Google Cloud AutoML Vision

    View Slide

  31. Google Cloud AutoML Vision - Dataset

    View Slide

  32. Google Cloud AutoML Vision - Google
    Cloud Storage
    https:/
    /console.cloud.google.com/storage

    View Slide

  33. Firebase AutoML Vision Edge

    View Slide

  34. Firebase AutoML Vision Edge -
    Dataset

    View Slide

  35. Firebase AutoML Vision Edge -
    Dataset

    View Slide

  36. Firebase AutoML Vision Edge - Train
    pod 'Firebase/MLModelInterpreter'

    View Slide

  37. Fritz AI

    View Slide

  38. CreateML
    • Activity, Sound, Image, Text, Tabular Classification
    • Word tagger
    • Recommendor
    • Object detection

    View Slide

  39. CreateML - Create Document

    View Slide

  40. CreateML - Data

    View Slide

  41. CreateML - Train

    View Slide

  42. CreateMLUI Playground
    • macOS Playground

    View Slide

  43. Turi Create
    • https:/
    /github.com/apple/turicreate
    • Open source Python framework
    • Latest version 5.8

    View Slide

  44. Turi Create

    View Slide

  45. Turi Create - SFrame
    import turicreate as tc
    import os
    # 1. Load images
    data = tc.image_analysis.load_images('dataset', with_path=True)
    # 2. Create label column based on folder name
    data['hero_name'] = data['path'].apply(lambda path: os.path.basename(os.path.dirname(path)))
    # 3. Save as .sframe
    data.save('turi.sframe')
    # 4. Explore
    data.explore()

    View Slide

  46. Turi Create - Visualization

    View Slide

  47. Turi Create - Training
    import turicreate as tc
    # 1. Load the data
    data = tc.SFrame('turi.sframe')
    # 2. Split to train and test data
    train_data, test_data = data.random_split(0.8)
    # 3. Create model
    model = tc.image_classifier.create(train_data, target='hero_name')
    # 4. Predictions
    predictions = model.predict(test_data)
    # 5. Evaluate the model and show metrics
    metrics = model.evaluate(test_data)
    print(metrics['accuracy'])
    # 6. Save the model
    model.save('turi.model')
    # 7. Export to CoreML format
    model.export_coreml('model/TuriCreate.mlmodel')

    View Slide

  48. Turi Create - Transfer Learning
    resnet-50
    model = tc.image_classifier.create(train_data, target='hero_name', model='squeezenet_v1.1')

    View Slide

  49. Writing
    • Machine Learning in iOS: IBM Watson and CoreML
    • Machine Learning in iOS: Azure Custom Vision and CoreML
    • Machine Learning in iOS: Turi Create and CoreML
    • Vision in iOS: Text detection and Tesseract recognition

    View Slide

  50. Thank you
    May your code continue to compile

    View Slide