Pragmatic Machine Learning for mobile apps

16bebb36e0e28572a316ba0450e190d1?s=47 Khoa Pham
November 06, 2019

Pragmatic Machine Learning for mobile apps

16bebb36e0e28572a316ba0450e190d1?s=128

Khoa Pham

November 06, 2019
Tweet

Transcript

  1. Pragmatic Machine Learning for iOS apps

  2. About • Khoa • iOS developer, at Shortcut • https:/

    /onmyway133.github.io
  3. Agenda • Deep Learning • Layer in Neural Network •

    Activation Functions • Training in Neural Network • Loss, Learning Rate • Train, Test & Validation Sets • Predicting • Overfitting
  4. Agenda (cont) • Convolutional Neural networks & visualizing • Zero

    padding, max pooling • Explain back propagation • Vanishing & Exploding Gradient • Weight Initialization & Bias • Learnable parameters • Regularizationm, batch size, fine tuning and batch normalization
  5. Pragmatic

  6. My Sentosa

  7. Pragmatic

  8. Pragmatic

  9. Pragmatic How I replicated an $86 million project in 57

    lines of code
  10. Farming

  11. CoreML

  12. CoreML • Computer Vision: image classification • Natural Language: language

    idenification, tokenization • Speech: speech recognition
  13. Image classification Preprocess photos using the Vision framework and classify

    them with a Core ML model. iOS app Avengers https:/ /github.com/onmyway133/avengers
  14. Avengers • SwiftUI • CoreML • Vision

  15. Tools • IBM Watson • Azure Custom Vision • Google

    AutoML • Firebase Vision Edge • CreateML • Turi Create
  16. Data set • Google images download https:/ /github.com/hardikvasa/google- images-download •

    Data augmentation
  17. IBM Watson Visual Recognition • https:/ /www.ibm.com/watson/services/visual-recognition/ • https:/ /cloud.ibm.com/developer/watson/services

    • https:/ /www.ibm.com/cloud/watson-studio • https:/ /dataplatform.cloud.ibm.com/home
  18. IBM Watson Visual Recognition - Service

  19. IBM Watson Visual Recognition - Assets

  20. IBM Watson Visual Recognition - Train

  21. IBM Watson Visual Recognition - Watson SDK let classifierID =

    "your-classifier-id" let failure = { (error: Error) in print(error) } visualRecognition.updateLocalModel(classifierID: classifierID, failure: failure) { print("model updated") }
  22. Vision

  23. Vision

  24. CoreML + Vision let model = try VNCoreMLModel(for: IBMWatson().model) let

    request = VNCoreMLRequest(model: model, completionHandler: { request, error in let results = request.results as? [VNClassificationObservation], let handler = VNImageRequestHandler(cgImage: image.cgImage!, options: [:]) try handler.perform([request])
  25. Azure Custom Vision • https:/ /www.customvision.ai/

  26. Azure Custom Vision - Project

  27. Azure Custom Vision - Assets

  28. Azure Custom Vision - Train

  29. Google Cloud AutoML Vision • https:/ /cloud.google.com/automl/ • https:/ /console.cloud.google.com/vision

  30. Google Cloud AutoML Vision

  31. Google Cloud AutoML Vision - Dataset

  32. Google Cloud AutoML Vision - Google Cloud Storage https:/ /console.cloud.google.com/storage

  33. Firebase AutoML Vision Edge

  34. Firebase AutoML Vision Edge - Dataset

  35. Firebase AutoML Vision Edge - Dataset

  36. Firebase AutoML Vision Edge - Train pod 'Firebase/MLModelInterpreter'

  37. Fritz AI

  38. CreateML • Activity, Sound, Image, Text, Tabular Classification • Word

    tagger • Recommendor • Object detection
  39. CreateML - Create Document

  40. CreateML - Data

  41. CreateML - Train

  42. CreateMLUI Playground • macOS Playground

  43. Turi Create • https:/ /github.com/apple/turicreate • Open source Python framework

    • Latest version 5.8
  44. Turi Create

  45. Turi Create - SFrame import turicreate as tc import os

    # 1. Load images data = tc.image_analysis.load_images('dataset', with_path=True) # 2. Create label column based on folder name data['hero_name'] = data['path'].apply(lambda path: os.path.basename(os.path.dirname(path))) # 3. Save as .sframe data.save('turi.sframe') # 4. Explore data.explore()
  46. Turi Create - Visualization

  47. Turi Create - Training import turicreate as tc # 1.

    Load the data data = tc.SFrame('turi.sframe') # 2. Split to train and test data train_data, test_data = data.random_split(0.8) # 3. Create model model = tc.image_classifier.create(train_data, target='hero_name') # 4. Predictions predictions = model.predict(test_data) # 5. Evaluate the model and show metrics metrics = model.evaluate(test_data) print(metrics['accuracy']) # 6. Save the model model.save('turi.model') # 7. Export to CoreML format model.export_coreml('model/TuriCreate.mlmodel')
  48. Turi Create - Transfer Learning resnet-50 model = tc.image_classifier.create(train_data, target='hero_name',

    model='squeezenet_v1.1')
  49. Writing • Machine Learning in iOS: IBM Watson and CoreML

    • Machine Learning in iOS: Azure Custom Vision and CoreML • Machine Learning in iOS: Turi Create and CoreML • Vision in iOS: Text detection and Tesseract recognition
  50. Thank you May your code continue to compile