$30 off During Our Annual Pro Sale. View Details »

Pragmatic Machine Learning for mobile apps

Khoa Pham
November 06, 2019

Pragmatic Machine Learning for mobile apps

Khoa Pham

November 06, 2019
Tweet

More Decks by Khoa Pham

Other Decks in Technology

Transcript

  1. Pragmatic Machine Learning for iOS apps

  2. About • Khoa • iOS developer, at Shortcut • https:/

    /onmyway133.github.io
  3. Agenda • Deep Learning • Layer in Neural Network •

    Activation Functions • Training in Neural Network • Loss, Learning Rate • Train, Test & Validation Sets • Predicting • Overfitting
  4. Agenda (cont) • Convolutional Neural networks & visualizing • Zero

    padding, max pooling • Explain back propagation • Vanishing & Exploding Gradient • Weight Initialization & Bias • Learnable parameters • Regularizationm, batch size, fine tuning and batch normalization
  5. Pragmatic

  6. My Sentosa

  7. Pragmatic

  8. Pragmatic

  9. Pragmatic How I replicated an $86 million project in 57

    lines of code
  10. Farming

  11. CoreML

  12. CoreML • Computer Vision: image classification • Natural Language: language

    idenification, tokenization • Speech: speech recognition
  13. Image classification Preprocess photos using the Vision framework and classify

    them with a Core ML model. iOS app Avengers https:/ /github.com/onmyway133/avengers
  14. Avengers • SwiftUI • CoreML • Vision

  15. Tools • IBM Watson • Azure Custom Vision • Google

    AutoML • Firebase Vision Edge • CreateML • Turi Create
  16. Data set • Google images download https:/ /github.com/hardikvasa/google- images-download •

    Data augmentation
  17. IBM Watson Visual Recognition • https:/ /www.ibm.com/watson/services/visual-recognition/ • https:/ /cloud.ibm.com/developer/watson/services

    • https:/ /www.ibm.com/cloud/watson-studio • https:/ /dataplatform.cloud.ibm.com/home
  18. IBM Watson Visual Recognition - Service

  19. IBM Watson Visual Recognition - Assets

  20. IBM Watson Visual Recognition - Train

  21. IBM Watson Visual Recognition - Watson SDK let classifierID =

    "your-classifier-id" let failure = { (error: Error) in print(error) } visualRecognition.updateLocalModel(classifierID: classifierID, failure: failure) { print("model updated") }
  22. Vision

  23. Vision

  24. CoreML + Vision let model = try VNCoreMLModel(for: IBMWatson().model) let

    request = VNCoreMLRequest(model: model, completionHandler: { request, error in let results = request.results as? [VNClassificationObservation], let handler = VNImageRequestHandler(cgImage: image.cgImage!, options: [:]) try handler.perform([request])
  25. Azure Custom Vision • https:/ /www.customvision.ai/

  26. Azure Custom Vision - Project

  27. Azure Custom Vision - Assets

  28. Azure Custom Vision - Train

  29. Google Cloud AutoML Vision • https:/ /cloud.google.com/automl/ • https:/ /console.cloud.google.com/vision

  30. Google Cloud AutoML Vision

  31. Google Cloud AutoML Vision - Dataset

  32. Google Cloud AutoML Vision - Google Cloud Storage https:/ /console.cloud.google.com/storage

  33. Firebase AutoML Vision Edge

  34. Firebase AutoML Vision Edge - Dataset

  35. Firebase AutoML Vision Edge - Dataset

  36. Firebase AutoML Vision Edge - Train pod 'Firebase/MLModelInterpreter'

  37. Fritz AI

  38. CreateML • Activity, Sound, Image, Text, Tabular Classification • Word

    tagger • Recommendor • Object detection
  39. CreateML - Create Document

  40. CreateML - Data

  41. CreateML - Train

  42. CreateMLUI Playground • macOS Playground

  43. Turi Create • https:/ /github.com/apple/turicreate • Open source Python framework

    • Latest version 5.8
  44. Turi Create

  45. Turi Create - SFrame import turicreate as tc import os

    # 1. Load images data = tc.image_analysis.load_images('dataset', with_path=True) # 2. Create label column based on folder name data['hero_name'] = data['path'].apply(lambda path: os.path.basename(os.path.dirname(path))) # 3. Save as .sframe data.save('turi.sframe') # 4. Explore data.explore()
  46. Turi Create - Visualization

  47. Turi Create - Training import turicreate as tc # 1.

    Load the data data = tc.SFrame('turi.sframe') # 2. Split to train and test data train_data, test_data = data.random_split(0.8) # 3. Create model model = tc.image_classifier.create(train_data, target='hero_name') # 4. Predictions predictions = model.predict(test_data) # 5. Evaluate the model and show metrics metrics = model.evaluate(test_data) print(metrics['accuracy']) # 6. Save the model model.save('turi.model') # 7. Export to CoreML format model.export_coreml('model/TuriCreate.mlmodel')
  48. Turi Create - Transfer Learning resnet-50 model = tc.image_classifier.create(train_data, target='hero_name',

    model='squeezenet_v1.1')
  49. Writing • Machine Learning in iOS: IBM Watson and CoreML

    • Machine Learning in iOS: Azure Custom Vision and CoreML • Machine Learning in iOS: Turi Create and CoreML • Vision in iOS: Text detection and Tesseract recognition
  50. Thank you May your code continue to compile