Pragmatic Machine Learning for mobile apps

Transcript

1. Pragmatic Machine
   Learning for iOS apps
   

   View Slide

2. About
   • Khoa
   • iOS developer, at Shortcut
   • https:/
   /onmyway133.github.io
   

   View Slide

3. Agenda
   • Deep Learning
   • Layer in Neural Network
   • Activation Functions
   • Training in Neural Network
   • Loss, Learning Rate
   • Train, Test & Validation Sets
   • Predicting
   • Overfitting
   

   View Slide

4. Agenda (cont)
   • Convolutional Neural networks & visualizing
   • Zero padding, max pooling
   • Explain back propagation
   • Vanishing & Exploding Gradient
   • Weight Initialization & Bias
   • Learnable parameters
   • Regularizationm, batch size, fine tuning and batch normalization
   

   View Slide

5. Pragmatic
   

   View Slide

6. My Sentosa
   

   View Slide

7. Pragmatic
   

   View Slide

8. Pragmatic
   

   View Slide

9. Pragmatic
   How I replicated an $86 million project in 57 lines of code
   

   View Slide

10. Farming
    

    View Slide

11. CoreML
    

    View Slide

12. CoreML
    • Computer Vision: image classification
    • Natural Language: language idenification, tokenization
    • Speech: speech recognition
    

    View Slide

13. Image classification
    Preprocess photos using the Vision framework and classify them with
    a Core ML model.
    iOS app Avengers https:/
    /github.com/onmyway133/avengers
    

    View Slide

14. Avengers
    • SwiftUI
    • CoreML
    • Vision
    

    View Slide

15. Tools
    • IBM Watson
    • Azure Custom Vision
    • Google AutoML
    • Firebase Vision Edge
    • CreateML
    • Turi Create
    

    View Slide

16. Data set
    • Google images download https:/
    /github.com/hardikvasa/google-
    images-download
    • Data augmentation
    

    View Slide

17. IBM Watson Visual Recognition
    • https:/
    /www.ibm.com/watson/services/visual-recognition/
    • https:/
    /cloud.ibm.com/developer/watson/services
    • https:/
    /www.ibm.com/cloud/watson-studio
    • https:/
    /dataplatform.cloud.ibm.com/home
    

    View Slide

18. IBM Watson Visual Recognition -
    Service
    

    View Slide

19. IBM Watson Visual Recognition -
    Assets
    

    View Slide

20. IBM Watson Visual Recognition -
    Train
    

    View Slide

21. IBM Watson Visual Recognition -
    Watson SDK
    let classifierID = "your-classifier-id"
    let failure = { (error: Error) in print(error) }
    visualRecognition.updateLocalModel(classifierID: classifierID, failure: failure) {
    print("model updated")
    }
    

    View Slide

22. Vision
    

    View Slide

23. Vision
    

    View Slide

24. CoreML + Vision
    let model = try VNCoreMLModel(for: IBMWatson().model)
    let request = VNCoreMLRequest(model: model, completionHandler: { request, error in
    let results = request.results as? [VNClassificationObservation],
    let handler = VNImageRequestHandler(cgImage: image.cgImage!, options: [:])
    try handler.perform([request])
    

    View Slide

25. Azure Custom Vision
    • https:/
    /www.customvision.ai/
    

    View Slide

26. Azure Custom Vision - Project
    

    View Slide

27. Azure Custom Vision - Assets
    

    View Slide

28. Azure Custom Vision - Train
    

    View Slide

29. Google Cloud AutoML Vision
    • https:/
    /cloud.google.com/automl/
    • https:/
    /console.cloud.google.com/vision
    

    View Slide

30. Google Cloud AutoML Vision
    

    View Slide

31. Google Cloud AutoML Vision - Dataset
    

    View Slide

32. Google Cloud AutoML Vision - Google
    Cloud Storage
    https:/
    /console.cloud.google.com/storage
    

    View Slide

33. Firebase AutoML Vision Edge
    

    View Slide

34. Firebase AutoML Vision Edge -
    Dataset
    

    View Slide

35. Firebase AutoML Vision Edge -
    Dataset
    

    View Slide

36. Firebase AutoML Vision Edge - Train
    pod 'Firebase/MLModelInterpreter'
    

    View Slide

37. Fritz AI
    

    View Slide

38. CreateML
    • Activity, Sound, Image, Text, Tabular Classification
    • Word tagger
    • Recommendor
    • Object detection
    

    View Slide

39. CreateML - Create Document
    

    View Slide

40. CreateML - Data
    

    View Slide

41. CreateML - Train
    

    View Slide

42. CreateMLUI Playground
    • macOS Playground
    

    View Slide

43. Turi Create
    • https:/
    /github.com/apple/turicreate
    • Open source Python framework
    • Latest version 5.8
    

    View Slide

44. Turi Create
    

    View Slide

45. Turi Create - SFrame
    import turicreate as tc
    import os
    # 1. Load images
    data = tc.image_analysis.load_images('dataset', with_path=True)
    # 2. Create label column based on folder name
    data['hero_name'] = data['path'].apply(lambda path: os.path.basename(os.path.dirname(path)))
    # 3. Save as .sframe
    data.save('turi.sframe')
    # 4. Explore
    data.explore()
    

    View Slide

46. Turi Create - Visualization
    

    View Slide

47. Turi Create - Training
    import turicreate as tc
    # 1. Load the data
    data = tc.SFrame('turi.sframe')
    # 2. Split to train and test data
    train_data, test_data = data.random_split(0.8)
    # 3. Create model
    model = tc.image_classifier.create(train_data, target='hero_name')
    # 4. Predictions
    predictions = model.predict(test_data)
    # 5. Evaluate the model and show metrics
    metrics = model.evaluate(test_data)
    print(metrics['accuracy'])
    # 6. Save the model
    model.save('turi.model')
    # 7. Export to CoreML format
    model.export_coreml('model/TuriCreate.mlmodel')
    

    View Slide

48. Turi Create - Transfer Learning
    resnet-50
    model = tc.image_classifier.create(train_data, target='hero_name', model='squeezenet_v1.1')
    

    View Slide

49. Writing
    • Machine Learning in iOS: IBM Watson and CoreML
    • Machine Learning in iOS: Azure Custom Vision and CoreML
    • Machine Learning in iOS: Turi Create and CoreML
    • Vision in iOS: Text detection and Tesseract recognition
    

    View Slide

50. Thank you
    May your code continue to compile
    

    View Slide