Pragmatic Machine Learning for mobile apps

Transcript

1. Pragmatic Machine Learning for iOS apps
2. About • Khoa • iOS developer, at Shortcut • https:/ /onmyway133.github.io
3. Agenda • Deep Learning • Layer in Neural Network • Activation Functions • Training in Neural Network • Loss, Learning Rate • Train, Test & Validation Sets • Predicting • Overfitting
4. Agenda (cont) • Convolutional Neural networks & visualizing • Zero padding, max pooling • Explain back propagation • Vanishing & Exploding Gradient • Weight Initialization & Bias • Learnable parameters • Regularizationm, batch size, fine tuning and batch normalization
5. Pragmatic
6. My Sentosa
7. Pragmatic
8. Pragmatic
9. Pragmatic How I replicated an $86 million project in 57 lines of code
10. Farming
11. CoreML
12. CoreML • Computer Vision: image classification • Natural Language: language idenification, tokenization • Speech: speech recognition
13. Image classification Preprocess photos using the Vision framework and classify them with a Core ML model. iOS app Avengers https:/ /github.com/onmyway133/avengers
14. Avengers • SwiftUI • CoreML • Vision
15. Tools • IBM Watson • Azure Custom Vision • Google AutoML • Firebase Vision Edge • CreateML • Turi Create
16. Data set • Google images download https:/ /github.com/hardikvasa/google- images-download • Data augmentation
17. IBM Watson Visual Recognition • https:/ /www.ibm.com/watson/services/visual-recognition/ • https:/ /cloud.ibm.com/developer/watson/services • https:/ /www.ibm.com/cloud/watson-studio • https:/ /dataplatform.cloud.ibm.com/home
18. IBM Watson Visual Recognition - Service
19. IBM Watson Visual Recognition - Assets
20. IBM Watson Visual Recognition - Train
21. IBM Watson Visual Recognition - Watson SDK let classifierID = "your-classifier-id" let failure = { (error: Error) in print(error) } visualRecognition.updateLocalModel(classifierID: classifierID, failure: failure) { print("model updated") }
22. Vision
23. Vision
24. CoreML + Vision let model = try VNCoreMLModel(for: IBMWatson().model) let request = VNCoreMLRequest(model: model, completionHandler: { request, error in let results = request.results as? [VNClassificationObservation], let handler = VNImageRequestHandler(cgImage: image.cgImage!, options: [:]) try handler.perform([request])
25. Azure Custom Vision • https:/ /www.customvision.ai/
26. Azure Custom Vision - Project
27. Azure Custom Vision - Assets
28. Azure Custom Vision - Train
29. Google Cloud AutoML Vision • https:/ /cloud.google.com/automl/ • https:/ /console.cloud.google.com/vision
30. Google Cloud AutoML Vision
31. Google Cloud AutoML Vision - Dataset
32. Google Cloud AutoML Vision - Google Cloud Storage https:/ /console.cloud.google.com/storage
33. Firebase AutoML Vision Edge
34. Firebase AutoML Vision Edge - Dataset
35. Firebase AutoML Vision Edge - Dataset
36. Firebase AutoML Vision Edge - Train pod 'Firebase/MLModelInterpreter'
37. Fritz AI
38. CreateML • Activity, Sound, Image, Text, Tabular Classification • Word tagger • Recommendor • Object detection
39. CreateML - Create Document
40. CreateML - Data
41. CreateML - Train
42. CreateMLUI Playground • macOS Playground
43. Turi Create • https:/ /github.com/apple/turicreate • Open source Python framework • Latest version 5.8
44. Turi Create
45. Turi Create - SFrame import turicreate as tc import os # 1. Load images data = tc.image_analysis.load_images('dataset', with_path=True) # 2. Create label column based on folder name data['hero_name'] = data['path'].apply(lambda path: os.path.basename(os.path.dirname(path))) # 3. Save as .sframe data.save('turi.sframe') # 4. Explore data.explore()
46. Turi Create - Visualization
47. Turi Create - Training import turicreate as tc # 1. Load the data data = tc.SFrame('turi.sframe') # 2. Split to train and test data train_data, test_data = data.random_split(0.8) # 3. Create model model = tc.image_classifier.create(train_data, target='hero_name') # 4. Predictions predictions = model.predict(test_data) # 5. Evaluate the model and show metrics metrics = model.evaluate(test_data) print(metrics['accuracy']) # 6. Save the model model.save('turi.model') # 7. Export to CoreML format model.export_coreml('model/TuriCreate.mlmodel')
48. Turi Create - Transfer Learning resnet-50 model = tc.image_classifier.create(train_data, target='hero_name', model='squeezenet_v1.1')
49. Writing • Machine Learning in iOS: IBM Watson and CoreML • Machine Learning in iOS: Azure Custom Vision and CoreML • Machine Learning in iOS: Turi Create and CoreML • Vision in iOS: Text detection and Tesseract recognition
50. Thank you May your code continue to compile