WWDC18 ML Overview

Transcript

1. 1.

   WWDC18 ML Overview ɹ 2018/06/15 - WWDC 2018 A.er Party

   @shingt (Shinichi Goto)
2. 2.

   shingt (Shinichi Goto) GitHub: @shingt Twi5er: @_shingt iOS Engineer at

   Mercari US @ Tokyo 2
3. 3.

   ML-related Sessions in WWDC18 • 609 Metal for Accelera0ng Machine

   Learning • 703 Introducing Create ML • 708 What's New in Core ML, Part 1 • 709 What's New in Core ML, Part 2 • 712 A Guide to Turi Create • 713 Introducing Natual Language Framework • 716 Object Tracking in Vision • 717 Vision with Core ML • 719 Core Image: Performance, Prototyping, and Python 3
4. 4.

   WWDC 18 ML Overview • Core ML 2 • Create

   ML • Turi Create • Metal • Vision / Natural Language 4
5. 5.

   WWDC 18 ML Overview • Core ML 2 • Create

   ML • Turi Create • Metal • Vision / Natural Language 5
6. 6.

   WWDC 18 ML Overview • Core ML 2 • Create

   ML • Turi Create • Metal • Vision / Natural Language 6
7. 7.

   Core ML 2 708 What's New in Core ML, Part

   1ɹɹ709 What's New in Core ML, Part 2 7
8. 8.

   Core ML (Recap) • Framework to integrate pre-trained models to

   apps • Inference only • Introduced Core ML model format (**.mlmodel) • Xcode automa9cally generates Swi; interface for model • Introduced coremltools 8
9. 9.

   Core ML problem examples • Model size • Unsupported machine

   learning components • Cannot convert model if it includes components Core ML doesn't support • Model data protec;on • Parameters are not encrypted / can be seen • Why Core ML will not work for your app (most likely) 9
10. 10.

    Core ML 2 Smaller. Faster. Customizable. 10

11. 11.

    Core ML 2 Updates • Smaller • Weights Quan.za.on •

    Faster • Batch predic.on • Customizable • Custom Layer Supports (since iOS 11.) • Custom Model Supports 11
12. 12.

    Understanding the Structure of Neural Networks (h7ps:/ /becominghuman.ai/understanding-the- structure-of-neural-networks-1fa5bd17fef0) 12

13. 13.

    13

14. 14.

    14

15. 15.

    15

16. 16.

    Weights Quan-za-on • Tradeoff between size and accuracy • Example

    in WWDC18 • Style Transfer • 6.7MB (32-bit) => 857KB (4-bit) • Without accuracy loss 16
17. 17.

    How to get quan,zed models • coremltools provides method •

    Post-training quan5za5on • quantize_weights(model, 8, "linear") • Train quan5zed 17
18. 18.

    coremltools 2.0beta1 • h#ps:/ /pypi.org/project/coremltools/2.0b1/ 18

19. 19.

    if macos_version() < (10, 14): print("WARNING! Unable to return a

    quantized MLModel instance since OS != macOS 10.14 or later") print("Returning quantized model specification instead") return qspec 19
20. 20.

    qmode_mapping = { "linear": _QUANTIZATION_MODE_LINEAR_QUANTIZATION, "kmeans": _QUANTIZATION_MODE_LOOKUP_TABLE_KMEANS, "linear_lut": _QUANTIZATION_MODE_LOOKUP_TABLE_LINEAR, "custom_lut":

    _QUANTIZATION_MODE_CUSTOM_LOOKUP_TABLE, "dequantization": _QUANTIZATION_MODE_DEQUANTIZE } 20
21. 21.

    Batch // Old: Loop over inputs for i in 0..<

    modelInputs.count { modelOutputs[i] = model.prediction( from: modelInputs[i], options: options ) } // New: Batch predictions // Remove GPU idle time / Keep high-performance modelOutputs = model.predictions( from: modelInputs, options: options ) 21
22. 22.

    Custom Layer / Model Supports • MLCustomLayer (Since iOS 11.2)

    • Can be used when Core ML doens't support desired neural net layer • MLCustomModel (Since iOS 12.0) • For ML other than neural net 22
23. 23.

    23

24. 24.

    Core ML 2 Updates • Smaller • Weights Quan2za2on •

    Faster • Batch predic2on • Customizable • Custom Layer Supports (since iOS 11.2) • Custom Model Supports 24
25. 25.

    WWDC 18 ML Overview • Core ML 2 • Create

    ML • Turi Create • Metal • Vision / Natural Language 25
26. 26.

    Create ML 703 Introducing Create ML 26

27. 27.

    Create ML • New ML framework for training phase •

    Developers don't need to define ML algorithms • Tasks are limited • Create MLUI • Framework to train classifiers in the UI • GPU accerelated 27
28. 28.

    Create ML Tasks • Image Classifica.on • Text classifica.on &

    Word tagging • Classical regression, classifica.on Tabular Data 28
29. 29.

    Create ML Tasks • Image Classifica-on • Text classifica-on &

    Word tagging • Classical regression, classifica-on Tabular Data 29
30. 30.

    Create ML Tasks • Image Classifica.on => Transfer Learning based

    • Text classifica.on & Word tagging • Classical regression, classifica.on Tabular Data 30
31. 31.

    31

32. 32.

    Transfer Learning Pros • Training with smaller data • Faster

    (compared to zero-based training) 32
33. 33.

    Transfer Learning Pros on Create ML • Training with smaller

    data • Faster (compared to zero-based training) • Smaller models (e.g. 94.7MB => 83KB ) • Because training is done on top the model that already exists on OS 33
34. 34.

    import Foundation import CreateML // Specify Data let trainDirectory =

    URL(fileURLWithPath: “/Users/createml/Desktop/Fruits“) let testDirectory = URL(fileURLWithPath: “/Users/createml/Desktop/TestFruits“) // Create Model let model = try MLImageClassifier(trainingData: .labeledDirectories(at: trainDirectory)) // Evaluate Model let evaluation = model.evaluation(on: .labeledDirectories(at: testDirectory)) // Save Model try model.write(to: URL(fileURLWithPath: “/Users/createml/Desktop/FruitClassifier.mlmodel“)) 34
35. 35.

    // `MLImageClassifier.ModelParameters` var augmentationOptions: MLImageClassifier.ImageAugmentationOptions // `MLImageClassifier.ImageAugmentationOptions` static let blur:

    MLImageClassifier.ImageAugmentationOptions static let exposure: MLImageClassifier.ImageAugmentationOptions static let flip: MLImageClassifier.ImageAugmentationOptions static let noise: MLImageClassifier.ImageAugmentationOptions static let rotation: MLImageClassifier.ImageAugmentationOptions static let shear: MLImageClassifier.ImageAugmentationOptions 35
36. 36.

    WWDC 18 ML Overview • Core ML 2 • Create

    ML • Turi Create • Metal • Vision / Natural Language 36
37. 37.

    Turi Create 712 A Guide to Turi Create 37

38. 38.

    Turi Create (apple/turicreate) • Python package for crea1ng Core ML

    models • Released in 2017/12 • Developers don't need to define ML algorithms (again) • Apple acquires Turi, a machine learning company | TechCrunch (posted Aug 5, 2016) 38
39. 39.

    Turi Create: Supported Tasks • Image classifica-on, Object detec-on •

    => Transfer Learning based • Image similarity • Recommender systems • Text classifier • Ac-vity classifica-on • Essen-al tasks (regression, classifica-on, etc.) 39
40. 40.

    # https://apple.github.io/turicreate/docs/userguide/image_classifier/introduction.html import turicreate as tc data = tc.SFrame('cats-dogs.sframe') train_data,

    test_data = data.random_split(0.8) # Automatically picks the right model based on your data. # Based on Transfer Learning model = tc.image_classifier.create(train_data, target='label') predictions = model.predict(test_data) metrics = model.evaluate(test_data) model.save('mymodel.model') model.export_coreml('MyCustomImageClassifier.mlmodel') 40
41. 41.

    Turi Create in WWDC18 • Basic usage explana/on • Turi

    Create 5.0 introduc/on 41
42. 42.

    Turi Create 5.0 • New Task • Style Transfer •

    New Deployments • Recommenders • Vision Feature Print powered models • GPU Accelera@on 42
43. 43.

    43

44. 44.

    44

45. 45.

    Create ML Turi Create 45

46. 46.

    Recap • WWDC18 ML overview • Core ML 2 •

    Create ML • Turi Create • Running ML on device has become much more realis?c 46
47. 47.

    Reference • 703 Introducing Create ML • 708 What's New

    in Core ML, Part 1 • 709 What's New in Core ML, Part 2 • 712 A Guide to Turi Create • Apple Developer DocumentaIon • Turi Create User Guide • Turi Create User Guide • Becoming Human : Understanding the Structure of Neural Networks • SmartNews Engineering Blog : χϡʔϥϧωοτϫʔΫͷྔࢠԽʹ͍ͭͯͷ࠷ۙͷݚڀͷਐలͱɺͦͷॏ ཁੑ 47