Build smart cross-platform mobile apps - Flutter & MLKit Integration

Building “smarter” applications is a popular and important topic in the mobile development world. Machine Learning is the primary tool used to create apps that improve their delivered value by dynamically adjusting as more relevant data is made available. ML Kit and Flutter, which is Google’s mobile SDK for building native Android and iOS apps from a single codebase, can build apps with machine learning capabilities. In this session, we will explore how to use Flutter to leverage ML Kit’s functionalities and how we can train custom models using Google Cloud AutoVision API. The demo includes how to export the models to infer in the mobile app using Flutter and Firebase MLKit.

Transcript

1. Build Smart
   Cross-Platform Apps
   Flutter & ML Kit
   SivamuthuKumar
   Byteconf Flutter, August 14, 2020
   Build Smart
   Cross-Platform Apps
   Flutter & ML Kit
   SivamuthuKumar
   Byteconf Flutter, August 14, 2020
   

   View Slide

2. Agenda
   • Flutter & Machine Learning
   • MLKit APIs
   • AutoML - Custom Model / TFLite
   • Flutter Integration
   • Demo
   

   View Slide

3. Sivamuthu Kumar
   Software Architect, Computer Enterprise Inc,
   Cloud, Mobile, IoT, ML
   Orlando, FL
   ksivamuthu ksivamuthu ksivamuthu
   

   View Slide

4. Flutter
   

   View Slide

5. View Slide

6. View Slide

7. Machine
   Learning
   Google’s AI Building Blocks
   

   View Slide

8. AI building blocks make
   it easy to add the human
   like capabilities of sight,
   language, and
   conversation to your
   applications.
   

   View Slide

9. Machine Learning
   APIs
   Pre trained models
   ML Engine /
   Deep Learning VMs
   Custom Models
   Cloud AutoML
   Application Developers Data scientists & Practitioners
   @ksivamuthu
   Spectrum of AI Building Blocks
   

   View Slide

10. Machine Learning APIs
    

    View Slide

11. Cloud AutoML
    

    View Slide

12. TensorFlow TPUs Google Machine Perception
    Pre-trained ML APIs and AutoML
    Energy Auto Finance Entertainment Media Manufacturing /
    Agriculture
    Retail
    @ksivamuthu
    

    View Slide

13. MLKit
    Machine Learning for Mobile Developers
    

    View Slide

14. MLKit
    • Optimized for Mobile – iOS / Android SDK, Flutter
    • Easy to use APIs – Pretrained and Custom models
    • On-device and Google Cloud AI Inference APIs
    • Fast inference time
    • Privacy of your data – On-Device ML Support
    

    View Slide

15. View Slide

16. Initialize the Detectors
    

    View Slide

17. Get the image
    Detect / Process Image
    

    View Slide

18. Extract the Labels
    

    View Slide

19. Demo
    Flutter + MLKit APIs
    

    View Slide

20. Boat
    Classification
    

    View Slide

21. Ensure Mask
    

    View Slide

22. Ensure Mask
    • Preparing Dataset
    • Training
    • Evaluation
    • Exporting model to run in Edge/Device
    • Running inference in device
    

    View Slide

23. View Slide

24. View Slide

25. Reference
    • Demo Repo -
    https://github.com/ksivamuthu/flutter_mlkit_demo
    • MLKit - https://developers.google.com/ml-kit
    • Firebase ML Vision Plugin -
    https://pub.dev/packages/firebase_ml_vision
    • AutoML / Coral Edge Demo -
    https://www.youtube.com/watch?v=sZBN04tprPs
    

    View Slide

26. Thank you
    Sivamuthu Kumar
    ksivamuthu
    

    View Slide