TensorFlow on mobile HWSH Budapest

When it comes to blending AI into mobile apps, there are a couple of tools at developers disposal. We will explore some of these available options, then linger on to the steps and challenges of integrating a TensorFlow model. In the end, you’ll be left with, a basic understanding of the process and work with TensorFlow on mobile. Hopefully, these new learnings will spark your curiosity and encourage you to experiment and implement your crazy, creative AI ideas. The session consists mainly of an Introduction to the TensorFlow framework from an Android developer’s point of view.

Transcript

1. on mobile HWSW mobile! 2018 Budapest
2. Attila Blénesi Android Project Lead @ Halcyon Mobile @ablenessy
3. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow
4. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow
5. Firebase ML Kit Source: firebase.google.com/products/ml-kit Image labelling Text Recognition Face Detection Barcode scanning Landmark detection NEW
6. Source: firebase.google.com/products/ml-kit Face Contour Detection NEW
7. Text Recognition
8. Text Recognition
9. val image = FirebaseVisionImage.fromBitmap(selectedImage) val detector = FirebaseVision.getInstance() .getVisionTextDetector() detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...)
10. Text Recognition
11. Text Recognition
12. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Lite
13. generic model = generic solution Do I even have to dig deeper ?
14. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow
16. TensorFlow #pix2pix Source: https://affinelayer.com/pixsrv/
17. #pix2pix TensorFlow
18. How does pix2pix work?
19. How does pix2pix work? Source: https://affinelayer.com/pix2pix/
20. How does pix2pix work? Source: https://affinelayer.com/pix2pix/
21. workflow Train > Convert > Integrate
22. Train > Convert > Integrate
23. On device ML is important Low latency, no server calls Works offline, no connection needed Privacy, data stays on device
24. Train > Convert > Integrate
25. Interpreter Core Converter TensorFlow Lite Format Operation Kernels Hardware acceleration ~75 KB vs 1.1 MB in TF ~2.4 MB vs 22.3 MB in TF Android
26. Converter TensorFlow Lite Format Android App (Java /C++ API) iOS App (C++ API) Trained TensorFlow Model Linux (e.g. Raspberry Pi) (C++ API)
27. Converter TensorFlow Lite Format Android App (Java /C++ API) iOS App (C++ API) Trained TensorFlow Model Linux (e.g. Raspberry Pi) (C++ API)
28. Converter TensorFlow Lite Format Android App (Java /C++ API) iOS App (C++ API) Trained TensorFlow Model Converter Core ML Format iOS App (Core ML) Linux (e.g. Raspberry Pi) (C++ API)
29. Train > Convert > Integrate
30. Demo
31. Machine learning is NOT just for experts
32. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Lite Machine learning is NOT just for experts
33. Select model ( i.e. Base API ) Business Analyst/Product Owner/Software Dev Predict Software Dev Gather, Massage, Split Data Bring or create Model Train, Tune, and Evaluate model ML expert / Data scientist Deploy Model DevOps Predict Software Dev ML Kit Simplifies Machine Learning
34. Machine learning is NOT just for experts attila.blenesi@halcyonmobile.com @ablenessy ML Kit Demos bit.ly/MLKitDemo Pix2Pix Model Implementation bit.ly/pix2pixModel TensorFlow Lite Codelab bit.ly/tfLiteCodeLab