Machine Learning in small, low-powered devices

Machine learning (ML) is a way of writing computer programs. The difference is that the rules of ML programs are not determined by a developer but by training the machine. Attend this session and we will collect a substantial set of training data from a device, feed it into a special kind of algorithm that will discover the rules then convert it to a TinyML model. We will deploy the TinyML model to a small, low-powered embedded device, test it and send output to Azure.

Transcript

1. Machine Learning in small, low-powered devices (TinyML) Ron Dagdag
2. Agenda •What is Machine Learning? •Embedded Machine Learning (TinyML) •Small, low-powered device •Challenges TinyML •Demo @rondagdag
3. world's tiniest chameleon @rondagdag
4. programming @rondagdag algorithm input answers
5. machine learning @rondagdag algorithm input answers
6. algorithm input answers algorithm input answers ML Primer Programming machine Learning Training Data Model Model Inferencing Training Framework Runtime
7. Anja Markiewicz - Artist that Creates Tiny Sculptures from Paper
8. Applications of Machine Learning 9
9. This Photo by Unknown Author is licensed under CC BY-NC-ND
10. 11 Cloud Mobile
11. 12 Mobile Google Assistant
12. Google Assistant nest 13
13. Google Assistant nest 14 IoT 1.0: Internet of Things
14. Google Assistant nest 15 IoT 2.0: Intelligence on Things
15. nest 16 IoT 2.0: Intelligence on Things Bandwidth Reliability Latency Privacy Energy
16. Motion Sensors Gyroscope, radar, magnetometer, accelerator Environmental Sensors Temperature, Humidity, Pressure, IR, etc. Touchscreen Sensors Capacitive, IR Image Sensors Thermal, Image Biometric Sensors Fingerprint, Heart rate, etc. TinyML Is All About Sensor Data Intelligence 17 Rotation Sensors Encoders Force Sensors Pressure, Strain Acoustic Sensors Ultrasonic, Microphones, Geophones, Vibrometers ...
17. Painting Miniatures By Hasan Kale
18. Keyword Spotting Visual Wake Words Anomaly Detection Image Classification
19. Board MCU / ASIC Clock Memory Sensors Radio Himax WE-I Plus EVB HX6537-A 32-bit EM9D DSP 400 MHz 2MB flash 2MB RAM Accelerometer, Mic, Camera None Arduino Nano 33 BLE Sense 32-bit nRF52840 64 MHz 1MB flash 256kB RAM Mic, IMU, Temp, Humidity, Gesture, Pressure, Proximity, Brightness, Color BLE SparkFun Edge 2 32-bit ArtemisV1 48 MHz 1MB flash 384kB RAM Accelerometer, Mic, Camera BLE Espressif EYE 32-bit ESP32-D0WD 240 MHz 4MB flash 520kB RAM Mic, Camera WiFi, BLE 20
20. • ARM Cortex-M4F core running at 120MHz(Boost up to 200MHz) • 4 MB External Flash, 192 KB RAM • WIFI，BT • LCD screen • Built-in Modules： • Accelerometer • Microphone • Speaker • Light Sensor • Infrared Emitter • MicroSD Card Slot，5-Way Switch， Programmable Buttons • Grove • Raspberry Pi 40-pin Compatible GPIO Wio Terminal
21. Codecraft • Graphical programming platform • Powered by Edge Impulse • Whole TinyML pipeline
23. Step 1 - Initialize Model Creation One-click model creation Step 2 - Data Acquisition Upload data collection program and collect data Step 3 - Training & Deployment Easily adjust parameters to visualize training results Step 4 - Programming One-click deployment to personal programs
24. DEMO
33. Philippine Tarsier 6 inches big and weigh less than half a pound
38. ML Development ML Training Continuous Training Model Deployment Prediction Serving Continuous Monitoring Data, Model Management Code & Config Training Pipeline Serving Logs Convert Model Tensor Flow Lite Model Serving Package Registered Model
39. @rondagdag ML Engineer ML Researcher Data Scientist Data Engineer Software Engineer DevOps Business Analyst
40. ML Expertise BASIC EXPERT ML Engineer ML Researcher Data Scientist Data Engineer Software Engineer DevOps Business Analyst
41. Deployment Expertise BASIC EXPERT ML Engineer ML Researcher Data Scientist Data Engineer Software Engineer DevOps Business Analyst
42. ML Engineer ML Researcher Data Scientist Data Engineer Software Engineer DevOps Business Analyst
43. Summary •What is Machine Learning? •training and inferencing •Embedded Machine Learning (TinyML) •Sensor Data Intelligence •Small, low-powered device •Wio Terminal •Challenges TinyML •Technical Debt, Embedded MLOps @rondagdag
44. Resources • Wio Terminal - Hello World of AI • https://www.seeedstudio.com/wio-terminal-tinyml.html • Codecraft connecting Azure IoT with Wio Terminal • https://wiki.seeedstudio.com/Azure_IoT_CC/ • TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-low-power Microcontrollers • https://tinymlbook.com/ • The Future of ML is Tiny and Bright • https://www.edx.org/professional-certificate/harvardx-tiny-machine-learning @rondagdag
45. @rondagdag https://speakerdeck.com/rondagdag/machine-learning-in-small-low-powered-devices
46. Miniature Pencil Lead Sculptures By Salavat Fidai
47. About Me Ron Dagdag @rondagdag Director of Software Engineering at SpaceE 6th year Microsoft MVP awardee www.dagdag.net ron@dagdag.net @rondagdag Linked In www.linkedin.com/in/rondagdag/ www.dagdag.net https://linktr.ee/rondagdag
48. Dag Dag Goodbye What did you learn that’s new?