Machine Learning in small, low-powered devices

Slide 1

Slide 1 text

Machine Learning in small, low-powered devices (TinyML) Ron Dagdag

Slide 2

Slide 2 text

Agenda •What is Machine Learning? •Embedded Machine Learning (TinyML) •Small, low-powered device •Challenges TinyML •Demo @rondagdag

Slide 3

Slide 3 text

world's tiniest chameleon @rondagdag

Slide 4

Slide 4 text

programming @rondagdag algorithm input answers

Slide 5

Slide 5 text

machine learning @rondagdag algorithm input answers

Slide 6

Slide 6 text

algorithm input answers algorithm input answers ML Primer Programming machine Learning Training Data Model Model Inferencing Training Framework Runtime

Slide 7

Slide 7 text

Anja Markiewicz - Artist that Creates Tiny Sculptures from Paper

Slide 8

Slide 8 text

Applications of Machine Learning 9

Slide 9

Slide 9 text

This Photo by Unknown Author is licensed under CC BY-NC-ND

Slide 10

Slide 10 text

11 Cloud Mobile

Slide 11

Slide 11 text

12 Mobile Google Assistant

Slide 12

Slide 12 text

Google Assistant nest 13

Slide 13

Slide 13 text

Google Assistant nest 14 IoT 1.0: Internet of Things

Slide 14

Slide 14 text

Google Assistant nest 15 IoT 2.0: Intelligence on Things

Slide 15

Slide 15 text

nest 16 IoT 2.0: Intelligence on Things Bandwidth Reliability Latency Privacy Energy

Slide 16

Slide 16 text

Motion Sensors Gyroscope, radar, magnetometer, accelerator Environmental Sensors Temperature, Humidity, Pressure, IR, etc. Touchscreen Sensors Capacitive, IR Image Sensors Thermal, Image Biometric Sensors Fingerprint, Heart rate, etc. TinyML Is All About Sensor Data Intelligence 17 Rotation Sensors Encoders Force Sensors Pressure, Strain Acoustic Sensors Ultrasonic, Microphones, Geophones, Vibrometers ...

Slide 17

Slide 17 text

Painting Miniatures By Hasan Kale

Slide 18

Slide 18 text

Keyword Spotting Visual Wake Words Anomaly Detection Image Classification

Slide 19

Slide 19 text

Board MCU / ASIC Clock Memory Sensors Radio Himax WE-I Plus EVB HX6537-A 32-bit EM9D DSP 400 MHz 2MB flash 2MB RAM Accelerometer, Mic, Camera None Arduino Nano 33 BLE Sense 32-bit nRF52840 64 MHz 1MB flash 256kB RAM Mic, IMU, Temp, Humidity, Gesture, Pressure, Proximity, Brightness, Color BLE SparkFun Edge 2 32-bit ArtemisV1 48 MHz 1MB flash 384kB RAM Accelerometer, Mic, Camera BLE Espressif EYE 32-bit ESP32-D0WD 240 MHz 4MB flash 520kB RAM Mic, Camera WiFi, BLE 20

Slide 20

Slide 20 text

• ARM Cortex-M4F core running at 120MHz(Boost up to 200MHz) • 4 MB External Flash, 192 KB RAM • WIFI，BT • LCD screen • Built-in Modules： • Accelerometer • Microphone • Speaker • Light Sensor • Infrared Emitter • MicroSD Card Slot，5-Way Switch， Programmable Buttons • Grove • Raspberry Pi 40-pin Compatible GPIO Wio Terminal

Slide 21

Slide 21 text

Codecraft • Graphical programming platform • Powered by Edge Impulse • Whole TinyML pipeline

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Step 1 - Initialize Model Creation One-click model creation Step 2 - Data Acquisition Upload data collection program and collect data Step 3 - Training & Deployment Easily adjust parameters to visualize training results Step 4 - Programming One-click deployment to personal programs

Slide 24

Slide 24 text

DEMO

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

Philippine Tarsier 6 inches big and weigh less than half a pound

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

ML Development ML Training Continuous Training Model Deployment Prediction Serving Continuous Monitoring Data, Model Management Code & Config Training Pipeline Serving Logs Convert Model Tensor Flow Lite Model Serving Package Registered Model

Slide 39

Slide 39 text

@rondagdag ML Engineer ML Researcher Data Scientist Data Engineer Software Engineer DevOps Business Analyst

Slide 40

Slide 40 text

ML Expertise BASIC EXPERT ML Engineer ML Researcher Data Scientist Data Engineer Software Engineer DevOps Business Analyst

Slide 41

Slide 41 text

Deployment Expertise BASIC EXPERT ML Engineer ML Researcher Data Scientist Data Engineer Software Engineer DevOps Business Analyst

Slide 42

Slide 42 text

ML Engineer ML Researcher Data Scientist Data Engineer Software Engineer DevOps Business Analyst

Slide 43

Slide 43 text

Summary •What is Machine Learning? •training and inferencing •Embedded Machine Learning (TinyML) •Sensor Data Intelligence •Small, low-powered device •Wio Terminal •Challenges TinyML •Technical Debt, Embedded MLOps @rondagdag

Slide 44

Slide 44 text

Resources • Wio Terminal - Hello World of AI • https://www.seeedstudio.com/wio-terminal-tinyml.html • Codecraft connecting Azure IoT with Wio Terminal • https://wiki.seeedstudio.com/Azure_IoT_CC/ • TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-low-power Microcontrollers • https://tinymlbook.com/ • The Future of ML is Tiny and Bright • https://www.edx.org/professional-certificate/harvardx-tiny-machine-learning @rondagdag

Slide 45

Slide 45 text

@rondagdag https://speakerdeck.com/rondagdag/machine-learning-in-small-low-powered-devices

Slide 46

Slide 46 text

Miniature Pencil Lead Sculptures By Salavat Fidai

Slide 47

Slide 47 text

About Me Ron Dagdag @rondagdag Director of Software Engineering at SpaceE 6th year Microsoft MVP awardee www.dagdag.net [email protected] @rondagdag Linked In www.linkedin.com/in/rondagdag/ www.dagdag.net https://linktr.ee/rondagdag

Slide 48

Slide 48 text

Dag Dag Goodbye What did you learn that’s new?