Slide 1
Slide 1 text
MACHINE LEARNING & THE IOT: LIVING ON THE EDGE
BRANDON SATROM | [email protected]
Slide 2
Slide 2 text
No content
Slide 3
Slide 3 text
Zeke
Slide 4
Slide 4 text
Zeke
Slide 5
Slide 5 text
To “Zeke”
Slide 6
Slide 6 text
To “Zeke”
Slide 7
Slide 7 text
Zeked!
Slide 8
Slide 8 text
Zeked!
Slide 9
Slide 9 text
Zeked!
Slide 10
Slide 10 text
Zeked!
Slide 11
Slide 11 text
Zeked!
Slide 12
Slide 12 text
Zeked!
Slide 13
Slide 13 text
Zeked!
Slide 14
Slide 14 text
Zeked!
Slide 15
Slide 15 text
Zeked!
Slide 16
Slide 16 text
No content
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
No content
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
No content
Slide 24
Slide 24 text
No content
Slide 25
Slide 25 text
No content
Slide 26
Slide 26 text
No content
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
2 ESSENTIAL TRUTHS OF MACHINE LEARNING
1 2
DATA DOESN’T COME FROM THE CLOUD MACHINE LEARNING IS HUMAN
Slide 31
Slide 31 text
2 ESSENTIAL TRUTHS OF MACHINE LEARNING
1 2
DATA DOESN’T COME FROM THE CLOUD MACHINE LEARNING IS HUMAN
Slide 32
Slide 32 text
2 ESSENTIAL TRUTHS OF MACHINE LEARNING
1 2
DATA DOESN’T COME FROM THE CLOUD MACHINE LEARNING IS HUMAN
Slide 33
Slide 33 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 34
Slide 34 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 35
Slide 35 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 36
Slide 36 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 37
Slide 37 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 38
Slide 38 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 39
Slide 39 text
AI, MACHINE LEARNING & DEEP LEARNING
AI
Machine learning
Representation learning
Deep learning
Example:
Knowledge
bases
Example:
Logistic
regression
Example:
Shallow
autoencoders
Example:
MLPs
Slide 40
Slide 40 text
ALGORITHMS VS. PREDICITONS
Slide 41
Slide 41 text
ALGORITHMS VS. PREDICITONS
“Learning” Happens Here
Slide 42
Slide 42 text
DEEP LEARNING: FINDING CORRELATION AMONG COMPLEX DATA
Slide 43
Slide 43 text
DEEP LEARNING AS FEATURE EXTRACTION
Slide 44
Slide 44 text
THE STREETLIGHT PROBLEM
Examples and images from “Grokking Deep Learning” by Andrew Trask
Slide 45
Slide 45 text
THE STREETLIGHT PROBLEM
Examples and images from “Grokking Deep Learning” by Andrew Trask
Some combination of
these three lights
tells us when its safe
to walk or stop
Slide 46
Slide 46 text
THE STREETLIGHT PROBLEM
Examples and images from “Grokking Deep Learning” by Andrew Trask
Some combination of
these three lights
tells us when its safe
to walk or stop
Slide 47
Slide 47 text
THE STREETLIGHT PROBLEM
Examples and images from “Grokking Deep Learning” by Andrew Trask
Some combination of
these three lights
tells us when its safe
to walk or stop
Slide 48
Slide 48 text
THE STREETLIGHT PROBLEM
Examples and images from “Grokking Deep Learning” by Andrew Trask
Some combination of
these three lights
tells us when its safe
to walk or stop
Slide 49
Slide 49 text
THE STREETLIGHT PROBLEM
Examples and images from “Grokking Deep Learning” by Andrew Trask
Some combination of
these three lights
tells us when its safe
to walk or stop
After a few rounds, a
human can infer that
the middle light is the
key
Slide 50
Slide 50 text
DEEP LEARNING = TEACHING COMPUTERS TO FIND CORRELATION
Examples and images from “Grokking Deep Learning” by Andrew Trask
Slide 51
Slide 51 text
DEEP LEARNING = TEACHING COMPUTERS TO FIND CORRELATION
Examples and images from “Grokking Deep Learning” by Andrew Trask
Slide 52
Slide 52 text
DEEP LEARNING = TEACHING COMPUTERS TO FIND CORRELATION
Examples and images from “Grokking Deep Learning” by Andrew Trask
Slide 53
Slide 53 text
DEEP LEARNING = TEACHING COMPUTERS TO FIND CORRELATION
Examples and images from “Grokking Deep Learning” by Andrew Trask
Slide 54
Slide 54 text
DEEP LEARNING = TEACHING COMPUTERS TO FIND CORRELATION
Examples and images from “Grokking Deep Learning” by Andrew Trask
import numpy as np
weights = np.array([0.5, 0.48, -0.7])
alpha = 0.1
streetlights = np.array([[1, 0, 1],[0, 1, 1],[0, 0, 1],[1, 1, 1],
[0, 1, 1],[1, 0, 1]])
walk_vs_stop = np.array([0, 1, 0, 1, 1, 0])
input = streetlights[0]
goal_prediction = walk_vs_stop[0]
for iteration in range(40):
error_for_all_lights = 0
for row_index in range(len(walk_vs_stop)):
input = streetlights[row_index]
goal_prediction = walk_vs_stop[row_index]
prediction = input.dot(weights)
error = (goal_prediction - prediction) "** 2
error_for_all_lights += error
delta = prediction - goal_prediction
weights = weights - (alpha * (input * delta))
print("Prediction:" + str(prediction))
print("Error:" + str(error_for_all_lights) + "\n")
Slide 55
Slide 55 text
DEEP LEARNING = TEACHING COMPUTERS TO FIND CORRELATION
Examples and images from “Grokking Deep Learning” by Andrew Trask
import numpy as np
weights = np.array([0.5, 0.48, -0.7])
alpha = 0.1
streetlights = np.array([[1, 0, 1],[0, 1, 1],[0, 0, 1],[1, 1, 1],
[0, 1, 1],[1, 0, 1]])
walk_vs_stop = np.array([0, 1, 0, 1, 1, 0])
input = streetlights[0]
goal_prediction = walk_vs_stop[0]
for iteration in range(40):
error_for_all_lights = 0
for row_index in range(len(walk_vs_stop)):
input = streetlights[row_index]
goal_prediction = walk_vs_stop[row_index]
prediction = input.dot(weights)
error = (goal_prediction - prediction) "** 2
error_for_all_lights += error
delta = prediction - goal_prediction
weights = weights - (alpha * (input * delta))
print("Prediction:" + str(prediction))
print("Error:" + str(error_for_all_lights) + "\n")
Slide 56
Slide 56 text
DEEP LEARNING = TEACHING COMPUTERS TO FIND CORRELATION
Examples and images from “Grokking Deep Learning” by Andrew Trask
import numpy as np
weights = np.array([0.5, 0.48, -0.7])
alpha = 0.1
streetlights = np.array([[1, 0, 1],[0, 1, 1],[0, 0, 1],[1, 1, 1],
[0, 1, 1],[1, 0, 1]])
walk_vs_stop = np.array([0, 1, 0, 1, 1, 0])
input = streetlights[0]
goal_prediction = walk_vs_stop[0]
for iteration in range(40):
error_for_all_lights = 0
for row_index in range(len(walk_vs_stop)):
input = streetlights[row_index]
goal_prediction = walk_vs_stop[row_index]
prediction = input.dot(weights)
error = (goal_prediction - prediction) "** 2
error_for_all_lights += error
delta = prediction - goal_prediction
weights = weights - (alpha * (input * delta))
print("Prediction:" + str(prediction))
print("Error:" + str(error_for_all_lights) + "\n")
Prediction: -0.19999999999999996
Prediction: -0.19999999999999996
Prediction: -0.5599999999999999
Prediction: 0.6160000000000001
Prediction: 0.17279999999999995
Prediction: 0.17552
Error: 2.6561231104
…
Prediction: -0.0022410273814405524
Prediction: 0.9978745386023716
Prediction: -0.016721264429884947
Prediction: 1.0151127459893812
Prediction: 0.9969492081270097
Prediction: -0.0026256193329783125
Error:0.00053373677328488
Slide 57
Slide 57 text
DEEP LEARNING = TEACHING COMPUTERS TO FIND CORRELATION
Examples and images from “Grokking Deep Learning” by Andrew Trask
import numpy as np
weights = np.array([0.5, 0.48, -0.7])
alpha = 0.1
streetlights = np.array([[1, 0, 1],[0, 1, 1],[0, 0, 1],[1, 1, 1],
[0, 1, 1],[1, 0, 1]])
walk_vs_stop = np.array([0, 1, 0, 1, 1, 0])
input = streetlights[0]
goal_prediction = walk_vs_stop[0]
for iteration in range(40):
error_for_all_lights = 0
for row_index in range(len(walk_vs_stop)):
input = streetlights[row_index]
goal_prediction = walk_vs_stop[row_index]
prediction = input.dot(weights)
error = (goal_prediction - prediction) "** 2
error_for_all_lights += error
delta = prediction - goal_prediction
weights = weights - (alpha * (input * delta))
print("Prediction:" + str(prediction))
print("Error:" + str(error_for_all_lights) + "\n")
Prediction: -0.19999999999999996
Prediction: -0.19999999999999996
Prediction: -0.5599999999999999
Prediction: 0.6160000000000001
Prediction: 0.17279999999999995
Prediction: 0.17552
Error: 2.6561231104
…
Prediction: -0.0022410273814405524
Prediction: 0.9978745386023716
Prediction: -0.016721264429884947
Prediction: 1.0151127459893812
Prediction: 0.9969492081270097
Prediction: -0.0026256193329783125
Error:0.00053373677328488
Slide 58
Slide 58 text
Machine Learning is Really Human Teaching
Slide 59
Slide 59 text
MACHINE LEARNING IS HUMAN TEACHING
1 2 3
CREATE INPUTS AND
OUTPUTS
DEFINE A MODEL
ARCHITECTURE
TRAIN, EVALUATE, TUNE
CHAPTER 5. MACHINE LEARNING BASICS
100 101 102 103 104 105
Number of training examples
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
Error (MSE)
Bayes error
Train (quadratic)
Test (quadratic)
Test (optimal capacity)
Train (optimal capacity)
10
15
20
ty (polynomial degree)
Slide 60
Slide 60 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 61
Slide 61 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 62
Slide 62 text
MACHINE LEARNING ISN’T “DONE” IN ANY ONE PLACE
Slide 63
Slide 63 text
ML = DEVELOPMENT, TRAINING, INFERENCE
1 2 3
DEVELOPMENT TRAINING INFERENCE
Slide 64
Slide 64 text
MODEL CREATION TOOLS
Slide 65
Slide 65 text
TRAINING, EXPERIMENTS, & TUNING
Slide 66
Slide 66 text
DEMO
TRAINING AN EMOTION MODEL
Slide 67
Slide 67 text
THE TRAIN, PREDICT, TUNE LOOP
Slide 68
Slide 68 text
THE TRAIN, PREDICT, TUNE LOOP
Define Task
Collect Data
Prepare Data
Test & Evaluate
Train Model
Deploy Model
Maintain, Update &
Refine
Tune
& Refine
Explore
Model
Slide 69
Slide 69 text
INFERENCE: PREDICTION IN REAL TIME
Slide 70
Slide 70 text
INFERENCE: PREDICTION IN REAL TIME
During training, the
network “learns” by
tuning weights to
reduce error
Slide 71
Slide 71 text
INFERENCE: PREDICTION IN REAL TIME
During training, the
network “learns” by
tuning weights to
reduce error
These weights are then applied
against data the network has never
seen before to make a prediction
Slide 72
Slide 72 text
INFERENCE: PREDICTION IN REAL TIME
During training, the
network “learns” by
tuning weights to
reduce error
These weights are then applied
against data the network has never
seen before to make a prediction
Slide 73
Slide 73 text
DEMO
RUNNING INFERENCE ON A MODEL
Slide 74
Slide 74 text
MODEL DEPLOYMENT
+
+
+
Slide 75
Slide 75 text
PRE-TRAINED MODELS AS APIS
Slide 76
Slide 76 text
DEMO
USING THE AZURE FACE API
Slide 77
Slide 77 text
LOCAL DEVELOPMENT, CLOUD TRAINING AND DEPLOYMENT
Define Task
Collect Data
Prepare Data
Test & Evaluate
Train Model
Deploy Model
Maintain, Update &
Refine
Tune
& Refine
Explore
Model
Slide 78
Slide 78 text
LOCAL DEVELOPMENT, CLOUD TRAINING AND DEPLOYMENT
Define Task
Collect Data
Prepare Data
Test & Evaluate
Train Model
Deploy Model
Maintain, Update &
Refine
Tune
& Refine
Explore
Model
Local Development
Training & Validation
Data
Slide 79
Slide 79 text
LOCAL DEVELOPMENT, CLOUD TRAINING AND DEPLOYMENT
Define Task
Collect Data
Prepare Data
Test & Evaluate
Train Model
Deploy Model
Maintain, Update &
Refine
Tune
& Refine
Explore
Model
Local Development
Cloud Training &
Deployment
Training & Validation
Data
Slide 80
Slide 80 text
LOCAL DEVELOPMENT, CLOUD TRAINING AND DEPLOYMENT
Define Task
Collect Data
Prepare Data
Test & Evaluate
Train Model
Deploy Model
Maintain, Update &
Refine
Tune
& Refine
Explore
Model
Local Development
Cloud Training &
Deployment
Training & Validation
Data
Real time Inference
Data
?
Slide 81
Slide 81 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 82
Slide 82 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 83
Slide 83 text
TRADITIONAL: DEVICES AND CLOUD
Slide 84
Slide 84 text
TRADITIONAL: DEVICES AND CLOUD
The Cloud
The Internet
of things
Slide 85
Slide 85 text
TRADITIONAL: DEVICES AND CLOUD
The Cloud
The Internet
of things
“Lots of
power”
Slide 86
Slide 86 text
TRADITIONAL: DEVICES AND CLOUD
The Cloud
The Internet
of things
“Lots of
power”
“Lots of
Devices”
Slide 87
Slide 87 text
THE CLOUD, THE “EDGE,” AND THE IOT
The Cloud
The Internet
of things
“Lots of
power”
“Lots of
Devices”
Slide 88
Slide 88 text
THE CLOUD, THE “EDGE,” AND THE IOT
The Cloud
The Internet
of things
The “Edge”
“Lots of
power”
“Lots of
Devices”
Slide 89
Slide 89 text
THE CLOUD, THE “EDGE,” AND THE IOT
The Cloud
The Internet
of things
The “Edge”
“Lots of
power”
“Lots of
Devices”
Slide 90
Slide 90 text
THE CLOUD, THE “EDGE,” AND THE IOT
The Cloud
The Internet
of things
The “Edge”
“Lots of
power”
“Lots of
Devices”
Slide 91
Slide 91 text
THE CLOUD, THE “EDGE,” AND THE IOT
The Cloud
The Internet
of things
The “Edge”
“Lots of
power”
“Lots of
Devices”
Slide 92
Slide 92 text
THE CLOUD, THE “EDGE,” AND THE IOT
The Cloud
The Internet
of things
The “Edge”
“Lots of
power”
“Lots of
Devices”
“A Bit of
both”
Slide 93
Slide 93 text
THE “EDGE” IS THE INTERNET OF THINGS!
The Cloud
The Internet
of things
The “Edge”
Slide 94
Slide 94 text
THE “EDGE” IS THE INTERNET OF THINGS!
The Cloud
The Internet
of things
The “Edge”
Slide 95
Slide 95 text
THE IOT IS THE SOURCE OF DATA FOR ML
Define Task
Collect Data
Prepare Data
Test & Evaluate
Train Model
Deploy Model
Maintain, Update &
Refine
Tune
& Refine
Explore
Model
Local Development
Cloud Training &
Deployment
Training & Validation
Data
Real time Inference
Data
?
Slide 96
Slide 96 text
THE IOT IS THE SOURCE OF DATA FOR ML
Define Task
Collect Data
Prepare Data
Test & Evaluate
Train Model
Deploy Model
Maintain, Update &
Refine
Tune
& Refine
Explore
Model
Local Development
Cloud Training &
Deployment
Training & Validation
Data
Real time Inference
Data
?
Slide 97
Slide 97 text
THE IOT IS THE SOURCE OF DATA FOR ML
Define Task
Collect Data
Prepare Data
Test & Evaluate
Train Model
Deploy Model
Maintain, Update &
Refine
Tune
& Refine
Explore
Model
Local Development
Cloud Training &
Deployment
Training & Validation
Data
Real time Inference
Data
?
Slide 98
Slide 98 text
WHY DO ML INFERENCING IN THE IOT?
1 2 3
VOLUME SPEED PRIVACY
Slide 99
Slide 99 text
MORE DEVICES == MORE DATA
Slide 100
Slide 100 text
No content
Slide 101
Slide 101 text
No content
Slide 102
Slide 102 text
No content
Slide 103
Slide 103 text
The Cloud may be infinite;
your pipe to it is not
Slide 104
Slide 104 text
ML LOVES DATA LIKE COOKIE MONSTER LOVES COOKIES
Slide 105
Slide 105 text
ML LOVES DATA LIKE COOKIE MONSTER LOVES COOKIES
Slide 106
Slide 106 text
DOING INFERENCE IN THE IOT MEANS LESS DATA IN THE CLOUD
Slide 107
Slide 107 text
DOING INFERENCE IN THE IOT MEANS LESS DATA IN THE CLOUD
Slide 108
Slide 108 text
INFERENCE FOR PREVENTATIVE MAINTENANCE
Slide 109
Slide 109 text
INFERENCE FOR PREVENTATIVE MAINTENANCE
Slide 110
Slide 110 text
INFERENCE FOR PREVENTATIVE MAINTENANCE
Slide 111
Slide 111 text
INFERENCE FOR PREVENTATIVE MAINTENANCE
Slide 112
Slide 112 text
INFERENCE FOR PREVENTATIVE MAINTENANCE
Slide 113
Slide 113 text
THERE IS NO CLOUD IF THE NETWORK IS OFFLINE
Slide 114
Slide 114 text
THERE IS NO CLOUD IF THE NETWORK IS OFFLINE
Slide 115
Slide 115 text
INFERENCE FOR PREVENTATIVE MAINTENANCE
Slide 116
Slide 116 text
INFERENCE FOR PREVENTATIVE MAINTENANCE
Slide 117
Slide 117 text
INFERENCE FOR PREVENTATIVE MAINTENANCE
Slide 118
Slide 118 text
IoT Inferencing == Built-In Privacy
Slide 119
Slide 119 text
VISION IN THE CLOUD
Slide 120
Slide 120 text
VISION IN THE CLOUD
Slide 121
Slide 121 text
VISION AT THE EDGE
Slide 122
Slide 122 text
VISION AT THE EDGE
Slide 123
Slide 123 text
VISION AT THE EDGE
[
{
"faceId": "b0f2804a-649f-4da3-b6cf-ff459c867ae8",
"faceRectangle": {
"width": 63,
"left": 168,
"height": 63,
"top": 110
},
"faceAttributes": {
"smile": 0.0,
"exposure": {
"exposureLevel": "overExposure",
"value": 0.75
},
"emotion": {
"sadness": 0.0,
"surprise": 0.0,
"neutral": 0.0,
"contempt": 0.001,
"anger": 0.666,
"disgust": 0.333,
"fear": 0.0,
"happiness": 0.0
}
"facialHair": {
"beard": 0.4,
"moustache": 0.4,
"sideburns": 0.4
},
"gender": "male",
"age": 33.0,
"headPose": {
"yaw": -24.3,
"pitch": -6.4,
"roll": -8.1
}
}
}
]
Slide 124
Slide 124 text
EDGE ML INFERENCING DEVICES
1 2
GOOGLE CORAL NVIDIA JETSON NANO
Slide 125
Slide 125 text
GOOGLE CORAL
• Edge TPU Module (SOM)
• NXP i.MX 8M SOC (Quad-core Cortex-A53, plus Cortex-M4F)
• Google Edge TPU ML accelerator coprocessor
• Wi-Fi 2x2 MIMO (802.11b/g/n/ac 2.4/5GHz)
• Bluetooth 4.1
• 8GB eMMC
• 1GB LPDDR4
• USB connections
• USB Type-C power port (5V DC)
• USB 3.0 Type-C OTG port
• USB 3.0 Type-A host port
• USB 2.0 Micro-B serial console port
• Audio & Video connections
• Digital PDM microphone (x2)
• HDMI 2.0a (full size)
• 39-pin FFC connector for MIPI DSI display (4-lane)
• 24-pin FFC connector for MIPI CSI-2 camera (4-lane)
• MicroSD card slot
• Gigabit Ethernet port
• 40-pin GPIO expansion header
• Supports Mendel Linux (derivative of Debian)
Slide 126
Slide 126 text
NVIDIA JETSON NANO
• GPU and CPU
• 128 Core Maxwell (472 GFLOPs)
• Quad-core Cortex-A57 @ 1.43 GHz
• 4 GB 64 bit LPDDR4 25.6 GB/s
• 16GB eMMC
• Video
• HDMI 2.0 or DP1.2
• 4K Encode and Decode
• 12-pin FFC connector for MIPI CSI-2 camera (1.1-lane)
• Connections
• 1x SDIO, 2x SPI, 5x SysIO, 13x GPIOs, 6x I2C
• USB connections
• 1 USB 3.0
• MicroSD card slot
• Gigabit Ethernet port
• Supports Ubuntu Linux
Slide 127
Slide 127 text
DEMO
FACE DETECTION WITH GOOGLE CORAL
Slide 128
Slide 128 text
Where does this leave the Internet of
Things?
Slide 129
Slide 129 text
THE INCREASING ROLE OF THE IOT IN ML
1 2
COLLECTION &
COORDINATION ON
DEVICES
ML INFERENCING ON
MICROCONTROLLERS!
Slide 130
Slide 130 text
No content
Slide 131
Slide 131 text
ML data doesn’t come from the cloud, or the
edge…
It comes from the IoT
Slide 132
Slide 132 text
IOT DEVICES SENSE AND SHARE
Slide 133
Slide 133 text
IOT DEVICES SENSE AND SHARE
Slide 134
Slide 134 text
IOT DEVICES TAKE ACTION
Slide 135
Slide 135 text
IOT DEVICES TAKE ACTION
Slide 136
Slide 136 text
IOT DEVICES TAKE ACTION
Slide 137
Slide 137 text
IOT DEVICES TAKE ACTION
Slide 138
Slide 138 text
Edge ML + Mesh Networking = Crazy
Delicious
Slide 139
Slide 139 text
TRADITIONAL IOT: HUB AND SPOKE
Wi-Fi Router
Slide 140
Slide 140 text
TRADITIONAL IOT: HUB AND SPOKE
Wi-Fi Router
Slide 141
Slide 141 text
TRADITIONAL IOT: HUB AND SPOKE
Wi-Fi Router
Slide 142
Slide 142 text
TRADITIONAL IOT: HUB AND SPOKE
Slide 143
Slide 143 text
ENTER MESH NETWORKING
Gateway
Slide 144
Slide 144 text
ENTER MESH NETWORKING
Gateway
Slide 145
Slide 145 text
ENTER MESH NETWORKING
Gateway
Slide 146
Slide 146 text
ENTER MESH NETWORKING
Gateway
Slide 147
Slide 147 text
ENTER MESH NETWORKING
Gateway
Slide 148
Slide 148 text
ENTER MESH NETWORKING
Slide 149
Slide 149 text
OPENTHREAD VS. ZIGBEE, ZWAVE AND LORA
Operating range 100 ft 35 ft
Max # of devices 232 65,000
Data rate 9.6-100 kbps 40-250 kbps
Cost $ $$
IP-Based Networking No No
Open Standard? No Yes
1-4 mi 100 ft
N/A 300
27 kbps 250 kbps
$$$ $
Yes Yes
No Yes
Slide 150
Slide 150 text
WHAT IS THREAD?
✴IPv6-based mesh
✴Wireless Personal Area Network
✴No single point of failure
✴Tailored to IoT Scenarios
✴Can be used in concert with Wi-Fi, Cellular and Bluetooth
is a low-power networking protocol
Slide 151
Slide 151 text
PARTICLE MESH DEVICES
Nordic nRF52840 SoC
✴ ARM Cortex-M4F 32-bit
✴ 1MB flash, 256KB RAM
✴ IEEE 802.15.4-2006
✴ Bluetooth 5: 2 Mbps, 1 Mbps, 500
Kbps, 125 Kbps
✴ ARM TrustZone Cryptographic
security module
✴ NFC-A tag
Argon
» Wi-Fi + BLE +Mesh
» Wi-Fi endpoint or mesh gateway
Xenon
» BLE + Mesh
» Mesh endpoint
Boron
» LTE-M1 + BLE + Mesh
» Cellular endpoint or mesh gateway
Slide 152
Slide 152 text
BRINGING AN EDGE DEVICE INTO A MESH NETWORK
The Internet
of things
Slide 153
Slide 153 text
BRINGING AN EDGE DEVICE INTO A MESH NETWORK
The Internet
of things
Slide 154
Slide 154 text
BRINGING AN EDGE DEVICE INTO A MESH NETWORK
The Internet
of things
We can bring an Edge ML
device into an IoT mesh
network
Slide 155
Slide 155 text
DEMO
EMOTION MESH
Slide 156
Slide 156 text
THE FUTURE OF ML AND THE IOT
1 2
ML INFERENCING ON
MICROCONTROLLERS!
FEDERATED AND
DISTRIBUTED LEARNING
Slide 157
Slide 157 text
TENSORFLOW LITE MICRO
Slide 158
Slide 158 text
MICROTENSOR = ML INFERENCING ON ARM-BASED DEVICES
Slide 159
Slide 159 text
MICROTENSOR = ML INFERENCING ON ARM-BASED DEVICES
Slide 160
Slide 160 text
TF LITE + MICROTENSOR == A ROBUST, OPEN APPROACH TO MCU ML INFERENCING
Slide 161
Slide 161 text
DEMO
ML ON MCUS
Slide 162
Slide 162 text
USING FEDERATED LEARNING TO MAXIMIZE IOT COMPUTE RESOURCES
Slide 163
Slide 163 text
USING FEDERATED LEARNING TO MAXIMIZE IOT COMPUTE RESOURCES
IoT Devices spend most
of their time idle…
Slide 164
Slide 164 text
USING FEDERATED LEARNING TO MAXIMIZE IOT COMPUTE RESOURCES
IoT Devices spend most
of their time idle…
Federated learning may enable models to be
distributed between the cloud, edge and devices for
the most efficient processing
Slide 165
Slide 165 text
OTA MODEL UPDATES!
Slide 166
Slide 166 text
OTA MODEL UPDATES!
Slide 167
Slide 167 text
OTA MODEL UPDATES!
Slide 168
Slide 168 text
THREE BIG IDEAS
1 2 3
MACHINE LEARNING
IS HUMAN TEACHING
TRAINING IS MEANT
FOR THE CLOUD
THE REAL WORK OF ML
IS IN THE IOT
Slide 169
Slide 169 text
Explore ML inferencing with SBCs
Slide 170
Slide 170 text
Explore Mesh networking for local network
intelligence
Slide 171
Slide 171 text
Explore TensorFlow Lite and µTensor for ML
on MCUs
Slide 172
Slide 172 text
THANK YOU!
BRANDON SATROM | [email protected]
GITHUB.COM/BSATROM/EMOTION-MESH