DevFest - Machine Learning on the Web

Machine Learning on the Web DevFest SG - November 30,
2019

Hi! I’m Galuh Twitter: @galuhsahid https://galuh.me

Machine learning gives computers the ability to learn without being
explicitly programmed. - Arthur Samuel (1959)

Computer Rules Output Data

Computer Data Output Rules Computer Rules Output Data

Ingredients of machine learning

Data: Pictures https://www.rainforest-alliance.org/sites/default/ﬁles/styles/750w_585h/public/2016-09/three-toed-sloth.jpg?itok=uWF-NdZZ https://www.thoughtco.com/thmb/DpVtY--UXVOtG68m9LhMY7be2w0=/768x0/ﬁlters:no_upscale():max_bytes(150000):strip_icc()/happy-red-panda-171399380-5b574325c9e77c005b690b41.jpg

Data: Voice

Data: Text

Features

Mathematical representation of a real life process. Model 2000*number of
floors + … + 5000*number of rooms = house price

2000*number of floors + … + 5000*number of rooms =
house price Model error = $5400

house price Model error = $5400 3000*number of floors + … + 5000*number of rooms = house price error = $4000

house price Model error = $5400 3000*number of floors + … + 5000*number of rooms = house price error = $4000 3000*number of floors + … + 2000*number of rooms = house price error = $2500

Machine learning on the web?

Machine learning on the web? “A WebGL accelerated JavaScript library
for training and deploying ML models.”

Why TensorFlow.js? Browser: !Without installation

Why TensorFlow.js? Browser: !Without installation !Interactive

Augmented Reality Speech Recognition Accessibility Education Conversational AI

Why TensorFlow.js? Browser: !Without installation !Interactive !Data stays in the
client

Why TensorFlow.js? Browser: !Without installation !Interactive !Data stays in the
client Server: !Nicely integrates with existing node.js stack

How come?

Machine learning on the web? A machine learning library for
JavaScript built on top of TensorFlow.js

Running pre-trained models

Image classification app

Input: file upload

MobileNet Mobile-first computer vision model Small, low-latency, low-power

Image classification with MobileNet MobileNet Prediction: Bicycle (0.7845)

https://teachablemachine.withgoogle.com/

Input: webcam

MobileNet

MobileNet DarkNet

Retraining existing models

New data MobileNet Prediction: ? Prediction: ?

Should we train from scratch? 14 million images

Transfer learning MobileNet trained on a new task Prediction: Angklung
(0.873)

Rock Paper Scissors

Rock Paper Scissors: Demo

Transfer learning MobileNet Bicycle (0.373) Cup (0.232) Vacuum (0.121)

Transfer learning MobileNet Bicycle (0.373) Cup (0.232) Vacuum (0.121) x

Transfer learning MobileNet Rock (0.923) Paper (0.422) Scissors (0.231)

Transfer learning Credit: MathWorks

Scroll with Your Voice

Scroll with Your Voice: Demo

Sound Classification: Data

Sound Classification: Model

Sound Classification: Model CNN Prediction: Up (0.86)

… and many more https://ml5js.org/reference/

Building new models from scratch

index.html <html> <head> <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs/dist/ tf.min.js"></script> </head> <body> <div id="output_field"></div>
</body> … Credit: Laurence Moroney

index.html … <script> async function learnLinear(){ const model = tf.sequential();
model.add(tf.layers.dense({units: 1, inputShape: [1]})); } learnLinear(); </script> Credit: Laurence Moroney

model.add(tf.layers.dense({units: 1, inputShape: [1]})); model.compile({ loss: 'meanSquaredError', optimizer: 'sgd' }); } learnLinear(); </script> Credit: Laurence Moroney

model.add(tf.layers.dense({units: 1, inputShape: [1]})); model.compile({ loss: 'meanSquaredError', optimizer: 'sgd' }); const xs = tf.tensor2d([-1, 0, 1, 2, 3, 4], [6, 1]); const ys = tf.tensor2d([-3, -1, 1, 3, 5, 7], [6, 1]); } learnLinear(); </script> Credit: Laurence Moroney

model.add(tf.layers.dense({units: 1, inputShape: [1]})); model.compile({ loss: 'meanSquaredError', optimizer: 'sgd' }); const xs = tf.tensor2d([-1, 0, 1, 2, 3, 4], [6, 1]); const ys = tf.tensor2d([-3, -1, 1, 3, 5, 7], [6, 1]); } learnLinear(); </script> Credit: Laurence Moroney y=2x-1

model.add(tf.layers.dense({units: 1, inputShape: [1]})); model.compile({ loss: 'meanSquaredError', optimizer: 'sgd' }); const xs = tf.tensor2d([-1, 0, 1, 2, 3, 4], [6, 1]); const ys = tf.tensor2d([-3, -1, 1, 3, 5, 7], [6, 1]); await model.fit(xs, ys, { epochs: 500 }); } learnLinear(); Credit: Laurence Moroney

index.html … model.compile({ loss: 'meanSquaredError', optimizer: 'sgd' }); const xs
= tf.tensor2d([-1, 0, 1, 2, 3, 4], [6, 1]); const ys = tf.tensor2d([-3, -1, 1, 3, 5, 7], [6, 1]); document.getElementById("output_field").innerText = model.predict( tf.tensor2d([10], [1, 1]) ); } learnLinear(); </script> </html> Credit: Laurence Moroney y=2x-1 2(10)-1 = 19

What’s next? ! A Beginner’s Guide to Machine Learning in
JavaScript by Daniel Shiffman ! Magenta.js - music & art with machine learning ! Coursera: Machine Learning by Andrew Ng ! Machine learning + hardware ! ml5.js source code ! Bias in machine learning - book reference: Weapons of Math Destruction oleh Cathy O’Neil

Thank you!

DevFest - Machine Learning on the Web

DevFest - Machine Learning on the Web

More Decks by Galuh Sahid

Other Decks in Technology

Featured

Transcript