Machine Learning on the Web

MA HINE LEARNING ON THE WEB @galuhsahid https://galuh.me

Halo! Saya Galuh. @galuhsahid https://galuh.me Data Engineer @ Gojek Mozilla
Te h Speaker o-host pod ast KartiniTeknologi.id

Pembelajaran mesin memberikan komputer kemampuan untuk belajar tanpa harus diprogram
se ara eksplisit Arthur Samuel (1959)

Data: Gambar https://www.bbc.com/news/world-asia-china-37273337 https://www.thoughtco.com/thmb/DpVtY--UXVOtG68m9LhMY7be2w0=/768x0/ﬁlters:no_upscale():max_bytes(150000):strip_icc()/happy-red-panda-171399380-5b574325c9e77c005b690b41.jpg

Data: Suara ommon Voi e

Data: Teks Wikipedia Berita Media Sosial

Data Satu Data Indonesia

Fitur ﬁtur- ﬁtur

Fitur target

Model Representasi matematis dari sebuah proses di dunia nyata

Model Representasi matematis dari sebuah proses di dunia nyata 2000*jumlah
lantai + … + 5000*jumlah kamar = harga rumah error=0.2

Model Representasi matematis dari sebuah proses di dunia nyata 2000*jumlah
lantai + … + 5000*jumlah kamar = harga rumah 3000*jumlah lantai + … + 6000*jumlah kamar = harga rumah error=0.2 error=0.1

Ma hine learning dengan JavaS ript?

“A WebGL a elerated JavaS ript library for training and
deploying ML models.” TensorFlow.js Ma hine learning dengan JavaS ript?

Tanpa instalasi

Tanpa instalasi Interaktif

Tanpa instalasi Interaktif Data tetap ada di lient

Loh, kok bisa?

Tensors Layers A tivation

Ma hine learning dengan JavaS ript?

“Friendly ma hine learning for the web.” Ml5js Ma hine
learning dengan JavaS ript?

Menjalankan model yang sudah dilatih sebelumnya 1

#1: Klasiﬁkasi Gambar

MobileNet • Model computer vision yang bersifat mobile-ﬁrst • Berukuran
kecil, low-latency, dan low-power

MobileNet

Klasiﬁkasi Gambar dengan MobileNet

Klasiﬁkasi Gambar dengan MobileNet MobileNet

MobileNet Prediction: Bicycle (0.7845) Klasiﬁkasi Gambar dengan MobileNet

<html> <head> <meta charset="UTF-8"> <title>Klasifikasi Gambar dengan ml5.js</title> <script src="https://unpkg.com/[email protected]/dist/
ml5.min.js"></script> </head> <body> … <script src="sketch.js"></script> </body> </html> index.html

sket h.js let classifier; const loadModel = async (modelName) =>
{ try { classifier = await ml5.imageClassifier(modelName); } catch (e) { console.error(e); } … };

sket h.js let classifier; const loadModel = async (modelName) =>
{ try { classifier = await ml5.imageClassifier(modelName); } catch (e) { console.error(e); } … }; Darknet Darknet-tiny MobileNet

sket h.js const image = document.getElementById("image"); … const getPrediction =
async (input) => { try { let results = await classifier.classify(input); return results; } catch (e) { console.error(e); } };

sket h.js const image = document.getElementById(“image"); … const getPrediction =
async (input, classifier) => { try { let results = await classifier.classify(input); return results; } catch (e) { console.error(e); } };

sket h.js const startWebcam = async () => { try
{ stream = await navigator.mediaDevices.getUserMedia( { video: true }); video.srcObject = stream; video.play(); getPrediction(video); } catch (e) { console.error(e); } };

sket h.js const startWebcam = async () => { try
{ stream = await navigator.mediaDevices.getUserMedia( { video: true }); video.srcObject = stream; video.play(); getPrediction(video); } catch (e) { console.error(e); } }; MediaDevi es.getUserMedia()

MobileNet

Darknet MobileNet

Menjalankan model yang sudah dilatih sebelumnya 1 Melatih ulang model
yang sudah ada 2

Melatih ulang model MobileNet ?

Melatih dari awal? 14 juta gambar

Transfer learning MobileNet yang dilatih ulang Prediction: Angklung (0.873)

#2: Ro k Paper S issors

index.html <html> … <img src="images/rock.svg" alt="Rock" width="100" height="100" /><br />
<p><span id="amountOfRockImages">0</ span> Rock Images</p> <p><button id="RockButton" class="button">Add Rock</button></p> </html>

index.html <html> … <img src="images/rock.svg" alt="Rock" width="100" height="100" /><br />
<p><span id="amountOfRockImages">0</ span> Rock Images</p> <p><button id="RockButton" class="button">Add Rock</button></p> … <button id="train" class="button">Train Model</button> </html>

sket h.js let classifier; const loadModel = async () =>
{ try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } };

{ try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } }; Jenis model

{ try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } }; Input

{ try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } }; jumlah kelas

{ try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; MobileNet Bicycle (0.373) Cup (0.232) Vacuum (0.121)

{ try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; MobileNet Bicycle (0.373) Cup (0.232) Vacuum (0.121) x

{ try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; MobileNet Rock (0.923) Paper (0.422) Scissors (0.231)

{ try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; jumlah kelas https://www.mathworks.com/solutions/deep-learning/convolutional-neural-network.html

sket h.js var RockButton = document.getElementById(“RockButton") … RockButton.onclick = function()
{ classifier.addImage(video, ”Rock”); amountOfRockImages.innerText = Number(amountOfRockImages.i nnerText) + 1; };

sket h.js const trainModel = async () => { try
{ await classifier.train(function(lossValue) { if (lossValue) { totalLoss = lossValue; loss.innerHTML = "Loss: " + totalLoss; } else { loss.innerHTML = "Training done! Final loss: " + tot alLoss; } }); } catch(e) { console.error(e); } };

sket h.js const getPlayerResult = async video => { try
{ let results = await classifier.classify(video); if (results && results[0]) { return results[0].label; } } catch (e) { console.error(err); } };

#3: S roll with Your Voi e

Klasiﬁkasi suara: Dataset Spee h ommands Dataset Yes No Up
Down Left Right Go Stop 0-9 Unknown word Ba kground noise

Klasiﬁkasi suara: Arsitektur Paper

Klasiﬁkasi suara NN Prediction: Up (0.86) Sound icon by Amiryshakiel
from the Noun Project https://www.tensorﬂow.org/tutorials/sequences/audio_recognition

sket h.js const options = { probabilityThreshold: 0.98 }; …
const loadModel = async() => { try { classifier = await ml5.soundClassifier('SpeechCommands18 w', options); } catch (e) { console.error(e); } };

sket h.js const getPrediction = async() => { try {
await classifier.classify(scrollPage); } catch (e) { console.error(e); } };

sket h.js function scrollPage(error, results) { if (results[0].label == 'down')
{ lastPosition = lastPosition + scrollBy; } else if (results[0].label == 'up') { lastPosition = lastPosition - scrollBy; } lastPosition = Math.max(0, lastPosition); window.scroll(0, lastPosition); };

Selanjutnya https://ml5js.org/reference/

Selanjutnya • A Beginner’s Guide to Machine Learning in JavaScript
oleh Daniel Shiﬀman • Magenta.js (generating music & art) • Kursus Coursera: Machine Learning oleh Andrew Ng • Beyond the web: machine learning + hardware • Source code ml5.js • Bias & etika pembelajaran mesin • Referensi buku: Weapons of Math Destruction oleh Cathy O’Neil

Terima kasih! @galuhsahid https://galuh.me

Machine Learning on the Web

Machine Learning on the Web

More Decks by Galuh Sahid

Other Decks in Programming

Featured

Transcript