Slide 1
Slide 1 text
MA HINE
LEARNING
ON THE WEB
@galuhsahid
https://galuh.me
Slide 2
Slide 2 text
Halo! Saya
Galuh.
@galuhsahid
https://galuh.me
Data Engineer @ Gojek
Mozilla Te h Speaker
o-host pod ast KartiniTeknologi.id
Slide 3
Slide 3 text
Pembelajaran mesin memberikan
komputer kemampuan untuk
belajar tanpa harus diprogram
se ara eksplisit
Arthur Samuel (1959)
Slide 4
Slide 4 text
Data: Gambar
https://www.bbc.com/news/world-asia-china-37273337
https://www.thoughtco.com/thmb/DpVtY--UXVOtG68m9LhMY7be2w0=/768x0/filters:no_upscale():max_bytes(150000):strip_icc()/happy-red-panda-171399380-5b574325c9e77c005b690b41.jpg
Slide 5
Slide 5 text
Data: Suara
ommon Voi e
Slide 6
Slide 6 text
Data: Teks
Wikipedia
Berita
Media Sosial
Slide 7
Slide 7 text
Data
Satu Data Indonesia
Slide 8
Slide 8 text
Fitur
Slide 9
Slide 9 text
Fitur fitur-
fitur
Slide 10
Slide 10 text
Fitur target
Slide 11
Slide 11 text
Model
Representasi matematis dari
sebuah proses di dunia nyata
Slide 12
Slide 12 text
Model
Representasi matematis dari sebuah
proses di dunia nyata
2000*jumlah lantai + … + 5000*jumlah
kamar = harga rumah
error=0.2
Slide 13
Slide 13 text
Model
Representasi matematis dari sebuah
proses di dunia nyata
2000*jumlah lantai + … + 5000*jumlah
kamar = harga rumah
3000*jumlah lantai + … + 6000*jumlah
kamar = harga rumah
error=0.2
error=0.1
Slide 14
Slide 14 text
Ma hine
learning
dengan
JavaS ript?
Slide 15
Slide 15 text
Ma hine
learning
dengan
JavaS ript?
Slide 16
Slide 16 text
“A WebGL a elerated
JavaS ript library for training
and deploying ML models.”
TensorFlow.js
Ma hine
learning
dengan
JavaS ript?
Slide 17
Slide 17 text
Tanpa
instalasi
Slide 18
Slide 18 text
Tanpa
instalasi
Interaktif
Slide 19
Slide 19 text
Tanpa
instalasi
Interaktif
Data tetap
ada di lient
Slide 20
Slide 20 text
Loh, kok
bisa?
Slide 21
Slide 21 text
Loh, kok
bisa?
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
Tensors
Layers
A tivation
Slide 24
Slide 24 text
Ma hine
learning
dengan
JavaS ript?
Slide 25
Slide 25 text
Ma hine
learning
dengan
JavaS ript?
Slide 26
Slide 26 text
“Friendly ma hine learning for
the web.”
Ml5js
Ma hine
learning
dengan
JavaS ript?
Slide 27
Slide 27 text
Menjalankan model
yang sudah dilatih
sebelumnya
1
Slide 28
Slide 28 text
#1: Klasifikasi Gambar
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
MobileNet • Model computer vision yang bersifat mobile-first
• Berukuran kecil, low-latency, dan low-power
Slide 31
Slide 31 text
MobileNet
Slide 32
Slide 32 text
Klasifikasi Gambar dengan
MobileNet
Slide 33
Slide 33 text
Klasifikasi Gambar dengan
MobileNet
MobileNet
Slide 34
Slide 34 text
MobileNet
Prediction:
Bicycle (0.7845)
Klasifikasi Gambar dengan
MobileNet
Slide 35
Slide 35 text
Klasifikasi Gambar dengan ml5.js
…
index.html
Slide 36
Slide 36 text
sket h.js
let classifier;
const loadModel = async (modelName) => {
try {
classifier = await ml5.imageClassifier(modelName);
} catch (e) {
console.error(e);
}
…
};
Slide 37
Slide 37 text
sket h.js
let classifier;
const loadModel = async (modelName) => {
try {
classifier = await ml5.imageClassifier(modelName);
} catch (e) {
console.error(e);
}
…
};
Darknet
Darknet-tiny
MobileNet
Slide 38
Slide 38 text
sket h.js
const image = document.getElementById("image");
…
const getPrediction = async (input) => {
try {
let results = await classifier.classify(input);
return results;
} catch (e) {
console.error(e);
}
};
Slide 39
Slide 39 text
sket h.js
const image = document.getElementById(“image");
…
const getPrediction = async (input, classifier) => {
try {
let results = await classifier.classify(input);
return results;
} catch (e) {
console.error(e);
}
};
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
sket h.js
const startWebcam = async () => {
try {
stream = await navigator.mediaDevices.getUserMedia(
{ video: true });
video.srcObject = stream;
video.play();
getPrediction(video);
} catch (e) {
console.error(e);
}
};
Slide 42
Slide 42 text
sket h.js
const startWebcam = async () => {
try {
stream = await navigator.mediaDevices.getUserMedia(
{ video: true });
video.srcObject = stream;
video.play();
getPrediction(video);
} catch (e) {
console.error(e);
}
};
MediaDevi es.getUserMedia()
Slide 43
Slide 43 text
MobileNet
Slide 44
Slide 44 text
Darknet
MobileNet
Slide 45
Slide 45 text
Menjalankan model
yang sudah dilatih
sebelumnya
1
Melatih ulang
model yang sudah
ada
2
Slide 46
Slide 46 text
Melatih ulang
model
MobileNet ?
Slide 47
Slide 47 text
Melatih dari
awal?
14 juta gambar
Slide 48
Slide 48 text
Transfer learning
MobileNet
yang
dilatih
ulang
Prediction:
Angklung (0.873)
Slide 49
Slide 49 text
#2: Ro k Paper S issors
Slide 50
Slide 50 text
No content
Slide 51
Slide 51 text
index.html
…
0 Rock Images
Add Rock
Slide 52
Slide 52 text
index.html
…
0 Rock Images
Add Rock
Slide 53
Slide 53 text
index.html
…
0 Rock Images
Add Rock
… Train ModelSlide 54
Slide 54 text
sket h.js
let classifier;
const loadModel = async () => {
try {
featureExtractor = await
ml5.featureExtractor("MobileNet");
classifier = await
featureExtractor.classification(video, { numLabels: 3 });
} catch (e) {
console.error(e);
}
};
Slide 55
Slide 55 text
sket h.js
let classifier;
const loadModel = async () => {
try {
featureExtractor = await
ml5.featureExtractor("MobileNet");
classifier = await
featureExtractor.classification(video, { numLabels: 3 });
} catch (e) {
console.error(e);
}
};
Jenis model
Slide 56
Slide 56 text
sket h.js
let classifier;
const loadModel = async () => {
try {
featureExtractor = await
ml5.featureExtractor("MobileNet");
classifier = await
featureExtractor.classification(video,
{ numLabels: 3 });
} catch (e) {
console.error(e);
}
};
Input
Slide 57
Slide 57 text
sket h.js
let classifier;
const loadModel = async () => {
try {
featureExtractor = await
ml5.featureExtractor("MobileNet");
classifier = await
featureExtractor.classification(video,
{ numLabels: 3 });
} catch (e) {
console.error(e);
}
};
jumlah kelas
Slide 58
Slide 58 text
sket h.js
let classifier;
const loadModel = async () => {
try {
featureExtractor = ml5.featureExtractor("MobileNet");
classifier = featureExtractor.classification(video, { nu
mLabels: 3 });
} catch (e) {
console.error(e);
}
};
MobileNet
Bicycle (0.373)
Cup (0.232)
Vacuum (0.121)
Slide 59
Slide 59 text
sket h.js
let classifier;
const loadModel = async () => {
try {
featureExtractor = ml5.featureExtractor("MobileNet");
classifier = featureExtractor.classification(video, { nu
mLabels: 3 });
} catch (e) {
console.error(e);
}
};
MobileNet
Bicycle (0.373)
Cup (0.232)
Vacuum (0.121)
x
Slide 60
Slide 60 text
sket h.js
let classifier;
const loadModel = async () => {
try {
featureExtractor = ml5.featureExtractor("MobileNet");
classifier = featureExtractor.classification(video, { nu
mLabels: 3 });
} catch (e) {
console.error(e);
}
};
MobileNet
Rock (0.923)
Paper (0.422)
Scissors (0.231)
Slide 61
Slide 61 text
sket h.js
let classifier;
const loadModel = async () => {
try {
featureExtractor = ml5.featureExtractor("MobileNet");
classifier = featureExtractor.classification(video, { nu
mLabels: 3 });
} catch (e) {
console.error(e);
}
};
jumlah kelas
https://www.mathworks.com/solutions/deep-learning/convolutional-neural-network.html
Slide 62
Slide 62 text
sket h.js
var RockButton = document.getElementById(“RockButton")
…
RockButton.onclick = function() {
classifier.addImage(video, ”Rock”);
amountOfRockImages.innerText = Number(amountOfRockImages.i
nnerText) + 1;
};
Slide 63
Slide 63 text
sket h.js
const trainModel = async () => {
try {
await classifier.train(function(lossValue) {
if (lossValue) {
totalLoss = lossValue;
loss.innerHTML = "Loss: " + totalLoss;
} else {
loss.innerHTML = "Training done! Final loss: " + tot
alLoss;
}
});
} catch(e) {
console.error(e);
}
};
Slide 64
Slide 64 text
sket h.js
const trainModel = async () => {
try {
await classifier.train(function(lossValue) {
if (lossValue) {
totalLoss = lossValue;
loss.innerHTML = "Loss: " + totalLoss;
} else {
loss.innerHTML = "Training done! Final loss: " + tot
alLoss;
}
});
} catch(e) {
console.error(e);
}
};
Slide 65
Slide 65 text
sket h.js
const getPlayerResult = async video => {
try {
let results = await classifier.classify(video);
if (results && results[0]) {
return results[0].label;
}
} catch (e) {
console.error(err);
}
};
Slide 66
Slide 66 text
sket h.js
const getPlayerResult = async video => {
try {
let results = await classifier.classify(video);
if (results && results[0]) {
return results[0].label;
}
} catch (e) {
console.error(err);
}
};
Slide 67
Slide 67 text
No content
Slide 68
Slide 68 text
#3: S roll with Your Voi e
Slide 69
Slide 69 text
No content
Slide 70
Slide 70 text
Klasifikasi
suara: Dataset
Spee h ommands
Dataset
Yes
No
Up
Down
Left
Right
Go
Stop
0-9
Unknown word
Ba kground noise
Slide 71
Slide 71 text
Klasifikasi
suara:
Arsitektur
Paper
Slide 72
Slide 72 text
Klasifikasi suara
NN
Prediction:
Up (0.86)
Sound icon by Amiryshakiel from the Noun Project
https://www.tensorflow.org/tutorials/sequences/audio_recognition
Slide 73
Slide 73 text
sket h.js
const options = { probabilityThreshold: 0.98 };
…
const loadModel = async() => {
try {
classifier = await ml5.soundClassifier('SpeechCommands18
w', options);
} catch (e) {
console.error(e);
}
};
Slide 74
Slide 74 text
sket h.js
const getPrediction = async() => {
try {
await classifier.classify(scrollPage);
} catch (e) {
console.error(e);
}
};
Slide 75
Slide 75 text
sket h.js
function scrollPage(error, results) {
if (results[0].label == 'down') {
lastPosition = lastPosition + scrollBy;
} else if (results[0].label == 'up') {
lastPosition = lastPosition - scrollBy;
}
lastPosition = Math.max(0, lastPosition);
window.scroll(0, lastPosition);
};
Slide 76
Slide 76 text
Selanjutnya
https://ml5js.org/reference/
Slide 77
Slide 77 text
Selanjutnya
• A Beginner’s Guide to Machine Learning in JavaScript oleh
Daniel Shiffman
• Magenta.js (generating music & art)
• Kursus Coursera: Machine Learning oleh Andrew Ng
• Beyond the web: machine learning + hardware
• Source code ml5.js
• Bias & etika pembelajaran mesin
• Referensi buku: Weapons of Math Destruction oleh Cathy
O’Neil
Slide 78
Slide 78 text
Terima kasih!
@galuhsahid
https://galuh.me