Machine Learning on the Web

2b6d7bdd43058e87f53866eb86538a59?s=47 Galuh Sahid
September 28, 2019
Programming
2
190

Machine Learning on the Web

Slide deck for the talk "Machine Learning on the Web" at JSDay ID

GitHub repository: https://github.com/galuhsahid/ml-on-the-web

2b6d7bdd43058e87f53866eb86538a59?s=128

Galuh Sahid

September 28, 2019
Tweet

More Decks by Galuh Sahid

See All by Galuh Sahid
2b6d7bdd43058e87f53866eb86538a59?s=48 galuhsahid
1
120
2b6d7bdd43058e87f53866eb86538a59?s=48 galuhsahid
0
91
2b6d7bdd43058e87f53866eb86538a59?s=48 galuhsahid
0
83
2b6d7bdd43058e87f53866eb86538a59?s=48 galuhsahid
0
79
2b6d7bdd43058e87f53866eb86538a59?s=48 galuhsahid
0
42
2b6d7bdd43058e87f53866eb86538a59?s=48 galuhsahid
0
69
2b6d7bdd43058e87f53866eb86538a59?s=48 galuhsahid
2
160
2b6d7bdd43058e87f53866eb86538a59?s=48 galuhsahid
0
34
2b6d7bdd43058e87f53866eb86538a59?s=48 galuhsahid
0
340

Other Decks in Programming

See All in Programming
C0eb1445cda9489ebf8c31c367fec3fb?s=48 exoego
1
290
40575ebf9c2f4a6e3bcb68630f446a3b?s=48 tomzoh
4
110
66988ef2ff1307f07c7c8d829a6dc4fa?s=48 rosariopfernandes
2
120
264b44c0c6d60336f8b6a0fbab118984?s=48 mmazour
0
130
A513bc1fa519d2fcf5e4e0d9b5500101?s=48 psibi
1
130
9da5d5cc4b6a9f28058152e28364b02a?s=48 prof18
1
120
E834b4b7cfcba6918f01f45d2916c302?s=48 f110
2
150
D79e3b0b78ca7027eb61d73028d45ad6?s=48 uchi_k
2
320
Fccbd625997f63e7b251d9725cb905a5?s=48 futo23
2
570
D8a3623b157508fecdae1f8e756f362f?s=48 cmota
0
110
1102d6e3810fcc652a8b2887d1642ade?s=48 47degacademy
0
200
0b26590a0a8b0f1da140ed5de9b68379?s=48 usamik26
0
160

Featured

See All Featured
168ccec72eee0530b818d44f3fedaacf?s=48 shlominoach
175
7.2k
245cee81a9c424266e5e401d844ea881?s=48 lara
590
60k
465724d73fe3a92c0879fdfb43a3a6f3?s=48 philhawksworth
190
17k
282d599b414022eba5096510f675b6e2?s=48 chrislema
231
16k
48c098b6579220a67a6ba949be6e4b5a?s=48 revolveconf
197
9.5k
D09fad3e36ae6841f54820be0e907f7a?s=48 rocio
155
11k
Fde6c3a50ca518a909ef35c22c0ee0e4?s=48 destraynor
145
19k
56c4053438af8e8b90d6f53cbb7573be?s=48 jakevdp
769
190k
C9b18d9dff88a9dd2393364c2b3b21bd?s=48 colly
65
2.9k
25c7c18223fb42a4c6ae1c8db6f50f9b?s=48 mojombo
356
62k
B83d5b590577969ee79a5ad845411a7d?s=48 ammeep
653
54k
78b475797a14c84799063c7cd073962f?s=48 holman
448
130k

Transcript

  1. MA HINE LEARNING ON THE WEB @galuhsahid https://galuh.me

  2. Halo! Saya Galuh. @galuhsahid https://galuh.me Data Engineer @ Gojek Mozilla

    Te h Speaker o-host pod ast KartiniTeknologi.id

  3. Pembelajaran mesin memberikan komputer kemampuan untuk belajar tanpa harus diprogram

    se ara eksplisit Arthur Samuel (1959)

  4. Data: Gambar https://www.bbc.com/news/world-asia-china-37273337 https://www.thoughtco.com/thmb/DpVtY--UXVOtG68m9LhMY7be2w0=/768x0/filters:no_upscale():max_bytes(150000):strip_icc()/happy-red-panda-171399380-5b574325c9e77c005b690b41.jpg

  5. Data: Suara ommon Voi e

  6. Data: Teks Wikipedia Berita Media Sosial

  7. Data Satu Data Indonesia

  8. Fitur

  9. Fitur fitur- fitur

  10. Fitur target

  11. Model Representasi matematis dari sebuah proses di dunia nyata

  12. Model Representasi matematis dari sebuah proses di dunia nyata 2000*jumlah

    lantai + … + 5000*jumlah kamar = harga rumah error=0.2

  13. Model Representasi matematis dari sebuah proses di dunia nyata 2000*jumlah

    lantai + … + 5000*jumlah kamar = harga rumah 3000*jumlah lantai + … + 6000*jumlah kamar = harga rumah error=0.2 error=0.1

  14. Ma hine learning dengan JavaS ript?

  15. Ma hine learning dengan JavaS ript?

  16. “A WebGL a elerated JavaS ript library for training and

    deploying ML models.” TensorFlow.js Ma hine learning dengan JavaS ript?

  17. Tanpa instalasi

  18. Tanpa instalasi Interaktif

  19. Tanpa instalasi Interaktif Data tetap ada di lient

  20. Loh, kok bisa?

  21. Loh, kok bisa?

  22. None

  23. Tensors Layers A tivation

  24. Ma hine learning dengan JavaS ript?

  25. Ma hine learning dengan JavaS ript?

  26. “Friendly ma hine learning for the web.” Ml5js Ma hine

    learning dengan JavaS ript?

  27. Menjalankan model yang sudah dilatih sebelumnya 1

  28. #1: Klasifikasi Gambar

  29. None

  30. MobileNet • Model computer vision yang bersifat mobile-first • Berukuran

    kecil, low-latency, dan low-power

  31. MobileNet

  32. Klasifikasi Gambar dengan MobileNet

  33. Klasifikasi Gambar dengan MobileNet MobileNet

  34. MobileNet Prediction: Bicycle (0.7845) Klasifikasi Gambar dengan MobileNet

  35. <html> <head> <meta charset="UTF-8"> <title>Klasifikasi Gambar dengan ml5.js</title> <script src="https://unpkg.com/ml5@0.3.1/dist/

    ml5.min.js"></script> </head> <body> … <script src="sketch.js"></script> </body> </html> index.html

  36. sket h.js let classifier; const loadModel = async (modelName) =>

    { try { classifier = await ml5.imageClassifier(modelName); } catch (e) { console.error(e); } … };

  37. sket h.js let classifier; const loadModel = async (modelName) =>

    { try { classifier = await ml5.imageClassifier(modelName); } catch (e) { console.error(e); } … }; Darknet Darknet-tiny MobileNet

  38. sket h.js const image = document.getElementById("image"); … const getPrediction =

    async (input) => { try { let results = await classifier.classify(input); return results; } catch (e) { console.error(e); } };

  39. sket h.js const image = document.getElementById(“image"); … const getPrediction =

    async (input, classifier) => { try { let results = await classifier.classify(input); return results; } catch (e) { console.error(e); } };
  40. None

  41. sket h.js const startWebcam = async () => { try

    { stream = await navigator.mediaDevices.getUserMedia( { video: true }); video.srcObject = stream; video.play(); getPrediction(video); } catch (e) { console.error(e); } };

  42. sket h.js const startWebcam = async () => { try

    { stream = await navigator.mediaDevices.getUserMedia( { video: true }); video.srcObject = stream; video.play(); getPrediction(video); } catch (e) { console.error(e); } }; MediaDevi es.getUserMedia()

  43. MobileNet

  44. Darknet MobileNet

  45. Menjalankan model yang sudah dilatih sebelumnya 1 Melatih ulang model

    yang sudah ada 2

  46. Melatih ulang model MobileNet ?

  47. Melatih dari awal? 14 juta gambar

  48. Transfer learning MobileNet yang dilatih ulang Prediction: Angklung (0.873)

  49. #2: Ro k Paper S issors

  50. None

  51. index.html <html> … <img src="images/rock.svg" alt="Rock" width="100" height="100" /><br />

    <p><span id="amountOfRockImages">0</ span> Rock Images</p> <p><button id="RockButton" class="button">Add Rock</button></p> </html>

  52. index.html <html> … <img src="images/rock.svg" alt="Rock" width="100" height="100" /><br />

    <p><span id="amountOfRockImages">0</ span> Rock Images</p> <p><button id="RockButton" class="button">Add Rock</button></p> </html>

  53. index.html <html> … <img src="images/rock.svg" alt="Rock" width="100" height="100" /><br />

    <p><span id="amountOfRockImages">0</ span> Rock Images</p> <p><button id="RockButton" class="button">Add Rock</button></p> … <button id="train" class="button">Train Model</button> </html>

  54. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } };

  55. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } }; Jenis model

  56. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } }; Input

  57. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } }; jumlah kelas

  58. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; MobileNet Bicycle (0.373) Cup (0.232) Vacuum (0.121)

  59. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; MobileNet Bicycle (0.373) Cup (0.232) Vacuum (0.121) x

  60. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; MobileNet Rock (0.923) Paper (0.422) Scissors (0.231)

  61. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; jumlah kelas https://www.mathworks.com/solutions/deep-learning/convolutional-neural-network.html

  62. sket h.js var RockButton = document.getElementById(“RockButton") … RockButton.onclick = function()

    { classifier.addImage(video, ”Rock”); amountOfRockImages.innerText = Number(amountOfRockImages.i nnerText) + 1; };

  63. sket h.js const trainModel = async () => { try

    { await classifier.train(function(lossValue) { if (lossValue) { totalLoss = lossValue; loss.innerHTML = "Loss: " + totalLoss; } else { loss.innerHTML = "Training done! Final loss: " + tot alLoss; } }); } catch(e) { console.error(e); } };

  64. sket h.js const trainModel = async () => { try

    { await classifier.train(function(lossValue) { if (lossValue) { totalLoss = lossValue; loss.innerHTML = "Loss: " + totalLoss; } else { loss.innerHTML = "Training done! Final loss: " + tot alLoss; } }); } catch(e) { console.error(e); } };

  65. sket h.js const getPlayerResult = async video => { try

    { let results = await classifier.classify(video); if (results && results[0]) { return results[0].label; } } catch (e) { console.error(err); } };

  66. sket h.js const getPlayerResult = async video => { try

    { let results = await classifier.classify(video); if (results && results[0]) { return results[0].label; } } catch (e) { console.error(err); } };
  67. None

  68. #3: S roll with Your Voi e

  69. None

  70. Klasifikasi suara: Dataset Spee h ommands Dataset Yes No Up

    Down Left Right Go Stop 0-9 Unknown word Ba kground noise

  71. Klasifikasi suara: Arsitektur Paper

  72. Klasifikasi suara NN Prediction: Up (0.86) Sound icon by Amiryshakiel

    from the Noun Project https://www.tensorflow.org/tutorials/sequences/audio_recognition

  73. sket h.js const options = { probabilityThreshold: 0.98 }; …

    const loadModel = async() => { try { classifier = await ml5.soundClassifier('SpeechCommands18 w', options); } catch (e) { console.error(e); } };

  74. sket h.js const getPrediction = async() => { try {

    await classifier.classify(scrollPage); } catch (e) { console.error(e); } };

  75. sket h.js function scrollPage(error, results) { if (results[0].label == 'down')

    { lastPosition = lastPosition + scrollBy; } else if (results[0].label == 'up') { lastPosition = lastPosition - scrollBy; } lastPosition = Math.max(0, lastPosition); window.scroll(0, lastPosition); };

  76. Selanjutnya https://ml5js.org/reference/

  77. Selanjutnya • A Beginner’s Guide to Machine Learning in JavaScript

    oleh Daniel Shiffman • Magenta.js (generating music & art) • Kursus Coursera: Machine Learning oleh Andrew Ng • Beyond the web: machine learning + hardware • Source code ml5.js • Bias & etika pembelajaran mesin • Referensi buku: Weapons of Math Destruction oleh Cathy O’Neil

  78. Terima kasih! @galuhsahid https://galuh.me