Machine Learning on the Web

2b6d7bdd43058e87f53866eb86538a59?s=47 Galuh Sahid
September 28, 2019

Machine Learning on the Web

Slide deck for the talk "Machine Learning on the Web" at JSDay ID

GitHub repository: https://github.com/galuhsahid/ml-on-the-web

2b6d7bdd43058e87f53866eb86538a59?s=128

Galuh Sahid

September 28, 2019
Tweet

Transcript

  1. MA HINE LEARNING ON THE WEB @galuhsahid https://galuh.me

  2. Halo! Saya Galuh. @galuhsahid https://galuh.me Data Engineer @ Gojek Mozilla

    Te h Speaker o-host pod ast KartiniTeknologi.id
  3. Pembelajaran mesin memberikan komputer kemampuan untuk belajar tanpa harus diprogram

    se ara eksplisit Arthur Samuel (1959)
  4. Data: Gambar https://www.bbc.com/news/world-asia-china-37273337 https://www.thoughtco.com/thmb/DpVtY--UXVOtG68m9LhMY7be2w0=/768x0/filters:no_upscale():max_bytes(150000):strip_icc()/happy-red-panda-171399380-5b574325c9e77c005b690b41.jpg

  5. Data: Suara ommon Voi e

  6. Data: Teks Wikipedia Berita Media Sosial

  7. Data Satu Data Indonesia

  8. Fitur

  9. Fitur fitur- fitur

  10. Fitur target

  11. Model Representasi matematis dari sebuah proses di dunia nyata

  12. Model Representasi matematis dari sebuah proses di dunia nyata 2000*jumlah

    lantai + … + 5000*jumlah kamar = harga rumah error=0.2
  13. Model Representasi matematis dari sebuah proses di dunia nyata 2000*jumlah

    lantai + … + 5000*jumlah kamar = harga rumah 3000*jumlah lantai + … + 6000*jumlah kamar = harga rumah error=0.2 error=0.1
  14. Ma hine learning dengan JavaS ript?

  15. Ma hine learning dengan JavaS ript?

  16. “A WebGL a elerated JavaS ript library for training and

    deploying ML models.” TensorFlow.js Ma hine learning dengan JavaS ript?
  17. Tanpa instalasi

  18. Tanpa instalasi Interaktif

  19. Tanpa instalasi Interaktif Data tetap ada di lient

  20. Loh, kok bisa?

  21. Loh, kok bisa?

  22. None
  23. Tensors Layers A tivation

  24. Ma hine learning dengan JavaS ript?

  25. Ma hine learning dengan JavaS ript?

  26. “Friendly ma hine learning for the web.” Ml5js Ma hine

    learning dengan JavaS ript?
  27. Menjalankan model yang sudah dilatih sebelumnya 1

  28. #1: Klasifikasi Gambar

  29. None
  30. MobileNet • Model computer vision yang bersifat mobile-first • Berukuran

    kecil, low-latency, dan low-power
  31. MobileNet

  32. Klasifikasi Gambar dengan MobileNet

  33. Klasifikasi Gambar dengan MobileNet MobileNet

  34. MobileNet Prediction: Bicycle (0.7845) Klasifikasi Gambar dengan MobileNet

  35. <html> <head> <meta charset="UTF-8"> <title>Klasifikasi Gambar dengan ml5.js</title> <script src="https://unpkg.com/ml5@0.3.1/dist/

    ml5.min.js"></script> </head> <body> … <script src="sketch.js"></script> </body> </html> index.html
  36. sket h.js let classifier; const loadModel = async (modelName) =>

    { try { classifier = await ml5.imageClassifier(modelName); } catch (e) { console.error(e); } … };
  37. sket h.js let classifier; const loadModel = async (modelName) =>

    { try { classifier = await ml5.imageClassifier(modelName); } catch (e) { console.error(e); } … }; Darknet Darknet-tiny MobileNet
  38. sket h.js const image = document.getElementById("image"); … const getPrediction =

    async (input) => { try { let results = await classifier.classify(input); return results; } catch (e) { console.error(e); } };
  39. sket h.js const image = document.getElementById(“image"); … const getPrediction =

    async (input, classifier) => { try { let results = await classifier.classify(input); return results; } catch (e) { console.error(e); } };
  40. None
  41. sket h.js const startWebcam = async () => { try

    { stream = await navigator.mediaDevices.getUserMedia( { video: true }); video.srcObject = stream; video.play(); getPrediction(video); } catch (e) { console.error(e); } };
  42. sket h.js const startWebcam = async () => { try

    { stream = await navigator.mediaDevices.getUserMedia( { video: true }); video.srcObject = stream; video.play(); getPrediction(video); } catch (e) { console.error(e); } }; MediaDevi es.getUserMedia()
  43. MobileNet

  44. Darknet MobileNet

  45. Menjalankan model yang sudah dilatih sebelumnya 1 Melatih ulang model

    yang sudah ada 2
  46. Melatih ulang model MobileNet ?

  47. Melatih dari awal? 14 juta gambar

  48. Transfer learning MobileNet yang dilatih ulang Prediction: Angklung (0.873)

  49. #2: Ro k Paper S issors

  50. None
  51. index.html <html> … <img src="images/rock.svg" alt="Rock" width="100" height="100" /><br />

    <p><span id="amountOfRockImages">0</ span> Rock Images</p> <p><button id="RockButton" class="button">Add Rock</button></p> </html>
  52. index.html <html> … <img src="images/rock.svg" alt="Rock" width="100" height="100" /><br />

    <p><span id="amountOfRockImages">0</ span> Rock Images</p> <p><button id="RockButton" class="button">Add Rock</button></p> </html>
  53. index.html <html> … <img src="images/rock.svg" alt="Rock" width="100" height="100" /><br />

    <p><span id="amountOfRockImages">0</ span> Rock Images</p> <p><button id="RockButton" class="button">Add Rock</button></p> … <button id="train" class="button">Train Model</button> </html>
  54. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } };
  55. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } }; Jenis model
  56. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } }; Input
  57. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = await ml5.featureExtractor("MobileNet"); classifier = await featureExtractor.classification(video, { numLabels: 3 }); } catch (e) { console.error(e); } }; jumlah kelas
  58. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; MobileNet Bicycle (0.373) Cup (0.232) Vacuum (0.121)
  59. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; MobileNet Bicycle (0.373) Cup (0.232) Vacuum (0.121) x
  60. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; MobileNet Rock (0.923) Paper (0.422) Scissors (0.231)
  61. sket h.js let classifier; const loadModel = async () =>

    { try { featureExtractor = ml5.featureExtractor("MobileNet"); classifier = featureExtractor.classification(video, { nu mLabels: 3 }); } catch (e) { console.error(e); } }; jumlah kelas https://www.mathworks.com/solutions/deep-learning/convolutional-neural-network.html
  62. sket h.js var RockButton = document.getElementById(“RockButton") … RockButton.onclick = function()

    { classifier.addImage(video, ”Rock”); amountOfRockImages.innerText = Number(amountOfRockImages.i nnerText) + 1; };
  63. sket h.js const trainModel = async () => { try

    { await classifier.train(function(lossValue) { if (lossValue) { totalLoss = lossValue; loss.innerHTML = "Loss: " + totalLoss; } else { loss.innerHTML = "Training done! Final loss: " + tot alLoss; } }); } catch(e) { console.error(e); } };
  64. sket h.js const trainModel = async () => { try

    { await classifier.train(function(lossValue) { if (lossValue) { totalLoss = lossValue; loss.innerHTML = "Loss: " + totalLoss; } else { loss.innerHTML = "Training done! Final loss: " + tot alLoss; } }); } catch(e) { console.error(e); } };
  65. sket h.js const getPlayerResult = async video => { try

    { let results = await classifier.classify(video); if (results && results[0]) { return results[0].label; } } catch (e) { console.error(err); } };
  66. sket h.js const getPlayerResult = async video => { try

    { let results = await classifier.classify(video); if (results && results[0]) { return results[0].label; } } catch (e) { console.error(err); } };
  67. None
  68. #3: S roll with Your Voi e

  69. None
  70. Klasifikasi suara: Dataset Spee h ommands Dataset Yes No Up

    Down Left Right Go Stop 0-9 Unknown word Ba kground noise
  71. Klasifikasi suara: Arsitektur Paper

  72. Klasifikasi suara NN Prediction: Up (0.86) Sound icon by Amiryshakiel

    from the Noun Project https://www.tensorflow.org/tutorials/sequences/audio_recognition
  73. sket h.js const options = { probabilityThreshold: 0.98 }; …

    const loadModel = async() => { try { classifier = await ml5.soundClassifier('SpeechCommands18 w', options); } catch (e) { console.error(e); } };
  74. sket h.js const getPrediction = async() => { try {

    await classifier.classify(scrollPage); } catch (e) { console.error(e); } };
  75. sket h.js function scrollPage(error, results) { if (results[0].label == 'down')

    { lastPosition = lastPosition + scrollBy; } else if (results[0].label == 'up') { lastPosition = lastPosition - scrollBy; } lastPosition = Math.max(0, lastPosition); window.scroll(0, lastPosition); };
  76. Selanjutnya https://ml5js.org/reference/

  77. Selanjutnya • A Beginner’s Guide to Machine Learning in JavaScript

    oleh Daniel Shiffman • Magenta.js (generating music & art) • Kursus Coursera: Machine Learning oleh Andrew Ng • Beyond the web: machine learning + hardware • Source code ml5.js • Bias & etika pembelajaran mesin • Referensi buku: Weapons of Math Destruction oleh Cathy O’Neil
  78. Terima kasih! @galuhsahid https://galuh.me