Tweet
Tweet

Slide 1

Slide 1 text

C L I E N T- S I D E M A C H I N E L E A R N I N G B R I N G I N G A I T O T H E F R O N T E N D @markusingvarssn

Slide 2

Slide 2 text

https://www.linkedin.com/posts/david-kriesel_nice-one-activity-7062736038179262464-OWdU/

Slide 3

Slide 3 text

Qualcomm image - https://www.qualcomm.com/news/onq/2024/02/what-is-an-npu-and-why-is-it-key-to-unlocking-on-device-generative-ai

Slide 4

Slide 4 text

Qualcom image - https://www.qualcomm.com/news/onq/2024/02/what-is-an-npu-and-why-is-it-key-to-unlocking-on-device-generative-ai Apple Neural Engine image - https://www.youtube.com/watch?v=zXJbdtxh0XE

Slide 5

Slide 5 text

Apple Intelligence on iPhone in 5 minutes - https://www.youtube.com/watch?v=PugKQZHPut8

Slide 6

Slide 6 text

B R I N G I N G A I TO T H E F R O N T E N D After this talk, we will know about:

Slide 7

Slide 7 text

B R I N G I N G A I TO T H E F R O N T E N D After this talk, we will know about: • Recent innovation that enables client-side machine learning

Slide 8

Slide 8 text

B R I N G I N G A I TO T H E F R O N T E N D After this talk, we will know about: • Recent innovation that enables client-side machine learning • What client-side machine learning can do for us

Slide 9

Slide 9 text

B R I N G I N G A I TO T H E F R O N T E N D After this talk, we will know about: • Recent innovation that enables client-side machine learning • What client-side machine learning can do for us • How we can start leveraging it today

Slide 10

Slide 10 text

HELLO! BRINGING AI TO THE FRONTEND I’m Markus @markusingvarssn

Slide 11

Slide 11 text

HELLO! BRINGING AI TO THE FRONTEND I’m Markus @markusingvarssn

Slide 12

Slide 12 text

HELLO! BRINGING AI TO THE FRONTEND I’m Markus @markusingvarssn

Slide 13

Slide 13 text

M A C H I N E L E A R N I N G I N 1 0 0 S E C O N D S https://www.linkedin.com/pulse/machine-learning-image-detectioncats-vs-dogs-amrith-kumar/

Slide 14

Slide 14 text

M A C H I N E L E A R N I N G I N 1 0 0 S E C O N D S https://www.linkedin.com/pulse/machine-learning-image-detectioncats-vs-dogs-amrith-kumar/

Slide 15

Slide 15 text

” R E C E N T ” M AC H I N E L E A R N I N G I N N OVAT I O N S

Slide 16

Slide 16 text

” R E C E N T ” M AC H I N E L E A R N I N G I N N OVAT I O N S • Hardware improvements – The GPU (and the NPU, and the TPU etc)

Slide 17

Slide 17 text

” R E C E N T ” M AC H I N E L E A R N I N G I N N OVAT I O N S • Hardware improvements – The GPU (and the NPU, and the TPU etc) • Software improvements • TensorFlow (and TensorFlow.js), PyTorch

Slide 18

Slide 18 text

P R E V I O U S TA L K S O N M L @ J S - P O L A N D

Slide 19

Slide 19 text

” R E C E N T ” M AC H I N E L E A R N I N G I N N OVAT I O N S • Hardware improvements – The GPU (and the NPU, and the TPU etc) • Software improvements • TensorFlow (and TensorFlow.js), PyTorch • WebGPU

Slide 20

Slide 20 text

W E B G P U B R O W S E R S U P P O RT

Slide 21

Slide 21 text

” R E C E N T ” M AC H I N E L E A R N I N G I N N OVAT I O N S • Hardware improvements – The GPU (and the NPU, and the TPU etc) • Software improvements • TensorFlow (and TensorFlow.js), PyTorch • WebGPU • Technique improvements https://arxiv.org/pdf/1706.03762

Slide 22

Slide 22 text

W H Y B R I N G I N G A I TO T H E C L I E N T ?

Slide 23

Slide 23 text

W H Y B R I N G I N G A I TO T H E C L I E N T ? • No more network delay!

Slide 24

Slide 24 text

W H Y B R I N G I N G A I TO T H E C L I E N T ? • No more network delay! https://www.keycdn.com/support/network-latency

Slide 25

Slide 25 text

B O N U S – O F F L I N E S U P P O RT https://developer.chrome.com/blog/improved-pwa-offline-detection

Slide 26

Slide 26 text

F A S T O N - D E V I C E M L W I T H T FJ S A N D M E D I A P I P E https://github.com/tensorflow/tfjs- models/tree/master/hand-pose-detection

Slide 27

Slide 27 text

F A S T O N - D E V I C E M L W I T H T FJ S A N D M E D I A P I P E • We will use TensorFlow.js and MediaPipe

Slide 28

Slide 28 text

F A S T O N - D E V I C E M L W I T H T FJ S A N D M E D I A P I P E • We will use TensorFlow.js and MediaPipe • Developed by Google

Slide 29

Slide 29 text

F A S T O N - D E V I C E M L W I T H T FJ S A N D M E D I A P I P E • We will use TensorFlow.js and MediaPipe • Developed by Google • MediaPipe Models gives us tons of light-weight ML models for free

Slide 30

Slide 30 text

F A S T O N - D E V I C E M L W I T H T FJ S A N D M E D I A P I P E • We will use TensorFlow.js and MediaPipe • Developed by Google • MediaPipe Models gives us tons of light-weight ML models for free • Can be customizable through MediaPipe Model Maker

Slide 31

Slide 31 text

F A S T O N - D E V I C E M L W I T H T FJ S A N D M E D I A P I P E • We will use TensorFlow.js and MediaPipe • Developed by Google • MediaPipe Models gives us tons of light-weight ML models for free • Can be customizable through MediaPipe Model Maker • We will use the @mediapipe/hands library in this demo.

Slide 32

Slide 32 text

W H Y B R I N G I N G A I TO T H E C L I E N T ?

Slide 33

Slide 33 text

W H Y B R I N G I N G A I TO T H E F R O N T E N D ? • No more network delay! • Privacy https://pub.towardsai.net/43-memorag-rag-agent-rag-fusion-and-more-7b8463590747

Slide 34

Slide 34 text

P R I V A T E A I W I T H A S K - M Y- P D F https://github.com/nico-martin/ask-my-pdf

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

S C E N A R I O – P E R S O N W I T H C H R O N I C B A C K P A I N

Slide 37

Slide 37 text

S C E N A R I O – P E R S O N W I T H C H R O N I C B A C K P A I N

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

https://www.youtube.com/watch?v=VAfy26xs6e0

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

W H Y B R I N G I N G A I TO T H E F R O N T E N D ? • No more network delay! • Privacy • Cost

Slide 46

Slide 46 text

M A C H I N E L E A R N I N G A N D C O S T https://ux- news.com/google- announces-new-virtual- try-on-tools/

Slide 47

Slide 47 text

M A C H I N E L E A R N I N G A N D C O S T https://www.reddit.com/r/Programm erHumor/comments/1cxfjws/tenouto ftendentistsrecommend/?rdt=61385

Slide 48

Slide 48 text

ng-pet-cam • Heavily inspired by Jason Mayes pet-cam app • Uses @tensorflow- models/coco-ssd for object- detection • An app to monitor your pet 🐶 • Is live on https://ngpetcam.web.app and code can be accessed on https://github.com/markusingv arsson/aiayn-ng-pet-cam

Slide 49

Slide 49 text

S U M M A RY

Slide 50

Slide 50 text

S U M M A RY • Dedicated hardware is required for client-side machine learning

Slide 51

Slide 51 text

S U M M A RY • Dedicated hardware is required for client-side machine learning • When client-side machine learning makes sense:

Slide 52

Slide 52 text

S U M M A RY • Dedicated hardware is required for client-side machine learning • When client-side machine learning makes sense: • Smooth on-device performance

Slide 53

Slide 53 text

S U M M A RY • Dedicated hardware is required for client-side machine learning • When client-side machine learning makes sense: • Smooth on-device performance • Offline Support

Slide 54

Slide 54 text

S U M M A RY • Dedicated hardware is required for client-side machine learning • When client-side machine learning makes sense: • Smooth on-device performance • Offline Support • Privacy

Slide 55

Slide 55 text

S U M M A RY • Dedicated hardware is required for client-side machine learning • When client-side machine learning makes sense: • Smooth on-device performance • Offline Support • Privacy • Enable ML when Server-Side ML is impractical (e.g. cost)

Slide 56

Slide 56 text

S U M M A RY • Dedicated hardware is required for client-side machine learning • When client-side machine learning makes sense: • Smooth on-device performance • Offline Support • Privacy • Enable ML when Server-Side ML is impractical (e.g. cost) • Optimistic future

Slide 57

Slide 57 text

G E T STA RT E D ?

Slide 58

Slide 58 text

G E T STA RT E D ?

Slide 59

Slide 59 text

G E T STA RT E D ?

Slide 60

Slide 60 text

G E T STA RT E D ?

Slide 61

Slide 61 text

C L I E N T- S I D E M A C H I N E L E A R N I N G B R I N G I N G A I T O T H E F R O N T E N D Connect with me!