Machine Learning with Angular and TensorFlow.js

Ideas for front-end machine learning features with Angular and TensorFlow.js. Two demo Angular apps using tfjs MobileNets, the Universal Sentence Encoder, and k-nearest neighbors. Talks given at St. Louis Angular Lunch and COMO DevFest 2019.

Transcript

1. Machine Learning with Angular and TensorFlow.js Andrew Wiens Oasis Digital Solutions
2. Roadmap 1. Background & Motivation 2. TensorFlow.js 3. Pre-demo ML Concepts 4. Demos
3. Background & Motivation
4. Relevant Projects
5. ?
6. TensorFlow.js
7. TensorFlow • Google’s framework for machine learning & AI • Usually used with Python, often under an abstraction layer like Keras • TensorFlow.js is the JavaScript version • npm package • Node.js or browser • Capabilities: ◦ Create models in the browser ◦ Use pre-trained models ◦ Re-train imported models • Demos
8. Brief History • TensorFlow - November 2015 • deeplearn.js - August 2017 ◦ Became TensorFlow.js Core • TensorFlow.js - March 2018 • TensorFlow.js 1.0 - March 2019 • Current: 1.2 https://playground.tensorflow.org
9. TensorFlow.js 1.0
10. Parts of TensorFlow.js • TensorFlow.js Core: Low-level API (deeplearn.js) • TensorFlow.js Layers: High-level API similar to Keras. • TensorFlow.js Converter: Tools to import a TensorFlow or Keras saved model • Importing @tensorflow/tfjs brings in all three ◦ Can import individually to potentially reduce bundle size • @tensorflow/tfjs works in the browser or Node.js • @tensorflow/tfjs-node: Faster Node.js version of tfjs that uses C++ bindings • @tensorflow/tfjs-node-gpu: Use Nvidia CUDA GPUs in Linux ◦ Absolute must if training large networks, e.g. convolutional neural nets (ConvNets) ◦ Not usually needed for inference • @tensorflow-models/<model-name>: Pre-made models
11. Official Models Available on npm • @tensorflow-models/mobilenet Google’s pre-trained image classifiers. Trained to detect these classes. Any browser image element => array of most likely predictions and their confidences. ~1 yr • @tensorflow-models/knn-classifier Predict what group a record belongs to based on how similar it is to others. ~1 yr • @tensorflow-models/coco-ssd Find where objects are located in an image. Supports 90 types of objects. ~1 yr • @tensorflow-models/universal-sentence-encoder Convert text into a 512-dimensional embedding. It can be used as a pre-processing step to do ML on natural language. 8 mo. • @tensorflow-models/toxicity Detect whether text contains toxic content. 7 mo.
12. Official Tools Available on npm • @tensorflow/tfjs-vis Visualization tools, e.g. graphs. Based on Vega. ~1 yr Left image src: https://i.redd.it/taia1x4xqea21.png
13. Official Tools Available on npm • @tensorflow/tfjs-tsne Automatically group data in two dimensions. Very useful for visualizing data. ~1 yr Image src: https://www.pnas.org/content/108/41/16916
14. Pre-Demo ML Concepts
15. Natural Language Bag of Words vs. Latent Spaces (1) John likes to watch movies. Mary likes movies too. (2) John also likes to watch football games. ====================================================== (1) [1, 2, 1, 1, 2, 1, 1, 0, 0, 0] (2) [1, 1, 1, 1, 0, 0, 0, 1, 1, 1] Srcs: Wikipedia (Bag of Words) & https://towardsdatascience.com/whats-new-in-deep-learning-research-inside-google-s-semantic-experiences-4536d57c685
16. Images Convolutional Neural Networks Srcs: https://machinethink.net/blog/googles-mobile-net-architecture-on-iphone/ and https://neurohive.io/en/popular-networks/vgg16/
17. Classification k-Nearest Neighbors (k-NN) k = 1 k = 5 Image src: Wikipedia “kNN”
18. Demos
19. Andrew Wiens andywiens.net andrew.wiens@oasisdigital.com Repos: github.com/adwiens/tfjs-knn-mobilenet github.com/adwiens/tfjs-knn-reddit