Machine Learning with Angular and TensorFlow.js Andrew Wiens Oasis Digital Solutions 

Roadmap 1. Background & Motivation 2. TensorFlow.js 3. Pre-demo ML Concepts 4. Demos 

Background & Motivation 

Relevant Projects 

? 

TensorFlow.js 

TensorFlow ● Google’s framework for machine learning & AI ● Usually used with Python, often under an abstraction layer like Keras ● TensorFlow.js is the JavaScript version ● npm package ● Node.js or browser ● Capabilities: ○ Create models in the browser ○ Use pre-trained models ○ Re-train imported models ● Demos 

Brief History ● TensorFlow - November 2015 ● deeplearn.js - August 2017 ○ Became TensorFlow.js Core ● TensorFlow.js - March 2018 ● TensorFlow.js 1.0 - March 2019 ● Current: 1.2 https://playground.tensorflow.org 

TensorFlow.js 1.0 

Parts of TensorFlow.js ● TensorFlow.js Core: Low-level API (deeplearn.js) ● TensorFlow.js Layers: High-level API similar to Keras. ● TensorFlow.js Converter: Tools to import a TensorFlow or Keras saved model ● Importing @tensorflow/tfjs brings in all three ○ Can import individually to potentially reduce bundle size ● @tensorflow/tfjs works in the browser or Node.js ● @tensorflow/tfjs-node: Faster Node.js version of tfjs that uses C++ bindings ● @tensorflow/tfjs-node-gpu: Use Nvidia CUDA GPUs in Linux ○ Absolute must if training large networks, e.g. convolutional neural nets (ConvNets) ○ Not usually needed for inference ● @tensorflow-models/: Pre-made models 

Official Models Available on npm ● @tensorflow-models/mobilenet Google’s pre-trained image classifiers. Trained to detect these classes. Any browser image element => array of most likely predictions and their confidences. ~1 yr ● @tensorflow-models/knn-classifier Predict what group a record belongs to based on how similar it is to others. ~1 yr ● @tensorflow-models/coco-ssd Find where objects are located in an image. Supports 90 types of objects. ~1 yr ● @tensorflow-models/universal-sentence-encoder Convert text into a 512-dimensional embedding. It can be used as a pre-processing step to do ML on natural language. 8 mo. ● @tensorflow-models/toxicity Detect whether text contains toxic content. 7 mo. 

Official Tools Available on npm ● @tensorflow/tfjs-vis Visualization tools, e.g. graphs. Based on Vega. ~1 yr Left image src: https://i.redd.it/taia1x4xqea21.png 

Official Tools Available on npm ● @tensorflow/tfjs-tsne Automatically group data in two dimensions. Very useful for visualizing data. ~1 yr Image src: https://www.pnas.org/content/108/41/16916 

Pre-Demo ML Concepts 

Natural Language Bag of Words vs. Latent Spaces (1) John likes to watch movies. Mary likes movies too. (2) John also likes to watch football games. ====================================================== (1) [1, 2, 1, 1, 2, 1, 1, 0, 0, 0] (2) [1, 1, 1, 1, 0, 0, 0, 1, 1, 1] Srcs: Wikipedia (Bag of Words) & https://towardsdatascience.com/whats-new-in-deep-learning-research-inside-google-s-semantic-experiences-4536d57c685 

Images Convolutional Neural Networks Srcs: https://machinethink.net/blog/googles-mobile-net-architecture-on-iphone/ and https://neurohive.io/en/popular-networks/vgg16/ 

Classification k-Nearest Neighbors (k-NN) k = 1 k = 5 Image src: Wikipedia “kNN” 

Demos 

Andrew Wiens andywiens.net [email protected] Repos: github.com/adwiens/tfjs-knn-mobilenet github.com/adwiens/tfjs-knn-reddit