Machine Learning for JavaScript developers Lee Boonstra Customer Engineer, Google Twitter: @ladysign 

2 Lee Boonstra @ladysign 

3 

4 What’s Machine Learning 

Why now? ● Amount of data Better Models More Computing Power 

Machine Learning to classify things Dog vs. Mop Oh, that’s easy... 

Wait what?! 

We would need machine learning to give us results Confidence level 

It’s inspired by how our brains work Neural Networks 

It’s easier to make computers learn than to build smarter computers Artificial Intelligence Process of building smarter computers Machine Learning Process of making computers learn 

Why Google? Data Scientists use Tensorflow Tensorflow is what we use for our own internal machine learning projects, and now it’s available to you! Google made it open source. ● More than 480 contributions ● 10,000 commits in a year ● 53k star rating http://www.tensorflow.org 

Google is an AI company Used across products: Unique project directories Time 

No content

No content

Machine Learning use cases • Predictive maintenance or condition monitoring • Warranty reserve estimation • Propensity to buy • Demand forecasting • Process optimization Manufacturing • Predictive inventory planning • Recommendation engines • Upsell and cross-channel marketing • Market segmentation and targeting • Customer ROI and lifetime value Retail • Alerts and diagnostics from real-time patient data • Disease identification and risk satisfaction • Patient triage optimization • Proactive health management • Healthcare provider sentiment analysis Healthcare and Life Sciences • Aircraft scheduling • Dynamic pricing • Social media – consumer feedback and interaction analysis • Customer complaint resolution • Traffic patterns and congestion management Travel and Hospitality • Risk analytics and regulation • Customer Segmentation • Cross-selling and upselling • Sales and marketing campaign management • Creditworthiness evaluation Financial Services • Power usage analytics • Seismic data processing • Carbon emissions and trading • Customer-specific pricing • Smart grid management • Energy demand and supply optimization Energy, Feedstock and Utilities 

There’s a lack of ML expertise in the industry 1000’s Deep Learning Researchers 21M Developers <1M Data Scientists 

To create your own machine learning model it takes a complex & time intensive process UPDATE DEPLOY EVALUATE TUNE ML MODEL PARAMETERS ML MODEL DESIGN DATA PREPROCESSING Large computational resource . Machine learning expertise . Manual data labeling 

So how can Google Cloud help? Pre-Trained Models Train your own Conversation Solutions 

Introducing Cloud AI Less ML expertise More ML expertise Cloud AI solutions Cloud Job Discovery Contact Center Document Understanding Cloud AI building blocks Cloud AI platform Cloud ML Engine Cloud Video Intelligence ML professionals & service partners ASL Professional Services Organization Cloud AutoML Vision Vision Cloud Vision Language Conversation Cloud Natural Language Cloud AutoML NL Dialogflow Enterprise Cloud Translation Cloud Speech-to-Text Cloud Text-to-Speech Cloud AutoML Translation New New New Cloud GPU Cloud TPU Cloud Dataflow Cloud Dataproc Machine & Deep Learning ML accelerators ML libraries Tensorflow Kubeflow Kaggle/datasets Datasets 

Train your own: Tensorflow on Cloud ML Engine ● Fully managed service ● Train using a custom TensorFlow graph for any ML use cases with CPUs/GPUs ● Training at scale to shorten dev cycle ● Automatically maximize predictive accuracy with HyperTune ● High throughput batch predictions ● Low latency online predictions (Beta) ● Integrated Datalab experience Train your own 

Train your own: Use your own data to train models Train your own Cloud Datalab Cloud Machine Learning Cloud Storage Google BigQuery Develop/Model/Test 

Pre-trained models ML APIs ● Developer Machine Learning APIs. Google trained the model for you. Your developers just need to call an URL endpoint. ○ Understanding Images ○ Understanding Videos ○ Understanding Speech ○ Converting Text to Speech ○ Understanding Text ○ Translating Text ○ Pre-Trained Models 

Vision API Detect broad sets of categories within an image, ranging from modes of transportation to animals. Analyze facial features to detect emotions: joy, sorrow, anger. Detect logos. Detect and extract text within an image, with support for a broad range of languages, along with support for automatic language identification. Extract text Detect different types of inappropriate content from adult to violent content. Powered by Google Safe Search Detect inappropriate content Object Recognition Facial sentiment & logos Pre-Trained Models 

Video Intelligence API Detect entities within the video, such as "dog", "flower" or "car". You can now search your video catalog the same way you search text documents.. Extract actionable insights from video files without requiring any machine learning or computer vision knowledge. Enable Video Search Label Detection Insights From Videos Pre-Trained Models 

Speech API Powered by deep learning neural networking to power your applications.. No need for signal processing or noise cancellation before calling API. Can handle noisy audio from a variety of environments. Noise Robustness Can provide context hints for improved accuracy. Especially useful for device and app use cases. Word Hints Speech Recognition Recognizes over 80 languages & variants. Can also filter inappropriate content in text results Over 80 languages Can stream text results, returning partial recognition results as they become available. Can also be run on buffered or archived audio files. Real-time results Pre-Trained Models 

Natural Language API Identify entities and label by types such as person, organization, location, events, products and media. Enables you to easily analyze text in multiple languages including English, Spanish and Japanese. Extract tokens and sentences, identify parts of speech (PoS) and create dependency parse trees for each sentence. Syntax analysis Entity Recognition Multi-Language Support Understand the overall sentiment expressed in a block of text. Sentiment Analysis Pre-Trained Models 

Translation API Supports more than 100 languages and thousands of language pairs. Behind the scenes, Translation API is learning from logs analysis and human translation examples. Existing language pairs improve and new language pairs come online at no additional cost. Sometimes you don’t know your source text language in advance. Can automatically identify languages with high accuracy. Automatic language detection The Premium edition is tailored for users who need precise, long-form translation services (e.g. livestream translations, high volume of emails, detailed articles and documents) Premium edition BETA Text Translation Continuous Updates Pre-Trained Models 

Translation API Pre-Trained Models 

No content

Auto ML Custom task Generic task Someone else has solved this before Trained on common classes Specific to your dataset ML APIs TensorFlow “cat” “bob” AUTO ML Developer Data Scientist Developer or Data scientist 

How it works AutoML Vision Photo dataset Train Deploy Serve Generate predictions with a REST API Auto ML 

Via a web interface you will go through these steps Auto ML 1 3 4 5 Upload Labeled Data Evaluate Predict Iterate (if needed) 2 Train 

Optimize the pre-trained models with your own data Auto ML ● Create models for your own domain, but use the Google pre-trained models as a base. ○ Vision, Translation, or NLP ● Web Interface, to upload a CSV with labeled data. ● Your use-case is: ○ Not supported by pre-built APIs AND ○ Has sufficient labeled training data ● You want to get to produce a model and predict quickly ● You don’t want to build a model from scratch Auto ML 

No content

35 50% of enterprises spend more time on creating bots than on mobile app development by 2021 —Gartner Conversation Solutions 

36 AI to improve your customer experience Voice Activated Speakers & smart assistants. Chatbots in websites, apps or social media platforms. AI in callcenters Conversation Solutions 

Meet the Google Assistant It’s your own personal Google, always ready to help. 

38 ● Write the conversation - Dialogflow (Enterprise) ● Deploy on GA+ UX components - Actions on Google What you need to build your own action Conversation Solutions 

39 ● Previously known as API.AI ○ (Sept 2016, acquired by Google) ● Powered by Machine Learning: ○ Natural Language Understanding (NLU) ○ Intent Matching ○ Conversation Training ● Cross platform ● Build faster with the Web UI ● Scalable: separate your conversation text from code ● Speech / Voice Integration ● Multi-lingual bot support (20+ languages) ● Part of Google Cloud (60+ cloud services) Development suite for building Conversational UIs. Conversation Solutions 

40 ● Bring your agents to smart speakers (Google Home) or phones (Android, iOS app) ● Actions on Google includes: ○ UI toolkit, ○ Audio toolkit ○ Account Linking API ○ SDKs ○ tutorial guides ● UI components such as: ○ Buttons, Images ○ Cards, Carousels, ○ Lists ○ Tables Program for developers of Actions (“apps”) that run via Google Assistant Actions on Google Conversation Solutions 

41 Terminology Google Assistant — The virtual assistant of Google. Out of the box on Android 6+. For iOS available as app. Action — A third party app, running on top of the Google Assistant. Google Home — Voice-activated speaker powered by Voice. Smart Display — Voice-activated speaker with screen powered by Voice and Touch. Conversation Solutions 

42 3rd party integration Extend the Google Assistant with your own custom actions. Hey Google, let me talk to Babs The Banking Bot Welcome, how can I help you? I want to transfer money. Let’s get Babs the Banking Bot How much do you want to transfer? 100 euro. Conversation Solutions 

Confidential + Proprietary Confidential + Proprietary Demo 

44 Sure! I’d like to transfer 100 euro to my wife her bank account.” A customer communicates with the Google Assistant 

45 What happens under the hood... 

Confidential + Proprietary Which customers are unhappy and why? 

Confidential + Proprietary How can I improve the user experience? 

48 Collect real-time chats from Dialogflow SDK 

49 Mask sensitive Information with DLP API 

50 Understand the text with NLP API 

51 Store all data in a data-warehouse 

52 Optimize your agent 

Confidential + Proprietary Advanced Chatflow with machine learning bot analytics User types to custom UI or channel Chatbot replies Dialogflow Enterprise Customer Client JS Angular 5 web front-end Kubernetes Engine Chat Server Dialogflow SDK / socket.io Kubernetes Engine Back-end CRM Python / Django Kubernetes Engine Container Registry Containers images can be stored in the Container Registry Messaging Publisher Pub/Sub Webhook Router Cloud Function Webhook Container Builder Building Dev Pipelines 

Confidential + Proprietary Advanced Chatflow with machine learning and bot analytics User types to custom UI or channel Chatbot replies Dialogflow Enterprise Customer Client JS Angular 5 web front-end Kubernetes Engine Chat Server Dialogflow SDK / socket.io Kubernetes Engine Back-end CRM Python / Django Kubernetes Engine Subscription Cloud Function Sensitivity Filter DLP API Sentiment Detector NLP API Data Warehouse BigQuery Messaging Publisher Pub/Sub Webhook Router Cloud Function Webhook 

Developers can use machine learning too! ● Tensorflow (JS or Python) ● With out of the box solutions for Speech, Video, Image or Text ● Training of custom models by uploading CSV data ● With solutions for call centers, smart speakers or chatbots. 

Lee Boonstra - @ladysign Thank you!