From Zero to ML on Google Cloud Platform

Slide 1

Slide 1 text

Confidential & Proprietary Zero to ML on GCP Sara Robinson Developer Advocate @SRobTweets

Slide 2

Slide 2 text

Confidential & Proprietary What is machine learning? @SRobTweets

Slide 3

Slide 3 text

Confidential & Proprietary Learning from examples and experience @SRobTweets

Slide 4

Slide 4 text

Confidential & Proprietary How did you learn your first language? @SRobTweets

Slide 5

Slide 5 text

Confidential & Proprietary ML is... Solving problems without explicitly knowing the solution Loosely based on how the human brain learns Enable systems that improve over time @SRobTweets

Slide 6

Slide 6 text

Confidential & Proprietary inputs, labels Model predictions Framing a machine learning problem @SRobTweets

Slide 7

Slide 7 text

Confidential & Proprietary How do we get from input to prediction? @SRobTweets

Slide 8

Slide 8 text

Confidential & Proprietary How do we get from input to prediction? “cat” @SRobTweets

Slide 9

Slide 9 text

Confidential & Proprietary How do we get from input to prediction? “cat” @SRobTweets

Slide 10

Slide 10 text

Confidential & Proprietary How do we get from input to prediction? @SRobTweets

Slide 11

Slide 11 text

Confidential & Proprietary How do we get from input to prediction? sports @SRobTweets

Slide 12

Slide 12 text

Confidential & Proprietary How do we get from input to prediction? sports baseball @SRobTweets

Slide 13

Slide 13 text

Confidential & Proprietary How do we get from input to prediction? sports baseball nytimes @SRobTweets

Slide 14

Slide 14 text

Confidential & Proprietary How do we get from input to prediction? @SRobTweets

Slide 15

Slide 15 text

Confidential & Proprietary How do we get from input to prediction? mountain pass nature birthday cake family @SRobTweets

Slide 16

Slide 16 text

Confidential & Proprietary Is machine learning only for experts?

Slide 17

Slide 17 text

Confidential & Proprietary First neural network 1957 @SRobTweets

Slide 18

Slide 18 text

Confidential & Proprietary First neural network 1957

Slide 19

Slide 19 text

Confidential & Proprietary

Slide 20

Slide 20 text

Confidential & Proprietary Democratizing ML @SRobTweets

Slide 21

Slide 21 text

Confidential & Proprietary What type of ML problem are you solving? Someone else has solved this before Specific to your dataset Custom task Generic task @SRobTweets

Slide 22

Slide 22 text

Confidential & Proprietary Custom task Generic task What type of ML problem are you solving? Someone else has solved this before Specific to your dataset “cat” “bob” Example: image classification @SRobTweets

Slide 23

Slide 23 text

Confidential & Proprietary Custom task Generic task What type of ML problem are you solving? Someone else has solved this before Specific to your dataset verb noun pronoun programming Google Cloud Example: NLP @SRobTweets

Slide 24

Slide 24 text

Confidential & Proprietary From zero… Machine Learning APIs ...to ML AutoML Custom model App developers Data scientists & ML practitioners @SRobTweets

Slide 25

Slide 25 text

Confidential & Proprietary What resources do you need to solve an ML problem? Training data Model code Training + serving infrastructure Prediction code Time Custom model: build from scratch Custom model: transfer learning AutoML ML APIs @SRobTweets

Slide 26

Slide 26 text

ML as an API Use a pre-trained model with a single REST API request

Slide 27

Slide 27 text

Confidential & Proprietary Use a pre-trained model to accomplish common ML tasks Cloud Vision Cloud Translation Cloud Natural Language Cloud Speech Cloud Video Intelligence @SRobTweets

Slide 28

Slide 28 text

Confidential & Proprietary Let’s look at Cloud Video & NL Cloud Vision Cloud Translation Cloud Natural Language Cloud Speech Cloud Video Intelligence @SRobTweets

Slide 29

Slide 29 text

Confidential & Proprietary Cloud Vision @SRobTweets

Slide 30

Slide 30 text

Confidential & Proprietary Label & web detection OCR Logo detection Explicit content detection Crop hints Landmark detection

Slide 31

Slide 31 text

Confidential & Proprietary Cloud Vision in production: GIPHY For more insight visit: Engineering.giphy.com @GIPHYEng

Slide 32

Slide 32 text

Confidential & Proprietary Let’s see a demo! @SRobTweets

Slide 33

Slide 33 text

Confidential & Proprietary Calling the Vision API from Node.js const gcloud = require('google-cloud'); const vision = gcloud.vision(); const types = [ 'face', 'label' ]; vision.detect('image.jpg', types, function(err, detections, apiResponse) { // detections = { // faces: [...], // labels: [...] // } }); @SRobTweets

Slide 34

Slide 34 text

Confidential & Proprietary Cloud Natural Language @SRobTweets

Slide 35

Slide 35 text

Extract entities Detect sentiment Analyze syntax Classify content

Slide 36

Slide 36 text

Confidential & Proprietary Analyze syntax The natural language API helps us understand text . Dependency Parse label det nn nn nsubj root nsubj ccomp dobj p Part of speech DET NOUN NOUN NOUN VERB PRON VERB NOUN PUNCT Lemma help Number = SINGULAR Proper = PROPER Number = SINGULAR Proper = PROPER Number = SINGULAR Proper = PROPER Mood = INDICATIVE Number = SINGULAR Person = THIRD Tense = Present Case = ACCUSATIVE Number = PLURAL Person = FIRST Number = SINGULAR Morphology @SRobTweets

Slide 37

Slide 37 text

Confidential & Proprietary Classify Content Rafael Montero Shines in Mets’ Victory Over the Reds. Montero, who was demoted at midseason, took a one-hitter into the ninth inning as the Mets continued to dominate Cincinnati with a win at Great American Ball Park. { categories: [ { name: '/Sports/Team Sports/Baseball', confidence: 0.99 } ] } @SRobTweets

Slide 38

Slide 38 text

Confidential & Proprietary Let’s see a demo! @SRobTweets

Slide 39

Slide 39 text

Confidential & Proprietary Calling the NL API from Node.js const gcloud = require('google-cloud'); const language = gcloud.language(); const bucket = gcs.bucket('my-bucket'); const file = bucket.file('my-file'); function callback(err, entities, apiResponse) {} language.annotate(file, callback); @SRobTweets

Slide 40

Slide 40 text

Confidential & Proprietary Use a pre-trained model to accomplish common ML tasks Cloud Vision Cloud Translation Cloud Natural Language Cloud Speech Cloud Video Intelligence @SRobTweets

Slide 41

Slide 41 text

Confidential & Proprietary What if we want to train these APIs on our own custom data? @SRobTweets

Slide 42

Slide 42 text

AutoML **alpha Use your own data to customize a pre-trained API

Slide 43

Slide 43 text

Confidential & Proprietary Let’s say I’m a meteorologist

Slide 44

Slide 44 text

Confidential & Proprietary I want to predict weather trends and flight plans from images

Slide 45

Slide 45 text

Confidential & Proprietary Can we use the cloud to analyze clouds?

Slide 46

Slide 46 text

Confidential & Proprietary There are 10+ different types of clouds

Slide 47

Slide 47 text

Confidential & Proprietary They all indicate different weather patterns

Slide 48

Slide 48 text

Confidential & Proprietary Let’s try the Vision API @SRobTweets

Slide 49

Slide 49 text

Confidential & Proprietary AutoML Vision to the rescue AutoML Vision Photo dataset Train Deploy Serve Generate predictions with a REST API @SRobTweets

Slide 50

Slide 50 text

Confidential & Proprietary Who is using AutoML Vision? In the past, ZSL used to track animals and understand their lives by having a human review photos from cameras deployed in the wild. Now they can tag these pictures automatically, enabling a deeper understanding across wider geographies to help protect the world’s wildlife. Build vision models to annotate shop Disney’s products with Disney characters, product category, and dominant color. Annotations are being integrated into Disney’s search engine to help users get more relevant results and find their ideal products faster. Building a model to create a comprehensive set of product attributes to improve product recommendations, search results, and product filters. Recognizing nuanced characteristics like patterns and neckline styles. @SRobTweets

Slide 51

Slide 51 text

Confidential & Proprietary What if you have a custom prediction task specific to your dataset or use case? @SRobTweets

Slide 52

Slide 52 text

Confidential & Proprietary Custom text classification @SRobTweets

Slide 53

Slide 53 text

Confidential & Proprietary Custom text classification javascript c++ sql @SRobTweets

Slide 54

Slide 54 text

Confidential & Proprietary Given demographic data, can we predict voting trends? @SRobTweets

Slide 55

Slide 55 text

Confidential & Proprietary Given demographic data, can we predict voting trends? inputs prediction @SRobTweets

Slide 56

Slide 56 text

Confidential & Proprietary Identifying the location of objects in an image @SRobTweets

Slide 57

Slide 57 text

Confidential & Proprietary Identifying the location of objects in an image sara 97% Kitten 99% @SRobTweets

Slide 58

Slide 58 text

Roll your own Build and train a custom model with your own data

Slide 59

Slide 59 text

Confidential & Proprietary Build train and serve your own models TensorFlow ML Engine @SRobTweets

Slide 60

Slide 60 text

Confidential & Proprietary Build train and serve your own models TensorFlow ML Engine @SRobTweets

Slide 61

Slide 61 text

Confidential & Proprietary TensorFlow Created by Google Brain team Most popular ML project on GitHub Multiple deployment options @SRobTweets

Slide 62

Slide 62 text

Confidential & Proprietary TensorFlow Distributed Execution Engine CPU GPU Android iOS ... C++ Frontend Python Frontend ... @SRobTweets

Slide 63

Slide 63 text

Confidential & Proprietary TensorFlow Distributed Execution Engine CPU GPU Android iOS ... C++ Frontend Python Frontend ... tf.layers Build models @SRobTweets

Slide 64

Slide 64 text

Confidential & Proprietary TensorFlow Distributed Execution Engine CPU GPU Android iOS ... C++ Frontend Python Frontend ... Estimator tf.layers Build models Keras @SRobTweets

Slide 65

Slide 65 text

Confidential & Proprietary TensorFlow Distributed Execution Engine CPU GPU Android iOS ... C++ Frontend Python Frontend ... Models in a box Estimator tf.layers Build models Keras Pre-built Estimators @SRobTweets

Slide 66

Slide 66 text

Confidential & Proprietary TensorFlow Distributed Execution Engine CPU GPU Android iOS ... C++ Frontend Python Frontend ... Layers Pre-built Estimators Estimator Keras @SRobTweets

Slide 67

Slide 67 text

Confidential & Proprietary Once you’ve got a TensorFlow model, what about…. Training Serving & @SRobTweets

Slide 68

Slide 68 text

Confidential & Proprietary Cloud Machine Learning Engine (ML Engine) Fully managed platform for TensorFlow Distributed Training with GPUs (and TPUs) Fast and scalable online/batch prediction cloud.google.com/ml-engine @SRobTweets

Slide 69

Slide 69 text

Confidential & Proprietary ML Engine in 3 steps Prepare your TensorFlow code and training data on Cloud Storage Run CMLE job with gcloud command Monitor the job on Cloud Console @SRobTweets

Slide 70

Slide 70 text

Confidential & Proprietary Example: building a Taylor Swift detector @SRobTweets

Slide 71

Slide 71 text

Confidential & Proprietary TensorFlow Object Detection to build the object recognition model @SRobTweets

Slide 72

Slide 72 text

Confidential & Proprietary ...and ML Engine to train, serve, and generate predictions @SRobTweets

Slide 73

Slide 73 text

Confidential & Proprietary TensorFlow Object Detection API @SRobTweets

Slide 74

Slide 74 text

Confidential & Proprietary TensorFlow Object Detection API @SRobTweets

Slide 75

Slide 75 text

Confidential & Proprietary Need a custom model but don’t have enough training data? Transfer learning Model trained on lots of data Your data Updated output using your training data @SRobTweets

Slide 76

Slide 76 text

Confidential & Proprietary How does it all fit together? Learn more: bit.ly/swift-detector @SRobTweets

Slide 77

Slide 77 text

Confidential & Proprietary What if we have a custom task and enough training data to build + train our model from scratch? @SRobTweets

Slide 78

Slide 78 text

Confidential & Proprietary Custom model: build from scratch Training data Model code Serving infrastructure Prediction code Time @SRobTweets

Slide 79

Slide 79 text

Confidential & Proprietary There are ~3,000 counties in the US Can we use demographic data to predict which way a county will vote? Example: a classification model to predict voting trends @SRobTweets

Slide 80

Slide 80 text

Confidential & Proprietary Get the data → Kaggle @SRobTweets

Slide 81

Slide 81 text

Confidential & Proprietary Select features from BigQuery @SRobTweets

Slide 82

Slide 82 text

Confidential & Proprietary From this... inputs prediction @SRobTweets

Slide 83

Slide 83 text

Confidential & Proprietary ...to this inputs prediction @SRobTweets

Slide 84

Slide 84 text

Confidential & Proprietary Preprocess our input data import pandas as pd df = pd.read_csv('data.csv', header=None) for idx, row in df.iterrows(): # Convert % to binary classes: 0 = Trump, 1 = Clinton classification_val = 0 if row[7] >= .50: classification_val = 1 classification_row = row.as_matrix() classification_row[7] = classification_val new_rows.append(classification_row) # Write new_rows to new csv file @SRobTweets

Slide 85

Slide 85 text

Confidential & Proprietary Define feature columns feature_names = [ 'PctUnder18', 'PctOver65', 'PctFemale', 'PctWhite', 'PctBachelors', 'PctDem', 'PctGop' ] feature_columns = [tf.feature_column.numeric_column(k) for k in feature_names] @SRobTweets

Slide 86

Slide 86 text

Confidential & Proprietary Define our input function def my_input_fn(file_path, repeat_count): def decode_csv(line): features = tf.decode_csv(line, [[0.],[0.],[0.],[0.],[0.],[0.],[0.],[0.]]) label = features[-1:] del features[-1] return dict(zip(feature_names, features)), label dataset = (tf.data.TextLineDataset(file_path) .map(decode_csv) .shuffle(buffer_size=256) .repeat(repeat_count) .batch(8)) iterator = dataset.make_one_shot_iterator() batch_features, batch_labels = iterator.get_next() return batch_features, batch_labels @SRobTweets

Slide 87

Slide 87 text

Slide 88

Slide 88 text

Confidential & Proprietary Define our input function def my_input_fn(file_path, repeat_count): def decode_csv(line): features = tf.decode_csv(line, [[0.],[0.],[0.],[0.],[0.],[0.],[0.],[0.]]) label = features[-1:] del features[-1] return dict(zip(feature_names, features)), label dataset = (tf.data.TextLineDataset(file_path) .map(decode_csv) .shuffle(buffer_size=256) .repeat(repeat_count) .batch(16)) iterator = dataset.make_one_shot_iterator() batch_features, batch_labels = iterator.get_next() return batch_features, batch_labels @SRobTweets

Slide 89

Slide 89 text

Confidential & Proprietary Define our input function def my_input_fn(file_path, repeat_count): def decode_csv(line): features = tf.decode_csv(line, [[0.],[0.],[0.],[0.],[0.],[0.],[0.],[0.]]) label = features[-1:] del features[-1] return dict(zip(feature_names, features)), label dataset = (tf.data.TextLineDataset(file_path) .map(decode_csv) .shuffle(buffer_size=256) .repeat(repeat_count) .batch(16)) iterator = dataset.make_one_shot_iterator() batch_features, batch_labels = iterator.get_next() return batch_features, batch_labels @SRobTweets

Slide 90

Slide 90 text

Confidential & Proprietary Create a classification model classifier = tf.estimator.LinearClassifier( feature_columns=feature_columns ) @SRobTweets

Slide 91

Slide 91 text

Confidential & Proprietary Run training & evaluation # Run training for 10 epochs classifier.train( input_fn=lambda: my_input_fn(train_file, 10)) # Evaluate the accuracy of our trained model results = classifier.evaluate( input_fn=lambda: my_input_fn(test_file, 1)) @SRobTweets

Slide 92

Slide 92 text

Confidential & Proprietary Run training & evaluation # Evaluation output accuracy: 0.965686 auc_precision_recall: 0.942694 average_loss: 0.110322 label/mean: 0.111111 prediction/mean: 0.130689 @SRobTweets

Slide 93

Slide 93 text

Confidential & Proprietary Generate predictions predict_results = classifier.predict( input_fn=lambda: my_input_fn(test_file, 4) ) @SRobTweets

Slide 94

Slide 94 text

Confidential & Proprietary Generate predictions { 'probabilities': array( [ 0.99163872, 0.00836132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object) } @SRobTweets

Slide 95

Slide 95 text

Confidential & Proprietary Generate predictions { 'probabilities': array( [ 0.99163872, 0.00836132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object) } @SRobTweets

Slide 96

Slide 96 text

Confidential & Proprietary Let’s see it in action! @SRobTweets

Slide 97

Slide 97 text

Confidential & Proprietary If you remember 3 things from this presentation

Slide 98

Slide 98 text

Confidential & Proprietary Use a pre-trained API to accomplish common ML tasks like image analysis, NLP, or translation @SRobTweets

Slide 99

Slide 99 text

Confidential & Proprietary To build an image classification API trained on your own data, use AutoML Vision @SRobTweets

Slide 100

Slide 100 text

Confidential & Proprietary For custom tasks, build a TensorFlow model with your data. Train and serve it on ML Engine. @SRobTweets

Slide 101

Slide 101 text

Confidential & Proprietary Video: cloud.google.com/video-intelligence Vision: cloud.google.com/vision Speech: cloud.google.com/speech Natural Language: cloud.google.com/natural-language API overview talk: bit.ly/ml-apis-overview Sign up for the alpha: cloud.google.com/automl Announcement blog post: bit.ly/announcing-automl Intro video: bit.ly/automl-intro-video Podcast: bit.ly/automl-podcast Object Detection: bit.ly/tf-obj-detection T-Swift detector blog: bit.ly/swift-detector Pet detector blog: bit.ly/pet-model-ml-engine TensorFlow: tensorflow.org ML Engine: cloud.google.com/ml-engine TF Estimators: bit.ly/tf-estimators Keras blog post: bit.ly/text-classification-keras ML APIs AutoML Transfer learning Custom model @SRobTweets

Slide 102

Slide 102 text

Confidential & Proprietary Thank you @SRobTweets medium.com/@SRobTweets