Slide 1

Slide 1 text

Machine Intelligence at Google Scale ML APIs, TensorFlow and Cloud ML

Slide 2

Slide 2 text

+Kazunori Sato @kazunori_279 Kaz Sato Staff Developer Advocate Tech Lead for Data & Analytics Cloud Platform, Google Inc.

Slide 3

Slide 3 text

What we’ll cover What is Neural Network and Deep Learning Machine Learning use cases at Google services Externalizing the power with ML APIs TensorFlow: the open source library for ML TensorFlow in the Wild Distributed training and prediction with Cloud ML

Slide 4

Slide 4 text

What is Neural Network and Deep Learning

Slide 5

Slide 5 text

Neural Network is a function that can learn

Slide 6

Slide 6 text

x n > b? w 1 w n x 2 x 1 Inspired by the behavior of biological neurons

Slide 7

Slide 7 text

How do you classify them?

Slide 8

Slide 8 text

weights bias (threshold) Programmers need to specify the parameters

Slide 9

Slide 9 text

Let’s see how neural network solves the problem

Slide 10

Slide 10 text

The computer tries to find the best parameters A neuron classifies a data point into two kinds

Slide 11

Slide 11 text

Gradient Descent: adjusting the params gradually to reduce errors From: Andrew Ng

Slide 12

Slide 12 text

How do you classify them?

Slide 13

Slide 13 text

What we see What the computer “sees”

Slide 14

Slide 14 text

28 x 28 gray scale image = 784 numbers

Slide 15

Slide 15 text

input vector (pixel data) output vector (probability)

Slide 16

Slide 16 text

How do you classify them?

Slide 17

Slide 17 text

More neurons = More features to extract

Slide 18

Slide 18 text

Hidden Layers: mapping inputs to a feature space, classifying with a hyperplane From: Neural Networks, Manifolds, and Topology, colah's blog

Slide 19

Slide 19 text

How about this?

Slide 20

Slide 20 text

More hidden layers = More hierarchies of features

Slide 21

Slide 21 text

How about this?

Slide 22

Slide 22 text

We need to go deeper neural network From: Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, Honglak Lee et al.

Slide 23

Slide 23 text

From: mNeuron: A Matlab Plugin to Visualize Neurons from Deep Models, Donglai Wei et. al.

Slide 24

Slide 24 text

Machine Learning use cases at Google services

Slide 25

Slide 25 text

25 signal for Search ranking, out of hundreds improvement to ranking quality in 2+ years #3 #1 Search machine learning for search engines RankBrain: a deep neural network for search ranking

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

WaveNet by Google DeepMind

Slide 28

Slide 28 text

28 [glacier] Google Photos 28

Slide 29

Slide 29 text

29 Smart reply in Inbox by Gmail 10% of all responses sent on mobile

Slide 30

Slide 30 text

http://googleresearch.blogspot.com/2016/05/announcing-syntaxnet-worlds-most.html

Slide 31

Slide 31 text

Saved Data Center cooling energy for 40% Improved Power Usage Effectiveness (PUE) for 15%

Slide 32

Slide 32 text

32 Android Apps Gmail Maps Photos Speech Search Translation YouTube and many others ... Used across products: 2012 2013 2014 2015 Deep Learning usage at Google

Slide 33

Slide 33 text

Externalizing the power with ML APIs

Slide 34

Slide 34 text

TensorFlow Cloud Machine Learning ML API Easy-to-Use, for non-ML engineers Customizable, for Data Scientists Machine Learning products from Google

Slide 35

Slide 35 text

Image analysis with pre-trained models No Machine Learning skill required REST API: receives an image and returns a JSON General Availability Cloud Vision API

Slide 36

Slide 36 text

Confidential & Proprietary Google Cloud Platform 36 Faces Faces, facial landmarks, emotions OCR Read and extract text, with support for > 10 languages Label Detect entities from furniture to transportation Logos Identify product logos Landmarks & Image Properties Detect landmarks & dominant color of image Safe Search Detect explicit content - adult, violent, medical and spoof

Slide 37

Slide 37 text

37 37 Demo

Slide 38

Slide 38 text

Pre-trained models. No ML skill required REST API: receives audio and returns texts Supports 80+ languages Streaming or non-streaming Public Beta - cloud.google.com/speech Cloud Speech API

Slide 39

Slide 39 text

Confidential & Proprietary Google Cloud Platform 39 Features Automatic Speech Recognition (ASR) powered by deep learning neural networking to power your applications like voice search or speech transcription. Recognizes over 80 languages and variants with an extensive vocabulary. Returns partial recognition results immediately, as they become available. Filter inappropriate content in text results. Audio input can be captured by an application’s microphone or sent from a pre-recorded audio file. Multiple audio file formats are supported, including FLAC, AMR, PCMU and linear-16. Handles noisy audio from many environments without requiring additional noise cancellation. Audio files can be uploaded in the request and, in future releases, integrated with Google Cloud Storage. Automatic Speech Recognition Global Vocabulary Inappropriate Content Filtering Streaming Recognition Real-time or Buffered Audio Support Noisy Audio Handling Integrated API

Slide 40

Slide 40 text

40 40 Demo

Slide 41

Slide 41 text

Pre-trained models. No ML skill required REST API: receives text and returns analysis results Supports English, Spanish and Japanese Public Beta - cloud.google.com/natural-language Cloud Natural Language API

Slide 42

Slide 42 text

Confidential & Proprietary Google Cloud Platform 42 Features Extract sentence, identify parts of speech and create dependency parse trees for each sentence. Identify entities and label by types such as person, organization, location, events, products and media. Understand the overall sentiment of a block of text. Syntax Analysis Entity Recognition Sentiment Analysis

Slide 43

Slide 43 text

43 43 Demo

Slide 44

Slide 44 text

TensorFlow: An open source library for Machine Intelligence

Slide 45

Slide 45 text

Google's open source library for machine intelligence tensorflow.org launched in Nov 2015 Used by many production ML projects What is TensorFlow?

Slide 46

Slide 46 text

Sharing our tools with researchers and developers around the world repository for “machine learning” category on GitHub #1 Released in Nov. 2015 From: http://deliprao.com/archives/168

Slide 47

Slide 47 text

Before Hire Data Scientists ↓ Understand the math model ↓ Impl with programming code ↓ Train with single GPU ↓ Build a GPU cluster ↓ Train with the GPU cluster ↓ Build a prediction server or Impl mobile/IoT prediction After Easy network design and impl ↓ Train with single machine ↓ Train on the cloud ↓ Prediction on the cloud or mobile/IoT devices many people stuck here

Slide 48

Slide 48 text

# define the network import tensorflow as tf x = tf.placeholder(tf.float32, [None, 784]) W = tf.Variable(tf.zeros([784, 10])) b = tf.Variable(tf.zeros([10])) y = tf.nn.softmax(tf.matmul(x, W) + b) # define a training step y_ = tf.placeholder(tf.float32, [None, 10]) xent = -tf.reduce_sum(y_*tf.log(y)) step = tf.train.GradientDescentOptimizer(0.01).minimize(xent)

Slide 49

Slide 49 text

TensorBoard: visualization tool

Slide 50

Slide 50 text

Portable and Scalable Training on: Mac/Windows GPU server GPU cluster / Cloud Prediction on: Android and iOS RasPi and TPU

Slide 51

Slide 51 text

Distributed Training with TensorFlow

Slide 52

Slide 52 text

TensorFlow in the Wild (or democratization of deep learning)

Slide 53

Slide 53 text

TensorFlow powered Fried Chicken Nugget Server From: http://www.rt-net.jp/karaage1/

Slide 54

Slide 54 text

TensorFlow powered Cucumber Sorter From: http://workpiles.com/2016/02/tensorflow-cnn-cucumber/

Slide 55

Slide 55 text

Autonomous Driving of RasPi car with Inception 3 on TensorFlow From: https://github.com/zxzhijia/GoPiGo-Driven-by-Te nsorflow

Slide 56

Slide 56 text

TV popstar classifier with 95% accuracy From: http://memo.sugyan.com/entry/2016/06/14/220624

Slide 57

Slide 57 text

TensorFlow+ RasPi for sorting garbages From: https://techcrunch.com/2016/09/13/auto-trash-sorts- garbage-automatically-at-the-techcrunch-disrupt-hacka thon/

Slide 58

Slide 58 text

TensorFlow + Drones for counting trucks From: http://www.brainpad.co.jp/news/2016/09/02/3454

Slide 59

Slide 59 text

From: http://otoro.net/ Generative Arts with TensorFlow

Slide 60

Slide 60 text

Distributed Training and Prediction with Cloud ML

Slide 61

Slide 61 text

From: Andrew Ng The Bigger, The Better

Slide 62

Slide 62 text

The Challenge: Computing Power DNN requires large training datasets Large models doesn't fit into a GPU Requires try-and-errors to find the best design, configs and params ↓ Need to spend a few days or weeks to finish a training

Slide 63

Slide 63 text

GPUs run at nanoseconds GPU cluster needs microsec network

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

Enterprise Google Cloud is The Datacenter as a Computer

Slide 66

Slide 66 text

Jupiter network 10 GbE x 100 K = 1 Pbps Consolidates servers with microsec latency

Slide 67

Slide 67 text

Borg No VMs, pure containers 10K - 20K nodes per Cell DC-scale job scheduling CPUs, mem, disks and IO

Slide 68

Slide 68 text

Distributed Training with TensorFlow by data parallelism split data, share model

Slide 69

Slide 69 text

● CPU/GPU scheduling ● Communications ○ Local, RPC, RDMA ○ 32/16/8 bit quantization ● Cost-based optimization ● Fault tolerance Distributed Systems for Large Neural Network

Slide 70

Slide 70 text

What's the scalability of Google Brain? "Large Scale Distributed Systems for Training Neural Networks", NIPS 2015 ○ Inception / ImageNet: 40x with 50 GPUs ○ RankBrain: 300x with 500 nodes

Slide 71

Slide 71 text

Fully managed distributed training and prediction Supports custom TensorFlow graphs Integrated with Cloud Dataflow and Cloud Datalab Limited Preview - cloud.google.com/ml Cloud Machine Learning (Cloud ML)

Slide 72

Slide 72 text

72 72 Ready to use Machine Learning models Use your own data to train models Cloud Vision API Cloud Speech API Cloud Translate API Cloud Machine Learning Develop - Model - Test Google BigQuery Stay Tuned…. Cloud Storage Cloud Datalab NEW Alpha GA Beta GA Alpha GA GA

Slide 73

Slide 73 text

Tensor Processing Unit ASIC for TensorFlow Designed by Google 10x better perf / watt latency and efficiency bit quantization

Slide 74

Slide 74 text

TPU on Production RankBrain AlphaGo Google Photos Speech and more

Slide 75

Slide 75 text

Thank you!

Slide 76

Slide 76 text

Links & Resources Large Scale Distributed Systems for Training Neural Networks, Jeff Dean and Oriol Vinals Cloud Vision API: cloud.google.com/vision Cloud Speech API: cloud.google.com/speech TensorFlow: tensorflow.org Cloud Machine Learning: cloud.google.com/ml Cloud Machine Learning: demo video