Machine Intelligence at Google Scale

What is Neural Network? Why is Deep Learning so important? What are the challenges for introducing those technologies to production services? In this session, we will look at the answers for those questions and how Google has been successfully deploying large scale neural network on services such as Google Photos, Android and Google Search. Also, we will introduce the new Google Cloud products such as Cloud Vision API, Speech API, TensorFlow and Cloud ML that allows developers to take advantage of the power of Google's machine intelligence with scalable and fully managed services.

「より使いやすい機械学習へ：Googleの機械学習テクノロジー」
ニューラルネットワークとは何か？　ディープラーニングはなぜ注目を集めているのか？　実開発での利用の課題は？　このセッションでは、これらの疑問について議論し、さらにGoogleフォトやAndroid、Google検索等で利用されているGoogleにおける大規模なニューラルネットワーク導入事例を紹介します。また後半では、Cloud Vision API、Speech API、TensorFlow、CloudMLなど、Googleが提供するスケーラブルでフルマネージドのクラウドサービスについて解説し、Googleの機械学習技術を活用する方法を紹介します。

Transcript

1. Machine Intelligence at Google Scale ML APIs, TensorFlow and Cloud ML
2. +Kazunori Sato @kazunori_279 Kaz Sato Staff Developer Advocate Tech Lead for Data & Analytics Cloud Platform, Google Inc.
3. What we’ll cover What is Neural Network and Deep Learning Machine Learning use cases at Google services Externalizing the power with ML APIs TensorFlow: the open source library for ML TensorFlow in the Wild Distributed training and prediction with Cloud ML
4. What is Neural Network and Deep Learning
5. Neural Network is a function that can learn
6. x n > b? w 1 w n x 2 x 1 Inspired by the behavior of biological neurons
7. How do you classify them?
8. weights bias (threshold) Programmers need to specify the parameters
9. Let’s see how neural network solves the problem
10. The computer tries to find the best parameters A neuron classifies a data point into two kinds
11. Gradient Descent: adjusting the params gradually to reduce errors From: Andrew Ng
12. How do you classify them?
13. What we see What the computer “sees”
14. 28 x 28 gray scale image = 784 numbers
15. input vector (pixel data) output vector (probability)
16. How do you classify them?
17. More neurons = More features to extract
18. Hidden Layers: mapping inputs to a feature space, classifying with a hyperplane From: Neural Networks, Manifolds, and Topology, colah's blog
19. How about this?
20. More hidden layers = More hierarchies of features
21. How about this?
22. We need to go deeper neural network From: Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, Honglak Lee et al.
23. From: mNeuron: A Matlab Plugin to Visualize Neurons from Deep Models, Donglai Wei et. al.
24. Machine Learning use cases at Google services
25. 25 signal for Search ranking, out of hundreds improvement to ranking quality in 2+ years #3 #1 Search machine learning for search engines RankBrain: a deep neural network for search ranking
27. WaveNet by Google DeepMind
28. 28 [glacier] Google Photos 28
29. 29 Smart reply in Inbox by Gmail 10% of all responses sent on mobile
30. http://googleresearch.blogspot.com/2016/05/announcing-syntaxnet-worlds-most.html
31. Saved Data Center cooling energy for 40% Improved Power Usage Effectiveness (PUE) for 15%
32. 32 Android Apps Gmail Maps Photos Speech Search Translation YouTube and many others ... Used across products: 2012 2013 2014 2015 Deep Learning usage at Google
33. Externalizing the power with ML APIs
34. TensorFlow Cloud Machine Learning ML API Easy-to-Use, for non-ML engineers Customizable, for Data Scientists Machine Learning products from Google
35. Image analysis with pre-trained models No Machine Learning skill required REST API: receives an image and returns a JSON General Availability Cloud Vision API
36. Confidential & Proprietary Google Cloud Platform 36 Faces Faces, facial landmarks, emotions OCR Read and extract text, with support for > 10 languages Label Detect entities from furniture to transportation Logos Identify product logos Landmarks & Image Properties Detect landmarks & dominant color of image Safe Search Detect explicit content - adult, violent, medical and spoof
37. 37 37 Demo
38. Pre-trained models. No ML skill required REST API: receives audio and returns texts Supports 80+ languages Streaming or non-streaming Public Beta - cloud.google.com/speech Cloud Speech API
39. Confidential & Proprietary Google Cloud Platform 39 Features Automatic Speech Recognition (ASR) powered by deep learning neural networking to power your applications like voice search or speech transcription. Recognizes over 80 languages and variants with an extensive vocabulary. Returns partial recognition results immediately, as they become available. Filter inappropriate content in text results. Audio input can be captured by an application’s microphone or sent from a pre-recorded audio file. Multiple audio file formats are supported, including FLAC, AMR, PCMU and linear-16. Handles noisy audio from many environments without requiring additional noise cancellation. Audio files can be uploaded in the request and, in future releases, integrated with Google Cloud Storage. Automatic Speech Recognition Global Vocabulary Inappropriate Content Filtering Streaming Recognition Real-time or Buffered Audio Support Noisy Audio Handling Integrated API
40. 40 40 Demo
41. Pre-trained models. No ML skill required REST API: receives text and returns analysis results Supports English, Spanish and Japanese Public Beta - cloud.google.com/natural-language Cloud Natural Language API
42. Confidential & Proprietary Google Cloud Platform 42 Features Extract sentence, identify parts of speech and create dependency parse trees for each sentence. Identify entities and label by types such as person, organization, location, events, products and media. Understand the overall sentiment of a block of text. Syntax Analysis Entity Recognition Sentiment Analysis
43. 43 43 Demo
44. TensorFlow: An open source library for Machine Intelligence
45. Google's open source library for machine intelligence tensorflow.org launched in Nov 2015 Used by many production ML projects What is TensorFlow?
46. Sharing our tools with researchers and developers around the world repository for “machine learning” category on GitHub #1 Released in Nov. 2015 From: http://deliprao.com/archives/168
47. Before Hire Data Scientists ↓ Understand the math model ↓ Impl with programming code ↓ Train with single GPU ↓ Build a GPU cluster ↓ Train with the GPU cluster ↓ Build a prediction server or Impl mobile/IoT prediction After Easy network design and impl ↓ Train with single machine ↓ Train on the cloud ↓ Prediction on the cloud or mobile/IoT devices many people stuck here
48. # define the network import tensorflow as tf x = tf.placeholder(tf.float32, [None, 784]) W = tf.Variable(tf.zeros([784, 10])) b = tf.Variable(tf.zeros([10])) y = tf.nn.softmax(tf.matmul(x, W) + b) # define a training step y_ = tf.placeholder(tf.float32, [None, 10]) xent = -tf.reduce_sum(y_*tf.log(y)) step = tf.train.GradientDescentOptimizer(0.01).minimize(xent)
49. TensorBoard: visualization tool
50. Portable and Scalable Training on: Mac/Windows GPU server GPU cluster / Cloud Prediction on: Android and iOS RasPi and TPU
51. Distributed Training with TensorFlow
52. TensorFlow in the Wild (or democratization of deep learning)
53. TensorFlow powered Fried Chicken Nugget Server From: http://www.rt-net.jp/karaage1/
54. TensorFlow powered Cucumber Sorter From: http://workpiles.com/2016/02/tensorflow-cnn-cucumber/
55. Autonomous Driving of RasPi car with Inception 3 on TensorFlow From: https://github.com/zxzhijia/GoPiGo-Driven-by-Te nsorflow
56. TV popstar classifier with 95% accuracy From: http://memo.sugyan.com/entry/2016/06/14/220624
57. TensorFlow＋ RasPi for sorting garbages From: https://techcrunch.com/2016/09/13/auto-trash-sorts- garbage-automatically-at-the-techcrunch-disrupt-hacka thon/
58. TensorFlow + Drones for counting trucks From: http://www.brainpad.co.jp/news/2016/09/02/3454
59. From: http://otoro.net/ Generative Arts with TensorFlow
60. Distributed Training and Prediction with Cloud ML
61. From: Andrew Ng The Bigger, The Better
62. The Challenge: Computing Power DNN requires large training datasets Large models doesn't fit into a GPU Requires try-and-errors to find the best design, configs and params ↓ Need to spend a few days or weeks to finish a training
63. GPUs run at nanoseconds GPU cluster needs microsec network
65. Enterprise Google Cloud is The Datacenter as a Computer
66. Jupiter network 10 GbE x 100 K = 1 Pbps Consolidates servers with microsec latency
67. Borg No VMs, pure containers 10K - 20K nodes per Cell DC-scale job scheduling CPUs, mem, disks and IO
68. Distributed Training with TensorFlow by data parallelism split data, share model
69. • CPU/GPU scheduling • Communications ◦ Local, RPC, RDMA ◦ 32/16/8 bit quantization • Cost-based optimization • Fault tolerance Distributed Systems for Large Neural Network
70. What's the scalability of Google Brain? "Large Scale Distributed Systems for Training Neural Networks", NIPS 2015 ◦ Inception / ImageNet: 40x with 50 GPUs ◦ RankBrain: 300x with 500 nodes
71. Fully managed distributed training and prediction Supports custom TensorFlow graphs Integrated with Cloud Dataflow and Cloud Datalab Limited Preview - cloud.google.com/ml Cloud Machine Learning (Cloud ML)
72. 72 72 Ready to use Machine Learning models Use your own data to train models Cloud Vision API Cloud Speech API Cloud Translate API Cloud Machine Learning Develop - Model - Test Google BigQuery Stay Tuned…. Cloud Storage Cloud Datalab NEW Alpha GA Beta GA Alpha GA GA
73. Tensor Processing Unit ASIC for TensorFlow Designed by Google 10x better perf / watt latency and efficiency bit quantization
74. TPU on Production RankBrain AlphaGo Google Photos Speech and more
75. Thank you!
76. Links & Resources Large Scale Distributed Systems for Training Neural Networks, Jeff Dean and Oriol Vinals Cloud Vision API: cloud.google.com/vision Cloud Speech API: cloud.google.com/speech TensorFlow: tensorflow.org Cloud Machine Learning: cloud.google.com/ml Cloud Machine Learning: demo video