Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Intelligence at Google Scale

Machine Intelligence at Google Scale

What is Neural Network? Why is Deep Learning so important? What are the challenges for introducing those technologies to production services? In this session, we will look at the answers for those questions and how Google has been successfully deploying large scale neural network on services such as Google Photos, Android and Google Search. Also, we will introduce the new Google Cloud products such as Cloud Vision API, Speech API, TensorFlow and Cloud ML that allows developers to take advantage of the power of Google's machine intelligence with scalable and fully managed services.

ニューラルネットワークとは何か? ディープラーニングはなぜ注目を集めているのか? 実開発での利用の課題は? このセッションでは、これらの疑問について議論し、さらにGoogleフォトやAndroid、Google検索等で利用されているGoogleにおける大規模なニューラルネットワーク導入事例を紹介します。また後半では、Cloud Vision API、Speech API、TensorFlow、CloudMLなど、Googleが提供するスケーラブルでフルマネージドのクラウドサービスについて解説し、Googleの機械学習技術を活用する方法を紹介します。

Kazunori Sato

April 28, 2016

More Decks by Kazunori Sato

Other Decks in Programming


  1. +Kazunori Sato @kazunori_279 Kaz Sato Staff Developer Advocate Tech Lead

    for Data & Analytics Cloud Platform, Google Inc.
  2. What we’ll cover What is Neural Network and Deep Learning

    Machine Learning use cases at Google services Externalizing the power with ML APIs TensorFlow: the open source library for ML TensorFlow in the Wild Distributed training and prediction with Cloud ML
  3. x n > b? w 1 w n x 2

    x 1 Inspired by the behavior of biological neurons
  4. The computer tries to find the best parameters A neuron

    classifies a data point into two kinds
  5. Hidden Layers: mapping inputs to a feature space, classifying with

    a hyperplane From: Neural Networks, Manifolds, and Topology, colah's blog
  6. We need to go deeper neural network From: Convolutional Deep

    Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, Honglak Lee et al.
  7. 25 signal for Search ranking, out of hundreds improvement to

    ranking quality in 2+ years #3 #1 Search machine learning for search engines RankBrain: a deep neural network for search ranking
  8. 32 Android Apps Gmail Maps Photos Speech Search Translation YouTube

    and many others ... Used across products: 2012 2013 2014 2015 Deep Learning usage at Google
  9. TensorFlow Cloud Machine Learning ML API Easy-to-Use, for non-ML engineers

    Customizable, for Data Scientists Machine Learning products from Google
  10. Image analysis with pre-trained models No Machine Learning skill required

    REST API: receives an image and returns a JSON General Availability Cloud Vision API
  11. Confidential & Proprietary Google Cloud Platform 36 Faces Faces, facial

    landmarks, emotions OCR Read and extract text, with support for > 10 languages Label Detect entities from furniture to transportation Logos Identify product logos Landmarks & Image Properties Detect landmarks & dominant color of image Safe Search Detect explicit content - adult, violent, medical and spoof
  12. Pre-trained models. No ML skill required REST API: receives audio

    and returns texts Supports 80+ languages Streaming or non-streaming Public Beta - cloud.google.com/speech Cloud Speech API
  13. Confidential & Proprietary Google Cloud Platform 39 Features Automatic Speech

    Recognition (ASR) powered by deep learning neural networking to power your applications like voice search or speech transcription. Recognizes over 80 languages and variants with an extensive vocabulary. Returns partial recognition results immediately, as they become available. Filter inappropriate content in text results. Audio input can be captured by an application’s microphone or sent from a pre-recorded audio file. Multiple audio file formats are supported, including FLAC, AMR, PCMU and linear-16. Handles noisy audio from many environments without requiring additional noise cancellation. Audio files can be uploaded in the request and, in future releases, integrated with Google Cloud Storage. Automatic Speech Recognition Global Vocabulary Inappropriate Content Filtering Streaming Recognition Real-time or Buffered Audio Support Noisy Audio Handling Integrated API
  14. Pre-trained models. No ML skill required REST API: receives text

    and returns analysis results Supports English, Spanish and Japanese Public Beta - cloud.google.com/natural-language Cloud Natural Language API
  15. Confidential & Proprietary Google Cloud Platform 42 Features Extract sentence,

    identify parts of speech and create dependency parse trees for each sentence. Identify entities and label by types such as person, organization, location, events, products and media. Understand the overall sentiment of a block of text. Syntax Analysis Entity Recognition Sentiment Analysis
  16. Google's open source library for machine intelligence tensorflow.org launched in

    Nov 2015 Used by many production ML projects What is TensorFlow?
  17. Sharing our tools with researchers and developers around the world

    repository for “machine learning” category on GitHub #1 Released in Nov. 2015 From: http://deliprao.com/archives/168
  18. Before Hire Data Scientists ↓ Understand the math model ↓

    Impl with programming code ↓ Train with single GPU ↓ Build a GPU cluster ↓ Train with the GPU cluster ↓ Build a prediction server or Impl mobile/IoT prediction After Easy network design and impl ↓ Train with single machine ↓ Train on the cloud ↓ Prediction on the cloud or mobile/IoT devices many people stuck here
  19. # define the network import tensorflow as tf x =

    tf.placeholder(tf.float32, [None, 784]) W = tf.Variable(tf.zeros([784, 10])) b = tf.Variable(tf.zeros([10])) y = tf.nn.softmax(tf.matmul(x, W) + b) # define a training step y_ = tf.placeholder(tf.float32, [None, 10]) xent = -tf.reduce_sum(y_*tf.log(y)) step = tf.train.GradientDescentOptimizer(0.01).minimize(xent)
  20. Portable and Scalable Training on: Mac/Windows GPU server GPU cluster

    / Cloud Prediction on: Android and iOS RasPi and TPU
  21. Autonomous Driving of RasPi car with Inception 3 on TensorFlow

    From: https://github.com/zxzhijia/GoPiGo-Driven-by-Te nsorflow
  22. The Challenge: Computing Power DNN requires large training datasets Large

    models doesn't fit into a GPU Requires try-and-errors to find the best design, configs and params ↓ Need to spend a few days or weeks to finish a training
  23. Jupiter network 10 GbE x 100 K = 1 Pbps

    Consolidates servers with microsec latency
  24. Borg No VMs, pure containers 10K - 20K nodes per

    Cell DC-scale job scheduling CPUs, mem, disks and IO
  25. • CPU/GPU scheduling • Communications ◦ Local, RPC, RDMA ◦

    32/16/8 bit quantization • Cost-based optimization • Fault tolerance Distributed Systems for Large Neural Network
  26. What's the scalability of Google Brain? "Large Scale Distributed Systems

    for Training Neural Networks", NIPS 2015 ◦ Inception / ImageNet: 40x with 50 GPUs ◦ RankBrain: 300x with 500 nodes
  27. Fully managed distributed training and prediction Supports custom TensorFlow graphs

    Integrated with Cloud Dataflow and Cloud Datalab Limited Preview - cloud.google.com/ml Cloud Machine Learning (Cloud ML)
  28. 72 72 Ready to use Machine Learning models Use your

    own data to train models Cloud Vision API Cloud Speech API Cloud Translate API Cloud Machine Learning Develop - Model - Test Google BigQuery Stay Tuned…. Cloud Storage Cloud Datalab NEW Alpha GA Beta GA Alpha GA GA
  29. Tensor Processing Unit ASIC for TensorFlow Designed by Google 10x

    better perf / watt latency and efficiency bit quantization
  30. Links & Resources Large Scale Distributed Systems for Training Neural

    Networks, Jeff Dean and Oriol Vinals Cloud Vision API: cloud.google.com/vision Cloud Speech API: cloud.google.com/speech TensorFlow: tensorflow.org Cloud Machine Learning: cloud.google.com/ml Cloud Machine Learning: demo video