Machine Learning @ Google Cloud by Robert Saxby

devCampNoord #02

Transcript

1. Machine Learning @ Google Cloud Robert Saxby - Big Data Product Specialist
4. Sustainability Google datacenters have half the overhead of typical industry data centers Largest private investor in renewables: $2 billion generating 3.2 GW Applying Machine Learning produced 40% reduction in cooling energy
5. 2016 Google Research 2008 2002 2004 2006 2010 2012 2014 2015 Open Source 2005 Google Cloud Products BigQuery Pub/Sub Dataflow Bigtable ML GFS Map Reduce BigTable Dremel Flume Java Millwheel Tensorflow 15+ years of solving Data Problems Apache Beam PubSub
6. Proprietary + Confidential Large Datasets Cutting Edge Models Compute at Scale Drivers of Success in AI/ML Projects
7. Confidential + Proprietary Industry Use-cases In-loop inferencing for trained models Cloud AI products Pre-trained ML APIs to Building custom ML models ML Framework Industry-standard & widely adopted Infrastructure Best-in class processors for ML/DL Proprietary + Confidential Google Cloud End-to End AI Platform Accelerate Business Outcomes with Enterprise-Ready Machine Learning Pipeline CPU GPU TPU Risk Analysis Customer Segmentation Predictive Inventory Mnagement Fraud Detection Demand Forecast Recommendation Engine Targeted marketing Predictive Analytics
8. App Developer Data Scientist Build custom models Use/extend OSS SDK Use pre-built models ML researcher Cloud MLE ML Perception services End to End: Google Cloud AI Spectrum Proprietary + Confidential
9. Proprietary + Confidential Cloud Video Intelligence API demo
10. Proprietary + Confidential How the demo works Cloud Storage (full length videos) Cloud Video Intelligence API Video Metadata Frontend built on App Engine Cloud Functions Cloud Storage (video annotation JSON) Video content Built by @SRobTweets and @AlexWolfe
11. Proprietary + Confidential NLP demo
12. App Developer Data Scientist Build custom models Use/extend OSS SDK Use pre-built models ML researcher Cloud MLE ML Perception services End to End: Google Cloud AI Spectrum Proprietary + Confidential
14. Proprietary + Confidential What is TensorFlow? • A system for distributed, parallel machine learning • It’s based on general-purpose dataflow graphs • It targets heterogeneous devices ◦ A single PC with CPU ◦ A single PC with GPU(s) ◦ A mobile device ◦ Clusters of 100s or 1000s of CPUs, GPUs and TPUs
15. Proprietary + Confidential Another data flow system MatMul Add Relu biases weights examples labels Xent Graph of Nodes, also called Operations or ops
16. Proprietary + Confidential With tensors MatMul Add Relu biases weights examples labels Xent Edges are N-dimensional arrays: Tensors
17. Proprietary + Confidential What’s in a name? 0 Scalar (magnitude only) s = 483 1 Vector (magnitude and direction) v = [1.1, 2.2, 3.3] 2 Matrix (table of numbers) m = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] 3 3-Tensor (cube of numbers) t = [[[2], [4], [6]], [[8], [10], [12]], [[14], [16], [18]]] 4 n-Tensor (you get the idea) ....
18. Proprietary + Confidential Convolutional layer W 1 [4, 4, 3] W 2 [4, 4, 3] +padding W[4, 4, 3, 2] filter size input channels output channels stride convolutional subsampling convolutional subsampling convolutional subsampling
19. Proprietary + Confidential With state Add Mul biases ... learning rate −= ... 'Biases' is a variable −= updates biases Some ops compute gradients
20. Proprietary + Confidential And distributed Add Mul biases ... learning rate −= ... Device B Device A
21. TensorFlow Distributed Execution Engine CPU GPU Android iOS ... C++ Frontend Python Frontend ... Layers Estimator Models in a box Train and evaluate models Build models Keras Model Canned Estimators
22. Proprietary + Confidential The popular imagination of what ML is Lots of data Magical results Complex mathematics in multidimensional spaces
23. Proprietary + Confidential In reality, ML is Collect data Create the model Refine the model Understand and prepare the data Serve the model Define objectives
24. Proprietary + Confidential In reality, ML is Collect data Create the model Refine the model Understand and prepare the data Serve the model Define objectives
25. Proprietary + Confidential Powerful Data Exploration Cloud Datalab Closely integrated with BigQuery and Cloud ML Notebook interface Leverage existing Jupyter modules and knowledge Suitable to interactive data science and machine learning
26. Proprietary + Confidential https://cloud.google.com/solutions/running-distributed-tensorflow-on-compute-engine Distributed TensorFlow on Compute Engine
27. Proprietary + Confidential Machine Learning on any data, of any size Cloud ML Engine Portable models with TensorFlow Services are designed to work together Managed distributed training infrastructure that supports CPUs and GPUs Automatic hyperparameter tuning
28. Custom Estimators: The Model https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/census ... def _model_fn(mode, features, labels): ... if mode == Modes.PREDICT: ... return tf.estimator.EstimatorSpec(mode, predictions=predictions, export_outputs=export_outputs) ... if mode == Modes.TRAIN: ... return tf.estimator.EstimatorSpec(mode, loss=loss, train_op=train_op) ...
29. https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/census Custom Estimators: The Task ... train_input = lambda: model.generate_input_fn(hparams.train_files, num_epochs=hparams.num_epochs, batch_size=hparams.train_batch_size) ... """This function is used by learn_runner to create an Experiment which executes model code provided in the form of an Estimator and input functions.""" def _experiment_fn(run_config, hparams): tf.estimator.Estimator( model.generate_model_fn( ... ), train_input_fn=train_input, eval_input_fn=eval_input, **experiment_args ) ...
30. Proprietary + Confidential Running locally gcloud ml-engine local train \ --module-name trainer.task --package-path trainer/ \ -- \ --train-files $TRAIN_DATA --eval-files $EVAL_DATA --train-steps 1000 --job-dir $MODEL_DIR training data evaluation data output directory train locally
31. Proprietary + Confidential Single trainer running in the cloud gcloud ml-engine jobs submit training $JOB_NAME --job-dir $OUTPUT_PATH \ --runtime-version 1.0 --module-name trainer.task --package-path trainer/ --region $REGION \ -- \ --train-files $TRAIN_DATA --eval-files $EVAL_DATA --train-steps 1000 --verbosity DEBUG train in the cloud region Google cloud storage location
32. Proprietary + Confidential Distributed training in the cloud gcloud ml-engine jobs submit training $JOB_NAME --job-dir $OUTPUT_PATH \ --runtime-version 1.0 --module-name trainer.task --package-path trainer/ --region $REGION \ --scale-tier STANDARD_1 -- \ --train-files $TRAIN_DATA --eval-files $EVAL_DATA --train-steps 1000 --verbosity DEBUG distributed
33. Proprietary + Confidential In reality, ML is Collect data Create the model Refine the model Understand and prepare the data Serve the model Define objectives
34. Proprietary + Confidential Refine the model Feature engineering Better algorithms More examples, more data Hyperparameter tuning
35. Proprietary + Confidential Hyperparameter tuning • Automatic hyperparameter tuning service • Build better performing models faster and save many hours of manual tuning • Google-developed search (Bayesian Optimisation) algorithm efficiently finds better hyperparameters for your model/dataset HyperParam #1 Objective We want to find this Not these https://cloud.google.com/blog/big-data/2017/08/hyperparameter-tuning-in-cloud-machine-learning-engine-using-bayesian-optimization
36. Proprietary + Confidential Hyperparameter tuning gcloud ml-engine jobs submit training $JOB_NAME --job-dir $OUTPUT_PATH \ --runtime-version 1.0 --module-name trainer.task --package-path trainer/ --region $REGION \ --scale-tier STANDARD_1 --config $HPTUNING_CONFIG -- \ --train-files $TRAIN_DATA --eval-files $EVAL_DATA --train-steps 1000 --verbosity DEBUG hypertuning
37. Proprietary + Confidential Hyperparameter tuning trainingInput: hyperparameters: goal: MAXIMIZE hyperparameterMetricTag: accuracy maxTrials: 4 maxParallelTrials: 2 params: - parameterName: first-layer-size type: INTEGER minValue: 50 maxValue: 500 scaleType: UNIT_LINEAR_SCALE ... ... # Construct layers sizes with exponetial decay hidden_units=[ max(2, int(hparams.first_layer_size * hparams.scale_factor**i)) for i in range(hparams.num_layers) ], ... parser.add_argument( '--first-layer-size', help='Number of nodes in the 1st layer of the DNN', default=100, type=int ) ... hptuning_config.yaml task.py
38. Proprietary + Confidential In reality, ML is Collect data Create the model Refine the model Understand and prepare the data Serve the model Define objectives
39. Proprietary + Confidential Deploying the model Creating model gcloud ml-engine models create $MODEL_NAME --regions=$REGION Creating versions gcloud ml-engine versions create v1 --model $MODEL_NAME --origin $MODEL_BINARIES \ --runtime-version 1.0 gcloud ml-engine models list
40. Proprietary + Confidential Predicting gcloud ml-engine predict --model $MODEL_NAME --version v1 --json-instances ../test.json Using REST: POST https://ml.googleapis.com/v1/{name=projects/**}:predict JSON format (in this case): {"age": 25, "workclass": "private", "education": "11th", "education_num": 7, "marital_status": "Never-married", "occupation": "machine-op-inspector", "relationship": "own-child", "gender": " male", "capital_gain": 0, "capital_loss": 0, "hours_per_week": 40, "native_country": " United-States"}