Machine Learning with Tensorflow (Study Jam)

This presentation was used for this meetup:

https://www.meetup.com/GDG-Bern/events/252678597/

Transcript

1. Machine Learning Crash Course (MLCC) with TensorFlow APIs Bern, 18 August 2018 GDG Bern 1
2. Goals of this workshop • Learn how to train a variety of standard ML models with TF • Cover a few algorithms and frameworks • Spoiler: this is not a general ML survey • First three modules and first steps with TF • Content: videos, docs, Playground and Colab exercises. • Complete list: https://developers.google.com/machine-learning/crash-co urse/exercises 2
3. Self Study The path is long but you are not alone :) https://developers.google.com/machine-learning/crash-course Bugs found in the course? Great! Send them to: ml-education@google.com May the force be with you! 3
4. Are you ready? Did you check the prerequisites? https://developers.google.com/machine-learning/crash-cours e/prereqs-and-prework • Python • Browser. Chrome? • Tensorflow https://www.tensorflow.org/install Colaboratory platform. Colaboratory is supported on most major browsers, and is most thoroughly tested on desktop versions of Chrome and Firefox. 4
5. Introduction to ML First video: https://developers.google.com/machine-learning/crash-cours e/ml-intro 5
6. Framing 6
7. Supervised Machine Learning • ML systems learn how to combine input to produce useful predictions on never-before-seen data. • We are the teachers and the algorithms are our students. • Typical algorithms: linear regression, logistic regression, multi-class classification, support vector machines... 7
8. Labels and features • Label: the true thing we are predicting, y. ◦ Variable y in basic linear regression • Feature: input variables describing our data: x i ◦ Variables {x 1 ,x 2 ,x 3 ,...x n } in basic linear regression 8
9. Examples and Models • Example: particular instance of data, x. • Labeled example: {features, label}, (x, y) ◦ To train the model • Unlabeled example: {features, ?}, (x, ?) ◦ To make predictions on the new data (test) • Model: maps examples to predict labels: y’ ◦ Defined by internal parameters (learned) 9
10. Check your understanding (5 min) Discussion in pairs: https://developers.google.com/machine-learning/crash-cours e/framing/check-your-understanding 10
11. Descending into ML 11
12. Learning from Data • Complex ways to learn from data • Best approach: start with something simple and familiar • This opens the door to some broadly useful methods 12
13. Linear Regression Approach: • Finds a line that fits the data • x 1 Input → y’ Output (prediction) 13
14. Fitting our data • y: the value we are trying to predict • x 1 : input data • w 1 and b are the values we are trying to learn from data 14
15. Not just fitting our Data • Linear methods are often used as models • ML is more than fitting the points • We need a line that works also well with new data points from the same distribution 15
16. Loss • Loss: penalty for an incorrect prediction • Lower loss is generally better • Choice of loss function guides model fitting 16
17. Convenient Loss function L 2 Loss for a given example is also called squared error = Square of the difference between prediction and label = (observation - prediction)2 = (y - y’)2 Penalizes large errors more strongly Mean Square Error: average squared loss per example over the whole dataset 17
18. Check your understanding (5 min) New partner: https://developers.google.com/machine-learning/crash-cours e/descending-into-ml/check-your-understanding 18
19. Reducing Loss 19
20. How to reduce loss? • Derivative of (y-y’)2 with respect to the weights and biases tells us how loss changes for a given example ◦ Simple to compute and convex • We can simply repeatedly take small steps in the direction that minimizes loss → Gradient Steps ◦ They are really negative gradient steps • Optimization strategy: Gradient Descent 20
21. Block Diagram of Gradient Descent 21
22. A Gradient Step 22
23. A Gradient Step 23
24. A Gradient Step 24
25. Learning rate too small 25
26. Learning rate too large 26
27. Learning at the right pace 27
28. Step size and Learning rates • Step size: how big is the gradient step ◦ Optimal value is data dependent • Learning rate: parameter that controls the step size • Example: gradient magnitude = 2.5 and learning rate = 0.01 → step size = 2.5*0.01= 0.025 • Experiment yourself (individually 10 min): https://developers.google.com/machine-learning/crash-c ourse/fitter/graph 28
29. Weight initialization • For convex problems ◦ weights can start anywhere (i.e. 0’s) ◦ Just one minimum • Not true for neural nets ◦ Non convex ◦ More than one minimum ◦ Strong dependencies on initial values 29
30. SGD and Mini-Batch Gradient Descent • Could compute gradient over entire dataset on each step, but this turns out to be not necessary • Computing gradient on small data samples works well ◦ On every step, get a new random sample • Stochastic gradient descent. One example at a time • Mini-Batch gradient descent. Batches 10-10000 ◦ Loss and gradients are averaged over the batch 30
31. Playground (10 min) In pairs: https://developers.google.com/machine-learning/crash-cours e/reducing-loss/playground-exercise 31
32. First steps with TF 32
33. Tensorflow API hierarchy • TF estimators provide a high level, object oriented API • tf.layers/tf.losses/tf.metrics provide reusable libraries for common model components • A low-level set of TF Python libraries provides operations, which wrap C++ kernels • Each kernel works on one or more platforms (GPU, CPU,...) 33
34. A quick look at the tf.estimator API import tensorflow as tf # Set up a linear classifier. classifier = tf.estimator.LinearClassifier(feature_columns) # Train the model on some example data. classifier.train(input_fn=train_input_fn, steps=2000) # Use it to predict. predictions = classifier.predict(input_fn=predict_input_fn) 34
35. Programming exercises (30 min) • First steps with Tensorflow • Synthetic Features and Outliers 35
36. Some Jupyter Notebooks https://bitbucket.org/hrojas/learn-tensorflow On NN’s: https://github.com/Hvass-Labs/TensorFlow-T utorials/blob/master/04_Save_Restore.ipynb 36
37. Join the community R Slack community https://www.jessemaegan.com/post/r4ds-the-next-iteration/ Python Slack community http://bit.ly/py4ds 37
38. Many thanks 38