Experimenting with TensorFlow (ConFoo Vancouver '16)

From implementing new research to validating results, running machine learning experiments can often be a process of trial and error. TensorFlow is an open source computing framework that places state of the art algorithms and tools into developers' hands, allowing you to rapidly iterate experiments, monitor their results, and scale into production. We'll show you how to train machine learning pipelines from conception to release with TensorFlow.

Transcript

1. Experimenting with TensorFlow Breandan Considine ConFoo Vancouver ‘16
2. Who am I? • Background in Computer Science, Machine Learning • Worked for a small ad-tech startup out of university • Spent two years as Developer Advocate at JetBrains • Interested machine learning and speech recognition • Enjoy writing code, traveling to conferences, reading • Say hello! @breandan | breandan.net | bre@ndan.co
3. So what’s a Tensor anyway? • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use tiny (and large) tensors every day!
4. So what’s a Tensor anyway? 't' • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use tiny (and large) tensors every day!
5. 't' • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use tiny (and large) tensors every day! So what’s a Tensor anyway?
6. So what’s a Tensor anyway? 't' • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use tiny (and large) tensors every day!
7. So what’s a Tensor anyway? 't' • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use tiny (and large) tensors every day!
8. So what’s a Tensor anyway? • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use tiny (and large) tensors every day! 't'
9. Amplitude Frequency Time
10. NxM image is a point in RNM
11. https://inst.eecs.berkeley.edu/~cs194-26/fa14/upload/files/proj5/cs194-dm/
12. https://inst.eecs.berkeley.edu/~cs194-26/fa14/upload/files/proj5/cs194-dm/
13. https://inst.eecs.berkeley.edu/~cs194-26/fa14/upload/files/proj5/cs194-dm/
14. What are they good for? • Modeling complex systems, data sets • Capturing higher order correlations • Representing dynamic relationships • Doing machine learning!
15. A quick taste of supervised learning • • < Live Code >
16. Let’s have a look at linear regression! • • < Live Code >
17. Classification in a nutshell
22. 0 1
23. Cool learning algorithm def classify(datapoint, weights):
24. Cool learning algorithm def classify(datapoint, weights): prediction = sum(x * y for x, y in zip([1] + datapoint, weights))
25. Cool learning algorithm def classify(datapoint, weights): prediction = sum(x * y for x, y in zip([1] + datapoint, weights)) if prediction < 0: return 0 else: return 1
26. Cool learning algorithm def classify(datapoint, weights): prediction = sum(x * y for x, y in zip([1] + datapoint, weights)) if prediction < 0: return 0 else: return 1
27. Cool learning algorithm def train(data_set):
28. Cool learning algorithm def train(data_set):
29. Cool learning algorithm def train(data_set): class Datum: def __init__(self, features, label): self.features = [1] + features self.label = label
30. Cool learning algorithm def train(data_set): weights = [0] * len(data_set[0].features) [0, 0, 0]
31. Cool learning algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1
32. Cool learning algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)
33. Cool learning algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)
34. Cool learning algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)
35. weights = [w + RATE * error * i for w, i in zip(weights, item.features)] Cool learning algorithm * 1 i1 i2 in * * * w0 w1 w2 wn Σ
36. Cool learning algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)
37. Cool learning algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)
41. Even Cooler Algorithm! (Backprop) train(trainingSet) : initialize network weights randomly until average error stops decreasing (or you get tired): for each sample in trainingSet: prediction = network.output(sample) compute error (prediction – sample.output) compute error of (hidden -> output) layer weights compute error of (input -> hidden) layer weights update weights across the network save the weights
46. Gradient Descent http://cs231n.github.io/
48. “A Neural Network Zoo,” Fjdor Van Neen http://www.asimovinstitute.org/neural-network-zoo/
49. Further resources • CS231 Course Notes • TensorFlow Models • An Interactive Tutorial on Numerical Optimization • Neural Networks and Deep Learning • Andrew Ng’s Machine Learning class • Awesome Public Datasets • Amy Unruh & Eli Bixby's TensorFlow Workshop
50. Thank You!