Experimenting with TensorFlow

Slide 1

Slide 1 text

Experimenting with TensorFlow Breandan Considine Python Ireland 2016

Slide 2

Slide 2 text

Who am I? • Background in Computer Science, Machine Learning • Worked for a small ad-tech startup out of university • Spent two years as Developer Advocate @JetBrains • Interested machine learning and speech recognition • Enjoy writing code, traveling to conferences, reading • Say hello! @breandan | breandan.net | [email protected]

Slide 3

Slide 3 text

So what’s a Tensor anyway? • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use (tiny) tensors every day!

Slide 4

Slide 4 text

So what’s a Tensor anyway? • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use (tiny) tensors every day! 't'

Slide 5

Slide 5 text

So what’s a Tensor anyway? • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use (tiny) tensors every day! 't'

Slide 6

Slide 6 text

So what’s a Tensor anyway? • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use (tiny) tensors every day! 't'

Slide 7

Slide 7 text

So what’s a Tensor anyway? • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use (tiny) tensors every day! 't'

Slide 8

Slide 8 text

So what’s a Tensor anyway? • A “tensor’ is just an n-dimensional array • Useful for working with complex data • We use tiny (and large) tensors every day! 't'

Slide 9

Slide 9 text

Amplitude Frequency Time

Slide 10

Slide 10 text

NxM image is a point in RNM

Slide 11

Slide 11 text

https://inst.eecs.berkeley.edu/~cs194-26/fa14/upload/files/proj5/cs194-dm/

Slide 12

Slide 12 text

https://inst.eecs.berkeley.edu/~cs194-26/fa14/upload/files/proj5/cs194-dm/

Slide 13

Slide 13 text

https://inst.eecs.berkeley.edu/~cs194-26/fa14/upload/files/proj5/cs194-dm/

Slide 14

Slide 14 text

What are they good for? • Modeling complex systems, data sets • Capturing higher order correlations • Representing dynamic relationships • Doing machine learning!

Slide 15

Slide 15 text

A quick taste of supervised learning

Slide 16

Slide 16 text

Let’s have a look at linear regression! • • < Live Code >

Slide 17

Slide 17 text

Classification in a nutshell

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

0 1

Slide 23

Slide 23 text

Cool learning algorithm def classify(datapoint, weights):

Slide 24

Slide 24 text

Cool learning algorithm def classify(datapoint, weights): prediction = sum(x * y for x, y in zip([1] + datapoint, weights))

Slide 25

Slide 25 text

Cool learning algorithm def classify(datapoint, weights): prediction = sum(x * y for x, y in zip([1] + datapoint, weights)) if prediction < 0: return 0 else: return 1

Slide 26

Slide 26 text

Cool learning algorithm def classify(datapoint, weights): prediction = sum(x * y for x, y in zip([1] + datapoint, weights)) if prediction < 0: return 0 else: return 1

Slide 27

Slide 27 text

Cool learning algorithm def train(data_set):

Slide 28

Slide 28 text

Cool learning algorithm def train(data_set):

Slide 29

Slide 29 text

Cool learning algorithm def train(data_set): class Datum: def __init__(self, features, label): self.features = [1] + features self.label = label

Slide 30

Slide 30 text

Cool learning algorithm def train(data_set): weights = [0] * len(data_set[0].features) [0, 0, 0]

Slide 31

Slide 31 text

Cool learning algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1

Slide 32

Slide 32 text

Cool learning algorithm def train(data_set): weights = [0] * len(data_set[0].features) total_error = threshold + 1 while total_error > threshold: total_error = 0 for item in data_set: error = item.label – classify(item.features, weights) weights = [w + RATE * error * i for w, i in zip(weights, item.features)] total_error += abs(error)

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

weights = [w + RATE * error * i for w, i in zip(weights, item.features)] Cool learning algorithm * 1 i1 i2 in * * * w0 w1 w2 wn Σ

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

Even Cooler Algorithm! (Backprop) train(trainingSet) : initialize network weights randomly until average error stops decreasing (or you get tired): for each sample in trainingSet: prediction = network.output(sample) compute error (prediction – sample.output) compute error of (hidden -> output) layer weights compute error of (input -> hidden) layer weights update weights across the network save the weights

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

Gradient Descent http://cs231n.github.io/

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

“A Neural Network Zoo,” Fjdor Van Neen http://www.asimovinstitute.org/neural-network-zoo/

Slide 49

Slide 49 text

Further resources • CS231 Course Notes • TensorFlow Models • Visualizing MNIST • Neural Networks and Deep Learning • Andrew Ng’s Machine Learning class • Awesome Public Datasets • Amy Unruh & Eli Bixby's TensorFlow Workshop

Slide 50

Slide 50 text

Thank You! Mary, Mark, Margaret, Hanneli