TensorFlow Introduction

A short introduction to TensorFlow that I used as part of an IGGI PhD CDT class.

Transcript

1. Neurons Networks MNIST Demo Convolution Tricks DQN Neural Networks in action Dino Ratcliffe January 3, 2017 Dino Ratcliffe Neural Networks in action
2. Neurons Networks MNIST Demo Convolution Tricks DQN Neurons Dino Ratcliffe Neural Networks in action
3. Neurons Networks MNIST Demo Convolution Tricks DQN Artificial Neuron Figure 1: Perceptron Dino Ratcliffe Neural Networks in action
4. Neurons Networks MNIST Demo Convolution Tricks DQN Artificial Neuron Single Neuron can compute logic AND, NOT, OR Flexibility in Activation Functions Step Continuous Log-Sigmoid Softmax Rectifier Bias Additional degree of freedom Dino Ratcliffe Neural Networks in action
5. Neurons Networks MNIST Demo Convolution Tricks DQN Artificial Neuron - TensorFlow input_data = tf.placeholder(tf.float32, [None, 2]) W = tf.Variable(tf.zeros([2, 1])) b = tf.Variable(tf.zeros([1])) output = tf.nn.sigmoid(tf.matmul(input_data, W) + b) Dino Ratcliffe Neural Networks in action
6. Neurons Networks MNIST Demo Convolution Tricks DQN Networks Dino Ratcliffe Neural Networks in action
7. Neurons Networks MNIST Demo Convolution Tricks DQN Feed Forward Figure 2: Three Layer Neural Network Dino Ratcliffe Neural Networks in action
8. Neurons Networks MNIST Demo Convolution Tricks DQN Feed Forward - TensorFlow input_data = tf.placeholder(tf.float32, [None, 3]) W_l1 = tf.Variable(tf.zeros([3, 4])) b_l1 = tf.Variable(tf.zeros([4])) h_l1 = tf.nn.sigmoid(tf.matmul(input_data, W_l1) + b_l1) W_l2 = tf.Variable(tf.zeros([4, 3])) b_l2 = tf.Variable(tf.zeros([3])) h_l2 = tf.nn.sigmoid(tf.matmul(h_l1, W_l2) + b_l2) ... Dino Ratcliffe Neural Networks in action
9. Neurons Networks MNIST Demo Convolution Tricks DQN Back Propagation - Gradient Decent Uses partial derivatives to obtain a gradient for the weights and biases Take small steps along the gradient to minimize the error Dino Ratcliffe Neural Networks in action
10. Neurons Networks MNIST Demo Convolution Tricks DQN Back Propagation - TensorFlow optimizer = tf.train.GradientDescentOptimizer(0.001) train_step = optimizer.minimize(error) Dino Ratcliffe Neural Networks in action
11. Neurons Networks MNIST Demo Convolution Tricks DQN MNIST Demo Dino Ratcliffe Neural Networks in action
12. Neurons Networks MNIST Demo Convolution Tricks DQN MNIST (Mixed National Institute of Standards and Technology) Handwritten digit dataset 28x28 images 60,000 training images 10,000 test images Figure 3: MNIST Examples Dino Ratcliffe Neural Networks in action
13. Neurons Networks MNIST Demo Convolution Tricks DQN DEMO Dino Ratcliffe Neural Networks in action
14. Neurons Networks MNIST Demo Convolution Tricks DQN Convolution Dino Ratcliffe Neural Networks in action
15. Neurons Networks MNIST Demo Convolution Tricks DQN Convolution Technique used in computer vision Used for many image filtering tasks Blurring Sharpening Embossing Edge Detection Dino Ratcliffe Neural Networks in action
16. Neurons Networks MNIST Demo Convolution Tricks DQN Convolution Figure 4: Convolution Kernel Dino Ratcliffe Neural Networks in action
17. Neurons Networks MNIST Demo Convolution Tricks DQN Convolution - TensorFlow W_conv = tf.Variable(tf.truncated_normal([3, 3, 1, 32], stddev=0.1)) b_conv = tf.Variable(tf.constant(0.1, shape=[32])) convolution = tf.nn.conv2d(input_data, W_conv, strides=[1, 1, 1, 1]) h_conv = tf.nn.relu(convolution + b_conv) Dino Ratcliffe Neural Networks in action
18. Neurons Networks MNIST Demo Convolution Tricks DQN Tricks Dino Ratcliffe Neural Networks in action
19. Neurons Networks MNIST Demo Convolution Tricks DQN Max Pooling h_pool = tf.nn.max_pool(h_conv, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1]) Dino Ratcliffe Neural Networks in action
20. Neurons Networks MNIST Demo Convolution Tricks DQN Dropout keep_prob = tf.placeholder(tf.float32) h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob) Dino Ratcliffe Neural Networks in action
21. Neurons Networks MNIST Demo Convolution Tricks DQN Saving Models saver = tf.train.Saver() sess = tf.Session() sess.run(tf.initialize_all_variables()) # training saver.save(sess, dir/to/save/model.ckpt , t_step) Dino Ratcliffe Neural Networks in action
22. Neurons Networks MNIST Demo Convolution Tricks DQN Loading Models ckpt = tf.train.get_checkpoint_state( dir/to/save ) if ckpt and ckpt.model_checkpoint_path: saver.restore(sess, ckpt.model_checkpoint_path) print loaded model at: dir/to/save else: print No model found at: dir/to/save Dino Ratcliffe Neural Networks in action
23. Neurons Networks MNIST Demo Convolution Tricks DQN Logging/Graphing score = tf.placeholder(tf.float32) tf.scalar_summary("score", score) v = tf.placeholder(tf.float32) tf.scalar_summary("v", v) summary_op = tf.merge_all_summaries() summary_writer = tf.train.SummaryWriter( dir/to/logs , sess.graph_def) summary_str = sess.run(summary_op, feed_dict = { score: 10.2, v: 0.2}) Dino Ratcliffe Neural Networks in action
24. Neurons Networks MNIST Demo Convolution Tricks DQN DQN Dino Ratcliffe Neural Networks in action
25. Neurons Networks MNIST Demo Convolution Tricks DQN Deep Q Networks Combines reinforcement learning and neural networks Input: game state Output: action-values Dino Ratcliffe Neural Networks in action
26. Neurons Networks MNIST Demo Convolution Tricks DQN Figure 5: DQN Network Dino Ratcliffe Neural Networks in action
27. Neurons Networks MNIST Demo Convolution Tricks DQN Target Ouput T = R + γ max θ(St+1) Dino Ratcliffe Neural Networks in action
28. Neurons Networks MNIST Demo Convolution Tricks DQN Error E = 1 N N i=1 (T − Va)2 action_value = tf.reduce_sum(tf.mul(output, action), reduction_indices = 1) error = tf.reduce_mean(tf.square(target - action_value)) Dino Ratcliffe Neural Networks in action
29. Neurons Networks MNIST Demo Convolution Tricks DQN Experiences St: initial state a: action taken from the initial state R: reward recieved after taking action a St+1: resulting state after taking action a t: is initial state terminal Dino Ratcliffe Neural Networks in action
30. Neurons Networks MNIST Demo Convolution Tricks DQN Experience Replay Store queue of experiences Train on a sample of experiences from buffer at each time step Breaks up the smoothness of the environment aiding convergence Dino Ratcliffe Neural Networks in action
31. Neurons Networks MNIST Demo Convolution Tricks DQN Choosing Actions Pick actions greedily During training explore with is the probability of picking a random action Dino Ratcliffe Neural Networks in action