Tensor Must Flow (#DEFVESTBY 26.11.2016)

During this talk we will figure out what is TensorFlow, how it works - key concepts and how it flow. We will find out what is hidden behind the magic words - "Artificial Neural Network". What additional tools you can use for visualisation and more stuff.

Transcript

1. Tensor Must Flow Sergey Kibish Transact Pro, GDG Riga @s_kibish
2. What is TensorFlow?
3. TensorFlow Is a library for distributed computations Dedicated for Machine Learning, but can be used not only for it Apache 2.0 3
4. Where it can work? 4
5. Who use it? Google - RankBrain (search ranking on google.com) Google - SmartReply (automatic email responses) Google - On-Device Vision for OCR (real- time translation) 5
6. Tensor
7. Tensor It is a matrix It is a 2-dimensional array It is a 2-dimensional Tensor 7 m = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
8. Flow
9. Flow Computations are described with graph Nodes – mostly mathematical operations (but not only) Edges – input/output relation between nodes 9
10. Flow x y mul 10
11. Flow 11 x = tf.constant([[2.,2.]], name="x") y = tf.constant([[3.],[3.]], name="y") out = tf.matmul(x, y, name="mul")
12. Flow 12 print out Tensor("mul:0", shape=(1, 1), dtype=float32) tf.get_default_graph().as_graph_def()
13. Flow node { name: "x" op: "Const" attr { key: "dtype" value { type: DT_FLOAT } } ... } node { name: "y" op: "Const" attr { key: "dtype" value { type: DT_FLOAT } } ... } node { name: "mul" op: "MatMul" input: "x" input: "y" ... } 13
14. Flow 14
15. Session
16. Session Graph must be placed in session to do computation Session places operations on Devices (CPU, GPU) 16
17. Session 17 x = tf.constant([[2.,2.]], name="x") y = tf.constant([[3.],[3.]], name="y") out = tf.matmul(x, y, name="mul") with tf.Session() as session: v = session.run(out) v array([[ 12.]], dtype=float32)
18. More TensorFlow way?
19. 19 x = tf.placeholder(dtype=tf.float32, shape=(1,2), name="x") y = tf.placeholder(dtype=tf.float32, shape=(2,1), name="y") v = tf.Variable([[0.]], dtype=tf.float32, name="out") out = tf.matmul(x, y, name="mul") update = tf.assign(v, out, name="update") init_op = tf.initialize_all_variables() with tf.Session() as sess: sess.run(init_op) sess.run(update, feed_dict={x: [[2.,2.]], y: [[3.],[3.]]}) print sess.run(v) [[ 12.]]
20. TensorFlow way 20
21. TensorBoard
22. TensorBoard 22
23. What else? Checkpoints 23
24. Checkpoint (Save) 24 saver = tf.train.Saver() … with tf.Session() as sess: …# training here saver.save(sess, "/tmp/simple_ann.ckpt")
25. Checkpoint (Restore) 25 saver = tf.train.Saver() … with tf.Session() as sess: saver.restore(sess, "/tmp/simple_ann.ckpt") … # training here
26. Artificial Neural Network
27. Problem 27
28. Data 28 x1 x2 y 2,557 3,676 0 10,567 15,323 1 4,873 5,212 0 … … …
29. ANN x2 b w2 w3 x1 w1 y 29
30. ANN x2 b ∑ w2 w3 x1 w1 σ y 30
31. Simple ANN (1/2) 31 x = tf.placeholder("float",[None, 2]) y = tf.placeholder("float",[None, 1]) W = tf.Variable(tf.zeros([2, 1])) b = tf.Variable(tf.ones([1, 1])) activation = tf.nn.sigmoid(tf.matmul(x, W)+b) cost = tf.reduce_sum(tf.square(activation - y))/100 optimizer = tf.train.RMSPropOptimizer(.01).minimize(cost) init = tf.initialize_all_variables()
32. Simple ANN (2/2) 32 with tf.Session() as sess: sess.run(init) for i in range(100): train_data = sess.run(optimizer, feed_dict={x: X, y: np.reshape(Y, [100, 1])}) result = sess.run(activation, feed_dict={x:X}) rounded = [round(x) for x in result] print rounded
33. What about image recognition? 33
34. MNIST MNIST for Beginners MNIST for Experts 34
35. Is it easy to start with? 35
36. No. 36
37. What to do? 37
38. Solutions http://tflearn.org http://keras.io 38
39. Example (Keras) 39 model = Sequential() model.add(Dense(1, input_dim=2, init='uniform', activation='sigmoid')) model.compile(optimizer='rmsprop', loss='binary_crossentropy') model.fit(X, Y, nb_epoch=100, batch_size=len(X), verbose=2)
40. Example (Keras) 40 predictions = model.predict(X) rounded = [round(x) for x in predictions]
41. Source http://bit.ly/tf-example-devfestby-2016 41
42. Another cool tool
43. http://playground.tensorflow.org/ 43
44. Production?
45. TensorFlow Serve 45
46. TensorFlow in cloud 46
47. Q&A Sergey Kibish Transact Pro, GDG Riga @s_kibish
48. Thank you! Sergey Kibish Transact Pro, GDG Riga @s_kibish