Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Intro to TensorFlow - Рустем Арзимбетов

GDG Ternopil
September 11, 2016

Intro to TensorFlow - Рустем Арзимбетов

Intro to TensorFlow - Рустем Арзимбетов

GDG Ternopil

September 11, 2016
Tweet

More Decks by GDG Ternopil

Other Decks in Programming

Transcript

  1. • Open source Machine Learning library • Especially useful for

    Deep Learning • For research and production • Apache 2.0 license
  2. • The modern reincarnation of Artificial Neural Networks from the

    1980s and 90s. • A collection of simple trainable mathematical units, which collaborate to compute a complicated function. • Compatible with supervised, unsupervised, and reinforcement learning. What is Deep Learning?
  3. Loosely inspired by what (little) we know about the biological

    brain. Higher layers form higher levels of abstraction. What is Deep Learning?
  4. Commonalities with real brains: • Each neuron is connected to

    a small subset of other neurons. • Based on what it sees, it decides what it wants to say. • Neurons learn to cooperate to accomplish a task. What is Deep Learning?
  5. Each neuron implements a relatively simple mathematical function: Y =

    T( W * x + b) But a composition of 106-109 such functions is surprisingly powerful. What is Deep Learning?
  6. Data flow graphs Computation is defined as a directed acyclic

    graph (DAG) to optimize an objective function • Graph is defined in high-level language (Python) • Graph is compiled and optimized • Graph is executed (in parts or fully) on available low level devices (CPU, GPU) • Data (tensors) flow through the graph • TensorFlow can compute gradients automatically
  7. Architecture • Core in C++ • Different front ends ◦

    Python and C++ today, community may add more Core TensorFlow Execution System CPU GPU Android iOS ... C++ front end Python front end ...
  8. Basic TensorFlow usage example Want to predict real estate price

    from square footage • Collect data (square footage, price) • Come up with a model (linear dependence: y = W * x + b) • Describe the model in TensorFlow • Train the model • Predict prices
  9. Defining the graph import tensorflow as tf W = tf.Variable(tf.ones([1,

    2])) b = tf.Variable(tf.ones([1])) x = tf.placeholder(tf.float32, shape=(2,1)) y = tf.add(tf.matmul(W, x), b) * + X W b y
  10. Defining the graph: variables import tensorflow as tf W =

    tf.Variable(tf.ones([1, 2])) b = tf.Variable(tf.ones([1])) x = tf.placeholder(tf.float32, shape=(2,1)) y = tf.add(tf.matmul(W, x), b) * + X W b y
  11. Defining the graph: placeholders import tensorflow as tf W =

    tf.Variable(tf.ones([1, 2])) b = tf.Variable(tf.ones([1])) x = tf.placeholder(tf.float32, shape=(2,1)) y = tf.add(tf.matmul(W, x), b) * + X W b y
  12. Defining the graph: formula import tensorflow as tf W =

    tf.Variable(tf.ones([1, 2])) b = tf.Variable(tf.ones([1])) x = tf.placeholder(tf.float32, shape=(2,1)) y = tf.add(tf.matmul(W, x), b) * + X W b y
  13. Executing the graph output = [[1 , 1]] * [[3]

    ,[4]] + 1 output = 8 with tf.Session() as sess: tf.initialize_all_variables().run() output = sess.run(y,feed_dict={x:[[3],[4]]}) x = tf.placeholder(tf.float32, shape=(2,1)) W x b
  14. Given (x, y) pairs learn W, b to minimize the

    distance from predicted y to known y. Updating variables: learning W and b W x + b X (footage) Y (price)
  15. Loss functions distance = y - labels sq_distance = distance

    * distance cost = tf.reduce_sum(sq_distance) X (footage) Y (price)
  16. Minimize the loss: build the graph distance = y -

    labels sq_distance = distance * distance cost = tf.reduce_sum(sq_distance) # Change W, b to minimize cost update_step = tf.GradientDescentOptimizer(learning_rate).minimize(cost)
  17. Minimize the loss: run the ops # Define x, W,

    b, y, cost as before update_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) with tf.Session() as sess: tf.initialize_all_variables().run() for i in xrange(100): sess.run(update_step, feed_dict={x: ..., labels: ...}) Your data
  18. Setting up TensorBoard In your python code: summary_writer = tf.train.SummaryWriter('/path/to/logs',

    sess.graph) # Tell the summary_writer what you want to log summary_writer.add_summary(measure, time_step) In the command line: % tensorboard --logdir=/path/to/logs
  19. (1) Use a Cloud-based API (2) Run your own pretrained

    model (3) Use an existing model architecture ◦ retrain or fine tune on your dataset (4) Develop your own machine learning models ◦ solve new problems More flexible, but more effort required Your options
  20. (2) Use pretrained models with tf.gfile.FastGFile(model_path, 'rb') as f: graph_def

    = tf.GraphDef() graph_def.ParseFromString(f.read()) _ = tf.import_graph_def(graph_def, name='') with tf.Session() as sess: softmax_tensor = sess.graph.get_tensor_by_name('softmax:0') predictions = sess.run(softmax_tensor, {'DecodeJpeg/contents:0': image_data}) load model inference
  21. (3) Training your own models cloud.google.com/ml OR Run open-source release

    on • your own physical machines • virtual machines in a cloud hosting environment