Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Getting Started with TensorFlow

Getting Started with TensorFlow

Avatar for Rebecca Murphy

Rebecca Murphy

March 21, 2016
Tweet

More Decks by Rebecca Murphy

Other Decks in Programming

Transcript

  1. Talk Overview • TensorFlow overview • Programming Model • Mechanics

    of TensorFlow • Installation • Model Definition • Fitting • Checkpointing • TensorBoard visualisations • Why TensorFlow?
  2. What Is TensorFlow? • Google’s 2nd generation deep learning library

    • Simple API (Python, C++) for: • Describing Machine Learning models • Implementing Machine Learning algorithms
  3. What Can We Do With TensorFlow? • Regression models •

    Neural networks • Deep learning: • Distributed representations • Convolutional Networks • Recurrent Neural Networks • LSTM Neural Networks
  4. TensorFlow: What Is a Tensor? • Tensor: n-dimensional array •

    Scalar: 0D Tensor • Vector: 1D Tensor • Matrix: 2D Tensor • Typed: • Int, double, complex, string
  5. Tensor Flows • Tensor Flow computations: stateful dataflow graphs •

    Deep learning model = Directed graph • Node: (mathematical) operation • Edge: • Control dependencies • Data flow • Describe graph -> initialize -> execute (parts of ) graph
  6. Installing TensorFlow • Python API • Python 2.7 • Python

    3.3+ • Setup instructions • pip install: • pip install --upgrade https://storage.googleapis. com/tensorflow/mac/tensorflow-0.7.1-cp27-none-any.whl • Docker: • docker run -it b.gcr.io/tensorflow/tensorflow
  7. Mechanics of Learning • Define model • Load data •

    Feed data • Make predictions • Evaluate • Visualise
  8. Model Definition: Key Features (1) • Tensor shapes are pre-defined:

    • Tensors support mathematical manipulation • Operations are nodes in the model graph
  9. Model Definition: Key Features (2) • Built-in functions for common

    Deep Learning operations: • See Neural Network API for more • Gradient descent optimisation: • Variables store current state of model
  10. Training the Model: Loading Data (1) • Load data into

    variables • Need to write custom functions to parse data
  11. Training the Model: Sessions • Model graph describes computations •

    Computations evaluated within a session: • Places graph onto CPU / GPU • Supplies methods to evaluate graph operations
  12. Training the Model: Evaluation • Pre-defined evaluation nodes compare predicted

    and true labels: • Evaluate accuracy function within a session:
  13. Checkpoints: Saving Models • Saver class allows model state to

    be stored and reloaded • Use checkpoints to periodically save the state of the model
  14. • Saver class allows model state to be stored and

    reloaded • Restore a previously trained model Checkpoints: Loading Saved Models
  15. Flags: Controlling Training • tf.app.flags: set command-line arguments • Wraps

    python gflags • tf.app.run() parses flags before calling main()
  16. TensorBoard: Basics • TensorFlow visualisation tool • View • Graph

    models • Training behaviour • Simple modifications to model code • Browser-based tool
  17. TensorBoard: Saving Output • Set up summary and writer objects

    • Periodically run evaluation and store output: • tensorboard --logdir=try_tf_logs/
  18. Why Use TensorFlow: Great Examples • TensorFlow Tutorials • Handwriting

    generation from @hardmaru • Next letter prediction from @karpathy
  19. TensorFlow: Future Developments • Improved memory usage in gradient calculations

    • JIT Compilation • Improved node execution scheduling • Support for parallelisation across many machines • Support for more languages (Java, Lua, Go, R …) • Source: TensorFlow Whitepaper