TensorFlow - Overview Ankit Bahuguna Software Engineer, . TensorFlow Müchen March 1st, 2016 

About Me • Software Engineer (R&D), CLIQZ GmbH. • Building a web scale search engine, optimized for German speaking community. • Areas: Large scale Information Retrieval, Machine Learning, Deep Learning and Natural Language Processing. • Masters in Informatics: TU Munich ‘15, 
 Work & Research Experience: Teradata, TUM, LMU and IIT Bombay. • Mozilla Representative (2012 - Present) Ankit Bahuguna (@codekee) 

Android + iOS Browsers Web Browsers based on Firefox + Cliqz Extension for Firefox 

Outline • What is TensorFlow, really? • What does TensorFlow bring to the table? • TensorFlow Mechanics • TensorBoard: Visualizing Learning • TF Demos: Basic and Advanced • Benchmarks and Likely scenarios for evolution of TensorFlow. • The new world ahead: TensorFlow - Today and tomorrow. • Q & A 

TensorFlow (TF) • An open source software library for Machine Intelligence by Google. • Single machine with single and multiple device [cpu/ gpu] support and distributed implementation. • Code written in C++ along with Python API. • Latest stable release v0.7.1 [Feb 19th, 2016] • License: Apache 2.0 Open Source {tensorflow.org} 

TensorFlow Usage • RankBrain: A large-scale deployment of deep neural nets for search ranking on www.google.com. • Inception Image Classification Model: Baseline model and follow on research into highly accurate computer vision models, Ex. 2014 Imagenet image classification challenge winning model. • SmartReply: Deep LSTM model to automatically generate email responses • Massively Multitask Networks for Drug Discovery: A deep neural network model for identifying promising drug candidates. • On-Device Computer Vision for OCR: On-device computer vision model to do optical character recognition to enable real-time translation. 

How is TensorFlow different? • Can be compiled/optimized on ARM architectures. • Deploy your trained models on a variety of devices (servers , sensors or mobile devices) • Multi-GPU Training. However not as fast as CNTK. • Clean, modular architecture with multiple front- ends and execution platforms. 

TensorFlow Mechanics • TF Computation: Directed Graph with set of nodes. • Graph represents a data-flow computation, provision for nodes to maintain and update states with branching and looping support. • Node: Represents - Instantiation of an operation. • Tensors: Arbitrary dimensional arrays or weights of the edges (flows across the nodes). • Control Dependencies: source node for the control dependence must finish executing before the destination node for the control dependence starts executing. 

TensorFlow Mechanics • Operation: An abstract computation: add, matrix multiply etc. • An operation can have attributes, must be provided at graph-construction time, to instantiate a node to perform the operation. • A kernel is a particular implementation of an operation that can be run on a particular type of device (e.g., CPU or GPU) • Clients programs interact with the TF system by creating a Session(). • session.extend(): augments the current graph managed by the session with additional nodes and edges. • session.run(): takes input: optional tensors and output names; Computes transitive closure of all nodes in order to get requested output. • Variable is a special operation: Returns handle to a persistent mutable tensor that survives across executions of a graph. • Passed to some special operations: Assign (=) and AssignAdd (+=) that mutate the referenced tensor. 

• Local and distributed implementations. • Devices: (type + name). Ex. Device Names: • "/job:localhost/device:cpu:0" or • “/job:worker/task:17/device:gpu:3" • Tensors: A typed, multidimensional array. • signed and unsigned integers (size: 8 bits to 64 bits), IEEE float and double types, a complex number type, and string type. TensorFlow Implementation 

TensorFlow Code & Computation Graph 

TensorBoard: 
 Visualizing Learning A: TensorBoard B: Model summary statistics time series data (A) (B) 

TensorBoard: 
 Graph Visualization 

TensorBoard: CNN Example 

Simple Demo Simple Linear Regression in TensorFlow 

Advanced Demo Visualizing Word Embeddings in TensorFlow 

Benchmarks (Conv-nets) * Soumith Chintala: https://github.com/soumith/convnet-benchmarks/issues/66 Current Version TF: 0.7.1 

Google Translate 
 Android App (WordLens) 

Final Words.. • TF aids in specifying a new networks fairly easily. • Can run on multiple devices (PC/ mobile/ IOT/ server). • C++ & Python: easy to prototype new models. • TensorBoard: useful tool for visualization of data flow and log summaries. • Built to work in both experimental and production environments. • Although, TF doesn’t yet support loop and condition controls in the graph definition. Hence, RNN’s slower. • Active development cycle with support from Google. • Future: TF + IOT = Super Smart Sensor Systems. 

References • TensorFlow Website: https://tensorflow.org • Evaluation of Deep Learning Toolkits: https://github.com/zer0n/ deepframeworks/blob/master/README.md • Udacity's Deep Learning Course: https://www.udacity.com/courses/ ud730 • TensorFlow Whitepaper: http://download.tensorflow.org/paper/ whitepaper2015.pdf • TensorFlow Convents Benchmark: https://github.com/soumith/convnet- benchmarks/issues/66
 

Thanks! Questions, or Feedback: [email protected] Twitter: @codekee LinkedIn: de.linkedin.com/in/ankitbahuguna