Machine learning with Tensorflow and Elasticsearch: Image Recognition MATTEO ZUCCON Data Scientist | Clinical Insights Ltd #ElasticMeetup 

Elastic 7 June 2017 About me MATTEO ZUCCON @matteo_zuccon [email protected] https://whiletrue.run Data Scientist | Clinical Insights Ltd #ElasticMeetup 

Elastic 7 June 2017 • What is Image Recognition? • Tensorflow • Case history architecture and implementation • Demo Agenda 

Elastic 7 June 2017 Image Recognition “an image is a visual representation of something” Process of identifying and detecting an object or a feature in a digital image or video For humans vision seems easy, our brains are incredibly good at understanding images 

Elastic 7 June 2017 What do you see? 

Elastic 7 June 2017 Characters, Barcode and Object instance recognition: easy task, algorithmical techniques, this is not machine learning (not too much intelligence) Object class recognition, scene understanding/interpretation: hard task, advanced machine learning techniques needed Different levels of difficulty 

Elastic 7 June 2017 Neural network that uses many identical copies of the same neuron "...a computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs”1 1. "Neural Network Primer: Part I" by Maureen Caudill, AI Expert, Feb. 1989 TL;DR A Basic Introduction To Neural Networks: http://pages.cs.wisc.edu/~bolo/shipyard/neural/local.html TL;DR Conv Nets: A Modular Perspective http://colah.github.io/posts/2014-07-Conv-Nets-Modular/ TL;DR Using neural nets to recognize handwritten digits http://neuralnetworksanddeeplearning.com/chap1.html Deep CNN: https://www.tensorflow.org/tutorials/deep_cnn Convolutional neural network 

Elastic 7 June 2017 ConvNet: neurons arranged in 3 dimensions: width, height, depth CS231n Convolutional Neural Networks for Visual Recognition – Architecture Overview http://cs231n.github.io/convolutional-networks/#architectures Convolutional neural network - Architecture Overview A regular 3-layer Neural Network ConvNet 

Elastic 7 June 2017 Notable CNNs • LeNet • first CNN 1990’s • read zip codes, digits • AlexNet • won the ImageNet ILSVRC challenge in 2012 • top 5 error of 16% • ZF Net • won the ImageNet ILSVRC challenge in 2013 • improvement on AlexNet by tweaking the architecture hyperparameters • GoogLeNet (aka Inception) • won the ImageNet ILSVRC challenge in 2014 • Developed at Google • dramatically reduced the number of parameters in the network (4M, compared to 60M AlexNet ) TL;DR Convolutional Neural Networks for Visual Recognition: http://cs231n.github.io/convolutional-networks/#case Justin Johnson Stanford Vision Lab - http://cs.stanford.edu/people/jcjohns/ Andrej Karpathy Research Scientist at OpenAI http://cs.stanford.edu/people/karpathy/ 

Elastic 7 June 2017 Training and validation using ImageNet - an academic benchmark for computer vision. Inception-v3 reaches 3.46% 1 (top-5 error rate) 1. Rethinking the Inception Architecture for Computer Vision https://arxiv.org/pdf/1512.00567.pdf TL;DR Inception in Tensorflow: https://github.com/tensorflow/models/tree/master/inception ImageNet http://image-net.org/ How well do humans do on ImageNet Challenge? http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/ Inception Model - convolutional neural network (CNN) 

Elastic 7 June 2017 • Open Source library https://github.com/tensorflow/tensorflow • Based on data flow graphs: • Nodes: mathematical operations • Graph edes: data arrays (tensors) communicated between them • Flexible architecture, deploy computation to one or more CPUs or GPUs • APIs for constructing and executing a TensorFlow graph: • Python • C++ • Java • Go • bindings for: C#, Haskell, Julia, Ruby, Rust Tensorflow - machine learning and deep neural network library 

Elastic 7 June 2017 Tensorflow - machine learning and deep neural network library 

Elastic 7 June 2017 TensorFlow-Examples: https://github.com/aymericdamien/TensorFlow-Examples Tensorflow – example 

Elastic 7 June 2017 Run image classification with Inception (200MB) model It runs inference on an input JPEG image. It outputs the top 5 predictions along with their accuracy probabilities. Source Code: https://github.com/tensorflow/models/blob/master/tutorials/image/imagenet/classify_image.py Image Recognition 

Elastic 7 June 2017 Computationally intensive task (Inception trained in 2 weeks with 8 GPUs NVIDIA Tesla K40s ) • Splits the batch across the GPUs • Updates model parameters synchronously by waiting for all GPUs to finish processing a batch of data How to train from scratch: https://github.com/tensorflow/models/blob/master/inception/README.md#how-to-train-from-scratch Train your own image classifier: https://research.googleblog.com/2016/03/train-your-own-image-classifier-with.html Codelabs by Google: https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/#0 Train from Scratch 

Elastic 7 June 2017 AWS Rekognition Google Cloud Vision (Tensorflow?) Azure Computer Vision API Kairos IBM Bluemix Visual Recognition Pros • RESTful APIs • Pay-as-you-go • Granted accuracy (Model updated) • Embedded system Cons • Hardly real-time (HTTPs calls) • ML Models are black box • No model customizations Tensorflow alternatives – cloud services 

Elastic 7 June 2017 Case History - Architecture 

Elastic 7 June 2017 Case History – Why? Why Elasticsearch ? • You Know, For Search! • Fast and Scalable • Nested Query - wildcard Why AWS S3 (or similar object storage)? • 99.99…% durability • Scalable • Up to 5TB for bucket • Control access • Low Price 

Elastic 7 June 2017 Case History – example of applications • Smart Photo Gallery: find things in your photos • Product Catalog: automatic classification of products • Content Moderation: get notified of NSFW contents • Accessibility camera app: help people with disability to see and take pictures 

Elastic 7 June 2017 Image type 

Elastic 7 June 2017 Index image classification 

Elastic 7 June 2017 Consume images and run classification Image Classification Source Code: https://github.com/tensorflow/models/blob/master/tutorials/image/imagenet/classify_image.py 

Elastic 7 June 2017 Index classification to Elasticsearch 

Elastic 7 June 2017 Search by Tag 

Elastic 7 June 2017 Search by Tag and accuracy 

Elastic 7 June 2017 Demo 

Elastic 7 June 2017 Machine learning with Tensorflow and Elasticsearch: Image recognition Thank you ☺ Any questions? @matteo_zuccon [email protected] https://whiletrue.run #ElasticMeetup 

No content

Elastic 7 June 2017 Demo - Search by Tag 

Elastic 7 June 2017 Demo - Search by Tag 

Elastic 7 June 2017 Demo - Search by Tag 

Elastic 7 June 2017 Demo - Search by Tag and Accuracy 

Elastic 7 June 2017 Demo – Upload image - classifications Image tags 

No content