Machine learning with Tensorflow and Elasticsearch: Image Recognition #EsInADay17 MATTEO ZUCCON Data Scientist | Clinical Insights Ltd 

About me MATTEO ZUCCON @matteo_zuccon [email protected] https://whiletrue.run Data Scientist | Clinical Insights Ltd #EsInADay17 

• What is Image Recognition? • Tensorflow • Case history architecture and implementation • Demo Agenda 

Image Recognition “an image is a visual representation of something” Process of identifying and detecting an object or a feature in a digital image or video For humans vision seems easy, our brains are incredibly good at understanding images 

What do you see? 

Characters, Barcode and Object instance recognition: easy task, algorithmical techniques, this is not machine learning (not too much intelligence) Object class recognition, scene understanding/interpretation: hard task, advanced machine learning techniques needed Different levels of difficulty 

Neural network that uses many identical copies of the same neuron "...a computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs”1 1. "Neural Network Primer: Part I" by Maureen Caudill, AI Expert, Feb. 1989 TL;DR A Basic Introduction To Neural Networks: http://pages.cs.wisc.edu/~bolo/shipyard/neural/local.html TL;DR Conv Nets: A Modular Perspective http://colah.github.io/posts/2014-07-Conv-Nets-Modular/ TL;DR Using neural nets to recognize handwritten digits http://neuralnetworksanddeeplearning.com/chap1.html Deep CNN: https://www.tensorflow.org/tutorials/deep_cnn Convolutional neural network 

ConvNet: neurons arranged in 3 dimensions: width, height, depth CS231n Convolutional Neural Networks for Visual Recognition – Architecture Overview http://cs231n.github.io/convolutional-networks/#architectures Convolutional neural network - Architecture Overview A regular 3-layer Neural Network ConvNet 

Notable CNNs • LeNet • first CNN 1990’s • read zip codes, digits • AlexNet • won the ImageNet ILSVRC challenge in 2012 • top 5 error of 16% • ZF Net • won the ImageNet ILSVRC challenge in 2013 • improvement on AlexNet by tweaking the architecture hyperparameters • GoogLeNet (aka Inception) • won the ImageNet ILSVRC challenge in 2014 • Developed at Google • dramatically reduced the number of parameters in the network (4M, compared to 60M AlexNet ) TL;DR Convolutional Neural Networks for Visual Recognition: http://cs231n.github.io/convolutional-networks/#case Justin Johnson Stanford Vision Lab - http://cs.stanford.edu/people/jcjohns/ Andrej Karpathy Research Scientist at OpenAI http://cs.stanford.edu/people/karpathy/ 

Training and validation using ImageNet - an academic benchmark for computer vision. Inception-v3 reaches 3.46% 1 (top-5 error rate) 1. Rethinking the Inception Architecture for Computer Vision https://arxiv.org/pdf/1512.00567.pdf TL;DR Inception in Tensorflow: https://github.com/tensorflow/models/tree/master/inception ImageNet http://image-net.org/ How well do humans do on ImageNet Challenge? http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/ Codelabs by Google: https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/#0 Inception Model - convolutional neural network (CNN) 

• Open Source library https://github.com/tensorflow/tensorflow • Based on data flow graphs: • Nodes: mathematical operations • Graph edes: data arrays (tensors) communicated between them • Flexible architecture, deploy computation to one or more CPUs or GPUs • APIs for constructing and executing a TensorFlow graph: • Python • C++ • Java • Go • bindings for: C#, Haskell, Julia, Ruby, Rust Tensorflow - machine learning and deep neural network library 

Run image classification with Inception (200MB) model It runs inference on an input JPEG image. It outputs the top 5 predictions along with their accuracy probabilities. Source Code: https://github.com/tensorflow/models/blob/master/tutorials/image/imagenet/classify_image.py Image Recognition 

AWS Rekognition Google Cloud Vision (Tensorflow?) Azure Computer Vision API Kairos Pros • RESTful APIs • Pay-as-you-go • Granted accuracy (Model updated) • Embedded system Cons • Hardly real-time (HTTPs calls) • ML Models are black box • No model customizations Tensorflow alternatives – cloud services 

Case History - Architecture 

Case History – Why? Why Elasticsearch ? • You Know, For Search! • Fast and Scalable • Nested Query - wildcard Why AWS S3 (or similar object storage)? • 99.99…% durability • Scalable • Up to 5TB for bucket • Control access • Low Price 

Case History – example of applications • Smart Photo Gallery: find things in your photos • Product Catalog: automatic classification of products • Content Moderation • Accessibility camera app 

Image type 

Index image classification 

Search by Tag 

Search by Tag and accuracy 

Demo 

Machine learning with Tensorflow and Elasticsearch: Image recognition Thank you ☺ Any questions? #EsInADay17 @matteo_zuccon [email protected] https://whiletrue.run 

No content

Demo - Search by Tag 

Demo - Search by Tag 

Demo - Search by Tag 

Demo - Search by Tag and Accuracy 

Demo – Upload image - classifications Image tags 

No content