Elastic Meetup - Tensorflow and Elasticsearch: Image Recognition

Slide of my speech about Tensorflow and Elasticsearch: image recognition @elasticMeetup in Milan / June 2017

Transcript

1. Machine learning with Tensorflow and Elasticsearch: Image Recognition MATTEO ZUCCON Data Scientist | Clinical Insights Ltd #ElasticMeetup
2. Elastic 7 June 2017 About me MATTEO ZUCCON @matteo_zuccon zuccon.matteo@gmail.com https://whiletrue.run Data Scientist | Clinical Insights Ltd #ElasticMeetup
3. Elastic 7 June 2017 • What is Image Recognition? • Tensorflow • Case history architecture and implementation • Demo Agenda
4. Elastic 7 June 2017 Image Recognition “an image is a visual representation of something” Process of identifying and detecting an object or a feature in a digital image or video For humans vision seems easy, our brains are incredibly good at understanding images
5. Elastic 7 June 2017 What do you see?
6. Elastic 7 June 2017 Characters, Barcode and Object instance recognition: easy task, algorithmical techniques, this is not machine learning (not too much intelligence) Object class recognition, scene understanding/interpretation: hard task, advanced machine learning techniques needed Different levels of difficulty
7. Elastic 7 June 2017 Neural network that uses many identical copies of the same neuron "...a computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs”1 1. "Neural Network Primer: Part I" by Maureen Caudill, AI Expert, Feb. 1989 TL;DR A Basic Introduction To Neural Networks: http://pages.cs.wisc.edu/~bolo/shipyard/neural/local.html TL;DR Conv Nets: A Modular Perspective http://colah.github.io/posts/2014-07-Conv-Nets-Modular/ TL;DR Using neural nets to recognize handwritten digits http://neuralnetworksanddeeplearning.com/chap1.html Deep CNN: https://www.tensorflow.org/tutorials/deep_cnn Convolutional neural network
8. Elastic 7 June 2017 ConvNet: neurons arranged in 3 dimensions: width, height, depth CS231n Convolutional Neural Networks for Visual Recognition – Architecture Overview http://cs231n.github.io/convolutional-networks/#architectures Convolutional neural network - Architecture Overview A regular 3-layer Neural Network ConvNet
9. Elastic 7 June 2017 Notable CNNs • LeNet • first CNN 1990’s • read zip codes, digits • AlexNet • won the ImageNet ILSVRC challenge in 2012 • top 5 error of 16% • ZF Net • won the ImageNet ILSVRC challenge in 2013 • improvement on AlexNet by tweaking the architecture hyperparameters • GoogLeNet (aka Inception) • won the ImageNet ILSVRC challenge in 2014 • Developed at Google • dramatically reduced the number of parameters in the network (4M, compared to 60M AlexNet ) TL;DR Convolutional Neural Networks for Visual Recognition: http://cs231n.github.io/convolutional-networks/#case Justin Johnson Stanford Vision Lab - http://cs.stanford.edu/people/jcjohns/ Andrej Karpathy Research Scientist at OpenAI http://cs.stanford.edu/people/karpathy/
10. Elastic 7 June 2017 Training and validation using ImageNet - an academic benchmark for computer vision. Inception-v3 reaches 3.46% 1 (top-5 error rate) 1. Rethinking the Inception Architecture for Computer Vision https://arxiv.org/pdf/1512.00567.pdf TL;DR Inception in Tensorflow: https://github.com/tensorflow/models/tree/master/inception ImageNet http://image-net.org/ How well do humans do on ImageNet Challenge? http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/ Inception Model - convolutional neural network (CNN)
11. Elastic 7 June 2017 • Open Source library https://github.com/tensorflow/tensorflow • Based on data flow graphs: • Nodes: mathematical operations • Graph edes: data arrays (tensors) communicated between them • Flexible architecture, deploy computation to one or more CPUs or GPUs • APIs for constructing and executing a TensorFlow graph: • Python • C++ • Java • Go • bindings for: C#, Haskell, Julia, Ruby, Rust Tensorflow - machine learning and deep neural network library
12. Elastic 7 June 2017 Tensorflow - machine learning and deep neural network library
13. Elastic 7 June 2017 TensorFlow-Examples: https://github.com/aymericdamien/TensorFlow-Examples Tensorflow – example
14. Elastic 7 June 2017 Run image classification with Inception (200MB) model It runs inference on an input JPEG image. It outputs the top 5 predictions along with their accuracy probabilities. Source Code: https://github.com/tensorflow/models/blob/master/tutorials/image/imagenet/classify_image.py Image Recognition
15. Elastic 7 June 2017 Computationally intensive task (Inception trained in 2 weeks with 8 GPUs NVIDIA Tesla K40s ) • Splits the batch across the GPUs • Updates model parameters synchronously by waiting for all GPUs to finish processing a batch of data How to train from scratch: https://github.com/tensorflow/models/blob/master/inception/README.md#how-to-train-from-scratch Train your own image classifier: https://research.googleblog.com/2016/03/train-your-own-image-classifier-with.html Codelabs by Google: https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/#0 Train from Scratch
16. Elastic 7 June 2017 AWS Rekognition Google Cloud Vision (Tensorflow?) Azure Computer Vision API Kairos IBM Bluemix Visual Recognition Pros • RESTful APIs • Pay-as-you-go • Granted accuracy (Model updated) • Embedded system Cons • Hardly real-time (HTTPs calls) • ML Models are black box • No model customizations Tensorflow alternatives – cloud services
17. Elastic 7 June 2017 Case History - Architecture
18. Elastic 7 June 2017 Case History – Why? Why Elasticsearch ? • You Know, For Search! • Fast and Scalable • Nested Query - wildcard Why AWS S3 (or similar object storage)? • 99.99…% durability • Scalable • Up to 5TB for bucket • Control access • Low Price
19. Elastic 7 June 2017 Case History – example of applications • Smart Photo Gallery: find things in your photos • Product Catalog: automatic classification of products • Content Moderation: get notified of NSFW contents • Accessibility camera app: help people with disability to see and take pictures
20. Elastic 7 June 2017 Image type
21. Elastic 7 June 2017 Index image classification
22. Elastic 7 June 2017 Consume images and run classification Image Classification Source Code: https://github.com/tensorflow/models/blob/master/tutorials/image/imagenet/classify_image.py
23. Elastic 7 June 2017 Index classification to Elasticsearch
24. Elastic 7 June 2017 Search by Tag
25. Elastic 7 June 2017 Search by Tag and accuracy
26. Elastic 7 June 2017 Demo
27. Elastic 7 June 2017 Machine learning with Tensorflow and Elasticsearch: Image recognition Thank you ☺ Any questions? @matteo_zuccon zuccon.matteo@gmail.com https://whiletrue.run #ElasticMeetup
29. Elastic 7 June 2017 Demo - Search by Tag
30. Elastic 7 June 2017 Demo - Search by Tag
31. Elastic 7 June 2017 Demo - Search by Tag
32. Elastic 7 June 2017 Demo - Search by Tag and Accuracy
33. Elastic 7 June 2017 Demo – Upload image - classifications Image tags