Machines Can Learn - a Practical Take on Machine Intelligence Using Spring Cloud Data Flow and TensorFlow

https://springoneplatform.io/2018/sessions/machines-can-learn-a-practical-take-on-machine-intelligence-using-spring-cloud-data-flow-and-tensorflow

Machine learning (ML) has brought unprecedented abilities to the software engineering field. ML allows you to reason about and to solve otherwise "un-programmable" tasks such as computer vision and language processing. If you're a Java developer and you're interested in leveraging ML to deliver richer business insights to your customers, in this talk you'll learn what it takes to build cloud-native applications to perform data-driven machine intelligence operations. This coding-centric talk walks through the different facets of iterative development and testing using Spring Cloud Stream and the orchestration of such applications into coherent data pipelines using Spring Cloud Data Flow. Specifically, we will also review TensorFlow, a popular Machine Learning toolkit, and how it is integrated in the overall design. This talk will showcase how building a complex use-cases such as real-time image recognition or object detection, can be simplified with the help of Spring Ecosystem and TensorFlow. More importantly, I'd will share the findings from the ML space; tips and tricks on what goes into developing such complex solutions.

Transcript

1. Machines Can Learn - a Practical Take on Machine Intelligence Using Spring Cloud Data Flow and TensorFlow By Christian Tzolov @christzolov
2. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Disclaimer
3. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Safe Harbor Statement The following is intended to outline the general direction of Pivotal's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding Pivotal's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for Pivotal's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation. 3
4. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Does It Matter?
5. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Rise of the machines • Machine learning brings unprecedented abilities to the Software Engineering field • Provides different way to reason about problems • Solves “un-programmable” tasks • But there is a technological gap (BLAS/CPU, CUDA/GPU, C++/Python) • And cultural difference J • How Machine Learning can enable us deliver richer business solutions? 5
6. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Real-Time Predictions with TensorFlow and Spring Cloud Data Flow (SCDF) • Supervised, Deep Learning (DL) • TensorFlow • Model Inference (e.g. Evaluation, Prediction) • (Near) Real-Time - Streaming • Java and Spring 6
7. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Can Machine Learn?
8. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Classical Programming 8 Classical Programming Rules Data Answers
9. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Machine Learning Paradigm 9 Machine Learning Rules’ Data* Answers* * Historical, annotated Datasets
10. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ ML Lifecycle 10 Model Inference Output Predictions MODEL (e.g. Rules) New Data Model Training Annotated answers Historical Data Offline Online (Production) Software Engineering & Ops Data Science
11. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Remote Server Model Serving & Embedding 11 Data Source HTTP/REST, RPC Processor(s) Data Sink Data Source Processor(s) Model Data Sink Runtime Model Remote Serving In-Pipeline Embedded
12. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Deep Learning Inference with Java • Deep Learning Java Runtime • Portable Models 12 DL Java Runtime Input Data Output Predictions Pre-trained Model (e.g. Rules)
13. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Deep Learning Frameworks
14. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Deep Learning Frameworks • Tensorflow (Google) • PyTorch • CNTK (Microsoft) • Caffe2 (Facebook) • MXNet (Amazon) • DeepLearning4J • Keras 14 Deep Learning Java Runtime
15. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Real-Time Model Evaluation with Spring Cloud Data Flow
16. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Transport Middleware Stream A Spring Cloud Data Flow (SCDF) Toolkit for building data integration, real-time, and batch data processing pipelines • Multi-platform support (PCF, Kubernetes) • Comprehensive app lifecycle (Skipper) • Pipes & Filters like DSL: Source | Processor … | Sink Source Processor Processor Sink
17. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Localizing and identifying multiple objects in a single image • TF Object Detection API: http://bit.ly/tensorflow-object-detection-api • Pre-trained models: http://bit.ly/object-detection-api-model-zoo • Object Detection Processor Doc: http://bit.ly/scdf-object-detection-processor • Blog: https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream Object Detection Processor file --directory=‘/tmp/scdf/od/input’ | object-detection --mode=header --model='file:/model.pb’ --labels='file:/label_map.pbtxt’ | image-viewer
18. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ TensorFlow SCDF Processor 21 TensorFlowService Message payload headers Output Converter Input Converter Message payload headers Map<String, Tensor> Load from file:// , http://, classpath:// TensorFlow Build/Train/Export TensorFlow Runtime TF Model Offline Real Time
19. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Object Detection - RT Demo
20. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Available TensorFlow Processors • Object Detection – TF Object Detection API: [12] [13] • Instance Segmentation, TF Object Detection API: [11] • Semantic Segmentation, TF API: [8][9][10] • Image Recognition (Inception V3), TensorFlow API: [7] • Pose Estimation (OpenPose), TensorFlow API [5], [6] • Twitter Sentiment Analysis, TensorFlow API: [4] • Face Detection (MTCNN), ND4J, ND4J-Tensorflow, JavaCV: [1], [2] • Face Recognition (FaceNet), TensorFlow API: [3]
21. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Quick Starters • TensorFlow Spring Cloud Stream App Starters: https://github.com/spring-cloud-stream-app-starters/tensorflow • Computer Vision Projects (experimental): https://github.com/tzolov/computer-vision • SCDF TensorFlow Demos (Docker): https://github.com/tzolov/scdf-tensorflow-demos • Real-time Object-Detection With Spring Cloud Stream https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream • Real-time Object-Detection With Spring Cloud Stream https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream • Real-Time Multi-Person, Human Pose Estimation with Spring Cloud Data Flow and TensorFlow http://bit.ly/scdf-pose-estimationn-blog
22. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Conclusions
23. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Convinced? • Data Science and Deep Learning mature at a rapid pace • Pivotal is committed to make them easily accessible for Spring/Java developers. • Way to start: Check the TensorFlow processors for Spring Cloud Data Flow • You feedback is appreciated!
24. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Next
25. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Unofficial agenda • Top level TensorFlow (or DL) project – move the utilities and existing implementations (such as Image Recognition, Object Detection, Sentiment Analysis) out of SCSt into a top level, generic projects. • Use with Spring Cloud Stream, Spring Cloud Function, Spring Cloud Batch • Research “Transfer Learning” possibilities • Evaluate Deeplearning4J, ND4J as Runtime alternative • ONNX Runtime to use models from other DL frameworks • REST, Spring Boot as remote DL serving application 39
26. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ References [1] Face Detection Lib: https://github.com/tzolov/mtcnn-java [2] Face Detection Processor: https://github.com/tzolov/computer-vision/blob/master/spring-cloud-starter-stream-processor-face-detection-mtcnn [3] Face Recognition: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-face-recognition [4] Twitter Sentiment Analysis: https://github.com/spring-cloud-stream-app-starters/tensorflow/blob/master/spring-cloud-starter-stream-processor-twitter-sentiment [5] Pose Estimation Processor: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-pose-estimation [6] Pose Estimation Blog: https://content.pivotal.io/blog/making-data-science-accessible-to-developers-real-time-multi-person-human-pose-estimation-with-spring-cloud- data-flow-and-tensorflow [7] Image Recognition: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-image-recognition [8] Semantic Segmentation Blog: https://goo.gl/dtwAfo [9] Semantic Segmentation - Java2D: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-semantic-segmentation [10] Semantic Segmentation - JavaCV: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-semantic-segmentation-cv [11] Instance Segmentation - Object Detection Processor with Masks: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter- stream-processor-object-detection [12] Object Detection Processor: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-object-detection [13] Object Detection Blog: https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream 40
27. > Stay Connected.