Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machines Can Learn - a Practical Take on Machine Intelligence Using Spring Cloud Data Flow and TensorFlow

Machines Can Learn - a Practical Take on Machine Intelligence Using Spring Cloud Data Flow and TensorFlow

https://springoneplatform.io/2018/sessions/machines-can-learn-a-practical-take-on-machine-intelligence-using-spring-cloud-data-flow-and-tensorflow

Machine learning (ML) has brought unprecedented abilities to the software engineering field. ML allows you to reason about and to solve otherwise "un-programmable" tasks such as computer vision and language processing. If you're a Java developer and you're interested in leveraging ML to deliver richer business insights to your customers, in this talk you'll learn what it takes to build cloud-native applications to perform data-driven machine intelligence operations. This coding-centric talk walks through the different facets of iterative development and testing using Spring Cloud Stream and the orchestration of such applications into coherent data pipelines using Spring Cloud Data Flow. Specifically, we will also review TensorFlow, a popular Machine Learning toolkit, and how it is integrated in the overall design. This talk will showcase how building a complex use-cases such as real-time image recognition or object detection, can be simplified with the help of Spring Ecosystem and TensorFlow. More importantly, I'd will share the findings from the ML space; tips and tricks on what goes into developing such complex solutions.

Christian Tzolov

September 25, 2018
Tweet

More Decks by Christian Tzolov

Other Decks in Programming

Transcript

  1. Machines Can Learn - a Practical Take on
    Machine Intelligence Using Spring Cloud
    Data Flow and TensorFlow
    By Christian Tzolov
    @christzolov

    View Slide

  2. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Disclaimer

    View Slide

  3. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Safe Harbor Statement
    The following is intended to outline the general direction of Pivotal's offerings. It is intended for information purposes
    only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal offerings,
    future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is subject to change.
    This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any
    material, code, or functionality, and should not be relied upon in making purchasing decisions regarding Pivotal's
    offerings. These purchasing decisions should only be based on features currently available. The development,
    release, and timing of any features or functionality described for Pivotal's offerings in this presentation remain at the
    sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation.
    3

    View Slide

  4. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Does It Matter?

    View Slide

  5. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Rise of the machines
    • Machine learning brings unprecedented abilities to the Software Engineering
    field
    • Provides different way to reason about problems
    • Solves “un-programmable” tasks
    • But there is a technological gap (BLAS/CPU, CUDA/GPU, C++/Python)
    • And cultural difference J
    • How Machine Learning can enable us deliver richer business solutions?
    5

    View Slide

  6. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Real-Time Predictions with TensorFlow and Spring Cloud
    Data Flow (SCDF)
    • Supervised, Deep Learning (DL)
    • TensorFlow
    • Model Inference (e.g. Evaluation, Prediction)
    • (Near) Real-Time - Streaming
    • Java and Spring
    6

    View Slide

  7. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Can Machine Learn?

    View Slide

  8. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Classical Programming
    8
    Classical
    Programming
    Rules
    Data
    Answers

    View Slide

  9. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Machine Learning Paradigm
    9
    Machine
    Learning
    Rules’
    Data*
    Answers*
    * Historical, annotated Datasets

    View Slide

  10. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    ML Lifecycle
    10
    Model Inference
    Output Predictions
    MODEL
    (e.g. Rules)
    New Data
    Model Training
    Annotated answers
    Historical Data
    Offline Online (Production)
    Software Engineering & Ops
    Data Science

    View Slide

  11. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Remote Server
    Model Serving & Embedding
    11
    Data Source
    HTTP/REST, RPC
    Processor(s)
    Data Sink
    Data Source Processor(s)
    Model
    Data Sink
    Runtime
    Model
    Remote Serving
    In-Pipeline Embedded

    View Slide

  12. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Deep Learning Inference with Java
    • Deep Learning Java Runtime
    • Portable Models
    12
    DL Java Runtime
    Input Data
    Output
    Predictions
    Pre-trained Model
    (e.g. Rules)

    View Slide

  13. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Deep Learning Frameworks

    View Slide

  14. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Deep Learning Frameworks
    • Tensorflow (Google)
    • PyTorch
    • CNTK (Microsoft)
    • Caffe2 (Facebook)
    • MXNet (Amazon)
    • DeepLearning4J
    • Keras
    14
    Deep Learning Java Runtime

    View Slide

  15. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Real-Time Model Evaluation with
    Spring Cloud Data Flow

    View Slide

  16. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Transport Middleware
    Stream A
    Spring Cloud Data Flow (SCDF)
    Toolkit for building data integration, real-time, and batch data processing pipelines
    • Multi-platform support (PCF, Kubernetes)
    • Comprehensive app lifecycle (Skipper)
    • Pipes & Filters like DSL: Source | Processor … | Sink
    Source Processor Processor Sink

    View Slide

  17. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Localizing and identifying multiple objects in a single image
    • TF Object Detection API: http://bit.ly/tensorflow-object-detection-api
    • Pre-trained models: http://bit.ly/object-detection-api-model-zoo
    • Object Detection Processor Doc: http://bit.ly/scdf-object-detection-processor
    • Blog: https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream
    Object Detection Processor
    file --directory=‘/tmp/scdf/od/input’
    | object-detection
    --mode=header
    --model='file:/model.pb’
    --labels='file:/label_map.pbtxt’
    | image-viewer

    View Slide

  18. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    TensorFlow SCDF Processor
    21
    TensorFlowService
    Message
    payload
    headers
    Output Converter
    Input Converter
    Message
    payload
    headers
    Map
    Load from file:// , http://, classpath://
    TensorFlow
    Build/Train/Export
    TensorFlow
    Runtime
    TF Model
    Offline
    Real Time

    View Slide

  19. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Object Detection - RT Demo

    View Slide

  20. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Available TensorFlow Processors
    • Object Detection – TF Object Detection API: [12] [13]
    • Instance Segmentation, TF Object Detection API: [11]
    • Semantic Segmentation, TF API: [8][9][10]
    • Image Recognition (Inception V3), TensorFlow API: [7]
    • Pose Estimation (OpenPose), TensorFlow API [5], [6]
    • Twitter Sentiment Analysis, TensorFlow API: [4]
    • Face Detection (MTCNN), ND4J,
    ND4J-Tensorflow, JavaCV: [1], [2]
    • Face Recognition (FaceNet), TensorFlow API: [3]

    View Slide

  21. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Quick Starters
    • TensorFlow Spring Cloud Stream App Starters:
    https://github.com/spring-cloud-stream-app-starters/tensorflow
    • Computer Vision Projects (experimental):
    https://github.com/tzolov/computer-vision
    • SCDF TensorFlow Demos (Docker):
    https://github.com/tzolov/scdf-tensorflow-demos
    • Real-time Object-Detection With Spring Cloud Stream
    https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream
    • Real-time Object-Detection With Spring Cloud Stream
    https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream
    • Real-Time Multi-Person, Human Pose Estimation with Spring Cloud Data Flow and
    TensorFlow
    http://bit.ly/scdf-pose-estimationn-blog

    View Slide

  22. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Conclusions

    View Slide

  23. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Convinced?
    • Data Science and Deep Learning mature at a rapid pace
    • Pivotal is committed to make them easily accessible for Spring/Java developers.
    • Way to start: Check the TensorFlow processors for Spring Cloud Data Flow
    • You feedback is appreciated!

    View Slide

  24. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Next

    View Slide

  25. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    Unofficial agenda
    • Top level TensorFlow (or DL) project – move the utilities and existing
    implementations (such as Image Recognition, Object Detection, Sentiment
    Analysis) out of SCSt into a top level, generic projects.
    • Use with Spring Cloud Stream, Spring Cloud Function, Spring Cloud Batch
    • Research “Transfer Learning” possibilities
    • Evaluate Deeplearning4J, ND4J as Runtime alternative
    • ONNX Runtime to use models from other DL frameworks
    • REST, Spring Boot as remote DL serving application
    39

    View Slide

  26. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
    Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
    References
    [1] Face Detection Lib: https://github.com/tzolov/mtcnn-java
    [2] Face Detection Processor: https://github.com/tzolov/computer-vision/blob/master/spring-cloud-starter-stream-processor-face-detection-mtcnn
    [3] Face Recognition: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-face-recognition
    [4] Twitter Sentiment Analysis: https://github.com/spring-cloud-stream-app-starters/tensorflow/blob/master/spring-cloud-starter-stream-processor-twitter-sentiment
    [5] Pose Estimation Processor: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-pose-estimation
    [6] Pose Estimation Blog: https://content.pivotal.io/blog/making-data-science-accessible-to-developers-real-time-multi-person-human-pose-estimation-with-spring-cloud-
    data-flow-and-tensorflow
    [7] Image Recognition: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-image-recognition
    [8] Semantic Segmentation Blog: https://goo.gl/dtwAfo
    [9] Semantic Segmentation - Java2D: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-semantic-segmentation
    [10] Semantic Segmentation - JavaCV: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-semantic-segmentation-cv
    [11] Instance Segmentation - Object Detection Processor with Masks: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-
    stream-processor-object-detection
    [12] Object Detection Processor: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-object-detection
    [13] Object Detection Blog: https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream
    40

    View Slide

  27. > Stay Connected.

    View Slide