Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Wie man eine Spring Boot Anwendung in weniger als 30 Minuten Machine Learning fähig macht (code.talks 2022)

Wie man eine Spring Boot Anwendung in weniger als 30 Minuten Machine Learning fähig macht (code.talks 2022)

Machine Learning ist im Mainstream angekommen. Immer mehr Unternehmen nutzen es, können so schneller Schlussfolgerungen aus Daten ziehen und Produktivitätsvorteile erlangen. Grund genug zu überlegen, wie man diese Fähigkeiten in bestehenden Anwendungen nutzen kann, selbst wenn man kein Data Science Experte ist.

In diesem Vortrag werden wir Machine Learning in eine gängige Spring Boot Anwendung integrieren, so dass diese mit bekannten Frameworks wie Apache MXNet, PyTorch und TensorFlow funktioniert. Als Beispiel für einen typischen Machine Learning Anwendungsfall führen wir im Anschluss Bilderkennung mit der Anwendung durch.

Dennis Kieselhorst

September 16, 2022
Tweet

More Decks by Dennis Kieselhorst

Other Decks in Programming

Transcript

  1. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Wie man eine Spring Boot Anwendung in weniger
    als 30 Minuten Machine Learning fähig macht
    Dennis Kieselhorst @ Amazon Web Services
    Artificial Intelligence

    View full-size slide

  2. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Wie man eine Spring Boot Anwendung
    in weniger als 30 Minuten Machine
    Learning fähig macht
    Dennis Kieselhorst
    Sr. Solutions Architect
    Amazon Web Services

    View full-size slide

  3. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Agenda
    • Intro
    ▪ Reach of Machine Learning
    ▪ Artificial Intelligence, Machine, Deep and Reinforcement learning
    • Why in Java?
    • Ways to integrate it
    • Visual recognition
    • Code samples
    • Use cases
    3

    View full-size slide

  4. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    The reach of Machine Learning is growing
    4
    INCREASED
    SPENDING
    By 2025, global spending on
    artificial intelligence will reach
    $204 billion
    —IDC
    IDC, “Worldwide Spending on Artificial Intelligence,”
    https://bit.ly/3y7hDoP.
    FROM PILOTING TO
    OPERATIONALIZING
    By the end of 2024, 75% of
    enterprises will shift from
    piloting to operationalizing AI
    —Gartner
    Gartner, “Gartner Identifies Top 10,”
    https://gtnr.it/3Bln3uU.
    AI
    TRANSFORMATION
    57% said that AI would
    transform their organization
    in the next three years
    —Deloitte
    Deloitte, “Thriving in the Era of Pervasive AI,”
    https://bit.ly/3CtGDqf.

    View full-size slide

  5. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Healthcare &
    life sciences
    Reduce costs, improve
    patient outcomes, and
    enable the discovery
    and manufacturing
    of new drugs
    Industrial &
    manufacturing
    Implement predictive
    maintenance, improve
    workplace safety,
    automate visual
    inspection, and optimize
    supply chains
    Financial
    services
    Detect fraud, process
    documents faster,
    personalize
    recommendations,
    and perform credit
    decisioning
    Retail
    Personalize product
    recommendations,
    forecast demand,
    and predict fraud
    Media &
    entertainment
    Accelerate
    content creation and
    curation, increase
    customer engagement,
    and improve monetization

    View full-size slide

  6. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Artificial Intelligence, Machine, Deep and Reinforcement
    learning
    8
    Artificial intelligence (AI)
    Any technique that enables computers to mimic human intelligence using logic,
    if-then statements, and machine learning
    Machine learning (ML)
    A subset of AI that uses machines to search for
    patterns in data to build logic models automatically
    Deep learning (DL)
    A subset of ML composed of deeply multi-layered neural networks that
    perform tasks like speech and image recognition
    Reinforcement learning (RL)
    Reinforcement learning trains models by using a reward function of
    a desired outcome

    View full-size slide

  7. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Why in Java?
    • Looking at indexes (like TIOBE and PYPL) Java is trending a bit
    down but still one of the most popular languages
    • Great language with a huge ecosystem with millions of
    developers
    • Existing APIs weren‘t Java-friendly (complex, „C flavor“)
    Image source: https://wiki.openjdk.java.net/display/duke/Gallery

    View full-size slide

  8. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Ways to integrate it
    Local call (self-managed/ using library)
    Add library dependency to Maven/ Gradle
    and use it, some examples:
    • Deep Netts
    • Deeplearning4J
    • Open Intelligent Multimedia Analysis for
    Java (OpenIMAJ)
    • …
    • Deep Java Library (DJL) → will be used
    today
    Remote call (to a managed service)
    Use a remote service, typically via HTTP calls
    Software Development Kits (SDKs) simplify
    integration
    Less compute resources on the client side
    required
    Highly scalable, no servers to manage
    Pay-per-use pricing model
    Example: Amazon Rekognition

    View full-size slide

  9. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Visual recognition
    11

    View full-size slide

  10. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Source: https://github.com/awslabs/djl/blob/master/examples/src/test/resources/dog-cat.jpg

    View full-size slide

  11. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/cat_dog_detected.jpg

    View full-size slide

  12. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/detected-dogs.jpg

    View full-size slide

  13. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/detected-dog_bike_car.png

    View full-size slide

  14. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    © 2022, Amazon Web Services, Inc. or its affiliates.
    How to integrate the Deep
    Java Library (DJL) in a
    Spring-Boot-App
    16

    View full-size slide

  15. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Dependency declaration

    ai.djl.spring
    djl-spring-boot-starter-
    autoconfigure
    0.18


    ai.djl.spring
    djl-spring-boot-starter-mxnet-
    auto
    0.18

    • Spring Boot Starter
    projects for DJL
    ▪ mxnet-auto
    ▪ pytorch-auto
    ▪ tensorflow-auto
    • -auto dependencies
    automatically download
    OS-dependent artifacts
    at runtime

    View full-size slide

  16. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Spring Boot auto configuration
    • Auto configuration for use
    cases like object detection,
    sample application.yml
    configuration →
    • Possible types:
    djl:
    # Define application type
    application-type: OBJECT_DETECTION
    # Define input data type, a model may
    accept multiple input data type
    input-class: ai.djl.modality.cv.Image
    # Define output data type, a model may
    generate different out put
    output-class:
    ai.djl.modality.cv.output.DetectedObjects
    arguments:
    threshold: 0.5 # Display all results with
    probability of 0.5 and above
    QUESTION_ANSWER
    TEXT_CLASSIFICATION
    SENTIMENT_ANALYSIS
    WORD_EMBEDDING
    MACHINE_TRANSLATION
    MULTIPLE_CHOICE
    LINEAR_REGRESSION
    SOFTMAX_REGRESSION
    IMAGE_CLASSIFICATION
    OBJECT_DETECTION
    SEMANTIC_SEGMENTATION
    INSTANCE_SEGMENTATION
    POSE_ESTIMATION
    ACTION_RECOGNITION
    WORD_RECOGNITION
    IMAGE_GENERATION
    IMAGE_ENHANCEMENT

    View full-size slide

  17. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Code for object detection
    Image image =
    ImageFactory.getInstance().fromInputStream(resource.getInputStream());
    try (var predictor = predictorProvider.get()) {
    var results = predictor.predict(image);
    for(var result : results.items()) {
    LOG.info("results for {}: {}", resource.getFilename(), result.toString());
    }
    saveBoundingBoxImage(image, results, resource.getFilename());
    }
    @Resource
    private Supplier> predictorProvider;

    View full-size slide

  18. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Links
    • Deep Java Library: https://djl.ai
    • Simple console application:
    https://github.com/deepjavalibrary/djl-spring-boot-
    starter/tree/master/djl-spring-boot-console-sample
    • More complex application (MVC frontend/ REST API backend):
    https://github.com/deepjavalibrary/djl-spring-boot-starter-demo
    • Blogpost: https://aws.amazon.com/blogs/opensource/adopting-
    machine-learning-in-your-microservices-with-djl-deep-java-
    library-and-spring-boot/

    View full-size slide

  19. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    © 2022, Amazon Web Services, Inc. or its affiliates.
    How to integrate Amazon
    Rekognition in a Spring-
    Boot-App
    21

    View full-size slide

  20. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    The AWS ML stack
    B R O A D E S T A N D M O S T C O M P L E T E S E T O F M A C H I N E L E A R N I N G C A P A B I L I T I E S
    AI
    SERVICES
    Code + DevOps
    Amazon CodeGuru
    Amazon CodeWhisperer
    Amazon DevOps Guru
    Business processes
    Amazon Personalize
    Amazon Forecast
    Amazon Fraud Detector
    Amazon Lookout for Metrics
    Search
    Amazon Kendra
    Industrial
    Amazon Monitron
    Amazon Lookout for Equipment
    Amazon Lookout for Vision
    Healthcare
    Amazon HealthLake
    Amazon Comprehend Medical
    Amazon Transcribe Medical
    SPECIALIZED
    Chatbots
    Amazon Lex
    Text & Documents
    Amazon Translate
    Amazon Comprehend
    Amazon Textract
    Speech
    Amazon Polly
    Amazon Transcribe
    Amazon Transcribe Call Analytics
    Vision
    Amazon Rekognition
    AWS Panorama
    CORE
    ML
    SERVICES Manage
    edge devices
    Label
    data
    Learn
    ML
    Prepare
    data
    Store
    features
    Detect
    bias
    Build with
    notebooks
    Manage
    & monitor
    Train
    models
    Deploy in
    production
    Tune
    parameters
    Explain
    predictions
    CI/CD
    No-code ML
    for business
    analysts
    SAGEMAKER
    STUDIO LAB
    SAGEMAKER
    GROUND
    TRUTH
    SAGEMAKER STUDIO | RSTUDIO
    ML FRAMEWORKS
    & INFRASTRUCTURE
    PyTorch, Apache MXNet,
    Hugging Face, TensorFlow
    Amazon EC2 CPUs GPUs AWS Trainium
    Elastic
    inference
    AWS Inferentia FPGA
    Habana
    Gaudi
    SAGEMAKER
    CANVAS

    View full-size slide

  21. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Amazon Rekognition Image and Video
    CELEBRITY RECOGNITION
    FACE COMPARE & SEARCH
    PATHING
    FACE DETECTION AND ANALYSIS
    LIVE STREAM VIDEO
    TEXT
    CONTENT MODERATION
    CUSTOM LABELS
    OBJECT, SCENE, AND ACTIVITY
    VIDEO SEGMENTS
    PPE DETECTION

    View full-size slide

  22. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Code (with AWS SDK for Java v2)
    API: https://docs.aws.amazon.com/rekognition/latest/dg/API_DetectLabels.html
    SDK code: https://docs.aws.amazon.com/code-samples/latest/catalog/javav2-rekognition-src-
    main-java-com-example-rekognition-DetectLabels.java.html



    software.amazon.awssdk
    bom
    2.17.268
    pom
    import





    software.amazon.awssdk
    rekognition


    RekognitionClient rekognitionClient =
    RekognitionClient.builder()
    .region(Region.EU_CENTRAL_1)
    .build();
    InputStream sourceStream = new FileInputStream(
    new File(sourceImageName));
    SdkBytes sourceBytes = SdkBytes.fromInputStream(sourceStream);
    Image sourceImage = Image.builder()
    .bytes(sourceBytes)
    .build();
    DetectLabelsRequest detectLabelsRequest =
    DetectLabelsRequest.builder()
    .image(sourceImage)
    .maxLabels(10)
    .build();
    DetectLabelsResponse labelsResponse =
    rekognitionClient.detectLabels(detectLabelsRequest);

    View full-size slide

  23. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Blog post: https://aws.amazon.com/blogs/machine-learning/automatically-detecting-personal-
    protective-equipment-on-persons-in-images-using-amazon-rekognition/
    API: https://docs.aws.amazon.com/rekognition/latest/dg/API_DetectProtectiveEquipment.html

    View full-size slide

  24. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Ingest Store Analyze Deliver
    Petabytes of
    images and
    video assets
    Centralized
    storage and
    global registry
    Metadata
    enrichment
    through deep
    learning
    Enhanced value
    and search
    experience
    FEEDBACK
    LOOP
    Augment
    Utilize humans to
    perform validation
    and tasks ML
    cannot yet do,
    include 3rd party
    datasets from
    Amazon Data
    Exchange
    https://aws.amazon.com/machine-learning/ml-use-cases/media-intelligence/
    Another use case: Media enrichment

    View full-size slide

  25. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Summary
    • It’s easy to integrate AI and ML in Java, no need to be a Data Scientist
    • Lots of potential use cases (especially for visual recognition)
    • The Java Specification Request (JSR) 381 standardizes visual recognition as
    part of the Java Community Process (JCP)
    • Engage with the DJL team on Slack for self-managed and an AWS
    Solutions Architect for managed services

    View full-size slide

  26. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Q&A
    29

    View full-size slide

  27. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Recommend to read: Dive into Deep Learning (D2L)
    • 150+ runnable Jupyter Notebooks from model architectures to applications (CV, NLP, etc.)
    • Adopted as a textbook or reference book at UC Berkeley, CMU, MIT, and 70+ universities worldwide
    • Wide theoretical coverage: statistics, optimization, machine learning basics, GPU parallel training, etc.
    Dive Into Deep Learning is an excellent text on deep learning
    and deserves attention from anyone who wants to learn why
    deep learning has ignited the AI revolution – the most
    powerful technology force of our time.
    --- Jensen Huang, CEO of NVIDIA
    https://d2l.djl.ai

    View full-size slide

  28. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Thank you!
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Dennis Kieselhorst
    kieselhorst

    View full-size slide