Wie man eine Spring Boot Anwendung in weniger als 30 Minuten Machine Learning fähig macht (code.talks 2022)

Machine Learning ist im Mainstream angekommen. Immer mehr Unternehmen nutzen es, können so schneller Schlussfolgerungen aus Daten ziehen und Produktivitätsvorteile erlangen. Grund genug zu überlegen, wie man diese Fähigkeiten in bestehenden Anwendungen nutzen kann, selbst wenn man kein Data Science Experte ist.

In diesem Vortrag werden wir Machine Learning in eine gängige Spring Boot Anwendung integrieren, so dass diese mit bekannten Frameworks wie Apache MXNet, PyTorch und TensorFlow funktioniert. Als Beispiel für einen typischen Machine Learning Anwendungsfall führen wir im Anschluss Bilderkennung mit der Anwendung durch.

Transcript

1. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Wie man eine Spring Boot Anwendung in weniger als 30 Minuten Machine Learning fähig macht Dennis Kieselhorst @ Amazon Web Services Artificial Intelligence
2. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. Wie man eine Spring Boot Anwendung in weniger als 30 Minuten Machine Learning fähig macht Dennis Kieselhorst Sr. Solutions Architect Amazon Web Services
3. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Agenda • Intro ▪ Reach of Machine Learning ▪ Artificial Intelligence, Machine, Deep and Reinforcement learning • Why in Java? • Ways to integrate it • Visual recognition • Code samples • Use cases 3
4. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. The reach of Machine Learning is growing 4 INCREASED SPENDING By 2025, global spending on artificial intelligence will reach $204 billion —IDC IDC, “Worldwide Spending on Artificial Intelligence,” https://bit.ly/3y7hDoP. FROM PILOTING TO OPERATIONALIZING By the end of 2024, 75% of enterprises will shift from piloting to operationalizing AI —Gartner Gartner, “Gartner Identifies Top 10,” https://gtnr.it/3Bln3uU. AI TRANSFORMATION 57% said that AI would transform their organization in the next three years —Deloitte Deloitte, “Thriving in the Era of Pervasive AI,” https://bit.ly/3CtGDqf.
5. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Healthcare & life sciences Reduce costs, improve patient outcomes, and enable the discovery and manufacturing of new drugs Industrial & manufacturing Implement predictive maintenance, improve workplace safety, automate visual inspection, and optimize supply chains Financial services Detect fraud, process documents faster, personalize recommendations, and perform credit decisioning Retail Personalize product recommendations, forecast demand, and predict fraud Media & entertainment Accelerate content creation and curation, increase customer engagement, and improve monetization
6. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Artificial Intelligence, Machine, Deep and Reinforcement learning 8 Artificial intelligence (AI) Any technique that enables computers to mimic human intelligence using logic, if-then statements, and machine learning Machine learning (ML) A subset of AI that uses machines to search for patterns in data to build logic models automatically Deep learning (DL) A subset of ML composed of deeply multi-layered neural networks that perform tasks like speech and image recognition Reinforcement learning (RL) Reinforcement learning trains models by using a reward function of a desired outcome
7. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Why in Java? • Looking at indexes (like TIOBE and PYPL) Java is trending a bit down but still one of the most popular languages • Great language with a huge ecosystem with millions of developers • Existing APIs weren‘t Java-friendly (complex, „C flavor“) Image source: https://wiki.openjdk.java.net/display/duke/Gallery
8. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Ways to integrate it Local call (self-managed/ using library) Add library dependency to Maven/ Gradle and use it, some examples: • Deep Netts • Deeplearning4J • Open Intelligent Multimedia Analysis for Java (OpenIMAJ) • … • Deep Java Library (DJL) → will be used today Remote call (to a managed service) Use a remote service, typically via HTTP calls Software Development Kits (SDKs) simplify integration Less compute resources on the client side required Highly scalable, no servers to manage Pay-per-use pricing model Example: Amazon Rekognition
9. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. Visual recognition 11
10. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Source: https://github.com/awslabs/djl/blob/master/examples/src/test/resources/dog-cat.jpg
11. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/cat_dog_detected.jpg
12. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/detected-dogs.jpg
13. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/detected-dog_bike_car.png
14. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. How to integrate the Deep Java Library (DJL) in a Spring-Boot-App 16
15. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Dependency declaration <dependency> <groupId>ai.djl.spring</groupId> <artifactId>djl-spring-boot-starter- autoconfigure</artifactId> <version>0.18</version> </dependency> <dependency> <groupId>ai.djl.spring</groupId> <artifactId>djl-spring-boot-starter-mxnet- auto</artifactId> <version>0.18</version> </dependency> • Spring Boot Starter projects for DJL ▪ mxnet-auto ▪ pytorch-auto ▪ tensorflow-auto • -auto dependencies automatically download OS-dependent artifacts at runtime
16. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Spring Boot auto configuration • Auto configuration for use cases like object detection, sample application.yml configuration → • Possible types: djl: # Define application type application-type: OBJECT_DETECTION # Define input data type, a model may accept multiple input data type input-class: ai.djl.modality.cv.Image # Define output data type, a model may generate different out put output-class: ai.djl.modality.cv.output.DetectedObjects arguments: threshold: 0.5 # Display all results with probability of 0.5 and above QUESTION_ANSWER TEXT_CLASSIFICATION SENTIMENT_ANALYSIS WORD_EMBEDDING MACHINE_TRANSLATION MULTIPLE_CHOICE LINEAR_REGRESSION SOFTMAX_REGRESSION IMAGE_CLASSIFICATION OBJECT_DETECTION SEMANTIC_SEGMENTATION INSTANCE_SEGMENTATION POSE_ESTIMATION ACTION_RECOGNITION WORD_RECOGNITION IMAGE_GENERATION IMAGE_ENHANCEMENT
17. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Code for object detection Image image = ImageFactory.getInstance().fromInputStream(resource.getInputStream()); try (var predictor = predictorProvider.get()) { var results = predictor.predict(image); for(var result : results.items()) { LOG.info("results for {}: {}", resource.getFilename(), result.toString()); } saveBoundingBoxImage(image, results, resource.getFilename()); } @Resource private Supplier<Predictor<Image, DetectedObjects>> predictorProvider;
18. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Links • Deep Java Library: https://djl.ai • Simple console application: https://github.com/deepjavalibrary/djl-spring-boot- starter/tree/master/djl-spring-boot-console-sample • More complex application (MVC frontend/ REST API backend): https://github.com/deepjavalibrary/djl-spring-boot-starter-demo • Blogpost: https://aws.amazon.com/blogs/opensource/adopting- machine-learning-in-your-microservices-with-djl-deep-java- library-and-spring-boot/
19. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. How to integrate Amazon Rekognition in a Spring- Boot-App 21
20. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. The AWS ML stack B R O A D E S T A N D M O S T C O M P L E T E S E T O F M A C H I N E L E A R N I N G C A P A B I L I T I E S AI SERVICES Code + DevOps Amazon CodeGuru Amazon CodeWhisperer Amazon DevOps Guru Business processes Amazon Personalize Amazon Forecast Amazon Fraud Detector Amazon Lookout for Metrics Search Amazon Kendra Industrial Amazon Monitron Amazon Lookout for Equipment Amazon Lookout for Vision Healthcare Amazon HealthLake Amazon Comprehend Medical Amazon Transcribe Medical SPECIALIZED Chatbots Amazon Lex Text & Documents Amazon Translate Amazon Comprehend Amazon Textract Speech Amazon Polly Amazon Transcribe Amazon Transcribe Call Analytics Vision Amazon Rekognition AWS Panorama CORE ML SERVICES Manage edge devices Label data Learn ML Prepare data Store features Detect bias Build with notebooks Manage & monitor Train models Deploy in production Tune parameters Explain predictions CI/CD No-code ML for business analysts SAGEMAKER STUDIO LAB SAGEMAKER GROUND TRUTH SAGEMAKER STUDIO | RSTUDIO ML FRAMEWORKS & INFRASTRUCTURE PyTorch, Apache MXNet, Hugging Face, TensorFlow Amazon EC2 CPUs GPUs AWS Trainium Elastic inference AWS Inferentia FPGA Habana Gaudi SAGEMAKER CANVAS
21. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Amazon Rekognition Image and Video CELEBRITY RECOGNITION FACE COMPARE & SEARCH PATHING FACE DETECTION AND ANALYSIS LIVE STREAM VIDEO TEXT CONTENT MODERATION CUSTOM LABELS OBJECT, SCENE, AND ACTIVITY VIDEO SEGMENTS PPE DETECTION
22. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Code (with AWS SDK for Java v2) API: https://docs.aws.amazon.com/rekognition/latest/dg/API_DetectLabels.html SDK code: https://docs.aws.amazon.com/code-samples/latest/catalog/javav2-rekognition-src- main-java-com-example-rekognition-DetectLabels.java.html <dependencyManagement> <dependencies> <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>bom</artifactId> <version>2.17.268</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>rekognition</artifactId> </dependency> </dependencies> RekognitionClient rekognitionClient = RekognitionClient.builder() .region(Region.EU_CENTRAL_1) .build(); InputStream sourceStream = new FileInputStream( new File(sourceImageName)); SdkBytes sourceBytes = SdkBytes.fromInputStream(sourceStream); Image sourceImage = Image.builder() .bytes(sourceBytes) .build(); DetectLabelsRequest detectLabelsRequest = DetectLabelsRequest.builder() .image(sourceImage) .maxLabels(10) .build(); DetectLabelsResponse labelsResponse = rekognitionClient.detectLabels(detectLabelsRequest);
23. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Blog post: https://aws.amazon.com/blogs/machine-learning/automatically-detecting-personal- protective-equipment-on-persons-in-images-using-amazon-rekognition/ API: https://docs.aws.amazon.com/rekognition/latest/dg/API_DetectProtectiveEquipment.html
24. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Ingest Store Analyze Deliver Petabytes of images and video assets Centralized storage and global registry Metadata enrichment through deep learning Enhanced value and search experience FEEDBACK LOOP Augment Utilize humans to perform validation and tasks ML cannot yet do, include 3rd party datasets from Amazon Data Exchange https://aws.amazon.com/machine-learning/ml-use-cases/media-intelligence/ Another use case: Media enrichment
25. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Summary • It’s easy to integrate AI and ML in Java, no need to be a Data Scientist • Lots of potential use cases (especially for visual recognition) • The Java Specification Request (JSR) 381 standardizes visual recognition as part of the Java Community Process (JCP) • Engage with the DJL team on Slack for self-managed and an AWS Solutions Architect for managed services
26. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Q&A 29
27. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Recommend to read: Dive into Deep Learning (D2L) • 150+ runnable Jupyter Notebooks from model architectures to applications (CV, NLP, etc.) • Adopted as a textbook or reference book at UC Berkeley, CMU, MIT, and 70+ universities worldwide • Wide theoretical coverage: statistics, optimization, machine learning basics, GPU parallel training, etc. Dive Into Deep Learning is an excellent text on deep learning and deserves attention from anyone who wants to learn why deep learning has ignited the AI revolution – the most powerful technology force of our time. --- Jensen Huang, CEO of NVIDIA https://d2l.djl.ai
28. CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT © 2022, Amazon Web Services, Inc. or its affiliates. Thank you! © 2022, Amazon Web Services, Inc. or its affiliates. Dennis Kieselhorst kieselhorst