Wie man eine Spring Boot Anwendung in weniger als 30 Minuten Machine Learning fähig macht (code.talks 2022)

CODE.TALKS 2022 - WIE MAN EINE SPRING BOOT ANWENDUNG IN WENIGER ALS 30 MINUTEN MACHINE LEARNING FÄHIG MACHT
© 2022, Amazon Web Services, Inc. or its affiliates.
Wie man eine Spring Boot Anwendung in weniger
als 30 Minuten Machine Learning fähig macht
Dennis Kieselhorst @ Amazon Web Services
Artificial Intelligence

View Slide

Wie man eine Spring Boot Anwendung
in weniger als 30 Minuten Machine
Learning fähig macht
Dennis Kieselhorst
Sr. Solutions Architect
Amazon Web Services

View Slide

Agenda
• Intro
▪ Reach of Machine Learning
▪ Artificial Intelligence, Machine, Deep and Reinforcement learning
• Why in Java?
• Ways to integrate it
• Visual recognition
• Code samples
• Use cases
3

View Slide

The reach of Machine Learning is growing
4
INCREASED
SPENDING
By 2025, global spending on
artificial intelligence will reach
$204 billion
—IDC
IDC, “Worldwide Spending on Artificial Intelligence,”
https://bit.ly/3y7hDoP.
FROM PILOTING TO
OPERATIONALIZING
By the end of 2024, 75% of
enterprises will shift from
piloting to operationalizing AI
—Gartner
Gartner, “Gartner Identifies Top 10,”
https://gtnr.it/3Bln3uU.
AI
TRANSFORMATION
57% said that AI would
transform their organization
in the next three years
—Deloitte
Deloitte, “Thriving in the Era of Pervasive AI,”
https://bit.ly/3CtGDqf.

View Slide

Healthcare &
life sciences
Reduce costs, improve
patient outcomes, and
enable the discovery
and manufacturing
of new drugs
Industrial &
manufacturing
Implement predictive
maintenance, improve
workplace safety,
automate visual
inspection, and optimize
supply chains
Financial
services
Detect fraud, process
documents faster,
personalize
recommendations,
and perform credit
decisioning
Retail
Personalize product
recommendations,
forecast demand,
and predict fraud
Media &
entertainment
Accelerate
content creation and
curation, increase
customer engagement,
and improve monetization

View Slide

Artificial Intelligence, Machine, Deep and Reinforcement
learning
8
Artificial intelligence (AI)
Any technique that enables computers to mimic human intelligence using logic,
if-then statements, and machine learning
Machine learning (ML)
A subset of AI that uses machines to search for
patterns in data to build logic models automatically
Deep learning (DL)
A subset of ML composed of deeply multi-layered neural networks that
perform tasks like speech and image recognition
Reinforcement learning (RL)
Reinforcement learning trains models by using a reward function of
a desired outcome

View Slide

Why in Java?
• Looking at indexes (like TIOBE and PYPL) Java is trending a bit
down but still one of the most popular languages
• Great language with a huge ecosystem with millions of
developers
• Existing APIs weren‘t Java-friendly (complex, „C flavor“)
Image source: https://wiki.openjdk.java.net/display/duke/Gallery

View Slide

Ways to integrate it
Local call (self-managed/ using library)
Add library dependency to Maven/ Gradle
and use it, some examples:
• Deep Netts
• Deeplearning4J
• Open Intelligent Multimedia Analysis for
Java (OpenIMAJ)
• …
• Deep Java Library (DJL) → will be used
today
Remote call (to a managed service)
Use a remote service, typically via HTTP calls
Software Development Kits (SDKs) simplify
integration
Less compute resources on the client side
required
Highly scalable, no servers to manage
Pay-per-use pricing model
Example: Amazon Rekognition

View Slide

Visual recognition
11

View Slide

Source: https://github.com/awslabs/djl/blob/master/examples/src/test/resources/dog-cat.jpg

View Slide

Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/cat_dog_detected.jpg

View Slide

Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/detected-dogs.jpg

View Slide

Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/detected-dog_bike_car.png

View Slide

How to integrate the Deep
Java Library (DJL) in a
Spring-Boot-App
16

View Slide

Dependency declaration

ai.djl.spring
djl-spring-boot-starter-
autoconfigure
0.18

ai.djl.spring
djl-spring-boot-starter-mxnet-
auto
0.18

• Spring Boot Starter
projects for DJL
▪ mxnet-auto
▪ pytorch-auto
▪ tensorflow-auto
• -auto dependencies
automatically download
OS-dependent artifacts
at runtime

View Slide

Spring Boot auto configuration
• Auto configuration for use
cases like object detection,
sample application.yml
configuration →
• Possible types:
djl:
# Define application type
application-type: OBJECT_DETECTION
# Define input data type, a model may
accept multiple input data type
input-class: ai.djl.modality.cv.Image
# Define output data type, a model may
generate different out put
output-class:
ai.djl.modality.cv.output.DetectedObjects
arguments:
threshold: 0.5 # Display all results with
probability of 0.5 and above
QUESTION_ANSWER
TEXT_CLASSIFICATION
SENTIMENT_ANALYSIS
WORD_EMBEDDING
MACHINE_TRANSLATION
MULTIPLE_CHOICE
LINEAR_REGRESSION
SOFTMAX_REGRESSION
IMAGE_CLASSIFICATION
OBJECT_DETECTION
SEMANTIC_SEGMENTATION
INSTANCE_SEGMENTATION
POSE_ESTIMATION
ACTION_RECOGNITION
WORD_RECOGNITION
IMAGE_GENERATION
IMAGE_ENHANCEMENT

View Slide

Code for object detection
Image image =
ImageFactory.getInstance().fromInputStream(resource.getInputStream());
try (var predictor = predictorProvider.get()) {
var results = predictor.predict(image);
for(var result : results.items()) {
LOG.info("results for {}: {}", resource.getFilename(), result.toString());
}
saveBoundingBoxImage(image, results, resource.getFilename());
}
@Resource
private Supplier> predictorProvider;

View Slide

Links
• Deep Java Library: https://djl.ai
• Simple console application:
https://github.com/deepjavalibrary/djl-spring-boot-
starter/tree/master/djl-spring-boot-console-sample
• More complex application (MVC frontend/ REST API backend):
https://github.com/deepjavalibrary/djl-spring-boot-starter-demo
• Blogpost: https://aws.amazon.com/blogs/opensource/adopting-
machine-learning-in-your-microservices-with-djl-deep-java-
library-and-spring-boot/

View Slide

How to integrate Amazon
Rekognition in a Spring-
Boot-App
21

View Slide

The AWS ML stack
B R O A D E S T A N D M O S T C O M P L E T E S E T O F M A C H I N E L E A R N I N G C A P A B I L I T I E S
AI
SERVICES
Code + DevOps
Amazon CodeGuru
Amazon CodeWhisperer
Amazon DevOps Guru
Business processes
Amazon Personalize
Amazon Forecast
Amazon Fraud Detector
Amazon Lookout for Metrics
Search
Amazon Kendra
Industrial
Amazon Monitron
Amazon Lookout for Equipment
Amazon Lookout for Vision
Healthcare
Amazon HealthLake
Amazon Comprehend Medical
Amazon Transcribe Medical
SPECIALIZED
Chatbots
Amazon Lex
Text & Documents
Amazon Translate
Amazon Comprehend
Amazon Textract
Speech
Amazon Polly
Amazon Transcribe
Amazon Transcribe Call Analytics
Vision
Amazon Rekognition
AWS Panorama
CORE
ML
SERVICES Manage
edge devices
Label
data
Learn
ML
Prepare
data
Store
features
Detect
bias
Build with
notebooks
Manage
& monitor
Train
models
Deploy in
production
Tune
parameters
Explain
predictions
CI/CD
No-code ML
for business
analysts
SAGEMAKER
STUDIO LAB
SAGEMAKER
GROUND
TRUTH
SAGEMAKER STUDIO | RSTUDIO
ML FRAMEWORKS
& INFRASTRUCTURE
PyTorch, Apache MXNet,
Hugging Face, TensorFlow
Amazon EC2 CPUs GPUs AWS Trainium
Elastic
inference
AWS Inferentia FPGA
Habana
Gaudi
SAGEMAKER
CANVAS

View Slide

Amazon Rekognition Image and Video
CELEBRITY RECOGNITION
FACE COMPARE & SEARCH
PATHING
FACE DETECTION AND ANALYSIS
LIVE STREAM VIDEO
TEXT
CONTENT MODERATION
CUSTOM LABELS
OBJECT, SCENE, AND ACTIVITY
VIDEO SEGMENTS
PPE DETECTION

View Slide

Code (with AWS SDK for Java v2)
API: https://docs.aws.amazon.com/rekognition/latest/dg/API_DetectLabels.html
SDK code: https://docs.aws.amazon.com/code-samples/latest/catalog/javav2-rekognition-src-
main-java-com-example-rekognition-DetectLabels.java.html

software.amazon.awssdk
bom
2.17.268
pom
import

software.amazon.awssdk
rekognition

RekognitionClient rekognitionClient =
RekognitionClient.builder()
.region(Region.EU_CENTRAL_1)
.build();
InputStream sourceStream = new FileInputStream(
new File(sourceImageName));
SdkBytes sourceBytes = SdkBytes.fromInputStream(sourceStream);
Image sourceImage = Image.builder()
.bytes(sourceBytes)
.build();
DetectLabelsRequest detectLabelsRequest =
DetectLabelsRequest.builder()
.image(sourceImage)
.maxLabels(10)
.build();
DetectLabelsResponse labelsResponse =
rekognitionClient.detectLabels(detectLabelsRequest);

View Slide

Blog post: https://aws.amazon.com/blogs/machine-learning/automatically-detecting-personal-
protective-equipment-on-persons-in-images-using-amazon-rekognition/
API: https://docs.aws.amazon.com/rekognition/latest/dg/API_DetectProtectiveEquipment.html

View Slide

Ingest Store Analyze Deliver
Petabytes of
images and
video assets
Centralized
storage and
global registry
Metadata
enrichment
through deep
learning
Enhanced value
and search
experience
FEEDBACK
LOOP
Augment
Utilize humans to
perform validation
and tasks ML
cannot yet do,
include 3rd party
datasets from
Amazon Data
Exchange
https://aws.amazon.com/machine-learning/ml-use-cases/media-intelligence/
Another use case: Media enrichment

View Slide

Summary
• It’s easy to integrate AI and ML in Java, no need to be a Data Scientist
• Lots of potential use cases (especially for visual recognition)
• The Java Specification Request (JSR) 381 standardizes visual recognition as
part of the Java Community Process (JCP)
• Engage with the DJL team on Slack for self-managed and an AWS
Solutions Architect for managed services

View Slide

Q&A
29

View Slide

Recommend to read: Dive into Deep Learning (D2L)
• 150+ runnable Jupyter Notebooks from model architectures to applications (CV, NLP, etc.)
• Adopted as a textbook or reference book at UC Berkeley, CMU, MIT, and 70+ universities worldwide
• Wide theoretical coverage: statistics, optimization, machine learning basics, GPU parallel training, etc.
Dive Into Deep Learning is an excellent text on deep learning
and deserves attention from anyone who wants to learn why
deep learning has ignited the AI revolution – the most
powerful technology force of our time.
--- Jensen Huang, CEO of NVIDIA
https://d2l.djl.ai

View Slide

Thank you!
Dennis Kieselhorst
kieselhorst

View Slide

Wie man eine Spring Boot Anwendung in weniger als 30 Minuten Machine Learning fähig macht (code.talks 2022)

Wie man eine Spring Boot Anwendung in weniger als 30 Minuten Machine Learning fähig macht (code.talks 2022)

Dennis Kieselhorst

More Decks by Dennis Kieselhorst

Other Decks in Programming

Featured

Transcript