Machine Learning in Java: Die Visual Recognition API (JSR381)

Machine Learning ist im Mainstream angekommen. Daher wurde im Rahmen des Java Community Process (JCP) im Java Specification Request (JSR) 381 auch ein standardisiertes Set an APIs zum Klassifizieren und Erkennen von Objekten in Bildern verabschiedet. In diesem Talk werden Umsetzungsmöglichkeiten anhand von Beispielen aufgezeigt und auch mögliche Alternativen beleuchtet.

Transcript

1. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
   Dennis Kieselhorst, Sr. Solutions Architect
   Machine Learning in Java:
   Die Visual Recognition API
   Java Specification Request (JSR) 381
   

   View Slide

2. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
   Table of contents
   • Intro Machine Learning, Artificial Intelligence and Deep Learning
   • Visual recognition
   • Why in Java?
   • Java Specification Request (JSR) 381
   • Reference implementations
   • Code samples
   • Alternatives
   

   View Slide

3. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
   The reach of ML is growing
   By 2021, global spending on AI
   and cognitive technologies will
   exceed $50 billion
   —IDC
   INCREASED SPENDING
   By the end of 2024, 75% of
   enterprises will shift from
   piloting to operationalizing AI
   —Gartner
   FROM PILOTING TO
   OPERATIONALIZING
   AI TRANSFORMATION
   57% said that AI would
   transform their organization in
   the next three years
   —Deloitte
   

   View Slide

4. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
   Artificial Intelligence, Machine and Deep learning
   Artificial intelligence
   Subset of AI that uses machines to search for patterns
   in data to build logic models automatically
   Subset of ML composed of deeply multi-layered neural
   networks that perform tasks like speech and image recognition
   

   View Slide

5. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
   Visual recognition
   

   View Slide

6. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Source: https://github.com/awslabs/djl/blob/master/examples/src/test/resources/dog-cat.jpg
   

   View Slide

7. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/cat_dog_detected.jpg
   

   View Slide

8. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/detected-dogs.jpg
   

   View Slide

9. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/detected-dog_bike_car.png
   

   View Slide

10. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Why in Java?
    • Looking at indexes (like TIOBE and PYPL) Java is trending a bit down but still
    one of the most popular languages
    • Great language with a huge ecosystem with millions of developers
    • Existing APIs weren‘t Java-friendly (complex, „C flavor“)
    Source: https://wiki.openjdk.java.net/display/duke/Gallery
    

    View Slide

11. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Java Specification Request (JSR) 381: Goals
    Goals from the Java Visual Recognition Specification doc (version 1.0):
    • Describe a standard, easy-to-use and flexible set of high-level APIs
    • Offer high-level abstractions for sustainable development of ML products and
    services
    • Have well-defined APIs essential for robust system architecture
    • Offer ease of development and portability
    • Provide thorough information to help create alternative implementations
    • Offer the ability to create custom Classifiers in addition to pre-trained
    Classifiers
    Source: https://jcp.org/en/jsr/detail?id=381
    

    View Slide

12. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Java Specification Request (JSR) 381: Architecture
    Source: https://jcp.org/en/jsr/detail?id=381
    

    View Slide

13. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Reference implementations
    • Deep Netts
    • DeepJavaLibrary (DJL)
    • Experimental:
    • Deeplearning4j
    • Open Intelligent Multimedia Analysis for Java (OpenIMAJ)
    • IBM Watson
    

    View Slide

14. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Code to train a model
    ImageClassifier classifier =
    NeuralNetImageClassifier.builder()
    .inputClass(BufferedImage.class)
    .imageHeight(128).imageWidth(128)
    .labelsFile(new File("labels.txt"))
    .trainingFile(new File("train.txt"))
    .networkArchitecture(new File("arch.json"))
    .exportModel(Paths.get("trained_model.dnet"))
    .maxError(0.03f)
    .maxEpochs(1000)
    .learningRate(0.01f)
    .build();
    BufferedImage image = ImageIO.read(new File("another_cat.png"));
    Map results = classifier.classify(image);
    

    View Slide

15. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Code to use an existing model
    ImageClassifier classifier =
    NeuralNetImageClassifier.builder()
    .inputClass(BufferedImage.class)
    .imageHeight(128).imageWidth(128)
    .importModel(Paths.get("trained_model.dnet"))
    .build();
    BufferedImage image = ImageIO.read(new File("another_cat.png"));
    Map results = classifier.classify(image);
    More sample code: https://github.com/JavaVisRec/visrec-api/wiki/Getting-Started-Guide#Examples
    

    View Slide

16. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    What about Spring Boot?
    • Spring Boot Starter project for
    DJL Deep Java Library, not
    specifically for JSR 381
    • Simplifies dependency
    management
    • Allows auto configuration for
    use cases like object detection
    (see application.yml sample)
    djl:
    # Define application type
    application-type: OBJECT_DETECTION
    # Define input data type, a model may accept multiple input data
    type
    input-class: java.awt.image.BufferedImage
    # Define output data type, a model may generate different out put
    output-class: ai.djl.modality.cv.output.DetectedObjects
    arguments:
    threshold: 0.5 # Display all results with probability of 0.5 and above
    Blog post: https://aws.amazon.com/blogs/opensource/adopting-machine-
    learning-in-your-microservices-with-djl-deep-java-library-and-spring-boot/
    

    View Slide

17. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Alternatives
    

    View Slide

18. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD DEVELOPMENT
    CONTACT
    CENTERS
    Ground
    Truth
    AWS Marketplace
    for ML
    Neo
    Augmented
    AI
    Built-in
    algorithms
    Notebooks Experiments Processing
    Model training
    and tuning
    Debugger Autopilot
    Model
    hosting
    Model
    Monitor
    Deep Learning
    AMIs & Containers
    GPUs &
    CPUs
    Elastic
    Inference
    Inferentia FPGA
    Amazon
    Rekognition
    Amazon
    Polly
    Amazon
    Transcribe
    +Medical
    Amazon
    Lex
    Amazon
    Personalize
    Amazon
    Forecast
    Amazon
    Comprehend
    +Medical
    AI SERVICES
    ML SERVICES
    ML FRAMEWORKS & INFRASTRUCTURE
    Amazon
    Textract
    Amazon
    Kendra
    Contact Lens
    For Amazon Connect
    SageMaker Studio IDE
    Amazon
    CodeGuru
    Amazon
    SageMaker
    DeepGraphLibrary
    Amazon
    Fraud Detector
    Amazon
    Translate
    The AWS ML Stack
    Broadest and most complete set of machine learning capabilities
    

    View Slide

19. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    The Amazon ML stack: Broadest & deepest set of
    capabilities
    ML FRAMEWORKS &
    INFRASTRUCTURE
    AI SERVICES
    Vision | Documents | Speech | Language | Chatbots | Forecasting | Recommendations |
    Fraud detection | Enterprise Search | Code Review
    ML SERVICES Data labeling | Pre-built algorithms & notebooks | One-click training and deployment
    Build, train, and deploy machine learning models fast
    Easily add intelligence to applications without machine
    learning skills
    Flexibility & choice, highest-performing infrastructure
    Support for ML frameworks | Compute options purpose-built for ML
    

    View Slide

20. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD DEVELOPMENT
    CONTACT
    CENTERS
    Ground
    Truth
    AWS Marketplace
    for ML
    Neo
    Augmented
    AI
    Built-in
    algorithms
    Notebooks Experiments Processing
    Model training
    and tuning
    Debugger Autopilot
    Model
    hosting
    Model
    Monitor
    Deep Learning
    AMIs & Containers
    GPUs &
    CPUs
    Elastic
    Inference
    Inferentia FPGA
    Amazon
    Rekognition
    Amazon
    Polly
    Amazon
    Transcribe
    +Medical
    Amazon
    Lex
    Amazon
    Personalize
    Amazon
    Forecast
    Amazon
    Comprehend
    +Medical
    AI SERVICES
    ML SERVICES
    ML FRAMEWORKS & INFRASTRUCTURE
    Amazon
    Textract
    Amazon
    Kendra
    Contact Lens
    For Amazon Connect
    SageMaker Studio IDE
    Amazon
    CodeGuru
    Amazon
    SageMaker
    DeepGraphLibrary
    Amazon
    Fraud Detector
    Amazon
    Translate
    The AWS ML Stack: Amazon Rekognition
    Automate your image and video analysis using machine learning
    

    View Slide

21. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Labels (object, scenes,
    and activities)
    Face detection and
    analysis
    Text in image
    Unsafe image and
    video detection
    Pathing
    Face search
    Real-time video
    analysis
    Celebrity recognition
    Amazon Rekognition
    Image and video analysis
    

    View Slide

22. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Amazon Rekognition
    Deep-Learning-Based Image and Video Analysis
    

    View Slide

23. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Code (with AWS SDK for Java v2)
    API: https://docs.aws.amazon.com/rekognition/latest/dg/API_DetectLabels.html
    SDK code: https://docs.aws.amazon.com/code-samples/latest/catalog/javav2-rekognition-src-
    main-java-com-example-rekognition-DetectLabels.java.html
    
    
    
    software.amazon.awssdk
    bom
    2.15.16
    pom
    import
    
    
    
    
    
    software.amazon.awssdk
    rekognition
    
    RekognitionClient rekognitionClient = RekognitionClient.builder()
    .region(Region.EU_CENTRAL_1)
    .build();
    InputStream sourceStream = new FileInputStream(
    new File(sourceImageName));
    SdkBytes sourceBytes = SdkBytes.fromInputStream(sourceStream);
    Image sourceImage = Image.builder()
    .bytes(sourceBytes)
    .build();
    DetectLabelsRequest detectLabelsRequest = DetectLabelsRequest.builder()
    .image(sourceImage)
    .maxLabels(10)
    .build();
    DetectLabelsResponse labelsResponse =
    rekognitionClient.detectLabels(detectLabelsRequest);
    

    View Slide

24. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Blog post: https://aws.amazon.com/blogs/machine-learning/automatically-detecting-personal-
    protective-equipment-on-persons-in-images-using-amazon-rekognition/
    API: https://docs.aws.amazon.com/rekognition/latest/dg/API_DetectProtectiveEquipment.html
    

    View Slide

25. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Summary
    • The current spec is a starting point but needs further refinement and
    evolution as stated in the final notes of the first release:
    • The long term vision of this JSR and specification is to provide a
    standard API to build machine learning applications using Java.
    • One of our major goals is to allow Java application developers to
    incorporate machine learning features without necessarily becoming
    experts in machine learning.
    • Please try it out and provide feedback using the GitHub issue tracker
    https://github.com/JavaVisRec or the mailing list https://groups.io/g/visrec
    Source: https://jcp.org/en/jsr/detail?id=381
    

    View Slide

26. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Q&A
    

    View Slide

27. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Recommend to read: Dive into Deep Learning (D2L)
    • 150+ runnable Jupyter Notebooks from model architectures to applications (CV, NLP, etc.)
    • Adopted as a textbook or reference book at UC Berkeley, CMU, MIT, and 70+ universities
    worldwide
    • Wide theoretical coverage: statistics, optimization, machine learning basics, GPU parallel
    training, etc.
    Dive Into Deep Learning is an excellent text on deep learning
    and deserves attention from anyone who wants to learn why
    deep learning has ignited the AI revolution – the most
    powerful technology force of our time.
    --- Jensen Huang, CEO of NVIDIA
    https://d2l.djl.ai
    

    View Slide

28. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Thank you!
    Dennis Kieselhorst, Sr. Solutions Architect
    dkieselh@amazon.de
    Feedback form: https://amzn.to/35cfKWx
    

    View Slide

29. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Appendix
    

    View Slide

30. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Amazon’s machine learning innovation at scale
    4,000 products per
    minute sold on
    Amazon.com
    1.6M packages every
    day
    Billions of Alexa
    interactions
    each week
    First Prime Air
    Delivery on
    Dec. 7, 2016
    

    View Slide