Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning in Java: Die Visual Recognition API (JSR381)

Machine Learning in Java: Die Visual Recognition API (JSR381)

Machine Learning ist im Mainstream angekommen. Daher wurde im Rahmen des Java Community Process (JCP) im Java Specification Request (JSR) 381 auch ein standardisiertes Set an APIs zum Klassifizieren und Erkennen von Objekten in Bildern verabschiedet. In diesem Talk werden Umsetzungsmöglichkeiten anhand von Beispielen aufgezeigt und auch mögliche Alternativen beleuchtet.

Dennis Kieselhorst

October 29, 2020
Tweet

More Decks by Dennis Kieselhorst

Other Decks in Programming

Transcript

  1. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Dennis Kieselhorst, Sr. Solutions Architect
    Machine Learning in Java:
    Die Visual Recognition API
    Java Specification Request (JSR) 381

    View full-size slide

  2. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Table of contents
    • Intro Machine Learning, Artificial Intelligence and Deep Learning
    • Visual recognition
    • Why in Java?
    • Java Specification Request (JSR) 381
    • Reference implementations
    • Code samples
    • Alternatives

    View full-size slide

  3. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    The reach of ML is growing
    By 2021, global spending on AI
    and cognitive technologies will
    exceed $50 billion
    —IDC
    INCREASED SPENDING
    By the end of 2024, 75% of
    enterprises will shift from
    piloting to operationalizing AI
    —Gartner
    FROM PILOTING TO
    OPERATIONALIZING
    AI TRANSFORMATION
    57% said that AI would
    transform their organization in
    the next three years
    —Deloitte

    View full-size slide

  4. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Artificial Intelligence, Machine and Deep learning
    Artificial intelligence
    Subset of AI that uses machines to search for patterns
    in data to build logic models automatically
    Subset of ML composed of deeply multi-layered neural
    networks that perform tasks like speech and image recognition

    View full-size slide

  5. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Visual recognition

    View full-size slide

  6. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Source: https://github.com/awslabs/djl/blob/master/examples/src/test/resources/dog-cat.jpg

    View full-size slide

  7. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/cat_dog_detected.jpg

    View full-size slide

  8. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/detected-dogs.jpg

    View full-size slide

  9. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Source: https://github.com/awslabs/djl/blob/master/examples/docs/img/detected-dog_bike_car.png

    View full-size slide

  10. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Why in Java?
    • Looking at indexes (like TIOBE and PYPL) Java is trending a bit down but still
    one of the most popular languages
    • Great language with a huge ecosystem with millions of developers
    • Existing APIs weren‘t Java-friendly (complex, „C flavor“)
    Source: https://wiki.openjdk.java.net/display/duke/Gallery

    View full-size slide

  11. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Java Specification Request (JSR) 381: Goals
    Goals from the Java Visual Recognition Specification doc (version 1.0):
    • Describe a standard, easy-to-use and flexible set of high-level APIs
    • Offer high-level abstractions for sustainable development of ML products and
    services
    • Have well-defined APIs essential for robust system architecture
    • Offer ease of development and portability
    • Provide thorough information to help create alternative implementations
    • Offer the ability to create custom Classifiers in addition to pre-trained
    Classifiers
    Source: https://jcp.org/en/jsr/detail?id=381

    View full-size slide

  12. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Java Specification Request (JSR) 381: Architecture
    Source: https://jcp.org/en/jsr/detail?id=381

    View full-size slide

  13. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Reference implementations
    • Deep Netts
    • DeepJavaLibrary (DJL)
    • Experimental:
    • Deeplearning4j
    • Open Intelligent Multimedia Analysis for Java (OpenIMAJ)
    • IBM Watson

    View full-size slide

  14. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Code to train a model
    ImageClassifier classifier =
    NeuralNetImageClassifier.builder()
    .inputClass(BufferedImage.class)
    .imageHeight(128).imageWidth(128)
    .labelsFile(new File("labels.txt"))
    .trainingFile(new File("train.txt"))
    .networkArchitecture(new File("arch.json"))
    .exportModel(Paths.get("trained_model.dnet"))
    .maxError(0.03f)
    .maxEpochs(1000)
    .learningRate(0.01f)
    .build();
    BufferedImage image = ImageIO.read(new File("another_cat.png"));
    Map results = classifier.classify(image);

    View full-size slide

  15. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Code to use an existing model
    ImageClassifier classifier =
    NeuralNetImageClassifier.builder()
    .inputClass(BufferedImage.class)
    .imageHeight(128).imageWidth(128)
    .importModel(Paths.get("trained_model.dnet"))
    .build();
    BufferedImage image = ImageIO.read(new File("another_cat.png"));
    Map results = classifier.classify(image);
    More sample code: https://github.com/JavaVisRec/visrec-api/wiki/Getting-Started-Guide#Examples

    View full-size slide

  16. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    What about Spring Boot?
    • Spring Boot Starter project for
    DJL Deep Java Library, not
    specifically for JSR 381
    • Simplifies dependency
    management
    • Allows auto configuration for
    use cases like object detection
    (see application.yml sample)
    djl:
    # Define application type
    application-type: OBJECT_DETECTION
    # Define input data type, a model may accept multiple input data
    type
    input-class: java.awt.image.BufferedImage
    # Define output data type, a model may generate different out put
    output-class: ai.djl.modality.cv.output.DetectedObjects
    arguments:
    threshold: 0.5 # Display all results with probability of 0.5 and above
    Blog post: https://aws.amazon.com/blogs/opensource/adopting-machine-
    learning-in-your-microservices-with-djl-deep-java-library-and-spring-boot/

    View full-size slide

  17. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Alternatives

    View full-size slide

  18. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD DEVELOPMENT
    CONTACT
    CENTERS
    Ground
    Truth
    AWS Marketplace
    for ML
    Neo
    Augmented
    AI
    Built-in
    algorithms
    Notebooks Experiments Processing
    Model training
    and tuning
    Debugger Autopilot
    Model
    hosting
    Model
    Monitor
    Deep Learning
    AMIs & Containers
    GPUs &
    CPUs
    Elastic
    Inference
    Inferentia FPGA
    Amazon
    Rekognition
    Amazon
    Polly
    Amazon
    Transcribe
    +Medical
    Amazon
    Lex
    Amazon
    Personalize
    Amazon
    Forecast
    Amazon
    Comprehend
    +Medical
    AI SERVICES
    ML SERVICES
    ML FRAMEWORKS & INFRASTRUCTURE
    Amazon
    Textract
    Amazon
    Kendra
    Contact Lens
    For Amazon Connect
    SageMaker Studio IDE
    Amazon
    CodeGuru
    Amazon
    SageMaker
    DeepGraphLibrary
    Amazon
    Fraud Detector
    Amazon
    Translate
    The AWS ML Stack
    Broadest and most complete set of machine learning capabilities

    View full-size slide

  19. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    The Amazon ML stack: Broadest & deepest set of
    capabilities
    ML FRAMEWORKS &
    INFRASTRUCTURE
    AI SERVICES
    Vision | Documents | Speech | Language | Chatbots | Forecasting | Recommendations |
    Fraud detection | Enterprise Search | Code Review
    ML SERVICES Data labeling | Pre-built algorithms & notebooks | One-click training and deployment
    Build, train, and deploy machine learning models fast
    Easily add intelligence to applications without machine
    learning skills
    Flexibility & choice, highest-performing infrastructure
    Support for ML frameworks | Compute options purpose-built for ML

    View full-size slide

  20. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD DEVELOPMENT
    CONTACT
    CENTERS
    Ground
    Truth
    AWS Marketplace
    for ML
    Neo
    Augmented
    AI
    Built-in
    algorithms
    Notebooks Experiments Processing
    Model training
    and tuning
    Debugger Autopilot
    Model
    hosting
    Model
    Monitor
    Deep Learning
    AMIs & Containers
    GPUs &
    CPUs
    Elastic
    Inference
    Inferentia FPGA
    Amazon
    Rekognition
    Amazon
    Polly
    Amazon
    Transcribe
    +Medical
    Amazon
    Lex
    Amazon
    Personalize
    Amazon
    Forecast
    Amazon
    Comprehend
    +Medical
    AI SERVICES
    ML SERVICES
    ML FRAMEWORKS & INFRASTRUCTURE
    Amazon
    Textract
    Amazon
    Kendra
    Contact Lens
    For Amazon Connect
    SageMaker Studio IDE
    Amazon
    CodeGuru
    Amazon
    SageMaker
    DeepGraphLibrary
    Amazon
    Fraud Detector
    Amazon
    Translate
    The AWS ML Stack: Amazon Rekognition
    Automate your image and video analysis using machine learning

    View full-size slide

  21. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Labels (object, scenes,
    and activities)
    Face detection and
    analysis
    Text in image
    Unsafe image and
    video detection
    Pathing
    Face search
    Real-time video
    analysis
    Celebrity recognition
    Amazon Rekognition
    Image and video analysis

    View full-size slide

  22. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Amazon Rekognition
    Deep-Learning-Based Image and Video Analysis

    View full-size slide

  23. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Code (with AWS SDK for Java v2)
    API: https://docs.aws.amazon.com/rekognition/latest/dg/API_DetectLabels.html
    SDK code: https://docs.aws.amazon.com/code-samples/latest/catalog/javav2-rekognition-src-
    main-java-com-example-rekognition-DetectLabels.java.html



    software.amazon.awssdk
    bom
    2.15.16
    pom
    import





    software.amazon.awssdk
    rekognition

    RekognitionClient rekognitionClient = RekognitionClient.builder()
    .region(Region.EU_CENTRAL_1)
    .build();
    InputStream sourceStream = new FileInputStream(
    new File(sourceImageName));
    SdkBytes sourceBytes = SdkBytes.fromInputStream(sourceStream);
    Image sourceImage = Image.builder()
    .bytes(sourceBytes)
    .build();
    DetectLabelsRequest detectLabelsRequest = DetectLabelsRequest.builder()
    .image(sourceImage)
    .maxLabels(10)
    .build();
    DetectLabelsResponse labelsResponse =
    rekognitionClient.detectLabels(detectLabelsRequest);

    View full-size slide

  24. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Blog post: https://aws.amazon.com/blogs/machine-learning/automatically-detecting-personal-
    protective-equipment-on-persons-in-images-using-amazon-rekognition/
    API: https://docs.aws.amazon.com/rekognition/latest/dg/API_DetectProtectiveEquipment.html

    View full-size slide

  25. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Summary
    • The current spec is a starting point but needs further refinement and
    evolution as stated in the final notes of the first release:
    • The long term vision of this JSR and specification is to provide a
    standard API to build machine learning applications using Java.
    • One of our major goals is to allow Java application developers to
    incorporate machine learning features without necessarily becoming
    experts in machine learning.
    • Please try it out and provide feedback using the GitHub issue tracker
    https://github.com/JavaVisRec or the mailing list https://groups.io/g/visrec
    Source: https://jcp.org/en/jsr/detail?id=381

    View full-size slide

  26. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Q&A

    View full-size slide

  27. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Recommend to read: Dive into Deep Learning (D2L)
    • 150+ runnable Jupyter Notebooks from model architectures to applications (CV, NLP, etc.)
    • Adopted as a textbook or reference book at UC Berkeley, CMU, MIT, and 70+ universities
    worldwide
    • Wide theoretical coverage: statistics, optimization, machine learning basics, GPU parallel
    training, etc.
    Dive Into Deep Learning is an excellent text on deep learning
    and deserves attention from anyone who wants to learn why
    deep learning has ignited the AI revolution – the most
    powerful technology force of our time.
    --- Jensen Huang, CEO of NVIDIA
    https://d2l.djl.ai

    View full-size slide

  28. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Thank you!
    Dennis Kieselhorst, Sr. Solutions Architect
    [email protected]
    Feedback form: https://amzn.to/35cfKWx

    View full-size slide

  29. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Appendix

    View full-size slide

  30. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Amazon’s machine learning innovation at scale
    4,000 products per
    minute sold on
    Amazon.com
    1.6M packages every
    day
    Billions of Alexa
    interactions
    each week
    First Prime Air
    Delivery on
    Dec. 7, 2016

    View full-size slide