Generative AI in practice: Concrete LLM use cases in Java, with the PaLM API

Large language models (LLMs) are a powerful new technology that can be used for a variety of tasks, including generating text, translating languages, and writing different kinds of creative content. However, LLMs can be difficult to use, especially for developers who are not proficient in Python, the lingua franca for AI. So what about us Java developers? How can we make use of Generative AI?
This presentation will go through how to use LLMs in Java without the need for Python. We will use the PaLM API, provided by Google Cloud’s Vertex AI services, to perform a variety of tasks, such as searching through documentation, generating kids stories, summarizing content, extracting keywords or entities, and more.

Transcript

1. Generative AI in practice:
   Concrete LLM use cases
   in Java with the PaLM API
   Guillaume Laforge
   Developer Advocate
   @glaforge@uwyn.net
   

   View Slide

2. Google Cloud Proprietary & Confidential
   01
   A few notions
   and vocabulary
   2
   @glaforge@uwyn.net
   

   View Slide

3. Google Cloud
   Artificial Intelligence
   NLP
   3
   Artificial Intelligence, Machine Learning,
   Data Science, Deep Learning…
   Data
   Science
   Machine Learning — unsupervised, supervised, reinforcement learning
   Deep Learning — ANN, CNN, RNN…
   Generative AI — GAN, VAE, Transformers…
   LLMs — Transformers
   Image Gen — GAN, VAE
   

   View Slide

4. Google Cloud 4
   Google invented the Transformer architecture
   Responsible AI at the foundation
   Google invents
   Transformer
   kickstarting LLM
   revolution
   Google’s
   groundbreaking
   large language
   model, BERT
   AlphaFold predicts
   3D models of
   protein structures
   Text-to-Text
   Transfer Transformer
   LLM 10B P model open
   sourced
   Google LaMDA
   model trained to
   converse
   Google PaLM
   single model to
   generalize across
   domains
   Google PaLM 2
   model is the SOTA
   LLM
   PaLM 2
   

   View Slide

5. Google Cloud 5
   So what are Large Language Models?
   ● Transformer-based neural network architecture that can
   recognize, predict, and generate human language
   ● Trained on huge corpuses of text, in various languages and domains
   ○ Ex: PaLM 2 learned 340 billion parameters,
   and trained over 3.6 trillions of tokens
   ● Learn the statistical relationships between words and phrases,
   as well as the patterns of human language
   ● Can be fine-tuned for specific tasks or domain knowledge
   

   View Slide

6. Google Cloud 6
   Language model sizes — March 2023
   https://lifearchitect.ai/models/
   

   View Slide

7. On the Dangers of
   Stochastic Parrots:
   Can Language
   Models Be Too Big?
   

   View Slide

8. Google Cloud 8
   With larger models emerge new capabilities
   https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html
   

   View Slide

9. Google Cloud Proprietary & Confidential 9
   02
   Google’s own
   LLMs and recent
   announcements
   at Cloud Next
   @glaforge@uwyn.net
   

   View Slide

10. Google Cloud
    Bard
    10
    PaLM, Codey, Imagen, MakerSuite, Vertex AI
    Vertex AI
    MakerSuite
    Model
    Garden
    Codey
    Imagen
    …
    Conversation
    Vertex AI
    Search
    Vector
    Search
    Notebooks
    Pipelines
    AutoML
    PaLM 2
    …
    

    View Slide

11. Google Cloud 11
    Cloud Next ‘23 announcements
    ● 100+ models in Model Garden — Llama 2, Code Llama, Falcon, Claude 2
    ● 32k token context for PaLM — 38 lang, higher quality, faster, cheaper for 8k
    ● Codey more performant
    ● Higher image quality for Imagen
    ○ SynthID: digital watermarking functionality
    ● Vertex AI extensions and Vertex AI data connectors
    ● GA for Vertex AI Enterprise Search and Conversation
    

    View Slide

12. Google Cloud Proprietary & Confidential 12
    03
    Time to have a
    look at some
    demos!
    @glaforge@uwyn.net
    

    View Slide

13. Google Cloud 13
    Let’s play with Bard and MakerSuite!
    Bard
    PaLM
    MakerSuite
    

    View Slide

14. Google Cloud 14
    Python is all the rage in AI…
    What’s in it for us,
    Java developers?
    

    View Slide

15. Google Cloud 15
    Generating bedtime stories for kids
    https://bed-time-stories.web.app/
    

    View Slide

16. Google Cloud 16
    Vertex AI Java API
    

    View Slide

17. Google Cloud 17
    LangChain4J
    

    View Slide

18. Google Cloud 18
    Searching the Apache Groovy documentation
    Apply the RAG pattern:
    Retrieval Augmented Generation
    

    View Slide

19. PaLM 2
    chunks
    DOCS
    Vector DB
    embedding
    vectors
    split calculate
    prompt
    embedding
    vector
    calculate
    find
    similar
    answer
    context +
    prompt +
    chunks
    store
    vector
    + chunk
    

    View Slide

20. Google Cloud Proprietary & Confidential 20
    04
    Summary
    @glaforge@uwyn.net
    

    View Slide

21. Google Cloud 21
    Summary
    ● PaLM 2 is a powerful LLM that allows you
    to build all sorts of generative AI ideas and projects
    ● However, using the PaLM API from Java is not necessarily ideal yet
    ● Fortunately, REST marshalling is pretty easy nowadays in Java
    ● Easy to deploy & serve containerized apps on Cloud Run
    ● And LangChain4J is pretty sweet!
    

    View Slide

22. Google Cloud 22
    What to build next?
    ● Chat Q&A with the Apache Groovy documentation
    ○ improve the quality of responses
    ● News curation that collects & summarize my social feeds
    What about you,
    what would you build?
    

    View Slide

23. Thanks!
    Guillaume Laforge
    Developer Advocate
    @glaforge@uwyn.net
    

    View Slide