Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Generative AI in practice: Concrete LLM use cases in Java, with the PaLM and Gemini APIs

Guillaume Laforge
January 25, 2024

Generative AI in practice: Concrete LLM use cases in Java, with the PaLM and Gemini APIs

Large language models (LLMs) are a powerful new technology that can be used for a variety of tasks, including generating text, translating languages, and writing different kinds of creative content. However, LLMs can be difficult to use, especially for developers who are not proficient in Python, the lingua franca for AI. So what about us Java developers? How can we make use of Generative AI?

This presentation will go through how to use LLMs in Java without the need for Python. We will use the PaLM and Gemini API, provided by Google Cloud’s Vertex AI services, to perform a variety of tasks, such as searching through documentation, generating kids stories, summarizing content, extracting keywords or entities, and more. In our journey through demos, we’ll discover LangChain4J, a wonderful LLM orchestrator for Java developers that simplifies the implementation of advanced LLM use cases.

Guillaume Laforge

January 25, 2024

More Decks by Guillaume Laforge


  1. Generative AI in practice: Concrete LLM use cases in Java

    with PaLM & Gemini Guillaume Laforge Developer Advocate @[email protected]
  2. Google Cloud Artificial Intelligence NLP 3 Artificial Intelligence, Machine Learning,

    Data Science, Deep Learning… Data Science Machine Learning — unsupervised, supervised, reinforcement learning Deep Learning — ANN, CNN, RNN… Generative AI — GAN, VAE, Transformers… LLMs — Transformers Image Gen — GAN, VAE
  3. Google Cloud 4 Google invented the Transformer architecture Responsible AI

    at the foundation Google invents Transformer kickstarting LLM revolution Google’s groundbreaking large language model, BERT AlphaFold predicts 3D models of protein structures Text-to-Text Transfer Transformer LLM 10B P model open sourced Google LaMDA model trained to converse Google PaLM single model to generalize across domains Google PaLM 2 model is the SOTA LLM PaLM 2
  4. Google Cloud 6 So what are Large Language Models? •

    Transformer-based neural network architecture that can recognize, predict, and generate human language • Trained on huge corpuses of text, in various languages and domains ◦ Ex: PaLM 2 learned 340 billion parameters, and trained over 3.6 trillions of tokens • Learn the statistical relationships between words and phrases, as well as the patterns of human language • Can be fine-tuned for specific tasks or domain knowledge
  5. Google Cloud Bard 10 PaLM, Codey, Imagen, MakerSuite, Vertex AI

    Vertex AI MakerSuite Model Garden Codey Imagen Llama 2, Claude 2, Falcon, Vicuna, Stable Diffusion … Search & Conversation Vector Search Notebooks Pipelines AutoML PaLM Gemini … Vision, Video, TTS / STT, NL APIs
  6. Google Cloud 11 What is Gemini? • Gemini is Google

    Deep Mind’s most capable AI model • It’s a multimodal large language model: text, images, videos • Comes in 3 sizes: Nano, Pro, and Ultra • Supports function calling • Ranks at the top of the various LLM benchmarks (general knowledge, translation, image understanding, reasoning, math, coding, and more…) • Will be integrated into Bard soon
  7. Google Cloud 19 Python is all the rage in AI…

    What’s in it for us, Java developers?
  8. Google Cloud 24 Searching the Apache Groovy documentation Apply the

    RAG pattern: Retrieval Augmented Generation
  9. LLM Vector DB embedding vectors chunks DOCS calculate prompt embedding

    vector split calculate find similar answer context + prompt + chunks store vector + chunk ❶ INGESTION ❷ QUERYING
  10. Google Cloud 29 Summary • PaLM 2 & Gemini are

    powerful LLM that allow you to build all sorts of generative AI ideas and projects • Using the PaLM API from Java is not necessarily ideal, but with REST or LangChain4j, it’s easy • Fortunately, Gemini has a much better SDK! (and is also supported in LangChain4j) • Easy to deploy & serve containerized apps on Cloud Run • And LangChain4J is pretty sweet!
  11. Google Cloud 31 What to build next? • Chat Q&A

    with the Apache Groovy documentation ◦ improve the quality of responses • News curation that collects & summarize my social feeds What about you, what would you build?