Generative AI in practice: Concrete LLM use cases in Java with the PaLM API Guillaume Laforge Developer Advocate @[email protected]

Google Cloud Proprietary & Confidential 01 A few notions and vocabulary 2 @[email protected]

Google Cloud Artificial Intelligence NLP 3 Artificial Intelligence, Machine Learning, Data Science, Deep Learning… Data Science Machine Learning — unsupervised, supervised, reinforcement learning Deep Learning — ANN, CNN, RNN… Generative AI — GAN, VAE, Transformers… LLMs — Transformers Image Gen — GAN, VAE

Google Cloud 4 Google invented the Transformer architecture Responsible AI at the foundation Google invents Transformer kickstarting LLM revolution Google’s groundbreaking large language model, BERT AlphaFold predicts 3D models of protein structures Text-to-Text Transfer Transformer LLM 10B P model open sourced Google LaMDA model trained to converse Google PaLM single model to generalize across domains Google PaLM 2 model is the SOTA LLM PaLM 2

Google Cloud 5 So what are Large Language Models? ● Transformer-based neural network architecture that can recognize, predict, and generate human language ● Trained on huge corpuses of text, in various languages and domains ○ Ex: PaLM 2 learned 340 billion parameters, and trained over 3.6 trillions of tokens ● Learn the statistical relationships between words and phrases, as well as the patterns of human language ● Can be fine-tuned for specific tasks or domain knowledge

Google Cloud 6 Language model sizes — March 2023 https://lifearchitect.ai/models/

On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?

Google Cloud 8 With larger models emerge new capabilities https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html

Google Cloud Proprietary & Confidential 9 02 Google’s own LLMs and recent announcements at Cloud Next @[email protected]

Google Cloud Bard 10 PaLM, Codey, Imagen, MakerSuite, Vertex AI Vertex AI MakerSuite Model Garden Codey Imagen … Conversation Vertex AI Search Vector Search Notebooks Pipelines AutoML PaLM 2 …

Google Cloud 11 Cloud Next ‘23 announcements ● 100+ models in Model Garden — Llama 2, Code Llama, Falcon, Claude 2 ● 32k token context for PaLM — 38 lang, higher quality, faster, cheaper for 8k ● Codey more performant ● Higher image quality for Imagen ○ SynthID: digital watermarking functionality ● Vertex AI extensions and Vertex AI data connectors ● GA for Vertex AI Enterprise Search and Conversation

Google Cloud Proprietary & Confidential 12 03 Time to have a look at some demos! @[email protected]

Google Cloud 13 Let’s play with Bard and MakerSuite! Bard PaLM MakerSuite

Google Cloud 14 Python is all the rage in AI… What’s in it for us, Java developers?

Google Cloud 15 Generating bedtime stories for kids https://bed-time-stories.web.app/

Google Cloud 16 Vertex AI Java API

Google Cloud 17 LangChain4J

Google Cloud 18 Searching the Apache Groovy documentation Apply the RAG pattern: Retrieval Augmented Generation

PaLM 2 chunks DOCS Vector DB embedding vectors split calculate prompt embedding vector calculate find similar answer context + prompt + chunks store vector + chunk

Google Cloud Proprietary & Confidential 20 04 Summary @[email protected]

Google Cloud 21 Summary ● PaLM 2 is a powerful LLM that allows you to build all sorts of generative AI ideas and projects ● However, using the PaLM API from Java is not necessarily ideal yet ● Fortunately, REST marshalling is pretty easy nowadays in Java ● Easy to deploy & serve containerized apps on Cloud Run ● And LangChain4J is pretty sweet!

Google Cloud 22 What to build next? ● Chat Q&A with the Apache Groovy documentation ○ improve the quality of responses ● News curation that collects & summarize my social feeds What about you, what would you build?

Thanks! Guillaume Laforge Developer Advocate @[email protected]