Hands-on Gemini, the Google DeepMind LLM

In this hands-on workshop, you will get to code using Gemini, the new Large Language Model from Google DeepMind. You will first start by familiarizing yourself with the model's capabilities. Then you will use Gemini in different concrete cases, such as extracting data from unstructured text, document classification, but also searching your own documents, or how to supplement the model by integrating the call to external APIs. The workshop will be conducted using the Java language and the LangChain4j library. Come equipped with a laptop. We will code together in the cloud, no need for any special installation on your machine.

Transcript

1. Hands-on Gemini with Vertex AI and LangChain4j Mete Atamel Valentin Deleplace Guillaume Laforge Developer Advocates
2. Google Cloud Introduction
3. Google Cloud Artificial Intelligence NLP AI Landscape Data Science Machine Learning — Unsupervised, Supervised, Reinforcement Learning Deep Learning — Artificial, Convolution, Recurrent Neural Networks Generative AI — GAN, VAE, Transformers LLMs — Transformers Image Gen — GAN, VAE
4. Google Cloud Google invented the Transformer architecture Responsible AI at the foundation Google invents Transformer kickstarting LLM revolution Google’s groundbreaking large language model, BERT AlphaFold predicts 3D models of protein structures Text-to-Text Transfer Transformer LLM 10B P model open sourced Google LaMDA model trained to converse Google PaLM single model to generalize across domains Google PaLM 2 model is the SOTA LLM PaLM 2
5. Google Cloud What are Large Language Models (LLM)? • Transformer-based neural network architecture that can recognize, predict, and generate human language • Trained on huge corpuses of text in various languages and domains ◦ PaLM 2 learned 340 billion parameters, and trained over 3.6 trillions of tokens • Learns the statistical relationships between words and phrases, as well as the patterns of human language • Can be fine-tuned for specific tasks or domain knowledge
6. Google Cloud LLM sizes
7. Google Cloud With larger models emerge new capabilities https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html
8. Google Cloud What does Google (Cloud) offer?
9. Google Cloud Gemini (brand) previously Bard, Duet AI, … Google (Cloud) Landscape Vertex AI Google AI Studio Model Garden Codey Imagen Llama 2, Claude 2, Falcon, Vicuna, Stable Diffusion … Search & Conversation Vector Search Notebooks Pipelines AutoML PaLM Gemini (model) … Vision, Video, TTS / STT, NL APIs
10. Google Cloud Duet AI Bard Gemini PaLM MakerSuite Google AI Studio
11. Google Cloud What is Gemini (brand)? Gemini is an umbrella brand used in Google products that utilize the Gemini model • Bard (consumer chat interface) ⇒ Gemini app • Duet AI (Google Cloud developer assistance) ⇒ Gemini Cloud Assist • Duet AI for developers ⇒ Gemini Code Assist • …
12. Google Cloud What is Gemini (model)? Gemini is Google Deep Mind’s most capable LLM • It’s a multimodal LLM: text, images, videos as inputs • Comes in 3 sizes: Nano, Pro, and Ultra • Supports function calling • Ranks at the top of the various LLM benchmarks (general knowledge, translation, image understanding, reasoning, math, coding, and more…)
13. Gemini 1.5
14. Open-weights model derived from Gemini
15. Google Cloud SDKs and libraries
16. Google Cloud Google AI Studio vs. Vertex AI You can use Gemini from Google AI Studio or Vertex AI in Google Cloud They’re separate products with separate libraries and language coverage In this workshop, we’ll be using Vertex AI in Google Cloud
17. Google Cloud 17 Python is the default language in AI… But what about Java developers?
18. Google Cloud Option 1: Vertex AI Java API for Gemini https://cloud.google.com/java/docs/reference/google-cloud-vertexai/
19. 19 Option 2: LangChain4j In this workshop, we’ll be using LangChain4j
20. Google Cloud Let’s get started!
21. Google Cloud Codelab: codelabs.developers.google.com/codelabs/gemini-java-developers Repo: github.com/glaforge/gemini-workshop-for-java-developers/tree/main Google Cloud Console: console.cloud.google.com/ bit.ly/gemini-devoxx-2024
22. Google Cloud Onramp TODO valentin screenshots, etc. Not using Onramp because of quota issues on step 10 (RAG) Instead, distribute frictionless accounts (email + password) Not using Onramp because of quota issues on step 10 (RAG) Instead, distribute frictionless accounts (email + password)
23. Thanks! Mete Atamel Valentin Deleplace Guillaume Laforge Developer Advocates