Gemini — Google’s Large Multimodal Model (for the Java developer) Guillaume Laforge Developer Advocate @[email protected]

Google Cloud 2 ♊ https://pixabay.com/illustr ations/constellation-const ellation-map-3301774/

https://en.wikipedia.org/wiki/Gemini_12#/ media/File:S66-63536.jpg

Google Research & DeepMind innovations

No content

Available in 3 sizes

Natively multimodal Advanced Coding Sophisticated reasoning

Google Cloud 8 Gemini Gemini everywhere

Multimodal

Gemini 1.5 model

Large context window in action: “One small step for a man…”

Large context window in action: videos!

Open model derived from Gemini

Google Cloud Gemma at a glance SOTA Excellent benchmark results Base & Instruction Tuned models 2B & 7B parameters Run it on Vertex AI, GKE, your laptop! Gemma is a family of lightweight, state-of-the art open models built from the same research and technology used to create Gemini 14

Google Cloud Gemma performance benchmark 15

Google Cloud 16 Gemini Gemma Type Closed, proprietary Open Size Very large Smaller (2B & 7B versions) Modality Text, image, video, speech Only text Languages 39 languages English-only Function calling ✅ ❌ Context window 32K for 1.0 Pro (8K out max) 1M+ for 1.5 Pro 8K tokens (in + out) Performance State-of-the-art in large models, high quality out-of-the-box State-of-the-art in its class, but can require fine-tuning Use cases Enterprise, scale, SLOs, model updates, etc. Experimentation, research, education Can run locally, privacy Pricing & Management Fully managed API Pay per character/token Manage yourself Pay for your own hardware & hosting Customization Through managed tuning: supervised, RLHF, distillation Programmatically modify underlying weights

Time for some 🥤Java!

Google Cloud 18 Python is all the rage in AI… What’s in it for us, Java developers? https://pixabay.com/photos/snake-repti le-python-boa-anaconda-7386684/

Google Cloud 19 Option 1⃣ → Gemini SDK ● https://github.com/googleapis/google-cloud-java/tree/main/java-vertexai ● https://github.com/GoogleCloudPlatform/java-docs-samples/ tree/main/vertexai/snippets/src/main/java/vertexai/gemini ● https://cloud.google.com/java/docs/reference/google-cloud-vertexai/

Google Cloud 20 Option 2⃣ → LangChain4j

Google Cloud More advanced use cases! What we’ll see ● Simple question / answer (streaming and non-streaming) ● Analyzing images with text prompts (multimodality) ● Maintain chat conversations ● Text classification with few-shot prompting ● Extract structured data from unstructured text ● Chat with your docs with Retrieval Augmented Generation ● Extend with Function Calling to access external APIs ● Gemma via Ollama, and TestContainers

From RAGs to riches

Google Cloud 23 Searching the Apache Groovy documentation Apply the RAG pattern: Retrieval Augmented Generation

LLM Vector DB vector embeddings chunks DOCS calculate split store vector + chunk ❶ INGESTION RAG

Chatbot app LLM Vector DB vector embeddings chunks DOCS calculate prompt vector embedding split calculate find similar answer context + prompt + chunks store vector + chunk ❶ INGESTION ❷ QUERYING RAG

Function calling “Don’t call me, I’ll call you!”

Chatbot app Gemini What’s the weather like in Paris? It’s sunny in Paris! External API or service user prompt + getWeather(String) function contract call getWeather(“Paris”) for me please 󰚦 getWeather(“Paris”) {“forecast”:”sunny”} function response is {“forecast”:”sunny”} Answer: “It’s sunny in Paris!” Function calling

Running Gemma via Ollama & Jlama https://github.com/tjake/Jlama https://ollama.com/ via

Gemma via Ollama in TestContainers Why is the sky blue? Chatbot app Ollama container Gemma Rayleigh scattering

No content

Thanks! Guillaume Laforge Developer Advocate @[email protected]