Gemini + Gemma from Java with LangChain4j

Transcript

1. Gemini + Gemma from Java with LangChain4j Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel codelabs.developers.google.com/codelabs/gemini-java-developers
2. Google Cloud Artificial Intelligence NLP AI Landscape Data Science Machine Learning — Unsupervised, Supervised, Reinforcement Learning Deep Learning — Artificial, Convolution, Recurrent Neural Networks Generative AI — GAN, VAE, Transformers LLMs — Transformers Image Gen — GAN, VAE
3. Google Research & DeepMind innovations
4. Google Cloud LLM sizes
5. Google Cloud With larger models emerge new capabilities https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html
6. Google Cloud Gemini (brand) Gemini App previously Bard Gemini Cloud Assist previously Duet AI Gemini Code Assist previously Duet AI for developers … Google AI Landscape Vertex AI Google AI Studio previously MakerSuite Model Garden Codey Imagen Gemma Llama 3 Claude 3 Falcon Vicuna Stable Diffusion … Search & Conversation Vector Search Notebooks Pipelines AutoML PaLM Gemini (model) … Vision, Video, TTS / STT, NL APIs
8. Available in 3 sizes
9. Natively multimodal Advanced Coding Sophisticated reasoning
11. Gemini 1.5
12. Open model derived from Gemini
13. Google Cloud 13 Gemini Gemma Type Closed, proprietary Open Size Very large Smaller (2B & 7B versions) Modality Text, image, video, speech Only text Languages 39 languages English-only Function calling ✅ ❌ Context window 32K for 1.0 Pro (8K out max) 1M+ for 1.5 Pro 8K tokens (in + out) Performance State-of-the-art in large models, high quality out-of-the-box State-of-the-art in its class, but can require fine-tuning Use cases Enterprise, scale, SLOs, model updates, etc. Experimentation, research, education Can run locally, privacy Pricing & Management Fully managed API Pay per character Manage yourself Pay for your own hardware & hosting Customization Through managed tuning: supervised, RLHF, distillation Programmatically modify underlying weights
14. Google Cloud 14 Python is the default language in AI… But what about Java developers?
15. Google Cloud 15 Option 1⃣ → Gemini SDK • github.com/googleapis/google-cloud-java/tree/main/java-vertexai • github.com/GoogleCloudPlatform/java-docs-samples/ tree/main/vertexai/snippets/src/main/java/vertexai/gemini • cloud.google.com/java/docs/reference/google-cloud-vertexai/
16. Google Cloud 16 Option 2⃣ → LangChain4j
17. Time for some demos! codelabs.developers.google.com/codelabs/gemini-java-developers
18. LLM Vector DB vector embeddings chunks DOCS calculate split store vector + chunk ❶ INGESTION RAG
19. Chatbot app LLM Vector DB vector embeddings chunks DOCS calculate prompt vector embedding split calculate find similar answer context + prompt + chunks store vector + chunk ❶ INGESTION ❷ QUERYING RAG
20. Chatbot app Gemini What’s the weather like in Sofia? It’s sunny in Sofia! External API or service user prompt + getWeather(String) function contract call getWeather(“Sofia”) for me please 󰚦 getWeather(“Sofia”) {“forecast”:”sunny”} function response is {“forecast”:”sunny”} Answer: “It’s sunny in Sofia!” Function calling
21. Gemma via Ollama in TestContainers Why is the sky blue? Chatbot app Ollama container Gemma Rayleigh scattering
22. Thank you! Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel codelabs.developers.google.com/codelabs/gemini-java-developers