Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Infusing Generative AI in Your Java Apps with L...

Infusing Generative AI in Your Java Apps with Langchain4j @ We Are Developers World Congress

Generative AI has taken the world by storm over the last year, and it seems like every executive leader out there is telling us “regular” Java application developers to “add AI” to our applications. Does that mean we need to drop everything we’ve built and become data scientists instead now?

Fortunately, we can infuse AI models built by actual AI experts into our applications in a fairly straightforward way, thanks to some new projects out there. We promise it’s not as complicated as you might think! Thanks to the ease of use and superb developer experience of Quarkus and the nice AI integration capabilities that the LangChain4j libraries offer, it becomes trivial to start working with AI and make your stakeholders happy :)

In this session, you’ll explore a variety of AI capabilities.

Kevin Dubois

July 16, 2024
Tweet

More Decks by Kevin Dubois

Other Decks in Programming

Transcript

  1. Supersonic, Subatomic intelligent applications. Infusing Generative AI in your Java

    Apps with LangChain4j Kevin Dubois(@kevindubois) Principal Developer Advocate
  2. Kevin Dubois ★ Principal Developer Advocate at Red Hat ★

    Java Champion ★ Based in Belgium 󰎐 ★ 🗣 Speak English, Dutch, French, Italian ★ Open Source Contributor (Quarkus, Camel, Knative, ..) @[email protected] youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com
  3. Set goals App developer IT operations Data engineer Data scientists

    ML engineer Gather and prepare data Develop model Integrate models in app dev Model monitoring & management Retrain models Business leadership AI as a team initiative
  4. Boring Stuff Deep Learning Artificial Intelligence “Vast volumes of data

    and complex algorithms to train a model using Neural Networks.” Machine Learning Deep Learning
  5. Boring Stuff Generative AI Artificial Intelligence “Focuses on creating new

    and original content ” Machine Learning Deep Learning Generative AI
  6. ▸ Transformers (recognize, predict and generate human language) ▸ Trained

    in a huge amount of data (trillions of tokens) ▸ Relationship between words and phrases ▸ Fine-tuned for specific domains What are Large Language Models (LLMs)?
  7. Prompts ▸ Interacting with the model for asking questions ▸

    Interpreting messages to get important information ▸ Populating Java classes from natural language ▸ Structuring output
  8. @RegisterAiService interface Assistant { String chat(String message); } -------------------- @Inject

    private final Assistant assistant; quarkus.langchain4j.openai.api-key=sk-... Configure API key Define Ai Service Create the instance
  9. Prompting @SystemMessage("You are a professional poet") @UserMessage(""" Write a poem

    about {topic}. The poem should be {lines} lines long. """) String writeAPoem(String topic, int lines); Add context to the calls Main message to send Placeholder
  10. class TransactionInfo { @Description("full name") public String name; @Description("IBAN value")

    public String iban; @Description("Date of the transaction") public LocalDate transactionDate; @Description("Amount in dollars of the transaction") public double amount; } interface TransactionExtractor { @UserMessage("Extract information about a transaction from {{it}}") TransactionInfo extractTransaction(String text); } Marshalling objects
  11. Chains & Memory ▸ Create conversations ▸ Refer to past

    answers ▸ Manage concurrent interactions
  12. @RegisterAiService(chatMemoryProviderSupplier = BeanChatMemoryProviderSupplier.class) interface AiServiceWithMemory { String chat(@UserMessage String msg);

    } --------------------------------- @Inject private AiServiceWithMemory ai; String userMessage1 = "Can you give a brief explanation of Kubernetes?"; String answer1 = ai.chat(userMessage1); String userMessage2 = "Can you give me a YAML example to deploy an app for this?"; String answer2 = ai.chat(userMessage2); Possibility to customize memory provider Remember previous interactions
  13. @RegisterAiService(/*chatMemoryProviderSupplier = BeanChatMemoryProviderSupplier.class*/) interface AiServiceWithMemory { String chat(@MemoryId Integer id,

    @UserMessage String msg); } --------------------------------- @Inject private AiServiceWithMemory ai; String answer1 = ai.chat(1,"I'm Frank"); String answer2 = ai.chat(2,"I'm Betty"); String answer3 = ai.chat(1,"Who Am I?"); default memory provider Refers to conversation with id == 1, ie. Frank keep track of multiple parallel conversations
  14. @RegisterAiService(tools = EmailService.class) public interface MyAiService { @SystemMessage("You are a

    professional poet") @UserMessage("Write a poem about {topic}. Then send this poem by email.") String writeAPoem(String topic); @ApplicationScoped public class EmailService { @Inject Mailer mailer; @Tool("send the given content by email") public void sendAnEmail(String content) { mailer.send(Mail.withText("[email protected]", "A poem", content)); } } Describe when to use the tool Register the tool Ties it back to the tool description
  15. Embedding Documents ▸ Adding terms of use rules into the

    model ▸ Asking questions of legal documents ▸ Natural queries
  16. @Inject RedisEmbeddingStore store; EmbeddingModel embeddingModel; public void ingest(List<Document> documents) {

    EmbeddingStoreIngestor ingestor = EmbeddingStoreIngestor.builder() .embeddingStore(store) .embeddingModel(embeddingModel) .documentSplitter(recursive(500, 0)) .build(); ingestor.ingest(documents); } Document from CSV, spreadsheet, text.. Ingested documents stored in Redis Ingest documents
  17. @ApplicationScoped public class DocumentRetriever implements Retriever<TextSegment> { private final EmbeddingStoreRetriever

    retriever; DocumentRetriever(RedisEmbeddingStore store, EmbeddingModel model) { retriever = EmbeddingStoreRetriever.from(store, model, 20); } @Override public List<TextSegment> findRelevant(String s) { return retriever.findRelevant(s); } } CDI injection Augmentation interface
  18. Easier way to retrieve docs: Easy RAG! $ quarkus extension

    add langchain4j-easy-rag quarkus.langchain4j.easy-rag.path=src/main/resources/catalog Path to documents
  19. Local Models ▸ Use models on-prem ▸ Evolve a model

    privately ▸ Sentiment Analysis of private data
  20. @RegisterAiService() public interface AiService { @SystemMessage("You are a Java developer")

    @UserMessage("Create a class about {topic}") @Fallback(fallbackMethod = "fallback") @Retry(maxRetries = 3, delay = 2000) public String chat(String topic); default String fallback(String topic){ return "I'm sorry, I wasn't able create a class about topic: " + topic; } } Handle Failure $ quarkus ext add smallrye-fault-tolerance Add MicroProfile Fault Tolerance dependency Retry up to 3 times
  21. Observability ▸ Collect metrics about your AI-infused app ▸ Trace

    through requests to see how long they took, and where they happened