Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build AI-infused applications with Quarkus

Build AI-infused applications with Quarkus

Kevin Dubois

June 10, 2024

More Decks by Kevin Dubois

Other Decks in Programming


  1. Kevin Dubois ★ Principal Developer Advocate at Red Hat ★

    Based in Belgium 󰎐 ★ 🗣 Speak English, Dutch, French, Italian ★ Open Source Contributor (Quarkus, Camel, Knative, ..) @[email protected] youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com
  2. Set goals App developer IT operations Data engineer Data scientists

    ML engineer Gather and prepare data Develop model Integrate models in app dev Model monitoring & management Retrain models Business leadership AI as a team initiative
  3. Boring Stuff Deep Learning Artificial Intelligence “Vast volumes of data

    and complex algorithms to train a model using Neural Networks.” Machine Learning Deep Learning
  4. Boring Stuff Generative AI Artificial Intelligence “Focuses on creating new

    and original content ” Machine Learning Deep Learning Generative AI
  5. ▸ Transformers (recognize, predict and generate human language) ▸ Trained

    in a huge amount of data (trillions of tokens) ▸ Relationship between words and phrases ▸ Fine-tuned for specific domains What are Large Language Models (LLMs)?
  6. Prompts ▸ Interacting with the model for asking questions ▸

    Interpreting messages to get important information ▸ Populating Java classes from natural language ▸ Structuring output
  7. @RegisterAiService interface Assistant { String chat(String message); } @Inject private

    final Assistant assistant; quarkus.langchain4j.openai.api-key=sk-... Configure API key Define Ai Service Create the instance
  8. Prompting @SystemMessage("You are a professional poet") @UserMessage(""" Write a poem

    about {topic}. The poem should be {lines} lines long. """) String writeAPoem(String topic, int lines); Add a context on the calls Chat message Placeholder
  9. class TransactionInfo { @Description("full name") public String name; @Description("IBAN value")

    public String iban; @Description("Date of the transaction") public LocalDate transactionDate; @Description("Amount in dollars of the transaction") public double amount; } interface TransactionExtractor { @UserMessage("Extract information about a transaction from {{it}}") TransactionInfo extractTransaction(String text); } Marshalling objects
  10. Chains & Memory ▸ Create conversations ▸ Refer to past

    answers ▸ Manage concurrent interactions
  11. @Inject private final ChatLanguageModel model; ConversationalChain chain = ConversationalChain.builder() .chatLanguageModel(model)

    .build(); String userMessage1 = "Can you give a brief explanation of Kubernetes, 3 lines max?"; String answer1 = chain.execute(userMessage1); String userMessage2 = "Can you give me a YAML example to deploy an application for that?"; String answer2 = chain.execute(userMessage2); Conversation like chat Remember previous interactions
  12. @RegisterAiService(tools = EmailService.class) public interface MyAiService { @SystemMessage("You are a

    professional poet") @UserMessage("Write a poem about {topic}. Then send this poem by email.") String writeAPoem(String topic); @ApplicationScoped public class EmailService { @Inject Mailer mailer; @Tool("send the given content by email") public void sendAnEmail(String content) { mailer.send(Mail.withText("[email protected]", "A poem", content)); } } Describe when to use the tool Register the tool Need to run code
  13. Embedding Documents ▸ Adding terms of use rules into the

    model ▸ Asking questions of legal documents ▸ Natural queries
  14. @Inject RedisEmbeddingStore store; @Inject EmbeddingModel embeddingModel; public void ingest(List<Document> documents)

    { EmbeddingStoreIngestor ingestor = EmbeddingStoreIngestor.builder() .embeddingStore(store) .embeddingModel(embeddingModel) .documentSplitter(recursive(500, 0)) .build(); ingestor.ingest(documents); } Loaded document from CSV, Spreadsheet, text.. Ingested documents stored in Redis Ingest documents
  15. @ApplicationScoped public class DocumentRetrieverExample implements Retriever<TextSegment> { private final EmbeddingStoreRetriever

    retriever; DocumentRetrieverExample(RedisEmbeddingStore store, EmbeddingModel model) { retriever = EmbeddingStoreRetriever.from(store, model, 20); } @Override public List<TextSegment> findRelevant(String s) { return retriever.findRelevant(s); } } CDI injection Augmentation interface
  16. Local Models ▸ Use models on-prem ▸ Evolve a model

    privately ▸ Sentiment Analysis of private data
  17. Run LLMs locally and build AI applications podman-desktop.io Download now

    at: Supported platforms: From getting started with AI, to experimenting with models and prompts, Podman AI Lab enables you to bring AI into your applications without depending on infrastructure beyond your laptop. Podman AI Lab