Engage 2024: Experimenting with Integrating Large Language Models into Domino Apps

Session Em03 from Engage User Group Conference 2024

"It's 2024, and hundreds of software enthusiasts gather at a conference, yet not a single word about ChatGPT is whispered. No such catastrophe will happen here! This session will explore the practical integration of Large Language Models directly into Domino applications.

With Domino's superior capabilities in storing non-structured rich text data, and the remarkable efficiency of Large Language Models in harvesting meaning, we're at a unique intersection of technology.
Join this session to discover how we can utilise Domino's integration capabilities, from the Domino RestAPI tooling to LotusScript agents, to incorporate cutting-edge technologies with Domino.
You'll uncover new possibilities and ideas to elevate your applications to the next level of sophistication."

(23/April/2024)

Transcript

1. #engageug Em03 Experimenting with Integrating Large Language Models into Domino Apps Serdar Basegmez Developi / OpenNTF
2. #engageug Serdar Basegmez ๏Developer/Half-blooded Admin ๏New(ish) Londoner - Ex-Istanbulite ๏Freelancer at Developi UK ๏Member Director at OpenNTF Board ๏Notes/Domino since 1999 ๏IBM Champion Alumni (2011-2018) ๏HCL Ambassador (2020-2024) ๏Blog: LotusNotus.com / Twitter: @serdar_basegmez ๏Also tweets/writes/speaks/podcasts on scientific skepticism
3. #engageug Today… ๏Glossary of Terms ๏Showcasing the Potential ๏Integration Scenarios ๏Assessing Our Toolbox ๏What is Next? ๏Conclusion
4. #engageug Impact
5. #engageug Understanding the Impact: Game Changer? ๏A new paradigm in programming? ๏Programming with prompts… ๏New ways to interact ๏Conversation - Chat or audio ๏Accessibility ๏Ability to use “unusable” data ๏Extract value from documents, audio, images ๏Global reach
6. #engageug …or is it Yet Another Big Hype? ๏Safety, security, privacy, compliance ๏Ethical issues ๏Bias and Fairness ๏“Glorified auto-complete”? ๏Lack of creativity and critical thinking ๏Indeterministic behaviour ๏“Temperature” trade-off ๏Scalability and Efficiency ๏Over-reliance
7. #engageug Foundations and Evolution Key concepts and their evolution
8. #engageug Glossary of Terms Source: https://www.techtimes.com/articles/297641/20231017/deepening-our-understanding-of-artificial-intelligence-from-machine-learning-to-generative-ai-large-language-models-and-beyond.htm ๏Artificial Intelligence ๏Machine Learning ๏Deep Learning ๏Natural Language Processing ๏Generative AI
9. #engageug Short History https://blogs.nvidia.com/blog/what-are-foundation-models/
10. #engageug Glossary of Terms ๏Transformers ๏BERT: Bidirectional Encoder Representations from Transformers ๏GPT: Generative pre-trained transformer https://vinija.ai/models/Transformers/
11. #engageug Glossary of Terms - Models ๏Large Language Models ๏Base / Foundation Models ๏Modalities ๏Tasks ๏Fine tuning https://blogs.nvidia.com/blog/what-are-foundation-models/ https://research.aimultiple.com/large-language-models/
12. #engageug Transformer Family Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond, Yang K. et a,. Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond 3 2023 2022 2019 2018 2020 2021 BERT ALBERT DeBERTa ELECTRA T5 BART Flan T5 mT5 T0 GPT-1 GPT-2 XLNet OPT BLOOM GPT-Neo GPT-NeoX GPT-J LLaMA YaLM Galactica RoBERTa Open-Source ERNIE Distill BERT Switch BLOOMZ Word2Vec GloVe FastText Tk OPT-IML Encoder-Only Decoder-Only Encoder-Decoder GLM Chat GLM ST-MoE ELMo ULMFiT Fig. 1. The evolutionary tree of modern LLMs traces the development of language models in recent years and highlights some of the
13. #engageug What is it Good For?
14. #engageug Large Language Model Tasks ๏Text summarisation / Simplification ๏Sentiment analysis ๏Chatbots / Conversational AI ๏Classification / Entity recognition ๏Semantic Search ๏Speech recognition ๏Recommendation ๏Text/Image/Audio/Video Generation ๏Text-to-speech synthesis ๏Spell/Grammar correction ๏Translation ๏Fraud detection ๏Code generation ๏AI Agents
15. #engageug Demo
16. #engageug Word Embeddings Vectors and Vector Search
17. #engageug Word Embeddings ๏Vector representation for words in multi- dimensional space https://www.cs.cmu.edu/~dst/WordEmbeddingDemo/tutorial.html
18. #engageug Word Embeddings 0.39805865 0.55423045 0.28632614 -0.6990865 -0.3808561 -0.1388 0.51647455 0.6454503 0.79717076 0.43035495 0.12107085 0.3470426 -0.21693653 0.1270209 -0.81142104 0.35026655 ... ... -0.13448396 -0.10078076 0.33276576 Large Language Model Hello World
19. #engageug Word Embeddings - with a Vector Store 0.39805865 0.55423045 0.28632614 -0.6990865 -0.3808561 -0.1388 0.51647455 0.6454503 0.79717076 0.43035495 0.12107085 0.3470426 -0.21693653 0.1270209 -0.81142104 0.35026655 ... ... -0.13448396 -0.10078076 0.33276576 Large Language Model Project Text Project Text Project Text Project Text Project Text Vector Database logging library query add/update store closest? Matching Vectors + metadata + Scores
20. #engageug Vector Database https://www.linkedin.com/pulse/navigating-landscape-vector-databases-comprehensive-analysis-bobbili-uuvre
21. #engageug Word Embeddings - Models Large Language Model Local Models (e.g. .onnx files) ✓ Your data won’t leave the server ✓ Most are free with permissive licenses ✓ No vendor lock-in ✓ No cost per operation 👎 Model files are huge. 👎 LLM tasks are resource-intensive 👎 Less capable models 👎 Programmability restrictions
22. #engageug Word Embeddings - Models Large Language Model Cloud Models (e.g. OpenAI, Vertex AI, etc.) ✓ Managed services ✓ Pay-per-use model ✓ Easy to use - RESTful API and native SDKs ✓ Scalable / Available ✓ High performance / High quality ✓ Much better in complicated tasks 👎 Privacy and security concerns 👎 Network latency 👎 High costs for very busy systems 👎 Vendor lock-in
23. #engageug Decide and Test the Model ๏Suggestion: Learn Python!
24. #engageug For Java Developers ๏LangChain4j is very promising Java meets AI How to Build LLM-Powered Applications with LangChain4j
25. #engageug Demo Prompts and Chat
26. #engageug Working with LLMs for Domino Apps LLM Integration is a simple REST API integration
27. #engageug Access LLMs using Java in Domino ๏XPages ๏OSGi Plugins ๏RESTful API (OpenNTF JakartaEE project by Jesse) ๏Java Agents (Notes Client or Server side) ๏DOTS ๏Java Addin .xsp
28. #engageug A New Project: Domino-LangChain4j ๏Experimental phase ๏Import langchain4j library into Domino ๏Utilise ChatModel w/ Local or Cloud LLM ๏Embedding ๏RAG ๏Server and Designer plugins ๏Add some utilities ๏Local Model Support ๏Managed beans ๏Configuration / Logging ๏RAG document loaders for Domino ๏Looking into Java Agent and DOTS support ๏Feedbacks are welcome! OpenNTF Discord
29. #engageug Bonus: Retrieval Augmented Generation https://medium.com/@tiro2000/chat-with-your-rag-97a142663d16
30. #engageug Access LLMs using LotusScript in Domino ๏For LotusScript, there are still options. ๏Use RESTful access using LotusScript ๏OpenNTF project for ChatGPT integration ๏Credits: Ayhan Sahin & Christian Sadeghi ๏Use Java Agent ๏LLM integration might be done with Java agents. LotusScript can call agents
31. #engageug Integration Outside of the Domino Server ๏Implement LLM logic in your favourite platform ๏Volt MX ๏Python ๏Java ๏JavaScript ๏… ๏Access to Domino Data ๏Using Domino REST API ๏Implement your own services with the OpenNTF Jakarta EE Project
32. #engageug Resources ➡ All the demo materials posted on Github: • https://github.com/sbasegmez/Engage24-Demo ➡ Domino-Langchain4j experimental version: • https://github.com/sbasegmez/domino-langchain4j
33. #engageug More Good Stuff: Odds and Ends ๏Further reading… ๏LangChain ๏RAG - Retrieval Augmented Generation ๏Multimodal approaches ๏Prompt Engineering ๏Courses, guides ๏Databricks - Large Language Models: Application through Production ๏NVidia - Large Language Model Ebooks ๏The Practical Guides for Large Language Models