KEYNOTE: Architecture in times of AI and LLMs

Artificial intelligence (AI) and large language models (LLMs) are taking the world by storm. While ChatGPT and Midjourney are making headlines and inspiring many private experiments, the question arises as to what these innovations will change in the professional environment.

In this keynote, we paint a picture of the future for the discipline of software architecture: How will LLM technologies affect important architectural issues such as security, maintainability, and usability? What new approaches to design and architecture will become possible? How will architectural tasks change, and what skills and knowledge will software architects need to have in the future?

This keynote provides inspiration and takes current developments further without completely losing touch with reality.

Transcript

1. 1 Architektur in Zeiten von KI und LLMs embarc.de Architecture in times of AI and LLMs
2. 2 Architektur in Zeiten von KI und LLMs embarc.de Hype curve...
3. 3 Architektur in Zeiten von KI und LLMs embarc.de The blog post that started it all… ;-)
4. 4 Architektur in Zeiten von KI und LLMs embarc.de What can AI do?
5. 5 Architektur in Zeiten von KI und LLMs embarc.de Machine Learning not new...
6. 6 Architektur in Zeiten von KI und LLMs embarc.de Learning Strategies in Machine Learning
7. 7 Architektur in Zeiten von KI und LLMs embarc.de Self-propelled cars – Tesla vs. LiDAR
8. 8 Architektur in Zeiten von KI und LLMs embarc.de Face recognition – e.g. Apple Face ID
9. 9 Architektur in Zeiten von KI und LLMs embarc.de Spotify NLP Models and Audio Vector Spaces
10. 10 Architektur in Zeiten von KI und LLMs embarc.de Large Language Models LLMs relatively new...
11. 11 Architektur in Zeiten von KI und LLMs embarc.de LLM – Transformer Architecture
12. 12 Architektur in Zeiten von KI und LLMs embarc.de Encoder: Tokens to vector embeddings “Max enjoyed nature and fresh land air to the fullest. ” “The daily working route to the full was always a torment for Julia. ” “Max enjoyed nature and fresh land air to the fullest. ” “The daily working route to the full was always a torment for Julia. ”
13. 13 Architektur in Zeiten von KI und LLMs embarc.de Decoder Bsp.
14. 14 Architektur in Zeiten von KI und LLMs embarc.de 1. Split text into components (tokens) with context (encoder) 2. Organize associations in a model (Vector Space Embeddings) 3. Generate representations if necessary in any form (decoder) LLMs Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and BeyondJ. YANG et al. (Amazon mit Texas A&M)
15. 15 Architektur in Zeiten von KI und LLMs embarc.de What does that mean? for the future...
16. 16 Architektur in Zeiten von KI und LLMs embarc.de Think LLMs? “If I didn’t know exactly what it was, which is this computer program we built recently, I’d think it was a seven-year-old, eight-year-old kid that happens to know physics.” - Blake Lemoine (LaMDA Eng.)
17. 17 Architektur in Zeiten von KI und LLMs embarc.de When is Artificial General Intelligence here? Necessary computing pow Expert opinion
18. 18 Architektur in Zeiten von KI und LLMs embarc.de Jobs!!
19. 19 Architektur in Zeiten von KI und LLMs embarc.de Software architecture ...in the next 5 years... not 2060...
20. 20 Architektur in Zeiten von KI und LLMs embarc.de What is software architecture? Software architecture := Important decisions important := ◼fundamental (as many) ◼in the further course only hard to change ◼decisive for the success of the software system
21. 21 Architektur in Zeiten von KI und LLMs embarc.de Architecture Theme Complexes Breakdown What architecture style? Modules + Dependencies Component formation ... Technology stack Programming language(s) Libraries, Frameworks Middleware Cross-section topics .... Procedure How do we work? Planning, development, testing building, documenting, Deliver, ... Objective environment Where's the software going? End User, RZ, Cloud Distribution, virtualization ...
22. 22 Architektur in Zeiten von KI und LLMs embarc.de What can be discussed? How does architecture work (methodic) change? How do the architectures (solution) change? How do things change (focus)? 1 2 3
23. 23 Architektur in Zeiten von KI und LLMs embarc.de 1. Methodology Architectural work in times of AI & LLMs
24. 24 Architektur in Zeiten von KI und LLMs embarc.de Architecture & Design by ChatGPT?
25. 25 Architektur in Zeiten von KI und LLMs embarc.de Architecture & Design by ChatGPT?
26. 27 Architektur in Zeiten von KI und LLMs embarc.de Architecture & Design by ChatGPT?
27. 28 Architektur in Zeiten von KI und LLMs embarc.de Today already Rubber tea asked?
28. 29 Architektur in Zeiten von KI und LLMs embarc.de Decision-making with AI
29. 30 Architektur in Zeiten von KI und LLMs embarc.de If we keep thinking... Learning technologies by asking and applying questions. Examples demand and use. Check compatibility, find good technology combinations. Prototyping can be implemented faster. without specific context: Document less yourself, instead encode solution and ask questions Write mappings/transformations for integration Architecture reviews with focus on best practices and vulnerabilities. with own context in the model: Architecture design based on own experience and successes.
30. 31 Architektur in Zeiten von KI und LLMs embarc.de Bringing your own context into a model... A) About prompt context in a trained model: ChatGPT 3 allows 4,000 tokens GPT 4 allows 32,000 Challenge: Data protection and protection of own intellectual property. Challenge: Model itself hosting can be expensive. Challenge: Context is not enough for very individual response.
31. 32 Architektur in Zeiten von KI und LLMs embarc.de Bringing your own context into a model... B) Retrain an existing model: Challenge: Cost. Challenge: Many good models are closed source. Challenge: Large models only partially retrain is devil's witness. Llama 2 70B was trained in 1.7million GPU hours. On average $2.8 million at GCP... Own HW with 1,000 GPUs costs 15million
32. 33 Architektur in Zeiten von KI und LLMs embarc.de Architecture Theme Complexes Breakdown What architecture style? Modules + Dependencies Component formation ... Technology stack Programming language(s) Libraries, Frameworks Middleware Cross-section topics .... Procedure How do we work? Planning, development, testing building, documenting, Deliver, ... Objective environment Where's the software going? End User, RZ, Cloud Distribution, virtualization ...
33. 34 Architektur in Zeiten von KI und LLMs embarc.de What can be discussed? How does architecture work (methodic) change? How do the architectures (solution) change? How do things change (focus)? 1 2 3
34. 35 Architektur in Zeiten von KI und LLMs embarc.de Two. Solution Architectures in times of AI & LLMs
35. 36 Architektur in Zeiten von KI und LLMs embarc.de Bringing your own context into a model...
36. 37 Architektur in Zeiten von KI und LLMs embarc.de ChatGPT makes architecture... Collision Detection System Warehouse management - Logistics Specialist System - Insurance Webshop system for fashion Microservices Layer architecture (+DDD) SOCIAL Event-Driven Architecture
37. 38 Architektur in Zeiten von KI und LLMs embarc.de If LLMs support us... Are architectures rather “mainstream”? An innovation for not innovative companies!? Very specific challenges are not solved out-of-the-box.
38. 39 Architektur in Zeiten von KI und LLMs embarc.de If we keep thinking... LLMs help with simpler and recurring problems. Little original solutions will soon be possible very quickly. without specific context: Homogenization and uniformity of your own solution and landscape becomes less important – the ideas can be extracted, the solution can be explained, mappings generated. New set of competencies for orchestration, data mgmt, plugins.. with own context in the model: Industry-specific LLMs will help to better implement best practices. (BloombergGPT) Rather no self-created LLMs as part of solutions
39. 40 Architektur in Zeiten von KI und LLMs embarc.de What can be discussed? How does architecture work (methodic) change? How do the architectures (solution) change? How do things change (focus)? 1 2 3
40. 41 Architektur in Zeiten von KI und LLMs embarc.de 3. Focus Architectural considerations in times of AI & LLMs
41. 42 Architektur in Zeiten von KI und LLMs embarc.de Ist das System verfügbar, tolerant gegenüber Fehlern, nach Abstürzen schnell wieder hergestellt? Wiederherstellbarkeit Zuverlässigkeit Verfügbarkeit Fehlertoleranz Ist die Software leicht zu ändern, erweitern, testen, verstehen? Lassen sich Teile wiederverwenden? Änderbarkeit Wartbarkeit Analysierbarkeit Erweiterbarkeit Ist das System sicher vor Angriffen? Sind Daten und Funktion vor unberechtigtem Zugriff geschützt? Authentizität Sicherheit Vertraulichkeit Datenintegrität Sind Personen, Tiere, Sachen oder Umwelt vor Schäden durch die Software geschützt? Verifizierbarkeit Safety Betriebssicherheit Regulatorische Compliance Ist die Software leicht auf andere Zielumgebungen übertragbar? Installierbarkeit Portierbarkeit Übertragbarkeit Austauschbarkeit Ist die Software konform zu Standards, arbeitet sie gut mit anderen zusammen? Interoperabilität Kompatibilität Koexistenz Versionierungsfähigkeit Ist die Software intuitiv zu bedienen, wiedererkennbar, leicht zu erlernen, attraktiv? Erlernbarkeit Benutzbarkeit Barrierefreiheit Bedienbarkeit Antwortet die Software schnell, hat sie einen hohen Durchsatz, einen geringen Ressourcenverbrauch? Verbrauchsverhalten Performanz Kapazität Zeitverhalten Sind die berechneten Ergebnisse genau genug / exakt, ist die Funktionalität angemessen? Korrektheit Funktionale Eignung Vollständigkeit Angemessenheit Quality characteristics as a target
42. 43 Architektur in Zeiten von KI und LLMs embarc.de usability Can't be taken over by LLMs, UX Research remains safe. Usability may also become the differentiation factor because standard solutions are created behind the UI very efficiently ▪ One could imagine recommendations based on usability patterns ▪ if necessary Help in evaluating user behavior (here classic ML, collecting data would then be more important than before) What can LLMs do?
43. 44 Architektur in Zeiten von KI und LLMs embarc.de Sind die berechneten Ergebnisse genau genug / exakt, ist die Funktionalität angemessen? Korrektheit Funktionale Eignung Vollständigkeit Angemessenheit Functional suitability - correctness LLMs are not always correct, not deterministic and haluzinate too. As a black box no "why?" possible. The less mainstream, the more inappropriate. Good scepticism, reviews, tests and validation / QS processes become more important. ▪ Model shading in which models check the output of other models ▪ Models search the sources of knowledge / reveal (not standard) What can LLMs do?
44. 45 Architektur in Zeiten von KI und LLMs embarc.de Ist das System sicher vor Angriffen? Sind Daten und Funktion vor unberechtigtem Zugriff geschützt? Authentizität Sicherheit Vertraulichkeit Datenintegrität Security LLMs means an additional resource (prompts, plugins, etc.) Data security is problematic because LLMs are not (only) running and training in their own environment, plus information about system setup can be tapped to create attack vectors. ▪ Vulnerability Checks to drive frequent attack vectors ▪ Support for reviews and inspections ▪ Detecting attacks in production early What can LLMs do?
45. 46 Architektur in Zeiten von KI und LLMs embarc.de Copyright and copyright... Identification Presentation Author:in Copyright: Protected is the representation, not the idea. Copyright: The representation is not transferable. LLM extracts ideas LLM creates a representation LLM output is Non-protected Ideas are not protected Prompts
46. 47 Architektur in Zeiten von KI und LLMs embarc.de Licenses https://sapling.ai/llm/bloom-vs-llama
47. 48 Architektur in Zeiten von KI und LLMs embarc.de Ist die Software leicht zu ändern, erweitern, testen, verstehen? Lassen sich Teile wiederverwenden? Änderbarkeit Wartbarkeit Analysierbarkeit Erweiterbarkeit Wartability Code change by LLMs is more invasive. Because of law of leaky abstraction, it is risky to use LLMs constantly for solution change. Short-term/medium-term rather generate 1x, then review, improve and later LLM-assisted maintenance. ▪ LLMs can make the understanding of solutions easier, assist with changes. ▪ Architecture decisions can be reviewed and questioned by an LLM (including contradictions, vulnerabilities, ...) What can LLMs do?
48. 49 Architektur in Zeiten von KI und LLMs embarc.de And this is just the beginning... ▪ How do we ensure that models remain “good”? ▪ How do we integrate ML and AI solutions into our landscape? ▪ What is it about ethical questions? ▪ What about resource use and performance? Model optimizations and reductions can also be done by (other) ML tools. ▪ How do we deal with sustainability? ▪ How do we use the support for compatibility and portability issues? ▪ ... There are plenty of architectural questions left. Architecture work changes, but remains important.
49. 50 Architektur in Zeiten von KI und LLMs embarc.de What can be discussed? How does architecture work (methodic) change? How do the architectures (solution) change? How do things change (focus)? 1 2 3
50. 51 Architektur in Zeiten von KI und LLMs embarc.de Thank you. stefan.toth@embarc.de xing.to/sto @st_toth Any questions are welcome!