AI: Will Architects Soon Become Obsolete?

Will all software architects soon be out of a job because AI systems have taken over? In this talk, we explore whether this concern is justified. We discuss where AI tools – especially Large Language Models (LLMs) – can already be effectively used in software architecture today, and where their limitations lie. Along the way, we’ll also take a look at how LLMs actually work – and what that means for their practical application.

Transcript

1. AI: Will Architects Soon Become Obsolete? Eberhard Wolff Head of Architecture https://swaglab.rocks/ https://ewolff.com/
2. What is the Most Important Skill in IT?
3. What is the Most Important Skill in IT? • We asked people at a technical conference. • Just an experiment • We did not really think about possible answers. • By far the most common answer: communication, soft skill …
4. https://software-architektur.tv/2024/08/16/episode228.html
5. What is the Main Challenge in Software Architecture? • Survey through social media What is the biggest challenge in #SoftwareArchitecture in your opinion?
6. Main Challenge in Software Architecture? • Communication • Organization and architecture • Evolution of systems
7. https://software-architektur.tv/2025/06/27/folge269.html
8. What does that mean? • Most important skill in IT: Communication • Main challenge in software architecture: Soft skills • Why should LLMs / AI improve impact architecture work massively? • LLMs might still be useful!
9. Software architecture is a people business! And you know it!
10. LLMs Prompt Context Window Probabilistic prediction of the next word Output Text generator Generating text is not thinking! …but we mistake generating text for thinking No explicit model of the world, truth…
11. No, actually it doesn’t use any message-oriented middleware. We review your architecture It uses message-oriented middleware! Architecture Review i.e. Apache ActiveMQ and Apache Artemis
12. No, actually it doesn’t use any message-oriented middleware. We reviews your architecture It uses message-oriented middleware! Architecture Review i.e. Apache ActiveMQ and Apache Artemis Would you trust the rest of the review? Would you consider us for other reviews? Would you consider us for other services? Would you consider this a minor or major mistake?
13. Architecture Review • Dirk Mahler (jQAssistant) mentioned exactly this as a result of an experiment with AI • jQAssistant: Architecture management tool • Claude with MCP access to jQAssistant data in Neo4J https://www.linkedin.com/posts/dirk-mahler-837a4b5_ai-neo4j-claude-activity-7321596753223340033-xz4r/
14. Architecture Review: Reaction • There was a short discussion in LinkedIn. • “It's a matter of context if speed or accuracy is more important…” • “Would be interesting to see where the wrong information came from.” • IMHO: It is quite questionable if AI is helpful at all for architecture. https://www.linkedin.com/posts/dirk-mahler-837a4b5_ai-neo4j-claude-activity-7321596753223340033-xz4r/
15. Architecture Review: Questions • This is just an experiment, no big deal. • But it leads to interesting questions. • Are people too easily convinced by AI? • Can LLMs be reliable?
16. Are People too Easily Convinced by AI? • ELIZA by Joseph Weizenbaum 1966 • Mimics a psychotherapist • I recall a listing in BASIC in the 1980s with ~100 lines. Picture: Andreas Scheper, CC https://www.flickr.com/photos/glueckauf/238430262/in/photolist-4rbRRe-n521A-4rbRLr-aa2Ggc https://de.wikipedia.org/wiki/ELIZA
17. ELIZA in Emacs
18. ELIZA in Emacs
19. ELIZA Results • Obviously, this system has absolutely no idea about anything. • It just trivially manipulates text. • People still think there is understanding. • Weizenbaum’s office manager asked him to not spy on her conversations with ELIZA. • ELIZA effect: People tend to assign human characteristics to chatbots. • Psychotherapists considered using ELIZA to automate therapy. • Experience made Weizenbaum an AI critic https://de.wikipedia.org/wiki/ELIZA-Effekt https://de.wikipedia.org/wiki/ELIZA
20. So Are People too Easily Convinced by AI? • ELIZA Effect • Nature paper: GPT-4 is more persuasive in debates than humans. • So: Yes, people are easily convinced by AI • And we know that since the 1960s. https://www.nature.com/articles/s41562-025-02194-6
21. How Is Persuasive Relevant? • LLMs make mistakes (as we have seen). • We must sceptic about results of LLMs. • Hard: They are persuasive. • We will have debates where LLMs are more persuasive than us. • LLMs can be used very well to spread misinformation. • (This should have political implications.)
22. But the LLM says otherwise. I think we shouldn’t do this. … Architecture Consulting Database of hallucinations in court cases: https://www.damiencharlotin.com/ hallucinations/
23. But some person at some conference says otherwise. I think we shouldn’t do this. … Architecture Consulting
24. Can LLMs be reliable? • LLMs produce “hallucinations”. • Term is IMHO misleading. • Hallucination: “perception in the absence of an external context stimulus”. • LLM do not have “perceptions”. • LLM generate fake information. https://en.wikipedia.org/wiki/Hallucination
25. https://chatgpt.com/share/68de48b6-96e4-800f-9d34-5050fe4173f0
26. https://chatgpt.com/share/68de48b6-96e4-800f-9d34-5050fe4173f0 Blueberry
27. https://chatgpt.com/share/68de48b6-96e4-800f-9d34-5050fe4173f0 Blueberry Counting characters has been fixed ChatGPT generates different output depending on version, random numbers… How can we reproduce output? Because it is using some external tool. We usually pin versions of dependencies..
28. Can Hallucinations Be Fixed? • The problem got worse. • OpenAI doesn’t know why. https://winfuture.de/news,150778.html
29. Surely This is Just Training Data? • Bad training data does not explain… …the blueberry problem. Data is in the prompt (input). …Dirk’s jQAssistant experiment. Data is in the prompt via MCP. • These are “intrinsic hallucinations”. generated output contradicts source content • Training data won’t fix this problem. https://arxiv.org/html/2202.03629v7 https://chatgpt.com/share/68ebc488-0b2c-800f-8455-193c32feccf3
30. LLMs Are For Conversation • LLMs are built for conversations. • Nobody checks everything they say in a conversation. • I.e. fake information keep a conversation going. • However, I do check information that I provide in a review.
31. LLMs Are Optimized For Tests • LLMs evaluation tests punish “I don’t know”. • So LLMs guess some answer. • Like a student in an exam who just gives a random answer. • Might be fixable with different rewards in tests. • Bullshit: If you just talk with no regards to truth. https://openai.com/index/why-language-models-hallucinate/
32. https://software-architektur.tv/2025/04/11/episode260.html
33. Are LLMs Optimized For Software Architecture? • How good is an architect who never says “I don’t know”? • How good is an architect with no concept of truth or the world?
34. AGI Prediction • AGI = Artificial general intelligence • 2030 (Sam Altman, OpenAI CEO) • Will take over 30-40% of tasks in the economy • OpenAI valuation: $300bn • This is probably what he has to say. https://www.techspot.com/news/109644-sam-altman-predicts-artificial-general-intelligence-2030-ai.html Picture: https://www.flickr.com/photos/jurvetson/54446118165/ Steve Jurvetson
35. AGI Prediction “In from 3-8 years we will have a machine with the general intelligence of an average human being. I mean a machine that will be able to read Shakespeare, grease a car, play office politics, tell a joke, have a fight.“ 1970, Marvin Minsky “Father of AI”, Founder MIT AI Lab https://en.wikiquote.org/wiki/Marvin_Minsky https://en.wikipedia.org/wiki/Marvin_Minsky Picture: Andreas Scheper, CC https://flic.kr/p/fSxyn
36. So How Can You Use AI for Software Architecture?
37. https://software-architektur.tv/2023/12/15/folge193.html
38. Result • LLMs can generate a persuasive text. • Even if they don’t take the requirements into account …because they went out of the context window.
39. Is this actually doing software architecture?
40. What Were We Doing? • Generate documentation • Documentation is not architecture!
41. What Is Architecture? • Architecture is what is implemented. • Writing a text alone will not help. • You must communicate the architecture to people. • You should get input from technical people. • You must understand requirements / business cases. • You must communicate with stakeholders. • Communication is the main challenges.
42. Architecture Theater • Is there such a thing as “too much architecture”? • I.e. a very hierarchical, bureaucratic, committee- driven approach • This is just “architecture theater”. • LLMs can help to produce documents… …for architecture theater. https://mastodon.social/@kevlin/112003129757159797
43. So How Can You Use AI for Software Architecture? • Search for information: not always correct but answers specific questions. • E.g. Generate a text about REST vs gRPC • Instead of Stackoverflow, web search…
44. So How Can You Use AI for Software Architecture? • Generating architecture documents in a dialog • I.e. humans check the results! • This is what people are doing in practice. • Humans review instead of creating themselves. • Is reviewing easier or harder than creating?
45. Let‘s Ask ChatGPT • ChatGPT generated a text about LLMs and software architecture. • LLM Benefits: o Speed up analysis and design o Better access to best practices o Communication and moderation (i.e. prepare workshops) • You could improve other aspects instead of using an LLM. https://chatgpt.com/share/68238ff5-4bac-800f-aea7-5d6aaee3b8bb
46. Let‘s Ask ChatGPT Without architectural skills, LLMs will only accelerate your missteps. https://chatgpt.com/share/68238ff5-4bac-800f-aea7-5d6aaee3b8bb
47. So How Can You Use AI for Software Architecture? • Another tool • No full automation. • Is this result surprising? • Does this differ from e.g. software development? • Software development should more deterministic and faster feedback (tests etc) • …but code is like architecture an artefact humans need to understand and change.
48. AI in Software Development: Science • Developers think they write more secure code but actually don’t. • Open-source developers experienced with a code base think they are faster with AI but actually are slower. • This overview is not comprehensive! https://arxiv.org/abs/2211.03622 https://arxiv.org/abs/2507.09089
49. https://software-architektur.tv/2025/11/05/folge287.html
50. https://software-architektur.tv/2025/11/05/folge287.html
51. AI in Software Development: Open Source • Open Source is where we can see publicly how well tools do. • cURL maintainer complains about AI slop w.r.t. hallucinated security issues. • Servo (Rust web browser engine) forbids AI-generated code. • Github Copilot (Microsoft) submits a pull request on Github (MS) for .NET (MS) and fails hilarously. https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/ https://book.servo.org/contributing.html#ai-contributions https://github.com/dotnet/runtime/pull/115762
52. AI in Software Development: Anecdotes • LLM deletes production database …even though it is told not to …and made excuses. • Simon Wardley had an LLM generate a fake test …executed no code, but generated the expect output. https://x.com/jasonlk/status/1946065483653910889 https://software-architektur.tv/2025/04/03/episode258.html
53. AI in Software Development: Companies • Developers are eager to use new cool tools. • MS has to force employees to use Copilot. • Why? https://www.businessinsider.com/microsoft-internal-memo-using-ai-no-longer-optional-github-copilot-2025-6
54. This is all so negative! https://chatgpt.com/share/68ebc4f9-f8b4-800f-887a-98dfa5760b2d
55. AI in Software Development: Positiv • Adrian Cockcroft (Ex-Netflix, Ex-AWS, Microservices pioneer) has vibe coded a useful smart house app using Claude code. • Multiple reports: Simple tasks (build a new element in the UI) can be done with AI. • Still needs work on the code level, not vibe coding. https://www.linkedin.com/posts/adriancockcroft_github-adriancothe-goodies-smart-home-activity-7355828333772750850-qCOo/
56. AI in Software Development: Positiv • I don’t want to code JavaScript, C++, EMACSLisp … without an LLM
57. How I use LLMs. • Every abstract is optimized by an LLM. • LLMs translate my blog into English. • We are using LLMs for transcripts / summaries for Software Architektur im Stream.
58. AI Is Not Just LLMs. • There might be other approaches that work better than LLMs.
59. Ethics • LLMs consume absurd amounts of energy. • LLM trainers have bad working conditions. • LLMs demand lots of investment i.e. they further monopolies. • These monopolies are already a threat to our society. • LLMs might make people loose skills?
60. https://software-architektur.tv/2025/02/21/folge251.html
61. Conclusion
62. Conclusion • Architecture and software development is a people business. • LLMs can’t really help with that. • There are ethical challenges.
63. Conclusion • LLMs by design provide fake information. • LLMs are persuasive. • I.e. results need to be checked …this is hard because the information is persuasive.
64. Conclusion • LLMs can be used for software architecture • LLMs can generate text e.g. documentation, ADRs … • In a dialog with constant checks and guidance. • Experience from software development supports this approach.
65. https://software-architektur.tv/2025/06/06/folge266.html
66. WIR SUCHEN MITGESTALTER:INNEN swaglab.rocks/karriere
67. DRINK A VIRTUAL COFFEE WITH ME! Eberhard Wolff Head of Architecture https://swaglab.rocks/ https://ewolff.com/ https://calendly.com/eberhard-wolff-swaglab/
68. Slides + more Powered by Amazon Lambda Email address logged for 14 days, wrong addressed emails handled manually Scan QR code or send email to wjax2025@ewolff.com