Share
Embed
Slide 1
Slide 1 text
LLMs as domain experts
@talyssonoc
Slide 2
Slide 2 text
Talysson Oliveira
Software architect
& Chief Learning Officer
at Codeminer42
@talyssonoc
beacons.ai/talyssonoc
Slide 3
Slide 3 text
codeminer42.com
@codeminer42
Slide 4
Slide 4 text
First of all: what is a domain?
Slide 5
Slide 5 text
Domain
- Finance
- Health
- Education
- Commerce
- Aviation
- Mining
- Transportation
- …
Problem
A system of abstractions that
describes selected aspects of a
domain and can be used to solve
problems related to that domain
Domain
Model
Slide 6
Slide 6 text
Airplane
(domain)
Plane model
- Business relevance
- Mentioned in conversations
- Involved in business rules
Slide 7
Slide 7 text
How is the model cultivated?
Slide 8
Slide 8 text
Domain experts
Development
experts
Slide 9
Slide 9 text
Domain experts
Development
experts
Domain
model
Slide 10
Slide 10 text
Domain experts
Development
experts
Domain
model
Slide 11
Slide 11 text
Domain experts
Development
experts
Domain
model
Ubiquitous language
Slide 12
Slide 12 text
Why domain experts may not be available
- Their available time is scarce
- The access to them is not easy or practical
- The communication is not in tune yet
- The team doesn't have domain experts (yet)
Slide 13
Slide 13 text
Can LLMs be used in these cases?
Slide 14
Slide 14 text
Yes!
Slide 15
Slide 15 text
Yes!
I mean, kind of!
Slide 16
Slide 16 text
The model exists in the shared
understanding of the team
Slide 17
Slide 17 text
LLMs can't cultivate a model
Slide 18
Slide 18 text
They can be used to provide
information about the domain
Slide 19
Slide 19 text
Using LLMs to enrich your
knowledge about the domain
Slide 20
Slide 20 text
Domain experts
Development
experts
Slide 21
Slide 21 text
Domain experts
Development
experts
Slide 22
Slide 22 text
Domain experts
Development
experts
Slide 23
Slide 23 text
Domain experts
Development
experts
We do this first
Slide 24
Slide 24 text
Domain experts
Development
experts
We do this first
To make the
best out of this
Slide 25
Slide 25 text
Domain
Other
Domain
Slide 26
Slide 26 text
If you don't know where you are going,
any road will get you there.
- Lewis Carroll
Slide 27
Slide 27 text
You can't "vibe" your way to something you
can't recognize when you get there
Slide 28
Slide 28 text
Domain
Problem
Slide 29
Slide 29 text
Example:
Special in-flight meals
Slide 30
Slide 30 text
In-flight meals
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
What problem are we solving?
Slide 34
Slide 34 text
Special in-flight meals
- Allow airlines to offer special in-flights meals paid in advance
- Customers order special meals up to 5 hours before the boarding
- A list of meals is sent to a specialized company, which will cook
and deliver them directly to the airports
- Customers can rate their experience and be part of a loyalty
program and get discounts and gifts
Slide 35
Slide 35 text
In-flight meals
Logistics
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
No content
Slide 38
Slide 38 text
No content
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
In-flight trolleys
(meal carts)
In-flight meal
shortage
Cold chain HACCP
(Hazard Analysis Critical Control Point)
Long-haul flights
Extra in-flight
meals
Turnaround
schedules
Slide 41
Slide 41 text
Then, ask about these concepts
in the same conversation
(within the same LLM context)
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
No content
Slide 44
Slide 44 text
Domain experts
Development
experts
HACCP
HACCP
Slide 45
Slide 45 text
Domain experts
Development
experts
HACCP
Slide 46
Slide 46 text
Use LLMs to optimize the conversations
with domain experts!
Slide 47
Slide 47 text
What if there are no domain experts?
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
No content
Slide 51
Slide 51 text
The development experts
become the domain experts
Slide 52
Slide 52 text
Use LLMs to empower your
team while the team
cultivates the domain model
Slide 53
Slide 53 text
Using LLMs to make the
access to the model easier
for the whole team
Slide 54
Slide 54 text
LLMs can't cultivate a model
Slide 55
Slide 55 text
You can't teach them the model
LLMs don't think
Slide 56
Slide 56 text
But you can document the
model as much as possible
and feed it to a LLM
Slide 57
Slide 57 text
[...] the programmer's knowledge transcends
that [knowledge] given in documentation [...]
- Programming as Theory Building, Peter Naur, 1985
Slide 58
Slide 58 text
But it can be good enough!
The documentation does not
contain the totality of the model
Slide 59
Slide 59 text
Documenting the domain model
- Sub-domains and bounded contexts
- Scope and responsibilities
- Ubiquitous language glossary
- Context map
- Use cases and user stories of your system
- Tactical patterns
- Aggregates, entities, value objects, repositories, domain services, …
Slide 60
Slide 60 text
Make the documentation LLM-friendly
- Don't leave important concepts implicit
- When images are needed, add a description as well
- Explicitly state possible misunderstandings
- E.g.: "Don't confuse with "
Slide 61
Slide 61 text
Use tools to make it even more LLM-friendly
- Use team workspaces in LLM chat tools
- Connect the LLM to MCP servers of your documentation platform
- GitHub
- Confluence
- Notion
- Use well-known LLM-friendly formats for coding tools
- AGENTS.md
- CLAUDE.md
Slide 62
Slide 62 text
Keep your documentation updated!
No documentation is better than
bad/outdated documentation
Slide 63
Slide 63 text
LLMs as domain experts
- LLMs can't be fully trusted, they can hallucinate
- Arguably, humans can make mistakes too
- The difference there is that humans can be held accountable
- Human domain experts provide domain expertise and
collaboratively cultivates the model
- LLMs can be used to the former, but should not be used for the latter
Slide 64
Slide 64 text
Use LLMs to empower your team,
not to replace it
Slide 65
Slide 65 text
¡Gracias!
@talyssonoc
beacons.ai/talyssonoc
@codeminer42
codeminer42.com