Coding with AI: How to trust the storyteller (AWS Community Day Adria, Ljubljana 2026)

Transcript

1. Do you still know how to write a "Hello World" in JavaScript or Python? Ok, Ok, that was a bit too much.
2. When was the last time you manually wrote a significant amount of code?
3. Do you read code generated by AI agents?
4. I still do*. But why?
5. How do you read that amount of code?
6. Custom software is cheap! So, I built a custom code review tool.
7. https://hawkshaw.dev
8. Ok, but a code review phase is too late. It's catching problems, not preventing them.
9. How do you make Claude Code, Codex or any other coding agent do what you actually want it to do?
10. The first obvious pick is to add instructions to Claude.md or Agents.md. That should work, right?
11. I found this in the Claude.md file of our marketing website
12. Should we give Claude Code a Twilio API key so it can call Dejan?
13. A year ago Also, back in the day, Claude Code sometimes lost context from Claude.md.
14. A friend added an instruction that tells Claude Code to start each reply with "Izvolte!" ("Here you go!" in Serbian)
15. So, Claude Code decided to start replying in… Russian?!
16. Coding with AI How to trust the storyteller
17. But who am I to talk about that?
18. Slobodan Stojanovic cofounder/CTO @ Vacation Tracker @slobodan_
20. let's get back to AI agents
21. Can you simply use AI agents to check the output of your AI agents?
22. Let's talk about deterministic vs non-deterministic checks
23. deterministic: same input = same output
24. deterministic: same input = same output
25. deterministic: same input = same output
26. deterministic: same input = same output
27. non-deterministic: same input = maybe different output
28. non-deterministic: same input = maybe different output
29. non-deterministic: same input = maybe different output
30. non-deterministic: same input = maybe different output
31. LLMs are storytellers. And they are very good ones.
32. But they tell a different story every time.
33. “Never trust the storyteller. Only trust the story.” Neil Gaiman
34. What's the alternative?
35. Custom linter rules?
36. btw, check OXlint Custom linter rules are so hard to write manually. But that's Claude's problem now, not mine.
37. Good Custom linter rules are even harder to write manually. Again, Claude's problem, not mine.
38. E.g., instead of an error: x is forbidden linter can say: avoid using X because of something use Y instead
39. E.g., instead of an error: unquoted-paren-in-label linter can say: an unquoted '(' in a bracket label starts a shape token, so the parse fails mid-label; restructure with a comma or colon, or quote the whole label.
40. We can run linter rules in a pre-commit hook. But it's still catching problems, not preventing them.
41. It would be great if Claude Code could run a "pre-commit" hook after each change...
42. And, of course it can.
46. Like automated code review We could tell Claude Code to run our linter on modified files and fix problems before we ever see them.
47. But this "PreToolUse" hook is especially interesting. Can we use it to run an automated check to determine whether a specific edit or action should be allowed?
49. What's a regulator?
50. Claude: Can I do "ls -la" in a workspace folder? Regulator: YES
51. Claude: Can I run this looong Python script? Regulator: No, simplify it.
53. This simplifies the review process. But we still need testing, right? Yes! More than ever. But can we automate part of it?
54. Can an agent do auto-verification? And maybe also record a demo video?
55. Yes, but it needs a test environment. And we definitely don't want it to access our production.
56. What should we trust an agent with?
57. NOTHING
58. Because of vulnerabilities in dependencies, prompt injections, Anthropic/OpenAI, storytelling, and many, many other things.
59. Check OrbStack on Mac I always run my agents in (Docker) containers. It's not perfectly safe, but it's better than nothing.
60. But what do you do if your laptop can't run all these local containers? I never said they have to be local.
61. An EC2 instance can be your "agent sandbox." Anything you can connect to remotely and run Claude Code in works.
62. But how to connect to your container? I use ghostty + tmux
63. But how to connect to your container? I use ghostty + tmux
64. But what do you do to access data in an AWS account or deploy an application for testing?
65. aws sso login --profile readonly
66. Or give your agents access to AWS sub-account (s). But be very careful with permissions and deployment flow, e.g., no direct access, but via some CICD process
67. Once agents have an environment, they can record a demo (provide proof) for you!
68. This gives you automation that reduces your time. But it's still not ideal.
69. This gives you automation that reduces your time. But it's still not ideal. 2 prompts = 100% of 5-hour limit 50% of weekly fable limit and 25% of weekly limit
70. It can be expensive and waste many, many tokens.
71. Unless we adjust our planning for human review and agent implementation. How do we do that?
72. 1. Define what we want. 2. Prepare an exact plan for Claude Code.
73. 1. Define what we want. Commander's Intent
74. Commander's Intent: - Purpose: Why personnel must complete the assignment. - Task: What the objective or goal entails. - End state: How the result should look.
75. Better version "Considerations in Communicating intent" Sources of Power: How People Make Decisions Book by Gary Klein
76. "There are seven types of information that a person could present to help the people receiving the request to understand what to do" Sources of Power Book by Gary Klein
77. 1. The purpose of the task (the higher level goals) 2.The objective of the task (an image of desired outcome) 3.The sequence of steps in the plan 4.The rationale for the plan 5.The key decisions that may have to be made 6.Antigoals 7.Constrains and other considerations
78. Prepare feature skill: Ask an agent to interview you until you have all 7, or enough to build a feature plan.
80. UX skill: Build a prototype to test it out (and apply Kathy Sierra's Badass principles)
82. Agent planning skill: Convert these two into a proper agent plan. What's a proper agent plan?
83. A parsable markdown file. + A workflow file
84. Claude Code defines agent workflows on the fly in state-function-like JavaScript code. And Claude Code knows how to write it.
86. So, our agent plan should prepare that and define the effort and model.
87. Remember this?
88. 3 hours of planning. 4 hours and 9 minutes of Claude Code work. 2 days of reviewing. It would take me at least 3-4 weeks to write it manually.
89. Custom software is cheap! So, I built a custom agent workflow review tool.
95. I lied, it was 290 agents instead of 207
96. SUMMARY
97. 1. There are no right or wrong things to do. Build a workflow that works for you. 2. Never trust the storyteller. Only trust the story. 3.
98. Questions? https://slobodan.me @slobodan_
99. Questions? https://slobodan.me @slobodan_