Share
Embed
Slide 1
Slide 1 text
How Swi''s Type System
Guides AI Agents
Yuta Koshizawa @koher
Slide 2
Slide 2 text
Yuta Koshizawa
• @koher
• Qoncept, Inc.
• try! Swi3 Tokyo 2016 Speaker
• Author of "Heart of Swi3"
• Swi3 Zoomin' Organizer
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
async/await in Swi&
func asyncGetInt(...) async -> Int {
...
}
let x: Int = await asyncGetInt(...)
Slide 6
Slide 6 text
throws/try
async/await
Slide 7
Slide 7 text
throws/try
async/await
AI
Slide 8
Slide 8 text
1. AI and Verifica/on
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
How I used to use an AI chatbot for coding
1. I explain to the AI what I want to implement
2. The AI writes code
3. I copy and paste that code into my IDE
4. The IDE reports type errors
5. I fix them myself, or send the errors back to the AI.
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
Give Claude a way to verify its work
Claude performs drama.cally be2er
when it can verify its own work1
— Best Prac+ces for Claude Code
1 h$ps:/
/code.claude.com/docs/en/best-prac7ces
Slide 13
Slide 13 text
A final 'p: probably the most
important thing to get great results
out of Claude Code -- give Claude a
way to verify its work. If Claude has
that feedback loop, it will 2-3x the
quality of the final result.2
— Boris Cherny
2 h$ps:/
/x.com/bcherny/status/2007179861115511237
Slide 14
Slide 14 text
2. throws
Slide 15
Slide 15 text
throws is part of a func-on's type
func foo() throws -> Int { // may throw
...
}
func bar() -> Int { // does not throw
...
}
Whether a func,on may throw is encoded in its signature as part of
Swi7's type system, and is checked sta,cally by the compiler.
Slide 16
Slide 16 text
What About Java?
Slide 17
Slide 17 text
Which line may throw an error?
// TypeScript
for (const rawName of requestedNames) {
const name = normalizeUserName(rawName);
const user = findUser(name);
if (user == null) {
missingNames.push(name);
continue;
}
const group = findGroup(user.id);
const summary = buildUserSummary(user, group);
members.push(summary);
activityLog.record(summary.displayName);
}
const pageTitle = formatPageTitle(members.length, missingNames.length);
const groupIds = new Set(members.map((member) => member.groupId));
At a glance, the poten.al sources of errors are unclear.
Slide 18
Slide 18 text
Which line may throw an error?
// TypeScript
for (const rawName of requestedNames) {
const name = normalizeUserName(rawName);
const user = findUser(name);
if (user == null) {
missingNames.push(name);
continue;
}
const group = findGroup(user.id);
const summary = buildUserSummary(user, group);
members.push(summary);
activityLog.record(summary.displayName);
}
const pageTitle = formatPageTitle(members.length, missingNames.length);
const groupIds = new Set(members.map((member) => member.groupId));
At a glance, the poten.al sources of errors are unclear.
Slide 19
Slide 19 text
The compiler cannot verify documenta4on
// TypeScript
/**
* Returns a user for the given name.
* May throw a database connection error.
*/
function findUser(name: string): User | null {
...
}
Slide 20
Slide 20 text
In Swi'
// throwing
func findUser(name: String) throws -> User? {
...
}
// non-throwing
func findUser(name: String) -> User? {
...
}
Whether a func,on may throw is visible in the declara,on and
checked by the compiler.
Slide 21
Slide 21 text
No content
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
async/await
Slide 24
Slide 24 text
3. Experiments
Slide 25
Slide 25 text
Generates Func,ons in Swi1 and TypeScript
// Swift
func foo(value: Int) throws
-> String? {
let value = value.count
let v0 = value * 2 + 1
let v1 = v0 + 4
let v2 = v1 << 3
let v3 = v2 > 30 ? v2 : v2 + 8
let v4 = abs(v3 - 32)
let op1Result = try op1(value: v4)
let v5 = op1Result.count * 2 + 1
let v6 = v5 + 4
let v7 = v6 << 3
let v8 = v7 > 30 ? v7 : v7 + 8
let v9 = abs(v8 - 32)
let op9Result = try op9(value: v9)
let v10 = op9Result.count * 2 + 1
let v11 = v10 + 4
let v12 = v11 << 3
let v13 = v12 > 30 ? v12 : v12 + 8
let v14 = abs(v13 - 32)
let v15 = v14 ^ 12
let v16 = v15 % 24 + 1
let op17Result = try op17(value: v16)
return op17Result
}
// TypeScript
export function foo(value: number)
: string | undefined {
const value = value.length;
const v0 = value * 2 + 1;
const v1 = v0 + 4;
const v2 = v1 << 3;
const v3 = v2 > 30 ? v2 : v2 + 8;
const v4 = Math.abs(v3 - 32);
const op1Result = op1(v4);
const v5 = op1Result.length * 2 + 1;
const v6 = v5 + 4;
const v7 = v6 << 3;
const v8 = v7 > 30 ? v7 : v7 + 8;
const v9 = Math.abs(v8 - 32);
const op9Result = op9(v9);
const v10 = op9Result.length * 2 + 1;
const v11 = v10 + 4;
const v12 = v11 << 3;
const v13 = v12 > 30 ? v12 : v12 + 8;
const v14 = Math.abs(v13 - 32);
const v15 = v14 ^ 12;
const v16 = v15 % 24 + 1;
const op17Result = op17(v16);
return op17Result;
}
Slide 26
Slide 26 text
Call Tree of a Generated Func1on
foo --+-> op1 --+-> op3 --+-> op5 --+-> op7
| | | +-> op9
| | | `-> op11
| | +-> op13 ---> ...
| | `-> op21 ---> ...
| +-> op29 ---> ...
| `-> op55 ---> ...
+-> op81 ---> ...
`-> op161 ---> ...
call depth = 4, branching factor = 3, 80 steps per func9on
Slide 27
Slide 27 text
Func%ons Used in the Experiment
// throwing
func findUser(name: String) throws -> User? {
...
}
func findGroup(userId: String) throws -> Group? {
...
}
Slide 28
Slide 28 text
Func%ons Used in the Experiment
// non-throwing
func findUser(name: String) -> User? {
...
}
func findGroup(userId: String) -> Group? {
...
}
Slide 29
Slide 29 text
Func%ons Used in the Experiment
// TypeScript
function findUser(name: string): User | undefined {
...
}
function findGroup(userId: string): Group | undefined {
...
}
In TypeScript, the func1on signatures are the same for both the
throwing and non-throwing variants.
Slide 30
Slide 30 text
Experiment Prompt
Implement a func-on listBelongingGroups(userNames:
[String]). This func-on should sa-sfy the following
specifica-on:
1. For each user name in the given array, retrieve the user using
findUser(name:).
2. If the user is found, retrieve the group that the user belongs to
using findGroup(userId:).
3. If a user or a group is not found, skip that element and con>nue
processing the rest.
Slide 31
Slide 31 text
Experiment Prompt
4. Remove duplicates from the resul5ng groups using their id.
5. Print each group's name to standard output, one per line.
6. Apply appropriate error handling. In par5cular, log errors so it is
clear where they occurred. However, do not add unnecessary
catch handling at call sites where errors are not thrown.
Slide 32
Slide 32 text
What I Measured
throwing non-throwing
caught ✅ ⛔
not caught ⛔ ✅
token_usage = input_tokens
+ cache_creation_input_tokens
+ cache_read_input_tokens
Slide 33
Slide 33 text
Error Handling Accuracy
Swi$ TypeScript
throwing 5/5 (100%) 0/5 (0%)
non-throwing 5/5 (100%) 2/5 (40%)
Claude Code v2.1.92, Opus 4.6 (1M context) with high effort
Slide 34
Slide 34 text
Generated Code
func listBelongingGroups(userNames: [String]) {
var seenGroupIds: Set = []
var groups: [Group] = []
for name in userNames {
let user: User?
do {
user = try findUser(name: name)
} catch {
print("Error finding user '\(name)': \(error)")
continue
}
guard let user else { continue }
let group: Group?
do {
group = try findGroup(userId: user.id)
} catch {
print("Error finding group for user '\(user.name)': \(error)")
continue
}
guard let group else { continue }
if seenGroupIds.insert(group.id).inserted {
groups.append(group)
}
}
for group in groups {
print(group.name)
}
}
import { findUser } from "./findUser";
import { findGroup } from "./findGroup";
import type { Group } from "./types";
export function listBelongingGroups(userNames: string[]): void {
const groups = new Map();
for (const name of userNames) {
let user;
try {
user = findUser(name);
} catch (error) {
console.log(`Error finding user for name "${name}": ${error}`);
continue;
}
if (user === undefined) {
continue;
}
let group;
try {
group = findGroup(user.id);
} catch (error) {
console.log(`Error finding group for user "${user.id}": ${error}`);
continue;
}
if (group === undefined) {
continue;
}
if (!groups.has(group.id)) {
groups.set(group.id, group);
}
}
for (const group of groups.values()) {
console.log(group.name);
}
}
Slide 35
Slide 35 text
Error Handling Accuracy
Swi$ TypeScript
throwing 5/5 (100%) 0/5 (0%)
non-throwing 5/5 (100%) 2/5 (40%)
Claude Code v2.1.92, Opus 4.6 (1M context) with high effort
Slide 36
Slide 36 text
Token Usage
Swi$ TypeScript
throwing 127.5k 119.6k
non-throwing 136.2k 117.3k
Claude Code v2.1.92, Opus 4.6 (1M context) with high effort
Slide 37
Slide 37 text
Addi$onal Prompt
If you conclude that errors are not thrown, read the code all the
way to the leaves and make sure that the relevant path truly does
not throw.
Slide 38
Slide 38 text
Error Handling Accuracy (Strict Prompt)
Swi$ TypeScript
throwing 5/5 (100%) 5/5 (100%)
non-throwing 5/5 (100%) 4/5 (80%)
Claude Code v2.1.92, Opus 4.6 (1M context) with high effort
Slide 39
Slide 39 text
Token Usage (Strict Prompt)
Swi$ TypeScript
throwing 161.1k 353.1k
non-throwing 187.9k 400.0k
Claude Code v2.1.92, Opus 4.6 (1M context) with high effort
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
No content
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
No content
Slide 44
Slide 44 text
4. Isola)on
Slide 45
Slide 45 text
No content
Slide 46
Slide 46 text
No content
Slide 47
Slide 47 text
5. Other Ways to Verify
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
No content
Slide 51
Slide 51 text
No content
Slide 52
Slide 52 text
No content
Slide 53
Slide 53 text
No content