Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
How we built an AI code reviewer with serverles...
Search
Yan Cui
February 12, 2025
Technology
0
99
How we built an AI code reviewer with serverless and Bedrock
Slides for my talk at the Serverless London meetup on 12-Feb-2025
Yan Cui
February 12, 2025
Tweet
Share
More Decks by Yan Cui
See All by Yan Cui
Money-saving tips for the frugal serverless developer (AWS Community Summit)
theburningmonk
1
180
Money-saving tips for the frugal serverless developer
theburningmonk
1
760
Why the fuzz about serverless (with CompassDigital)
theburningmonk
0
100
Money-saving tips for the frugal serverless developer
theburningmonk
0
120
Efficient patterns for serverless development (AWS Summit London)
theburningmonk
0
140
7 ways to solve Lambda cold starts
theburningmonk
0
63
Saving Money on Serverless: Common Mistakes and How to Avoid Them
theburningmonk
0
58
3 Ways to Improve Serverless Performance
theburningmonk
0
45
Smart and efficient ways to test serverless architectures
theburningmonk
1
280
Other Decks in Technology
See All in Technology
fukabori.fm 出張版: 売上高617億円と高稼働率を陰で支えた社内ツール開発のあれこれ話 / 20250704 Yoshimasa Iwase & Tomoo Morikawa
shift_evolve
PRO
2
8.1k
OSSのSNSツール「Misskey」をさわってみよう(右下ワイプで私のOSCの20年を振り返ります) / 20250705-osc2025-do
akkiesoft
0
170
AWS CDKの仕組み / how-aws-cdk-works
gotok365
8
370
[SRE NEXT] ARR150億円_エンジニア140名_27チーム_17プロダクトから始めるSLO.pdf
satos
2
770
開発生産性を測る前にやるべきこと - 組織改善の実践 / Before Measuring Dev Productivity
kaonavi
14
6.5k
TableauLangchainとは何か?
cielo1985
1
120
QuickSight SPICE の効果的な運用戦略~S3 + Athena 構成での実践ノウハウ~/quicksight-spice-s3-athena-best-practices
emiki
0
140
アクセスピークを制するオートスケール再設計: 障害を乗り越えKEDAで実現したリソース管理の最適化
myamashii
1
160
United™️ Airlines®️ Customer®️ USA Contact Numbers: Complete 2025 Support Guide
flyunitedguide
0
430
CDKTFについてざっくり理解する!!~CloudFormationからCDKTFへ変換するツールも作ってみた~
masakiokuda
1
180
VS CodeとGitHub Copilotで爆速開発!アップデートの波に乗るおさらい会 / Rapid Development with VS Code and GitHub Copilot: Catch the Latest Wave
yamachu
2
190
How to Quickly Call American Airlines®️ U.S. Customer Care : Full Guide
flyaahelpguide
0
150
Featured
See All Featured
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Embracing the Ebb and Flow
colly
86
4.7k
Scaling GitHub
holman
460
140k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
YesSQL, Process and Tooling at Scale
rocio
173
14k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.4k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Speed Design
sergeychernyshev
32
1k
GitHub's CSS Performance
jonrohan
1031
460k
GraphQLとの向き合い方2022年版
quramy
49
14k
How to train your dragon (web standard)
notwaldorf
96
6.1k
Transcript
How we built an AI Code Reviewer with Serverless and
Bedrock
Yan Cui http://theburningmonk.com @theburningmonk AWS user since 2010
Yan Cui http://theburningmonk.com @theburningmonk running serverless in production since 2016
Developer Advocate @ Yan Cui http://theburningmonk.com @theburningmonk
Yan Cui http://theburningmonk.com @theburningmonk independent consultant
None
evolua.io Demo
Architecture
API Gateway EventBridge Webhook
API Gateway DynamoDB Bedrock EventBridge Webhook
API Gateway DynamoDB Bedrock EventBridge Webhook
API Gateway DynamoDB Bedrock EventBridge Webhook evolua.io
None
API Gateway DynamoDB Bedrock EventBridge Webhook AppSync evolua.io
API Gateway DynamoDB Bedrock EventBridge Webhook AppSync evolua.io
None
API Gateway DynamoDB Bedrock EventBridge Webhook AppSync evolua.io Authoriser
API Gateway DynamoDB Bedrock EventBridge Webhook AppSync evolua.io Authoriser
API Gateway DynamoDB Bedrock EventBridge Webhook AppSync evolua.io Authoriser
API Gateway DynamoDB Bedrock EventBridge Webhook AppSync evolua.io Authoriser
API Gateway DynamoDB Bedrock EventBridge Webhook AppSync evolua.io Authoriser
Challenges (for an AI code reviewer) Handling sensitive data for
customers
Challenges (for an AI code reviewer) Large fi les. Large
PRs with many fi les. Handling sensitive data for customers
Why Bedrock?
Security
Security Data is encrypted at rest.
www.wiz.io/blog/wiz-research-uncovers-exposed-deepseek-database-leak
aws.amazon.com/bedrock/faqs
Security Data is encrypted at rest. Inputs & Outputs are
not shared with model providers. Inputs & Outputs are not used to train other models.
API Gateway DynamoDB Bedrock EventBridge Webhook AppSync evolua.io Authoriser Fallback
Primary
privacy.anthropic.com/en/articles/7996885-how-do-you-use-personal-data-in-model-training
Serverless
Serverless Usage-based AND provisioned throughput pricing
None
None
1M Input Tokens 1M Output Tokens $0.14 v3 r1 $0.28
$0.55 $2.19 Sonnet $3.75 $15.0 Haiku $0.80 $4.00
Very cost ef fi cient!
Very cost ef fi cient! Data is stored in China.
Very cost ef fi cient! Data is stored in China.
Data might be used to train other models.
www.wiz.io/blog/wiz-research-uncovers-exposed-deepseek-database-leak
Very cost ef fi cient! Data is stored in China.
Data might be used to train other models. Operationally immature.
None
No token-based pricing yet
No token-based pricing yet “GPU-based instance type like ml.p5e.48xlarge is
recommended”
ml.p5e.48xlarge 💰💰💰💰💰💰💰💰💰💰 💰💰💰💰💰💰💰💰💰💰 💰💰💰💰💰💰💰💰💰💰 💰💰💰💰💰💰💰💰💰💰 💰💰💰💰💰💰💰💰
Other capabilities Guardrails Knowledge base (managed RAG) Agents Cross-region inference
Model evaluations
None
None
None
API Gateway DynamoDB Bedrock EventBridge Webhook AppSync evolua.io Authoriser Fallback
Primary
Lessons
Webhook
Webhook Analyse changes
Webhook Analyse changes Feedback
Condensed view…
None
Lambda timed out after 15 mins
Succeeded on automatic retry
Webhook Analyse changes Feedback LLM limits GitHub limits AWS limits
Lesson: AI is 10% of the problem
None
Reasoning ability
Context window Max response tokens API rate limit Reasoning ability
Context window Max response tokens API rate limit Reasoning ability
Cost Performance
Context window Max response tokens API rate limit Reasoning ability
Cost Performance Important selection criteria for LLMs
Doing cool AI stuff! Working around AI limits
Doing cool AI stuff! Working around AI limits Stop playing
with my bowl…
Context window Max response tokens API rate limit Reasoning ability
Cost Performance
Claude 3.5 Sonnet’s default throughput is 50 per minute
Claude 3.5 Sonnet’s default throughput is 50 per minute Can
be raised to 1,000 per minute
Claude 3.5 Sonnet’s default throughput is 50 per minute Can
be raised to 1,000 per minute Bedrock has cross- region inference
Mitigate API rate limit Raise account limits. Use Bedrock cross-region
inference.
Mitigate API rate limit Raise account limits. Use Bedrock cross-region
inference. Limit no. of parallel requests per PR.
Mitigate API rate limit Raise account limits. Use Bedrock cross-region
inference. Limit no. of parallel requests per PR. Fallback to Anthropic & less powerful models (Claude 3 Sonnet, Claude 3.5 Haiku)
Future work: incorporate other models (Nova, DeepSeek, etc.)
Future work: incorporate other models (Nova, DeepSeek, etc.) Also good
for cost control!
Lesson: LLMs are still quite expensive
None
Dif fi cult to build a sustainable and competitive business
Cost control Only analyse changed lines.
Cost control Only analyse changed lines. Good for cost control
Good for UX
Cost control Only analyse changed lines. Limit free users to
few PRs per month.
API Gateway DynamoDB Bedrock EventBridge Webhook
API Gateway DynamoDB Bedrock EventBridge Webhook Built-in retries & DLQ
Lambda timed out after 15 mins
Lambda timed out after 15 mins Reprocess fi les on
retry…
Lambda timed out after 15 mins Reprocess fi les on
retry… Duplicated side- effects (e.g. Github comments)
Cost control Only analyse changed lines. Limit free users to
few PRs per month. Use checkpoints to avoid re-processing fi les on retries
const issues = await executeIdempotently( `${event-id}-${filename}-analyze`, () => analyzeFile(file) );
... await executeIdempotently( `${event-id}-${filename}-add-gh-comment`, () => addReviewComment(filename, comment) );
Webhook Analyse changes Feedback Why not Step Functions?
Webhook Analyse changes Feedback Why not Step Functions? Checkpoints is
just easier 🤷
Lesson: Latency is a challenge
Models take 10s of seconds to analyse each fi le
Wasted CPU cycles in Lambda
Future work: try other models
Future work: make use of these CPU cycles
Lesson: Be ware of hallucinations
“Give me JSON in this format”
None
“Give me JSON in this format” “Nope!”
None
Non-existent codes, invalid URLs
Non-existent line numbers
Future works
Go to evolua.io to try it out. We’d love your
feedback!
Questions?