Slide 1
Slide 1 text
LEADING
EFFECTIVE
ENGINEERING
TEAMS IN THE
AI ERA
ADDY OSMANI
Slide 2
Slide 2 text
FOLLOW
ALONG SLIDES
https://leetai.addy.ie
Slide 3
Slide 3 text
Maybe joke at the start “I am happy to
announce we have all been replaced by
AI”
Slide 4
Slide 4 text
WILL EVERY
ENGINEER BECOME
AN ORCHESTRATOR
OF MULTIPLE
CODING AGENTS?
THIS SHIFTS US FROM IMPLEMENTORS TO
DIRECTORS OF HUMAN-AI COLLABORATION
Slide 5
Slide 5 text
ENGINEER AS CONDUCTOR:
FOCUSES ON A SINGLE TASK, STEERING
THE AGENT’S BEHAVIOR DYNAMICALLY
- TWEAKING, INTERVENING, AND
GUIDING WHILE IT RUNS.
ENGINEER AS ORCHESTRATOR:
MANAGES A FLEET OF AGENTS WORKING
IN CONCERT ON MULTIPLE,
INTERDEPENDENT TASKS.
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
Claude Code
Gemini CLI
Cursor
Copilot
Bolt / Lovable / v0
Slide 8
Slide 8 text
90% OF DEVELOPERS USE AI TOOLS
84% PER 2025
STACK OVERFLOW
DEV SURVEY
Slide 9
Slide 9 text
AI IS NOT ABOUT
WRITING MORE
CODE FASTER. IT'S
ABOUT BUILDING
BETTER SOFTWARE.
Slide 10
Slide 10 text
AI-ASSISTED
ENGINEERING
VIBE CODING
USE HIGH-LEVEL PROMPTS TO RAPIDLY
GENERATE CODE BY TRUSTING AN AI'S
OUTPUT, PRIORITIZING SPEED OVER
RIGOROUS REVIEW.
USING AI AS A SUPERVISED TOOL WITHIN
A STRUCTURED DEVELOPMENT PROCESS
WHERE THE HUMAN MAINTAINS FULL
CONTROL AND ACCOUNTABILITY.
Slide 11
Slide 11 text
WHERE CAN AI IMPROVE DX?
Slide 12
Slide 12 text
WHERE CAN AI IMPROVE DX?
Slide 13
Slide 13 text
WHERE CAN AI IMPROVE DX?
Slide 14
Slide 14 text
WHERE CAN AI IMPROVE DX?
Slide 15
Slide 15 text
A GOOGLE
TEAM’S AI
CODING
JOURNEY
Slide 16
Slide 16 text
THE AI-NATIVE
ENGINEER:
ASKS FOR EVERY TASK:
“COULD AI HELP ME DO
THIS FASTER, BETTER,
OR DIFFERENTLY?”.
Slide 17
Slide 17 text
THE PILOT IS
THE FIRST
CHAPTER OF A
MUCH LARGER
AI STORY Credit: henrythe9th
Slide 18
Slide 18 text
VIBE CODING
PLAUSIBLE
DENIABILITY FOR ALL
YOUR TECHNICAL DEBT
Slide 19
Slide 19 text
SKEPTIC:
I ASKED THE AI TO
REFACTOR OUR
CODEBASE. IT'S
NOW ALL EQUALLY
TERRIBLE.
Slide 20
Slide 20 text
Source: Infosys
AI SKEPTICS TO EVANGELISTS
Slide 21
Slide 21 text
HACK WEEKS
ARE A SAFE
SPACE TO LEARN,
PLAY & BUILD
CONFIDENCE
OR JUST BLAME AI FOR MISTAKES
Slide 22
Slide 22 text
AI TOOLS DON'T REPLACE
EXPERTISE - THEY AMPLIFY
IT. THE MORE SKILL YOU
BRING AS AN ENGINEER,
THE MORE POWERFUL THE
RESULTS YOU'LL GET.
Slide 23
Slide 23 text
10%
30%
21%
13%
MEASURED VELOCITY GAIN FOR
DEVELOPERS ATTRIBUTED TO AI
GENERATED BY AI (AND THEN REVIEWED
AND ACCEPTED BY ENGINEERS)
FASTER CODING ATTRIBUTED TO AI, IN A
CONTROLLED UX STUDY
MONTHLY CODE CHANGES SUBMITTED FOR DEVELOPERS
USING AI CODING TOOLS
Slide 24
Slide 24 text
PROMPT ENGINEER:
A FANCY WAY TO SAY
“AI WHISPERER”
Slide 25
Slide 25 text
PROMPT ENGINEERING FAILURES
STEM FROM CONTEXT MISMANAGEMENT
https://www.instagram.com/p/DMc4ujyTuk6/?img_index=6
Slide 26
Slide 26 text
CONTEXT IS KING.
CONTEXT-
ENGINEERING IS
THE NEW
ESSENTAL SKILL. .
SPEC, FILES, EXAMPLES, CONSTRAINTS.
IT’S MORE THAN JUST A PROMPT.
Slide 27
Slide 27 text
ADD CONTEXT CONTEXT WINDOW USE
CONTEXT IN EDITORS & IDES
Slide 28
Slide 28 text
https://www.youtube.com/watch?v=ysPbXH0LpIE
Slide 29
Slide 29 text
https://context.addy.ie
Slide 30
Slide 30 text
PLAN BEFORE YOU
ACT FOR BETTER AI
OUTCOMES. DON’T
JUMP INTO CODING
RIGHT AWAY
SPEC-DRIVEN
DEVELOPMENT
Slide 31
Slide 31 text
CREATE A MEMORY ANCHOR
- A COMPOUNDING
LEARNING LOOP FOR AI
AGENTS, HAVE THEM DISTILL
INSIGHTS AFTER EACH TASK.
THIS MAKES THE NEXT RUN
FASTER AND HIGHER
QUALITY.
LEARNINGS.MD
Lots of
fl
exibility here for data (context.md, learnings.md, decisions.md) that persists across sessions
Slide 32
Slide 32 text
NEVER COMMIT
CODE YOU
CAN’T EXPLAIN
TO OTHERS
OR EVEN YOURSELF
Slide 33
Slide 33 text
READING CODE
BECOMES THE
JOB. BUILD
THIS MUSCLE.
YOU WILL ALWAYS READ
MORE CODE THAN YOU WRITE
Slide 34
Slide 34 text
VIBE-CODING IS
A GIFT FOR
PROTOTYPING.
INTERACTIVE
PROTOTYPES >
STATIC MOCKS
Slide 35
Slide 35 text
SHOPIFY:
AI AS
CULTURAL
CATALYST
LEADERSHIP-LED, INFRASTRUCTURE-FIRST,
DEMOCRATIZED ACCESS
Slide 36
Slide 36 text
VIBE CODE
WITH YOUR
VOICE
SPEAKING PROMPTS ALOUD IS OFTEN FASTER THAN
TYPING. HUMANS SPEAK 3-5X FASTER THAN THEY
TYPE (150+ WPM VS. 40-80 WPM).
Slide 37
Slide 37 text
AI IS A FORCE
MULTIPLIER.
WHAT HELPS THE
HUMAN HELPS
THE AI.
TESTS, LINTING, CI
Slide 38
Slide 38 text
TESTS ARE A
SAFETY NET.
THEY DE-RISK
AI CODING.
BTW…IF YOU PLAN TO USE AI FOR
WRITING TESTS, IT'S CRUCIAL TO ADOPT A
STRATEGY THAT EMPHASIZES HUMAN
OVERSIGHT
Slide 39
Slide 39 text
AI
REALITY
CHECK
WHAT THE DATA SAYS ABOUT
PRODUCTIVITY, TRUST, AND TEAM
VELOCITY
Source 2025 DORA and Faros AI Productivity
fi
ndings
Slide 40
Slide 40 text
ADOPTION
IS UP, TRUST
IS DOWN
FAVORABLE VIEWS FELL FROM 70 TO 60%
IN TWO YEARS
46% ACTIVELY DISTRUST AI ACCURACY
30% REPORT LITTLE TO NO TRUST IN AI
GENERATED CODE
Slide 41
Slide 41 text
VIBE CODING VIBE DEBUGGING
Slide 42
Slide 42 text
CHROME DEVTOOLS
MCP LETS AGENTS "SEE"
AND INTERACT WITH
THE LIVE BROWSER
ENVIRONMENT.
Slide 43
Slide 43 text
INDIVIDUAL OUTPUT
IS SURGING
WITH AI CODING,
DEVELOPERS
COMPLETE 21%
MORE TASKS
2025 DORA/FAROS AI PRODUCTIVITY REPORT
MERGE 98%
MORE PULL
REQUESTS
OVER 80% SAY AI
HELPS THEIR
PERSONAL
PRODUCTIVITY
TASKS PRS SENTIMENT
Slide 44
Slide 44 text
“I FEEL LIKE AI
SOMETIMES WRITES
BETTER CODE THAN
I DO FOR CERTAIN
THINGS, MAINLY
BECAUSE I FEEL LIKE
IT’S BEEN TRAINED
REALLY WELL”
Slide 45
Slide 45 text
WITH AI CODING
PULL REQUEST
REVIEW TIMES
INCREASED BY 91%
ALL PRODUCTIVITY GAINS ABSORBED BY HUMAN VERIFICATION
2025 DORA/FAROS AI PRODUCTIVITY REPORT
Slide 46
Slide 46 text
“AI SOLUTIONS THAT
ARE ALMOST RIGHT, BUT NOT QUITE,
ARE NOW MY BIGGEST TIME SINK. THE
CODE LOOKS PLAUSIBLE BUT I END UP
SPENDING MORE TIME FIXING THOSE
‘HELPFUL’ SUGGESTIONS.”
TOP FRUSTRATION: ALMOST RIGHT CODE. 66%
CITE IT AS THE MAIN TIME SINK.
Slide 47
Slide 47 text
WHERE AI HELPS MOST
GREENFIELD AND PROTOTYPING
BOILERPLATE AND REPETITIVE CODE
UNIT TEST GENERATION
DOCUMENTATION AND SIMPLE API INTEGRATIONS
LOW-CONTEXT, HIGH-REPETITION WORK
Slide 48
Slide 48 text
WHERE AI STRUGGLES
COMPLEX LEGACY CODEBASES
SECURITY CRITICAL IMPLEMENTATIONS
AMBIGUOUS REQUIREMENTS
CROSS SYSTEM LOGIC
HIGH CONTEXT, NOVEL PROBLEMS
Slide 49
Slide 49 text
KEEP VIBE-CODING. WE
CAN ALWAYS FIX IT LATER.
Slide 50
Slide 50 text
THE
LEADERSHIP
PLAYBOOK
FOR AI-ASSISTED ENGINEERING
Slide 51
Slide 51 text
STRATEGY 1:
MITIGATE SKILL
EROSION.
GROW CRITICAL
THINKING.
Slide 52
Slide 52 text
SOFTWARE ENGINEERS ARE
PAID TO SOLVE PROBLEMS,
NOT JUST WRITE CODE.
BUILDING SOFTWARE IS 10%
CODING AND 90% CRITICAL
THINKING - A TASK YOU
CAN'T COMPLETELY DELEGATE
TO AN LLM.
Slide 53
Slide 53 text
No content
Slide 54
Slide 54 text
SOMETIMES THAT
"LAST 30%" IS WHAT
THE SENIORS SEE AND
THE JUNIORS DON'T.
THE 70% PROBLEM
Slide 55
Slide 55 text
54% OF LEADERS
EXPECT TO REDUCE
JUNIOR HIRING
THE BAR IS RISING, THE ROLE IS EVOLVING
THE LEADDEV AI IMPACT REPORT 2025
Slide 56
Slide 56 text
EVOLVE CODE
REVIEWS INTO
LEARNING
MOMENTS
SHIFT FOCUS FROM "DOES IT
WORK?" TO "DO YOU UNDERSTAND
WHY IT WORKS?"
Slide 57
Slide 57 text
SENIOR GUIDES STRATEGY, JUNIOR
DRIVES IMPLEMENTATION, AI
ASSISTS
PAIR
PROGRAMMING
+ AI = TRIO
PROGRAMMING
Slide 58
Slide 58 text
REGULAR
"NO-AI
CHALLENGES"
KEEP SKILLS
SHARP
LIKE PILOTS TRAINING WITHOUT
AUTOPILOT—ESSENTIAL FOR WHEN
YOU NEED IT MOST
Slide 59
Slide 59 text
AI’S BASELINE IS
“HEY, IT WORKS”:
ENGINEERS MUST
ENSURE IT'S SECURE,
MAINTAINABLE,
AND PRODUCTION-
READY
MICROSOFT COPILOT STUDY: 26%
FASTER TASK COMPLETION WITH NO
QUALITY DEGRADATION
Slide 60
Slide 60 text
MOVE BEYOND VANITY METRICS TO MEANINGFUL INDICATORS
STRATEGY 2:
MEASURE
WHAT TRULY
MATTERS
Slide 61
Slide 61 text
60% CITE LACK OF
CLEAR METRICS AS
BIGGEST AI CHALLENGE
ONLY 18% MEASURE IMPACT SYSTEMATICALLY
2025 LEADDEV AI IMPACT REPORT
Slide 62
Slide 62 text
No content
Slide 63
Slide 63 text
A THREE-PILLAR
MEASUREMENT FRAMEWORK
THROUGHPUT & VELOCITY QUALITY & STABILITY DEVELOPER EXPERIENCE
PULL REQUEST
CYCLE TIME IS
A TRUTH
TELLING
METRIC
MEASURE TOTAL TIME
FROM FIRST COMMIT TO
MERGE
CHECK
FAILURE
RATES AND
THE USER
EXPERIENCE
WHAT PERCENTAGE OF
DEPLOYMENTS REQUIRE
REMEDIATION?
QUALITATIVE
SURVEYS
REVEAL THE
HUMAN
STORY
"DID AI HELP YOU COMPLETE WORK FASTER THIS WEEK?"
"HOW CONFIDENT ARE YOU IN THE QUALITY OF CODE YOU
SHIPPED?"
"RATE YOUR FRUSTRATION LEVEL WITH CURRENT TOOLS"
Slide 64
Slide 64 text
ENABLE SAFE EXPERIMENTATION WITH CLEAR BOUNDARIES
STRATEGY 3:
ESTABLISH
HUMAN-
CENTERED
GUARDRAILS
Slide 65
Slide 65 text
SAFETY
STANDARDS
BACK IN
THE DAY
Slide 66
Slide 66 text
MOST DON'T KNOW THEIR COMPANY'S AI POLICY
Slide 67
Slide 67 text
LEADERSHIP'S ROLE :
RESPONSIBLE AI USAGE
"SET CLEAR AI BOUNDARIES: NO PROPRIETARY CODE IN PUBLIC
TOOLS, VERIFY LICENSING, MAINTAIN HUMAN
ACCOUNTABILITY.”
"ESTABLISH AI POLICIES BEFORE PROBLEMS ARISE. ETHICS,
SECURITY, AND LEGAL COMPLIANCE CAN'T BE
AFTERTHOUGHTS.”
"YOUR JOB: CREATE GUARDRAILS THAT LET TEAMS INNOVATE
SAFELY WITH AI TOOLS."
Slide 68
Slide 68 text
THE MOST VALUABLE
SKILLS IN THE AI ERA
CRITICAL THINKING AND PROBLEM ANALYSIS
ARCHITECTURAL DESIGN AND SYSTEM THINKING
COMPLEX PROBLEM-SOLVING
CLEAR COMMUNICATION AND SPECIFICATION
Slide 69
Slide 69 text
CHANGE
MANAGEMENT IS
AS IMPORTANT
AS TECHNOLOGY:
PEOPLE,
PROCESS, AND
CULTURE.
Slide 70
Slide 70 text
You’re
absolutely right
Slide 71
Slide 71 text
USE AI TO BUILD BETTER
SOFTWARE FASTER BUT
ALWAYS WITH HUMAN
OVERSIGHT, CREATIVITY
AND PURPOSE GUIDING
THE WAY
QUESTIONS & DISCUSSION @ADDYOSMANI
Slide 72
Slide 72 text
THANK YOU & LEARN MORE
BEYOND VIBE
CODING
BUILDING WEB
APPS WITH BOLT
LEADING EFFECTIVE
ENG TEAMS
ADDYOSMANI.COM