Beyond Copilot: How AI coding tools change productivity, quality and the developer role | BCN Data Nights @ Appodeal

Transcript

1. Beyond Copilot How AI coding tools change productivity, quality and the developer role Daniel Sogl @sogldaniel Consultant @ Thinktecture
2. About me Daniel Sogl Consultant @ Thinktecture AG MVP – Developer & Web Technologies Focus: Developer Productivity & Generative AI Socials: linktr.ee/daniel_sogl 2 Beyond Copilot How AI coding tools change productivity, quality and the developer role
3. A Quick Show of Hands Who in this room… 1. …used an AI coding assistant this morning? 2. …shipped AI-generated code to production in the last 7 days? 3. …let an autonomous agent open a PR for you? 3 Beyond Copilot How AI coding tools change productivity, quality and the developer role
4. So Here’s the Talk 5 Acts in 40 Minutes Act 1 — The Discussion Is Over AI-generated code is no longer a question of "if" Act 2 — The Productivity Paradox Why adoption ≠ success Act 3 — From Local to Cloud Async, team-based AI engineering Act 4 — From Coder to Product Engineer Why implementation becomes a commodity Act 5 — Communication is the new Craft Specs, intent, and what remains human 4 Beyond Copilot How AI coding tools change productivity, quality and the developer role
5. The Data Is Unambiguous We’re past the inflection point 90% of developers use AI at work DORA 2025/2026 · JetBrains AI Pulse Jan 2026 80% of new GitHub developers use Copilot in week one GitHub Octoverse 2025 73% of engineering teams use AI tools daily (up from 41% in 2025) Pragmatic Engineer Survey Feb 2026 · 15K devs 1M+ PRs authored by GitHub Copilot's coding agent in 5 months GitHub Octoverse 2025 · May–Sept 2025 5 Beyond Copilot How AI coding tools change productivity, quality and the developer role
6. A Generational Marker AUGUST 2025 TypeScript overtook Python. The biggest language reshuffle in over a decade — and it wasn't a question of preference. It was a question of which language AI agents produce more reliable code in. +66%TypeScript contributors YoY +48%Python contributors YoY Strong types → fewer AI hallucinations → developers vote with their feet. SOURCE — GITHUB OCTOVERSE 2025 6 Beyond Copilot How AI coding tools change productivity, quality and the developer role
7. But adoption ≠ success. 7 Beyond Copilot How AI coding tools change productivity, quality and the developer role
8. Act 2 The Productivity Paradox 8 Beyond Copilot How AI coding tools change productivity, quality and the developer role
9. THE TRUST GAP 84% use AI tools at work 29% actually trust them — down 11 points YoY We use it. We don't trust it. We use it anyway. 66% spend more time fixing "almost-right" AI code — and it's their #1 frustration. SOURCE — STACK OVERFLOW DEVELOPER SURVEY 2025 · 49K RESPONDENTS 9 Beyond Copilot How AI coding tools change productivity, quality and the developer role
10. THE STORY EVERY AI TALK CITES — AND HOW IT UPDATED JULY 2025 −19% Senior open-source devs slower with AI. They thought they were 20% faster. → FEBRUARY 2026 UPDATE −4% New cohort, 57 devs, 800+ tasks. CI: −15% to +9%. 30–50% of developers refused tasks they'd have to do without AI. METR's own words: "an unreliable signal of current productivity effects." — METR, February 2026 — selection bias killed the RCT design SOURCE — METR.ORG · 2025-07-10 & 2026-02-24 10 Beyond Copilot How AI coding tools change productivity, quality and the developer role
11. SECURITY — TWO YEARS. ZERO PROGRESS. 45% of AI-generated code ships with known security vulnerabilities — unchanged since 2024. APRIL 2026 · VULNPOCALYPSE Anthropic withholds Mythos Preview — too capable at finding vulnerabilities. SOURCE — VERACODE SPRING 2026 GENAI CODE SECURITY UPDATE 11 Beyond Copilot How AI coding tools change productivity, quality and the developer role
12. The Productivity Gap AI’s gains collapse where the real work lives GREENFIELD · SIMPLE TASKS 35–40% productivity gain new projects · clean slate · isolated tasks COMPLEX LEGACY CODE ≤10% productivity gain — or less existing systems · the 90% of real work AI is up to 4× more productive on greenfield than on code you maintain. METR's 2025 RCT confirms it: experienced devs on mature repos were 19% slower with AI. SOURCE — STANFORD SEP (CITED IN DORA 2026 ROI REPORT) · METR 2025 RCT 12 Beyond Copilot How AI coding tools change productivity, quality and the developer role
13. Technical Debt: The Compound Interest Problem 304K AI-authored commits across 6,275 repos · Jan 2024 – Oct 2025 110K surviving AI-introduced issues across tracked repos up from <500 in early 2025 24.2% of AI-introduced issues still alive at HEAD ≈ 1 in 4 ships and stays 37.25 surviving issues per 100 AI-authored commits 41.1% of AI-introduced security issues survive at HEAD worst class — vs. 30.3% bugs · 22.7% smells "AI-induced tech debt" — velocity skyrockets, the cost shows up weeks later, not in the PR. DORA 2026 ROI Report quantifies it: change failure rate jumps from 5% → 6% post-AI, eating into the productivity gains. Sources: arXiv 2603.28592 (Feb 2026) · DORA ROI of AI-Assisted Software Development 2026 13 Beyond Copilot How AI coding tools change productivity, quality and the developer role
14. DORA’s One-Sentence Diagnosis "AI's primary role in software development is that of an amplifier. It magnifies the strengths of high-performing organisations and the dysfunctions of struggling ones." Strong teams get stronger. Struggling teams get worse — faster. The bottleneck has shifted from writing code to everything around it. SOURCE — DORA 2025 STATE OF AI · REAFFIRMED IN DORA 2026 ROI REPORT · ~5,000 RESPONDENTS 14 Beyond Copilot How AI coding tools change productivity, quality and the developer role
15. THE DORA TRADE-OFF ↑Throughput +39% first-year ROI · ~8-month payback ↓Stability Change failure rate 5% → 6% post-AI WHAT SEPARATES WINNERS FROM VICTIMS Three capabilities turn AI velocity into AI value: 01 User-centric focus 02 Quality internal platforms 03 Small-batch work SOURCE — DORA ROI OF AI-ASSISTED SOFTWARE DEVELOPMENT 2026 · 2025 STATE OF AI 15 Beyond Copilot How AI coding tools change productivity, quality and the developer role
16. Act 3 From Local to Cloud 16 Beyond Copilot How AI coding tools change productivity, quality and the developer role
17. The 4-Year Shift Where does AI code actually come from? 2022 Autocomplete in IDE 2023 Chat side panel 2024 Agents in IDE 2025/26 Async Cloud Agents in your PR queue The interface to AI is no longer the cursor. It's the pull request. 17 Beyond Copilot How AI coding tools change productivity, quality and the developer role
18. The Cloud Agent Landscape (May 2026) Devin · Cognition Full async sandbox. Production at Goldman Sachs, Citi, Dell, Nubank. Valuation $10.2B → talks of $25B (Apr 2026). OpenAI Codex Cloud 4M+ weekly developers (Apr 2026) · 10× since Aug 2025. Post GPT-5.2-Codex. Bundled in ChatGPT Plus. GitHub Copilot Coding Agent Assign an issue → get a PR. 1M+ PRs in 5 months. CODEOWNERS, branch protection apply. Cursor Cloud Agents Cursor 3 (Apr 2026). Sandboxed VMs, up to 8 in parallel. 30% of Cursor's own merged PRs now from background agents. Also in the field: Claude Code (GitHub Action · ~4% of public GitHub commits Claude-authored) · Google Jules · Sourcegraph Amp · Tembo (multi-agent orchestrator). Sources: Vendor docs · Morph LLM State of AI Coding Agents 2026 · SemiAnalysis Feb 2026 18 Beyond Copilot How AI coding tools change productivity, quality and the developer role
19. DEVIN · COGNITION — FROM DEMO TO PRODUCTION VALUATION · 13 MONTHS $4B → $10.2B → $25B Mar 2025 · Sep 2025 · Apr 2026 (in talks) PRODUCTION AT FORTUNE-100 SCALE Goldman Sachs · Citi · Dell · Cisco · Nubank Nubank: 6M-line ETL monolith → 100K+ data classes migrated by parallel Devin sessions. An 18-month project shipped in weeks. THE HONEST CAVEAT Async is powerful. It's not autopilot. ~14% SWE-bench autonomous (Cognition) — up to 30% on narrower, well-scoped tasks $1,800 / yr median real spend — sticker says $20/mo, ACU billing decides Narrow + well-specified → ships. Ambiguous + cross-cutting → senior engineer reviewing every step. SOURCES — COGNITION.AI · SILICONANGLE APR 2026 · MORPH LLM 2026 BENCHMARKS 19 Beyond Copilot How AI coding tools change productivity, quality and the developer role
20. THE PR IS THE NEW INTERFACE YESTERDAY Author → TODAY Editor-in-Chief OLD LOOP open IDE · write · test · commit · push · PR · wait for review NEW LOOP write spec · assign agent · do other work · agent opens PR · you review · iterate · merge 20 Beyond Copilot How AI coding tools change productivity, quality and the developer role
21. MULTI-AGENT WORKFLOWS · FEBRUARY 2026 Running multiple AI tools side-by- side is the new normal. COMMON COMBINATION Claude Code · GitHub Copilot · one open- source CLI "Reached the stage of parallel agent psychosis where I've lost a whole feature — I know I had it yesterday, but I can't seem to find the branch or worktree or cloud instance…" — Simon Willison · creator of Datasette · co-creator of Django · Feb 2026 Practical: start with 2–3 agents max. Clear merge strategy. Use worktrees. SOURCE — SIMONWILLISON.NET/2026/FEB/19/RECOVERING-LOST-CODE 21 Beyond Copilot How AI coding tools change productivity, quality and the developer role
22. When code becomes commodity, what’s your edge? 22 Beyond Copilot How AI coding tools change productivity, quality and the developer role
23. Act 4 From Coder to Product Engineer 23 Beyond Copilot How AI coding tools change productivity, quality and the developer role
24. The Companies Already Moved Product Engineers, not "developers" Linear No traditional PMs. Product engineers own outcomes. PostHog Same playbook. Published the playbook. Vercel "Code-last" philosophy. Outcomes > commits. Stripe Early pioneer. High-ownership engineering. Shopify Product engineers shipping product, not features. incident.io JD: "outcomes and impact > exact implementation" Hacker News "Who's Hiring": "Product Engineer" mentions have become noticeably more common over the last two years. 24 Beyond Copilot How AI coding tools change productivity, quality and the developer role
25. In an AI-first era, product engineering is more important than ever. Dare I say — it's basically the only thing left. Lee Robinson · Cursor (prev. Vercel) PARAPHRASED — LEEROB.COM/PRODUCT-ENGINEERS, 2025 25 Beyond Copilot How AI coding tools change productivity, quality and the developer role
26. COMMODITY ↔ EDGE COMMODITY AI is good enough. Boilerplate · syntax CRUD endpoints Language translation Refactoring within known patterns Documentation drafts EDGE What only you can do. Specifying intent precisely Domain & user context Architecture & system design Code review at high signal Taste & product judgment Multi-agent orchestration Knowing when not to use AI. Vercel's Code-Last mantra: make it work → make it right → make it fast. In that order. 26 Beyond Copilot How AI coding tools change productivity, quality and the developer role
27. A COUNTER-INTUITIVE TAKE The riskiest seat in the room? Mid-career. Juniors Learn fast, no bad habits. Often surprisingly safe. Mid-Career Routine implementation evaporates first. Depth not yet earned. Seniors Context, taste, judgment now leveraged 10×. Simon Willison's reading. The audience-take: don't be a senior implementer. Be a junior product engineer. 27 Beyond Copilot How AI coding tools change productivity, quality and the developer role
28. Act 5 Communication is the new Craft 28 Beyond Copilot How AI coding tools change productivity, quality and the developer role
29. The Four Pillars of AI Coding Red Hat’s framework — and why most of us are stuck on pillar 1 Specs Explicit intent. Repeatable. Team- shareable. Skills Reusable agent capabilities. Composable. Agents Plan, execute, iterate. Autonomously. Most teams ship from Vibes. The wins are in Specs. Vibes Intuitive, conversational. Fast — until it isn't. Source: Red Hat Developer · "Vibes, specs, skills, and agents" · March 2026 29 Beyond Copilot How AI coding tools change productivity, quality and the developer role
30. SPEC-DRIVEN DEVELOPMENT — THE BIGGEST LEVERAGE UPGRADE OF 2026 Stop coding. Start specifying. "Vibe coding" hits a wall around a few hundred lines (community rule of thumb). Agents guess at unstated requirements. The fix: define behavior, constraints, and acceptance criteria — then let the agent implement against that contract. THE INVERSION YESTERDAY prompt → code → fix → re-prompt TODAY spec → generate → verify → merge "Once the spec is solid, AI agents become interchangeable. The speedup comes from alignment — not faster typing." SOURCE — MICROSOFT DEVELOPER BLOG · "DIVING INTO SPEC-DRIVEN DEVELOPMENT WITH GITHUB SPEC KIT" · SEPT 2025 30 Beyond Copilot How AI coding tools change productivity, quality and the developer role
31. THE WORKFLOW — FROM IDEA TO PR IN 4 PHASES 01 Specify Business context. Success criteria. The what. spec.md 02 Plan Architectural decisions. Tech stack. The how. plan.md 03 Tasks Decomposition. Testable units. The steps. tasks.md 04 Implement Agent under contract. You review. The PR. → Pull Request THE CONSTITUTION Immutable principles across every session — your persistent contract with the agents. CHECKPOINTS Cross-artifact consistency runs before implement — not after. SOURCE — GITHUB/SPEC-KIT · KIRO SPECS DOCUMENTATION 31 Beyond Copilot How AI coding tools change productivity, quality and the developer role
32. SDD IN PRACTICE — THREE LEVELS OF RIGOR · MAY 2026 LEVEL 1 Spec-first Persistent context for every session. No automation. Where most teams start. AGENTS.md CLAUDE.md .cursorrules LEVEL 2 — SWEET SPOT Spec-anchored Spec evolves with code. Slash commands, checkpoints, cross-artifact consistency. GitHub Spec Kit 93k★ · 30+ agents supported Kiro (AWS) Agentic IDE · EARS notation · concurrent tasks LEVEL 3 Spec-as-source Humans only edit specs — never generated code. Generated files marked DO NOT EDIT . Tessl Private beta · spec is the source Specifications are increasingly treated as code artifacts — versioned, reviewed, enforced. Where most teams sit today: between Level 1 and Level 2. SOURCE — "SPEC-DRIVEN DEVELOPMENT: FROM CODE TO CONTRACT" · ARXIV · FEB 2026 32 Beyond Copilot How AI coding tools change productivity, quality and the developer role
33. Two Rules Worth Stealing Simon Willison · creator of Datasette · co- creator of Django: "I won't commit code I couldn't explain to someone else." → Forces understanding. Kills hallucinated dependencies. Catches silent bugs. Addy Osmani · Director, Google Cloud AI: Beware "house of cards code". → Fragile AI output that collapses under scrutiny. Specs in workflows prevent it. 33 Beyond Copilot How AI coding tools change productivity, quality and the developer role
34. The Lethal Trifecta Why AI coding agents need new guardrails Private Data Agent reads your private repo, secrets, customer data Untrusted Content Issue from a stranger · web page fetched · external doc External Comms Agent can push, open PRs, send requests — exfiltrate All three together = an attacker can trick your agent into reading your private data and sending it to them. Real-world incident: a GitHub MCP integration combined all three — and exfiltrated private repo content via a malicious issue. Source: Simon Willison · creator of Datasette · co-creator of Django · "The lethal trifecta for AI agents", Jun 2025 34 Beyond Copilot How AI coding tools change productivity, quality and the developer role
35. WHAT STAYS HUMAN AI replaces tasks. Not people. The realistic risk isn't being replaced by AI. It's being out- competed by someone who uses it better. 35 Beyond Copilot How AI coding tools change productivity, quality and the developer role
36. So what do you do on Monday? 36 Beyond Copilot How AI coding tools change productivity, quality and the developer role
37. Three Concrete Things. Starting Monday. 1 Write your first real spec A CLAUDE.md , AGENTS.md , or .cursorrules for your most active repo. Treat it like onboarding for a new hire. 2 Run a cloud agent on one real backlog item Pick a Copilot/Codex/Devin/Jules task. Let it open the PR. Review like a senior would. Notice what you'd actually ship. 3 Stop calling yourself a "developer" For the next PR, start with the user problem — not the implementation. Force yourself into the product seat. 37 Beyond Copilot How AI coding tools change productivity, quality and the developer role
38. You won't be replaced by AI. You'll be replaced by the engineer next to you who learned to use it. 38 Beyond Copilot How AI coding tools change productivity, quality and the developer role
39. Thank you! Questions? linktr.ee/daniel_sogl thinktecture.com daniel.sogl@thinktecture.com Slides & socials