How to measure AI developer productivity in 2025 | Nicole Forsgren

Can a team move faster without breaking everything?

What feels like speed sometimes masks fragility. AI can generate reams of code in minutes, but that burst of output is not the same as sustainable velocity. Nicole Forsgren, who built the modern playbook for measuring engineering performance, argues that the new challenge is less about getting machines to type and more about learning when to trust them — and how to redesign the work around that new reality.

Why developer experience matters more than ever

Think of developer experience as the operating system of your product org. When it hums, teams ship ideas that actually solve customer problems. When it creaks, all the best tools and incentives fail to create value. Nicole pushes back on the narrow obsession with raw output; she wants leaders to invest in flow, feedback loops, and the human context that turns speed into business results.

Flow, cognitive load, and the paradox of assistance

I was struck by how visceral her description of flow felt — the kind of deep concentration that produces elegant solutions. AI can both interrupt that flow and enable it. Some engineers, she says, have assembled intelligent toolchains that keep them in the strategic groove: sketch the architecture, dispatch parallel agents to prototype parts, then return and stitch a near-production artifact together. It looked almost orchestral — conductor plus ensemble — and it made me imagine a very different engineering day.

Measuring what matters (and ditching the vanity metrics)

Lines of code are now actively misleading. Nicole is blunt: a metric that can be gamed should be treated as suspect. She recommends taking a broader, outcome-focused view: speed to experiment, feature-to-customer timelines, cost savings from fewer failing tests. Traditional DORA metrics still help, but only when used with clear intent — especially as AI inserts new feedback loops at unexpected stages.

Trust becomes primary

One sentence kept resonating with me: trust is now front and center. Machine-generated code is non-deterministic. Teams must learn to evaluate hallucinations, reliability, and stylistic fit. That evaluation consumes time. The human labor of reading and validating AI output is becoming part of the work fabric. So the question shifts: how do you design workflows where review is efficient and meaningful?

Practical steps that actually move the needle

Nicole's recommendations were refreshingly pragmatic. Start by listening — literally talk to engineers and map the day-to-day friction. Ship a quick win. Use surveys to capture the frequency and pain of specific blockers. Then decide strategy, sell it to stakeholders, drive change at the right scale, and measure progress. It's a product-management mindset applied to the tooling and processes that shape developer lives.

• Quick wins can be process changes — sometimes the biggest gains need no new code.
• Telemetry and surveys provide a baseline; ask engineers to name their top three blockers.
• Attribution honesty matters — if AI and DevEx improvements both contributed, say so.

The organizational payoff and the J-curve

Nicole described the impact curve honestly: early wins can look huge, then plateau while teams build instrumentation and infrastructure, and then benefits compound. That J-curve is familiar to anyone who’s led transformation. The wins are real — cost savings, faster experiments, fewer blockers — but they require patience and communication so leadership understands what changes are happening and why.

When faster is actually worse

One sobering reminder: speed without strategy is shipping trash faster. The product decisions still matter. AI accelerates the mechanics, but human judgment must guide what gets built. That tension felt, to me, like the moral of the story: tools amplify intent. If intent is weak, speed becomes risk.

Seven steps toward a frictionless developer experience

Nicole outlines a seven-step blueprint: start with listening and mapping, get a quick win, build a data foundation, prioritize strategy, sell the plan, scale change thoughtfully, and evaluate progress. The approach treats developer experience as a product — with customers, M.V.P.s, and measurable outcomes — and that perspective felt both intuitive and powerful.

There’s a quiet hope under all of this: you don’t need to rip everything out to improve velocity. Small changes — a cleaner test suite, a simpler approval process, a thoughtful survey — can unlock surprising capacity. What lingers is the idea that the future of engineering work may not be coders versus machines, but conductors coaching an ensemble of AI assistants — and learning how to measure the music they make.