In January, Anthropic published a randomized controlled trial in which 52 engineers learned an unfamiliar Python library. One group had an AI assistant that could write the solution for them. The other coded by hand. Afterwards, everyone took the same quiz on the library they had just used.
The AI-assisted group scored 50%. The hand-coding group scored 67%. The widest gap was in debugging, the skill you need when something breaks. And the AI group wasn't meaningfully faster: about two minutes quicker on average, a difference too small to be statistically significant (Anthropic).
That study was small, and its participants were mostly junior engineers learning something new, not seniors working in code they already knew. But it puts a number on something many engineering leaders have started to sense: a team can ship more code while understanding less of it.
The debt your dashboards can't see
Technical debt usually means code that's hard to change: duplication, missing tests, shortcuts. Tools can find a lot of that. Addy Osmani has called the newer problem comprehension debt: the gap between how much code exists in a system and how much of it anyone on the team actually understands (O'Reilly Radar).
That kind of debt doesn't show up on a static-analysis report. A function can be clean, tested and correctly typed, and still be one that nobody on the team can confidently change, because nobody reasoned through it. The usual delivery metrics won't catch it either. PRs merged, cycle time and lines shipped can all improve while understanding falls.
Engineers themselves aren't reliable judges of this. In METR's 2025 trial, experienced open-source developers took 19% longer on real tasks with AI tools allowed, yet afterwards believed the tools had sped them up by 20% (METR). METR has since said that result is out of date. Its later data points to a modest speedup, and METR calls that data weak evidence, partly because many developers no longer want to do their work without AI (METR, Feb 2026). The precise productivity number is contested. What holds up is that a team's own sense of how things are going is a poor gauge.
Meanwhile, adoption is no longer a question. In Stack Overflow's 2026 Developer Survey, published last week with more than 30,000 respondents, 66% of developers who use AI tools use coding agents, and 73% use AI assistants or agents daily. Only 6.6% would trust AI for important decisions (Stack Overflow). Most developers who use AI now use it daily, yet very few would trust it with an important decision.
Where it shows up first
Comprehension debt is hard to measure directly, and we haven't found a study that puts a number on it in a real production codebase. You can still see its effects.
Review gets skipped. Code review is where a team builds shared understanding of new code. In data from high-AI-adoption teams, Faros reports PRs 71.8% larger, 52% more files touched per PR, and 76.3% more PRs merged without review, alongside a 125.4% rise in monthly incidents (Faros AI). That is one vendor's data, and it doesn't give a baseline or sample size for those figures, so treat it as an early signal rather than an industry benchmark. The direction is the worrying part: larger changes, reviewed less often.
Security flaws get through. In 2025, Veracode tested more than 100 language models on 80 coding tasks and found AI-generated code introduced a security vulnerability in 45% of cases, rising past 70% for Java. Larger models did not do meaningfully better than smaller ones (Veracode). Georgia Tech researchers tracking public vulnerabilities traced back to AI coding tools counted 6 in January 2026, 15 in February and 35 in March. They describe their count as a floor, and private enterprise code isn't included (Cloud Security Alliance).
Scanners catch some of these flaws. What scanners can't do is make sure someone on the team understands the code well enough to fix the rest. In the Anthropic trial, debugging was exactly the skill that dropped most.
A composite scenario
Picture a mid-market team six months into heavy AI-assisted development. This is a composite of a common pattern, not a specific client.
Throughput looks excellent. The backlog is moving. Then an incident lands in a service that was mostly written with an agent over a few busy weeks. The engineer on call opens the code and finds it readable, tested and plausible. But no one remembers why it handles a particular edge case the way it does, the PR that introduced it was large and approved quickly, and the person who prompted it has moved to another project.
So the fix starts with someone working out what the code does, under pressure, during the incident. The team pays the comprehension cost it deferred, with interest, at the worst possible time.
The difference was how the tool was used
The most useful finding in the Anthropic study is the variation inside the AI group.
Participants who used the assistant to explain, by asking conceptual questions, requesting code with an explanation, or generating code and then questioning it, averaged 65% or more on the quiz. That is close to the hand-coding group. Participants who handed over all the code writing, drifted into full delegation, or had the AI debug for them averaged below 40% (Anthropic). The subgroups were small, so this is a direction, not a law. But it points the same way as everything else here.
The tool was the same for everyone. The outcome depended on how the work was structured around it. That is the same mechanism we wrote about in our first post: AI amplifies whatever engineering system it is dropped into. A team with real review and clear ownership gets more out of it. A team without those things ships faster toward code no one can maintain.
What to do about it, starting Monday
None of this means stopping AI-assisted development. It means building understanding back into the process on purpose, because it no longer happens as a side effect of writing the code.
Keep review real. Require review on AI-assisted changes and keep PRs small enough to review properly. Large PRs merged without review are the clearest warning sign in the Faros data.
Measure what velocity hides. Track incidents, rework and reverted changes next to throughput. If throughput rises while those rise too, the speed is borrowed.
Move security checks into the dev loop. Run static analysis, dependency scanning and secret detection on every commit, before merge, as the Cloud Security Alliance note recommends. Don't rely on a gate at the end of the pipeline.
Set norms for how engineers use AI. Ask it to explain as well as to produce. Have the person who prompted a change be able to walk a reviewer through it. That is the pattern that preserved comprehension in the trial.
Name an owner for every service. Someone should be able to say why the code works the way it does. If no one can, that is the debt, whatever the dashboard says.
These are cheap compared to a codebase that only the tool can work in.
We build AI-assisted engineering with structured review because we have seen what happens without it. If your team's velocity is up and its confidence in the codebase is down, let's talk.