The 2025 Stack Overflow Developer Survey asked more than 49,000 developers across 177 countries what frustrates them about AI tools. The top answer, at 66%, was not that AI is useless or that it refuses to help. It was that the solutions arrive close to correct without ever landing. The second, at 45%, followed directly from the first: debugging AI-generated code eats more time than it saves.

Over the same period trust fell. Stack Overflow found favourable sentiment towards AI tools dropping from over 70% in 2023 and 2024 to 60% in 2025, and the survey team noted that 75% of developers would still go and ask another human the moment they stop trusting the answer they were given.

Adoption kept climbing anyway. We are using tools more while believing them less, and I do not think that gap closes by waiting for a better model.

The uncomfortable study, and the honest footnote

The most cited result in this argument is METR's randomised controlled trial from July 2025. Sixteen experienced open-source developers, 246 real tasks on repositories they knew well. Developers using AI tools took 19% longer. Afterwards, those same developers estimated AI had made them roughly 20% faster.

That perception gap is the finding worth carrying, and it is the one people usually stop at. So here is the part that gets left out. METR revisited the work in February 2026 and changed the study design, because selection effects had made the follow-up numbers hard to interpret. Developers who benefited most from AI were reluctant to enrol in trials where they might be assigned to work without it. METR now labels the original result as historical.

I am including that because a blog post that quotes the 19% and omits the retraction is doing exactly what this post is about: presenting a confident conclusion built on evidence it did not fully read.

What survives the caveat is the perception gap. In that trial, how fast developers felt and how fast they actually were pointed in opposite directions. That is a claim about human judgement, and no revision to the productivity estimate touches it.

A near miss is what thin evidence produces

Here is the reframe I have come round to. When an AI answer is confidently, subtly wrong, the reasoning is usually fine. The evidence was thin, and the model filled the gap with the most plausible thing rather than the true thing.

Think about what you actually hand over when you paste a log snippet into a chat window. You hand over some lines of text. You do not hand over what deployed forty minutes earlier, what the memory limit on that pod is, what the connection pool is configured to, whether this is the first occurrence or the four-thousandth, or which of your services calls the one that broke. None of that is in the snippet. It is not that the model overlooked it. It was never there.

An answer built on that will be structurally sound and specifically wrong. Which is precisely what 66% of developers reported.

The distinction a snippet cannot make

Rather than argue this in the abstract, here is something from our own instance that most tools do not do and most engineers do not notice is missing.

I asked Vörr for the application logs on one of our deployment jobs. It came back empty. But it did not simply say empty. It reported that it had queried the Loki datasource with both the service and the app selectors, across a thirty-day window, and described the outcome as a verified zero rather than a mapping gap.

That distinction is not cosmetic, and it is the single most underrated property a grounded read has. "There are no errors" and "I could not find any errors" lead to opposite decisions, and they look identical in a pasted screenshot of an empty log view. An empty result is one of the most dangerous inputs you can hand an assistant, because a model given silence will reason from it as though it were evidence of health. It will tell you the service looks fine. It will sound entirely reasonable doing so.

The failure modes behind an empty log panel are not rare. The label selector does not match how that service actually tags its streams. The retention window closed before you looked. The datasource is connected but that particular service was never mapped to it. Every one of those produces the same blank rectangle, and none of them means what a blank rectangle appears to mean.

A grounded read can tell you which kind of nothing you are looking at. A snippet cannot, and neither can you.

Three questions before you paste

I am not arguing against using AI for debugging. I use it constantly. I am arguing that the input deserves the scrutiny we currently reserve for the output.

Before pasting, ask whether the snippet contains the answer or merely the symptom. A stack trace tells you where the process gave up, which is frequently nowhere near where it went wrong.

Ask what a colleague would need that you have not included. If a competent engineer looking at those same lines would say "when did this start, and what shipped before it", the model needs that too, and it will not say so. It will simply guess.

Ask whether an empty result means nothing happened or nothing was looked at. If you cannot tell, the assistant certainly cannot.

The industry framing of this problem has been that models hallucinate and we must therefore verify their output. That is true and insufficient. A large share of what we call hallucination is a model reasoning competently over evidence too thin to support a conclusion, and being obliging enough not to mention it. Fix the evidence and a surprising amount of the near-miss problem stops being a model problem at all.

Author note

I am Mohan Gopi, an Associate DevOps Engineer at Frigga Cloud Labs, where I work on Vörr, connecting code, cloud, and monitoring so that AI answers from the state of a system rather than from a guess. I wrote this because the debate about AI reliability has focused almost entirely on the model and almost not at all on what we hand it. The pattern I keep seeing is teams treating a bad answer as proof the tool is not ready, or accepting a good-sounding answer because it arrived formatted and confident. Both skip the same question, which is what evidence the answer was built on. Checked honestly, the input will usually tell you why the output was wrong, long before you go shopping for a better model. Left unchecked, you will keep receiving answers that are structurally sound, entirely plausible, and about the wrong machine.

Let us connect on LinkedIn → Mohan Gopi

Stop Pasting Logs Into Your AI