Most Failed AI Projects Fail on Data, Not Model Choice

This one's speculative — a thought experiment about design principles, not a report on something I've built.

The easy thing to blame

When an AI system underperforms, the instinct is to swap the model — try a newer version, a different provider, a bigger context window. That's the easiest lever to pull, and it's very often not the actual problem, because most underperforming AI systems are being fed inconsistent, incomplete, or poorly structured data, and no model swap fixes a data problem.

Garbage in still applies, it just sounds fluent now

A traditional system fed bad data usually fails visibly — a null pointer, an obviously wrong number. A language model fed the same bad data produces a fluent, confident-sounding answer built on that bad foundation, which is a much more dangerous failure mode because it doesn't announce itself as a failure at all.

The audit nobody wants to do first

Before touching the model, the more useful question is almost always: is the data this system relies on actually consistent, current, and complete for the cases it's failing on. That audit is less interesting than trying a new model and produces less immediately visible progress, which is exactly why it tends to get skipped in favor of the more exciting lever.

Better data plumbing beats a better model more often than expected

Time spent fixing how data gets structured, validated, and kept current for an AI system to draw on typically produces a bigger accuracy improvement than time spent chasing the newest model release — a genuinely unglamorous finding, and one worth actually testing before assuming the model is the problem.

I'm Jesse Myers — Marine veteran, 32 years in enterprise IT, now building production AI systems. This site is where I write about what I've actually built, and occasionally about ideas I haven't built yet but think are worth taking seriously.