Why AI projects fail comes down to five root causes in RAND's 2024 study, built on interviews with 65 experienced data scientists and engineers: the problem is misunderstood, the data isn't there, the technology comes before the problem, the infrastructure to deploy is missing, or the task is too hard for AI. Most show up before a pilot starts.
Below: what the repeated failure figures measured, RAND's five causes in its own words, why pilots stall before production, and what to check first.
- RAND's 2024 report quotes "more than 80 percent of AI projects fail" as an estimate from elsewhere. RAND's own research is 65 interviews, not a failure count.
- RAND's five root causes: the wrong problem, not enough data, technology ahead of the problem, missing infrastructure, and problems too hard for AI.
- Pilots stall in the same places: no owner, no real data, no evaluation set, no exception path, no production plumbing.
- An AI readiness assessment checks those before the pilot starts. One honest answer is that the pilot shouldn't start yet.
What does the AI project failure rate actually measure?
Less than people assume. The most repeated figure appears in RAND's report The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed (RR-A2680-1, published 13 August 2024), but RAND didn't measure it. The report says "By some estimates, more than 80 percent of AI projects fail," and adds, "This is twice the already-high rate of failure in corporate information technology (IT) projects that do not involve AI." RAND cites the 80 percent estimate to a business article, "Want Your Company's AI Project to Succeed?"
What RAND did was ask. "We interviewed 65 experienced data scientists and engineers," each with "at least five years of experience building AI/ML models in industry or academia," and it calls the result an "exploratory analysis." One scope note matters now: "Projects that simply used pretrained LLMs (sometimes known as prompt engineering) were not included in the scope of this work." Many pilots today are exactly that kind.
Other figures circulate, some from reports behind a sign-up form. Before repeating one, answer three questions:
| Question | Why it matters |
|---|---|
| Who was counted or asked? | Practitioners' views are evidence about causes, not a failure rate |
| What did "fail" mean? | Abandoned, never deployed, over budget and no measurable return are different outcomes |
| What kind of AI was in scope? | RAND left out projects that only used pretrained LLMs |
Why do most AI projects fail? RAND's five root causes
RAND's interviewees named five root causes. In the report's words:
- The wrong problem. "Too often, trained AI models are deployed that have been optimized for the wrong metrics or do not fit into the overall business workflow and context."
- Not enough data. "Many AI projects fail because the organization lacks the necessary data to adequately train an effective AI model."
- Technology first. The organization "focuses more on using the latest and greatest technology than on solving real problems for its intended users."
- Missing infrastructure. Organizations "might not have adequate infrastructure to manage their data and deploy completed AI models."
- Too hard for AI. "The technology is applied to problems that are too difficult for AI to solve."
The first cause dominates. RAND says failures driven by business leadership's decisions and expectations "were far and away the most frequent causes of project failure," and "Eighty-four percent of our interviewees cited one or more of these root causes as the primary reason that AI projects would fail." Its advice on commitment is blunt: "leaders should be prepared to commit each product team to solving a specific problem for at least a year."
AI pilot to production: where pilots stall
An AI pilot program is a limited trial of an AI system on one real process, with real data and a small group of users, to decide whether it's worth running in production. A pilot can pass its demo and still stall before daily use. RAND's interviewees saw cases where "AI models could not be deployed from test environments to production environments because the production environments were incompatible with the requirements of the model."
| Failure cause | What it looks like in week 6 | What to check before starting |
|---|---|---|
| Unclear problem or owner | The demo works, and nobody can say what decision it changes | One process, one owner, a written definition of a useful result |
| Missing data | The pilot runs on an export; the live feed is "phase two" | Read access to the real source, and a sample of messy records |
| No evaluation set | Opinions about whether answers are good, but no scores | Past cases with known right answers, agreed with the owner |
| No exception path | The first wrong answer pauses everything | Where low-confidence and wrong cases go, and who reviews them |
| No production plumbing | It runs on one developer's laptop and API key | Service credentials, logging, monitoring, cost tracking, an on-call owner |
| Too hard for AI | Accuracy stalls below what the process needs | Whether rules, simpler automation or no change would do |
Production data often sits in an old system with no API; see connecting automation to legacy software. Evaluation sets matter most for assistants that answer from documents, such as an AI knowledge base. Every AI system gets some cases wrong, so the exception path is the approval step covered in AI agents for business.
What is an AI readiness assessment?
What is a readiness assessment, in general? A check, before committing money to a change, that the organization has what the change needs. An AI readiness assessment applies that to one process: is the problem defined and owned, can the data be reached, can results be scored against real cases, where will wrong answers go, can it run in production, and is AI the right tool at all.
Some processes need rules-based automation, not a model; our guide to business process automation services covers that choice, and automating data entry is a common case where it applies.
The AI implementation process, in order
- Pick one process and one owner, and write down what a useful result is.
- Get read access to the real data, messy records included.
- Build an evaluation set from past cases with known answers.
- Design the exception path before choosing a model.
- Pilot against the evaluation set, with real users, for a fixed period.
- Build the production plumbing, then roll out in stages, testing each change against the evaluation set.
Before the pilot: what the 30-day assessment covers
Every engagement starts here, before anyone proposes a build. The 30-day assessment is fixed in scope, at a price agreed before work starts. We spend 30 days inside your systems and process, with read-only access, under an NDA. Week 1: we agree the scope, with a go or no-go decision before any access is given. Week 2: NDA, read-only access, and we trace the workflows, data and dependencies. Week 3: senior engineers verify the findings in working sessions with your team and capture what isn't written down. Week 4: written findings and a readout to your decision makers.
The findings cover where the process stands, the practical options (including doing less than planned, or nothing), which AI and automation tools fit your stack and which would be hype for your case, where handoffs and approvals slow the work regardless of technology, and a phased roadmap with durations. You keep the findings. Your own team can build from them, or we build it as a separate project with its own scope and price. The findings stand on their own either way.
Pick a time to talk, and bring the process you're thinking of piloting. If a narrower check would answer your question, we'll say so.