AI Automation

AI Slop Is Costing Businesses Real Money: The Documented Cases, and Why Most AI Projects Fail

By the AiVirex Team, AiVirex Innovations LLP 12 min read

AI slop is careless, unreviewed AI output shipped as finished work, and it is no longer a cosmetic problem. A tribunal made Air Canada pay for its chatbot inventing a refund policy. Deloitte refunded part of an AUD $440,000 government contract over fabricated citations. Zillow wrote down over $500 million on algorithmic home buying. RAND research puts the AI project failure rate above 80%, roughly double that of ordinary IT projects, and nearly every documented disaster traces back to the same root cause: nobody checked the output before it reached a customer, a court, or a balance sheet.

The problem nobody invoices for

The internet is full of AI experts who have never shipped anything that survived contact with reality

Something strange happened over the last three years. AI became the easiest expertise in the world to claim and the hardest to verify. Anyone can wire a language model into a chat widget in an afternoon, call it an AI transformation, and invoice for it. The gap between that and a system a business can actually stake its reputation on is enormous, and the businesses paying the price for that gap are rarely the vendors. They are the clients.

Gartner has a name for one flavor of this: agent washing, vendors rebranding ordinary chatbots and automation scripts as AI agents. Of the thousands of vendors calling themselves agentic AI companies, Gartner estimates only around 130 are building the real thing. The rest are selling packaging. And the results are now well documented enough that nobody gets to claim they were not warned. What follows is not speculation or fear of new technology. Every case below is on the public record, with sources linked at the end of this post.

The public record

When the chatbot speaks for the company, the company pays for what it says

Four documented cases of customer facing bots shipped without guardrails, and what each one cost.

CompanyWhat the AI didWhat it cost
Air CanadaWebsite chatbot invented a bereavement refund policy that did not exist. The airline argued in tribunal that the bot was a separate legal entity responsible for its own actions.Lost the case in February 2024, ordered to pay CAD $812.02, and became the global precedent that a company owns every word its bot says.
DPDAfter a system update, the delivery firm's support bot swore at a customer, wrote a poem about its own uselessness, and called DPD the worst delivery firm in the world.The AI chat was pulled the same day, January 2024, after screenshots went viral. The poem outlived the bot.
New York CityThe official MyCity business chatbot told landlords they could refuse housing voucher tenants and told employers they could take a cut of worker tips. Both are illegal in NYC.Years of documented false legal advice from a government source before the city finally shut it down in 2026.
Chevrolet dealershipA ChatGPT powered sales widget was prompt injected into agreeing to sell a $76,000 Tahoe for one dollar, cheerfully adding that it was a legally binding offer.The offer was not enforceable, but the dealership pulled the bot and became the canonical example of shipping an unguarded model straight to customers.

The common thread: none of these were model failures. They were review failures. Nobody adversarially tested what the bot would say before pointing it at the public.

Bigger bets, bigger craters

When AI replaces judgment instead of assisting it

The chatbot stories are embarrassing. The operational ones are expensive. Zillow bet its balance sheet on a pricing model, buying thousands of homes based on algorithmic estimates. When the market shifted in 2021, the model kept overpaying, and Zillow shut the entire division down, took write downs exceeding $500 million, and laid off around a quarter of its workforce. McDonald's spent three years piloting AI drive through ordering with IBM across more than a hundred locations, then ended the partnership in 2024 after viral videos of the system adding 260 Chicken McNuggets to an order and putting bacon on ice cream.

Klarna is the most instructive case because the company said the quiet part out loud, twice. In early 2024 it announced its AI assistant was doing the work of 700 customer service agents and froze hiring. By May 2025 the CEO was publicly reversing course and recruiting humans again, admitting that organizing around cost as the dominant factor produced lower quality service. And iTutorGroup paid $365,000 to settle the first EEOC lawsuit over AI hiring discrimination, after its recruiting software automatically rejected older applicants, more than 200 of them, based on age alone.

None of these companies lacked budget, talent, or access to the best models available. What they lacked was scope discipline: each one handed AI an entire judgment heavy job instead of the bounded, verifiable piece of it the technology could actually carry.

The professionals did it too

AI slop shipped as expert work, at expert prices

The most damaging category is not startups cutting corners. It is established firms billing professional rates for unreviewed model output.

WhoWhat shippedThe consequence
Deloitte AustraliaA 237 page government report, contract value AUD $440,000, containing fabricated academic references, a made up quote attributed to a court judgment, and citations to books that do not exist.A researcher caught it, Deloitte refunded the final contract installment in October 2025, and the revised report quietly disclosed that GPT 4o had been used in drafting.
Two New York lawyersA federal court brief citing multiple cases that ChatGPT invented, complete with fake quotes. When challenged, they asked ChatGPT if the cases were real and filed its reassurance.A $5,000 sanction in 2023, national embarrassment, and the start of a trend: a public database now tracks roughly 1,490 court decisions worldwide involving hallucinated citations.
CNETDozens of finance articles quietly written by an internal AI tool under a staff byline, including basic compound interest math done wrong.Corrections on 41 of 77 articles after an audit, a plagiarism admission, and a lasting downgrade to its reliability reputation.
Sports IllustratedProduct reviews bylined by writers who did not exist, with AI generated headshots and fabricated bios, supplied by a third party vendor.The content was deleted, the vendor dropped, and within weeks the CEO and three other executives were out.

Every one of these is an unreviewed output scandal, not an AI capability scandal. The tool did what tools do. The professionals skipped the part they were being paid for.

The macro picture

The failure rate is not anecdotal, it is measured

Independent research from four separate organizations converges on the same uncomfortable shape.

>80%
Of AI projects fail by some estimates, roughly twice the failure rate of ordinary IT projects, per RAND Corporation research based on interviews with 65 experienced data scientists and engineers
95%
Of enterprise generative AI pilots deliver no measurable profit and loss impact, per the MIT NANDA State of AI in Business 2025 study
42%
Of companies abandoned most of their AI initiatives in 2025, up from 17% a year earlier, per an S&P Global survey of over 1,000 enterprises
>40%
Of agentic AI projects will be canceled by the end of 2027 over cost, unclear value, or inadequate risk controls, per Gartner

The pattern

Five failures that keep repeating, case after case

Read enough of these disasters and the pattern stops being subtle. The same five mistakes account for nearly all of them.

01

Accountability does not outsource

The Air Canada ruling settled this: a company owns what its AI says, full stop. Deloitte still owed a human quality report no matter what tool drafted it. Blaming the model is not a defense any tribunal, regulator, or client has accepted.

02

Unreviewed output is the actual scandal

CNET, Sports Illustrated, Deloitte, and the sanctioned lawyers are all the same story. The model produced plausible text, and no qualified human checked it before it shipped. Human review is the difference between an incident and a lawsuit.

03

Cost first automation backfires

Klarna optimized for headcount reduction and got what its own CEO later called lower quality. Cutting the human out is not the goal of automation. Cutting the repetitive work out from under the human is.

04

No guardrails, no adversarial testing

DPD shipped a system update with no regression testing on the bot. The Chevy dealership had zero prompt injection hardening. NYC never evaluated its bot for legal accuracy before launch. Customer facing AI has to be attacked by its own builders before the public gets the chance.

05

Scope too wide to survive reality

McDonald's tried full order taking in noisy drive throughs. Zillow bet billions on one model's price predictions. The projects that survive start with a bounded, low blast radius task, prove themselves with evidence, and expand from there.

What the successful few do

The 5% that works looks nothing like the 95% that fails

The same research that documents the failure rate also documents the escape route. McKinsey tested 25 attributes and found workflow redesign has the single biggest effect on whether generative AI produces real earnings impact: the winners redesign the process around the tool instead of bolting the tool onto the old process. CEO level oversight of AI governance is the factor most correlated with bottom line impact at large companies, yet only 28% of organizations have it. And the MIT study found external partnerships with specialists succeed around 67% of the time, roughly double the success rate of internal builds, mostly because specialists have already made these mistakes on someone else's pilot.

RAND's prescription list is almost boring, which is exactly the point: pick the problem before the technology, invest in data quality up front, make sure leadership understands what AI cannot do, and commit past the pilot phase. Nothing in that list requires a frontier model. All of it requires discipline that the overnight AI expert selling a chat widget has no incentive to bring.

Where we stand on this

How we build AI systems that do not end up in posts like this one

We build AI systems for a living, so this post is not an argument against the technology. It is an argument against shipping it carelessly, and the discipline shows up concretely in our own work. The loan approval system we built holds 85% model accuracy and cut processing time by 40%, but the design decision we are most confident in is that no application is ever decided by the model alone. The AI recommends with a stated reason, a human verifies every single case, and the verified decisions feed back into the model. That is the exact opposite of the unreviewed output pattern running through every disaster above.

The same restraint applies to scope. The trading automation we built cut manual effort by 80% and lifted revenue 18%, and it still does not choose the strategy. The client supervises strategy, the system executes it flawlessly at a speed no human can match. When a client asks us for a customer facing bot, the first deliverable is not the bot, it is the boundary: what it is allowed to say, what it must refuse, what gets escalated to a human, and what happens when someone deliberately tries to break it. If a vendor cannot describe that boundary for the system they are proposing, they are selling you a future incident report.

Protect yourself

Six questions that expose an AI slop vendor in one meeting

None of these require technical knowledge to ask. All of them are hard to fake answers to.

1

Ask what the system will refuse to do

A real builder has thought hard about the boundary and can describe refusal behavior, escalation paths, and failure modes in detail. A slop vendor talks only about capabilities.

2

Ask who reviews the output, and when

If the answer implies the model output goes straight to customers, courts, or financial decisions with no human checkpoint, you are looking at the Air Canada pattern with your name on it.

3

Ask how it was tested adversarially

Someone on the build team should have actively tried to make the system swear, lie, hallucinate policy, or agree to sell a truck for a dollar. If nobody attacked it before launch, your customers will be the first to.

4

Ask what happens when the model is wrong

Not if. When. The honest answer includes monitoring, logging, a correction path, and who is accountable. The dishonest answer is that the model is very accurate.

5

Ask for the smallest version first

A trustworthy partner proposes a bounded first scope with measurable results before expansion. A vendor pushing the full transformation in phase one is selling the pattern behind the 80% failure rate.

6

Ask what the running costs look like at ten times the volume

API costs, review costs, and error handling costs all scale with usage. A proposal with no answer here is a proposal that was never modeled past the demo.

The lesson of every case in this post is the same one sentence: AI does not remove the need for judgment, it concentrates that need at the point of review. The businesses getting burned are not the ones using AI. They are the ones who stopped checking its work.

Most agentic AI projects right now are early stage experiments or proof of concepts that are mostly driven by hype and are often misapplied.

Anushree Verma, Senior Director Analyst, Gartner

The takeaway for buyers

Real AI engineering is not more expensive than slop, it just fails less

The projects in this post did not fail because their budgets were too small. Most of them had enormous budgets. They failed because the work was not engineered for the messy reality of production, and that discipline is available at every price point, including from smaller studios that charge a fraction of what the failed projects cost. What guardrails and proper scoping cost for your use case is a question with a specific answer, once you share the use case.

We build AI systems with the boring parts included: fallbacks, human review where it matters, and testing against the inputs real users actually produce. Tell us what you want AI to do in your business, and we will quote the version that survives contact with production, priced against the return it has to generate.

AI Agents & Automations

This is exactly what we build.

See how AiVirex approaches ai agents & automations, and what it looks like to work with us.

Explore AI Agents & Automations

FAQ

Questions, answered

What does AI slop actually mean?

AI slop is low quality, unreviewed AI output shipped as if it were finished, verified work: chatbots inventing policy, reports with fabricated citations, articles with wrong math under a human byline. The defining feature is not that AI was used, it is that nobody qualified checked the result before it reached a customer, a court, or the public.

Is a company legally responsible for what its AI chatbot says?

Yes. The Moffatt v. Air Canada tribunal ruling in February 2024 rejected the argument that a chatbot is a separate entity responsible for its own statements, and ordered the airline to honor what its bot promised. The precedent is widely cited: information from a bot is information from the company.

Why do most AI projects fail?

The research is consistent: misunderstanding the problem the AI is meant to solve, poor data quality, chasing technology instead of a business outcome, and scope far wider than the system can reliably carry. RAND puts the failure rate above 80%, roughly double ordinary IT projects, and almost none of it comes down to the models themselves.

How do I tell a real AI builder from an AI slop vendor?

Ask about boundaries instead of capabilities: what the system refuses to do, who reviews its output, how it was adversarially tested, and what happens when it is wrong. Real builders answer those in detail because they designed for them. Vendors who only talk about what the AI can do have usually never thought about what it should not.

Does this mean businesses should avoid AI automation?

No. The same studies documenting the failures show the successful minority earning real returns by scoping narrowly, redesigning the workflow around the tool, keeping humans at the review point, and partnering with specialists rather than improvising internally. The technology works. Careless implementation is what fails.

Sources

The research behind this post

Want help with this?

Tell us where you're stuck and we'll tell you what's actually possible, then scope it and give you a clear, tailored quote.