Skip to content

The Tests Passed and Production Still Went Down — How That Keeps Happening

AI writes the code and the tests, so both share the same blind spots. A real experiment, the bug it shipped, and how to catch the missing 5%.

• Regressionly Team
Title image: The tests passed. Production still went down. A CI window shows 2,000 tests passed, including a delete test edited by AI; after deploy, a production chart shows database size climbing because deleted rows are still there
Table of Contents

TL;DR: When AI writes both the code and the tests, the tests share the code’s blind spots, so they pass even when a feature is wrong. In our own experiment, AI rewrote a failing test to match a bug instead of fixing the bug, and it reached production with every test green. Small gaps like this add up over three to six months, and the fix is a human who verifies every feature before it ships. That is the job of Regressionly’s testing pods. To see it on your own product, start with a free two-week pilot.

Every test was green. The pipeline passed. The feature was merged and shipped. And a few weeks later, something in production was quietly wrong.

If you run an engineering team that builds with AI, you have probably seen some version of this. Maybe not a full outage. Maybe a slow page, a wrong number in a report, or data that should not be there. And when you go back to check, the tests still pass.

We at Regressionly see this every week, with our clients and in our own work. So we want to explain, in simple words, why it keeps happening.

Here is the short version. When AI writes the code, it does its best. It gets you maybe 90 to 95% of the way. That is impressive. But it still skips some paths, some scenarios, some small workflows. Then AI writes the tests too, and the tests have the same blind spots as the code.

So the tests pass. Not because everything is right, but because nobody tested the part that was missed.

The answer is not to stop using AI. It is to put a human back at the end of the line, before production. That is what we do at Regressionly. Your team keeps building fast with AI, and our testing pod checks every feature the way a real user and an experienced tester would. What ships is a human-verified feature, not just an AI-verified one.

In this article we will cover:

Quick summary

What we sawResult
AI-written tests for one feature128 test cases, then 27 more, then 20 more, each time “everything covered”
Extra tests that covered real scenarios3 of the 47 extra tests
Tests at the end of our no-human-testing sprintAround 2,000 unit tests, 15 of them failing
How AI “fixed” the failing delete testIt changed the test to match the soft-delete bug, not the code
Time until we noticed the bug in productionAbout a week
Chance every feature is right at 95% per feature95% after 1 feature, below 50% after 14, around 8% after 50
When the gaps start to show in a vibe-coded codebaseIn our experience, after about a quarter, three to six months in

When AI writes the feature and AI tests the feature

Unit tests, integration tests and end-to-end UI tests exist to catch what the code missed. Today, many teams ask AI to write those tests too. We do it as well. It is a good technique and it saves a lot of time.

But think about what happens. The same AI, with the same understanding and the same context, writes the code and then writes the tests for that code. If it did not think of a scenario while writing the feature, there is a good chance it will not think of it while writing the test either.

So the tests check what the AI already believed was right. They do not check what it never thought about.

The “write more tests” experiment

Here is something we see every day. Ask AI to write unit tests for a feature and to “cover all the scenarios”. It will do it, and it will tell you everything is covered.

In one real example, AI wrote 128 test cases for a single feature. Then, without changing a single line of code, we asked it to write more, because we felt the coverage could be better. It added 27 more. We asked again. It added 20 more. We stopped there, but it would have kept going.

So AI said “everything is covered” three times, and each time it found more.

We went through those 47 extra tests. Most of them were the same scenario written a different way, with slightly different data. But 3 of them were real scenarios that the first 128 had not covered at all.

Bar chart: 128 tests from prompt 1, 27 more from prompt 2, 20 more from prompt 3. Of the 47 extra tests, 44 repeated existing scenarios and 3 covered real gaps
AI said "all scenarios covered" three times. Of the 47 extra tests, only 3 covered real gaps.

Three tests sounds small, about 2% on top of the original set. But those three were exactly the kind of gaps that become bugs in production. And nobody would have known, because the first test run was all green.

The lesson is simple. When AI says it has covered everything, it means it has covered everything it thought of. That is not the same thing.

Our own experiment: two sprints with no human testing

Our founder, Sunil Kumar, wanted to see what really happens when you trust AI completely. So we ran an experiment on our own internal applications, not on any client’s product.

We did it twice, each time for a one-week sprint. The rules were simple:

  • AI builds the features on its own
  • AI writes the unit tests, integration tests and UI tests
  • AI writes as many tests as possible
  • Tests must run before every commit, and we had a pre-commit hook for that
  • No human testing of the features

At the end of the sprint we had around 2,000 unit tests. 15 of them were failing.

How failing tests got committed

The pre-commit hook was there, but it was not mandatory. Git lets you skip hooks with --no-verify, and AI used it.

What happened was very human. AI ran the tests before committing, saw some failures, and told the developer: these failing tests are not related to the feature you are working on, so you can commit now and fix them later. The developer trusted that, and the commit went in with --no-verify.

Any employee could make that mistake. It is not the big lesson here. The big lesson came next.

AI fixed the test, not the bug

At the start of the next sprint, we asked AI to fix those 15 failing tests so we would have a clean, all-green test suite. It did. Every test passed again.

But look at how it fixed one of them.

We had a feature to delete a row from a table, from the UI and from the database. Somewhere while building a different feature, AI had assumed that “delete” in our app means “soft delete”. In a soft delete, the row is not removed. It stays in the database with a flag like is_deleted set to true. Nobody asked for this. Nobody was asked about it. AI just assumed it and changed the behavior.

The original delete test expected the row to be gone from the database, so it failed. That failure was the test doing its job. It was telling us the truth.

When we asked AI to fix the failing test, it did not fix the code. It changed the test. The new test checked that is_deleted was set, instead of checking that the row was actually removed. The misunderstanding was now inside the test too, and the test was green.

So the bug went to production with all tests passing.

How we found it

From the user’s side, nothing looked wrong. The row disappeared from the UI, just as expected.

The table was a logs table. Entries kept coming in, and we used to review them and clear them from the UI. Now “clear” only marked them as deleted, so every log stayed in the database. Our production database kept growing in a way we did not expect. Within about a week we noticed the growth, traced it back and fixed it.

We were lucky. It was a logs table, and the growth was visible. Now imagine the same thing with user records. A customer asks you to delete their data, the screen says it is gone, and it is still sitting in your database. Under privacy laws like the GDPR’s right to erasure, that is not just a storage problem, it is a compliance problem. And there may be no growth chart to warn you.

This is a classic example. AI misunderstood something. Then, when asked to fix the failing test, it misunderstood it even further and rewrote the test to match the wrong behavior. The one thing that was warning us got switched off.

Seven steps of how a bug shipped with every test green, with step 4, AI rewriting the test, highlighted

Why 5% does not stay 5%

If AI misses 5% on one feature, that sounds fine. 95% is a great score in most exams.

But software is not one feature. Picture your whole team vibe coding every day, and every person skipping a little bit every time. Those small misses do not cancel each other out. They add up.

Here is some simple math to show it. Say every feature has a 95% chance of being fully right. Assume, to keep it simple, that the misses are independent. Then:

  • After 1 feature, the chance that everything is right is 95%
  • After 14 features, it drops below 50%
  • After 50 features, it is around 8%
Line chart of the chance every feature is fully right: 95% after 1 feature, 49% after 14, 8% after 50

Real life is not this clean, but the direction is the point. A team shipping several features a week passes 50 features very fast.

It does not show up on day one

The hard part is timing. These problems do not appear in a day, a week or even a month. In our experience, they start to show after about a quarter, three to six months in.

By then, the small things AI left out have piled up. Code you wrote a quarter ago stops working the way you thought. New code breaks old code. And when your team starts digging, the problem is very hard and slow to trace, even for experienced engineers, because the tests around it all pass.

Credit limits make it worse

There is another thing we see that teams do not talk about much. Many developers have a limited budget of AI credits per week or per month. When they run low, they switch to a cheaper, lower-end model so they can keep working until the budget resets.

A lower-end model still writes code, as well as it can. But it is not as careful as a stronger model, and not as careful as a good human developer. Nobody on the team gets a warning when this happens. The code still looks fine and the tests it writes still pass. The quality just quietly drops for those weeks.

What AI cannot see

Even a careful AI, on its best model, has limits that come from how it works. These are the gaps a human tester covers almost without thinking.

It never sees the whole project at once

You cannot give AI your whole project’s context in every prompt and every session. AI coding tools work inside a limited context window. They read the files they think are relevant and work from that.

Most of the time that is good enough. But a real product is connected. A user signs up, makes a payment, the system processes some data, adds records, deletes records. Everything is linked. A human on your team knows that flow by heart. AI tries to work it out from the files it opened, and it does not always get it right.

So the feature AI built works fine, and the tests AI wrote for it pass. But something related, in a different feature of the same workflow, breaks.

It does not know who else uses your code

Now take it one step further. Say AI writes a shared library, and that library is used in 10 different projects across your organization.

AI knows the library. It can make sure the library works well on its own. But unless someone tells it, it does not know that it should check how those 10 projects use it. A human tester can import the new version into each project, build them and test them. AI will not think of doing that by default.

As your number of projects grows, this gap grows with it.

It does not notice what feels wrong

Some problems are not in any test at all. A human tester might say, “This page has been loading slower for the last two or three days.” Nobody wrote a test for that. It comes from using the product every day and having a feel for how it should behave.

AI will not stop and think, “The feature I just built made this page slower,” unless someone asks it to check. That kind of judgment comes from human experience and intuition.

To be fair to AI: almost all of this can be handled with AI, if it is set up right. Good context files, clear standards, strict rules. But in most of the codebases we see, it is not set up right. Things are written loosely, tested loosely, and still sent to production.

Warning signs for founders and engineering leaders

If you are a founder, a Director or a VP of Engineering, you will not read every commit. But there are signals you can watch. These are the ones that showed up in our own experiment and that we look for at Regressionly.

  • How many existing tests fail when a new feature is written. This is the best early signal. Those failures show you the real impact of the feature on the rest of the system. Every one of them needs a human to look at it, at the very least.
  • Tests being edited to pass. When AI changes a test that was failing, ask: did the code change, or did the expectation change? If the expectation changed, a human must approve it.
  • Failures called “not related”. If AI or a developer says failing tests are unrelated to the current work, treat that as something to check, not something to accept.
  • Skipped checks. Any commit made with --no-verify, or any branch where CI can be skipped, is a hole in your safety net.
  • Numbers that move without a reason. Database size, storage, error logs, page load times. If they change and nobody can explain why, something changed underneath.
  • The three-to-six-month mark. If your team has been heavily vibe coding for a quarter or more with little human verification, assume some gaps have already piled up.

What we changed after the experiment

That experiment taught us more than any article could. This is what experience looks like. Here is how we work now, and what we recommend to teams building with AI.

  1. Tests run in CI, and CI cannot be skipped. A local pre-commit hook is useful, but anyone, including AI, can skip it with --no-verify. So we run the same tests in CI and use branch protection to block merges when they fail. This now applies to every branch, including the dev branch. Before, dev could skip CI. Note that the soft-delete bug never went to production with failing tests. It went with passing tests that had been rewritten. CI alone would not have caught it, which is why the next rules matter.
  2. AI may not change a test without human approval. AI can write code freely. But tests are the thing that tells you everything is right. If AI also rewrites the tests whenever they fail, there is no human in the loop at all. Any change to an existing test’s expectation must be reviewed by a person.
  3. Proper context for AI. We keep context files and written standards in each project, so AI works from the same rules every time instead of guessing, like deciding on its own that delete means soft delete.
  4. Guardrails for weird behavior. Every time we saw AI do something strange, we added a rule or a check for it.
  5. A human approves every feature. Even with unit tests, integration tests and UI tests in place, a human tests the AI-generated feature before it goes to production. It can be two people, one who understands the feature and one who reviews it. But in the end, the understanding and the testing are done by a human.
Flow: AI writes the code and tests, CI on every branch with branch protection, a human approves test changes and verifies the feature, then production

None of this means using less AI. We still use AI to write code and tests every day. It just means AI does not get to approve its own work.

Frequently asked questions

Why do tests pass but production still breaks?

Tests only check the scenarios someone thought of. When AI writes both the code and the tests, the same blind spots end up in both. So the tests pass, but the missed scenario, a related workflow or another project using the code can still break in production.

Can AI write reliable unit tests?

AI writes useful tests quickly, and we use it for that every day. But “cover all scenarios” is not a guarantee. In one of our runs, AI wrote 128 tests, then 47 more when asked twice. Three of those extra tests covered real scenarios the first set had missed.

Should AI be allowed to fix failing tests?

AI can propose a fix, but a human should approve any change to what a test expects. A failing test is often telling the truth. In our experiment, AI “fixed” a failing delete test by changing it to match a bug, and the bug reached production with all tests green.

Is a pre-commit hook enough to stop bad code?

No. Git lets anyone skip local hooks with --no-verify, including AI agents. Run the same tests in CI and use branch protection to block merges when they fail. Even then, CI only enforces the tests you have. It cannot tell you that a test was rewritten to pass.

How long before problems show up in a vibe-coded codebase?

In our experience, usually after a quarter, around three to six months of heavy AI coding with little human verification. Small misses pile up across features, and old code starts behaving in ways nobody expected.

What is human-in-the-loop testing for AI-generated code?

AI writes the code and the tests, and a human then verifies the feature before release. The human runs real user workflows, checks related features and dependent projects, reviews any test AI changed, and notices things no test covers, like a page getting slower.

What does Regressionly do?

Regressionly provides a testing pod: human testers who work alongside AI-driven development teams. They make sure every feature that reaches production has been verified by a human, not only by AI.

Where Regressionly fits

The fix for this problem is actually simple. Let AI write the feature. Let AI write the tests. Then, before it goes live, have a human cover and test all the expected scenarios.

The hard part is doing that every time, for every feature, while your team keeps shipping fast. That is what we do at Regressionly.

We give you a testing pod, a team of human testers who work alongside your AI-driven development. Their job is to make sure that what goes to production is a human-verified feature, not an AI-verified feature. They:

  • Test the real user workflows end to end, not just the function that changed
  • Check the related features and the other projects that depend on your code
  • Review tests that AI has changed, and ask whether the code or the expectation moved
  • Notice the things no test was written for, like a page that got slower this week

Your users trust that their payments, messages and data go through a tested app, not a hacked-together, vibe-coded one. If you have millions of users, that trust is your business.

AI will get you to 95%. It is fast, and it is good. 95% is right. The other 5% is where you need Regressionly.

If your team has been building with AI for a few months and you are starting to see the warning signs above, talk to us. We will help you find the 5% before your users do.

References

About the authors

Regressionly provides human-in-the-loop testing pods for teams that build with AI: AI speed, human judgment, every release verified by a person.

Sunil Kumar is the founder of Regressionly. He ran the two-sprint experiment described in this article on Regressionly’s own internal applications.

Share Article

Read more by topic: