The Tests Passed and Production Still Went Down — How That Keeps Happening
AI writes the code and the tests, so both share the same blind spots. A real experiment, the bug it shipped, and how to catch the missing 5%.
Table of Contents
- Quick summary
- When AI writes the feature and AI tests the feature
- The “write more tests” experiment
- Our own experiment: two sprints with no human testing
- How failing tests got committed
- AI fixed the test, not the bug
- How we found it
- Why 5% does not stay 5%
- It does not show up on day one
- Credit limits make it worse
- What AI cannot see
- It never sees the whole project at once
- It does not know who else uses your code
- It does not notice what feels wrong
- Warning signs for founders and engineering leaders
- What we changed after the experiment
- Frequently asked questions
- Why do tests pass but production still breaks?
- Can AI write reliable unit tests?
- Should AI be allowed to fix failing tests?
- Is a pre-commit hook enough to stop bad code?
- How long before problems show up in a vibe-coded codebase?
- What is human-in-the-loop testing for AI-generated code?
- What does Regressionly do?
- Where Regressionly fits
- Related reading
- References
- About the authors
TL;DR: When AI writes both the code and the tests, the tests share the code’s blind spots, so they pass even when a feature is wrong. In our own experiment, AI rewrote a failing test to match a bug instead of fixing the bug, and it reached production with every test green. Small gaps like this add up over three to six months, and the fix is a human who verifies every feature before it ships. That is the job of Regressionly’s testing pods. To see it on your own product, start with a free two-week pilot.
Every test was green. The pipeline passed. The feature was merged and shipped. And a few weeks later, something in production was quietly wrong.
If you run an engineering team that builds with AI, you have probably seen some version of this. Maybe not a full outage. Maybe a slow page, a wrong number in a report, or data that should not be there. And when you go back to check, the tests still pass.
We at Regressionly see this every week, with our clients and in our own work. So we want to explain, in simple words, why it keeps happening.
Here is the short version. When AI writes the code, it does its best. It gets you maybe 90 to 95% of the way. That is impressive. But it still skips some paths, some scenarios, some small workflows. Then AI writes the tests too, and the tests have the same blind spots as the code.
So the tests pass. Not because everything is right, but because nobody tested the part that was missed.
The answer is not to stop using AI. It is to put a human back at the end of the line, before production. That is what we do at Regressionly. Your team keeps building fast with AI, and our testing pod checks every feature the way a real user and an experienced tester would. What ships is a human-verified feature, not just an AI-verified one.
In this article we will cover:
- Why AI-written tests can pass while the feature is still wrong
- A real experiment we ran on our own internal apps, and the bug it caused
- Why a small 5% gap turns into a big problem after a quarter
- What AI cannot see, even when it tries
- Warning signs for founders and VPs of Engineering
- What we changed, and what we recommend
Quick summary
| What we saw | Result |
|---|---|
| AI-written tests for one feature | 128 test cases, then 27 more, then 20 more, each time “everything covered” |
| Extra tests that covered real scenarios | 3 of the 47 extra tests |
| Tests at the end of our no-human-testing sprint | Around 2,000 unit tests, 15 of them failing |
| How AI “fixed” the failing delete test | It changed the test to match the soft-delete bug, not the code |
| Time until we noticed the bug in production | About a week |
| Chance every feature is right at 95% per feature | 95% after 1 feature, below 50% after 14, around 8% after 50 |
| When the gaps start to show in a vibe-coded codebase | In our experience, after about a quarter, three to six months in |
When AI writes the feature and AI tests the feature
Unit tests, integration tests and end-to-end UI tests exist to catch what the code missed. Today, many teams ask AI to write those tests too. We do it as well. It is a good technique and it saves a lot of time.
But think about what happens. The same AI, with the same understanding and the same context, writes the code and then writes the tests for that code. If it did not think of a scenario while writing the feature, there is a good chance it will not think of it while writing the test either.
So the tests check what the AI already believed was right. They do not check what it never thought about.
The “write more tests” experiment
Here is something we see every day. Ask AI to write unit tests for a feature and to “cover all the scenarios”. It will do it, and it will tell you everything is covered.
In one real example, AI wrote 128 test cases for a single feature. Then, without changing a single line of code, we asked it to write more, because we felt the coverage could be better. It added 27 more. We asked again. It added 20 more. We stopped there, but it would have kept going.
So AI said “everything is covered” three times, and each time it found more.
We went through those 47 extra tests. Most of them were the same scenario written a different way, with slightly different data. But 3 of them were real scenarios that the first 128 had not covered at all.
Three tests sounds small, about 2% on top of the original set. But those three were exactly the kind of gaps that become bugs in production. And nobody would have known, because the first test run was all green.
The lesson is simple. When AI says it has covered everything, it means it has covered everything it thought of. That is not the same thing.
Our own experiment: two sprints with no human testing
Our founder, Sunil Kumar, wanted to see what really happens when you trust AI completely. So we ran an experiment on our own internal applications, not on any client’s product.
We did it twice, each time for a one-week sprint. The rules were simple:
- AI builds the features on its own
- AI writes the unit tests, integration tests and UI tests
- AI writes as many tests as possible
- Tests must run before every commit, and we had a pre-commit hook for that
- No human testing of the features
At the end of the sprint we had around 2,000 unit tests. 15 of them were failing.
How failing tests got committed
The pre-commit hook was there, but it was not mandatory. Git lets you skip hooks with --no-verify, and AI used it.
What happened was very human. AI ran the tests before committing, saw some failures, and told the developer: these failing tests are not related to the feature you are working on, so you can commit now and fix them later. The developer trusted that, and the commit went in with --no-verify.
Any employee could make that mistake. It is not the big lesson here. The big lesson came next.
AI fixed the test, not the bug
At the start of the next sprint, we asked AI to fix those 15 failing tests so we would have a clean, all-green test suite. It did. Every test passed again.
But look at how it fixed one of them.
We had a feature to delete a row from a table, from the UI and from the database. Somewhere while building a different feature, AI had assumed that “delete” in our app means “soft delete”. In a soft delete, the row is not removed. It stays in the database with a flag like is_deleted set to true. Nobody asked for this. Nobody was asked about it. AI just assumed it and changed the behavior.
The original delete test expected the row to be gone from the database, so it failed. That failure was the test doing its job. It was telling us the truth.
When we asked AI to fix the failing test, it did not fix the code. It changed the test. The new test checked that is_deleted was set, instead of checking that the row was actually removed. The misunderstanding was now inside the test too, and the test was green.
So the bug went to production with all tests passing.
How we found it
From the user’s side, nothing looked wrong. The row disappeared from the UI, just as expected.
The table was a logs table. Entries kept coming in, and we used to review them and clear them from the UI. Now “clear” only marked them as deleted, so every log stayed in the database. Our production database kept growing in a way we did not expect. Within about a week we noticed the growth, traced it back and fixed it.
We were lucky. It was a logs table, and the growth was visible. Now imagine the same thing with user records. A customer asks you to delete their data, the screen says it is gone, and it is still sitting in your database. Under privacy laws like the GDPR’s right to erasure, that is not just a storage problem, it is a compliance problem. And there may be no growth chart to warn you.
This is a classic example. AI misunderstood something. Then, when asked to fix the failing test, it misunderstood it even further and rewrote the test to match the wrong behavior. The one thing that was warning us got switched off.
Why 5% does not stay 5%
If AI misses 5% on one feature, that sounds fine. 95% is a great score in most exams.
But software is not one feature. Picture your whole team vibe coding every day, and every person skipping a little bit every time. Those small misses do not cancel each other out. They add up.
Here is some simple math to show it. Say every feature has a 95% chance of being fully right. Assume, to keep it simple, that the misses are independent. Then:
- After 1 feature, the chance that everything is right is 95%
- After 14 features, it drops below 50%
- After 50 features, it is around 8%
Real life is not this clean, but the direction is the point. A team shipping several features a week passes 50 features very fast.
It does not show up on day one
The hard part is timing. These problems do not appear in a day, a week or even a month. In our experience, they start to show after about a quarter, three to six months in.
By then, the small things AI left out have piled up. Code you wrote a quarter ago stops working the way you thought. New code breaks old code. And when your team starts digging, the problem is very hard and slow to trace, even for experienced engineers, because the tests around it all pass.
Credit limits make it worse
There is another thing we see that teams do not talk about much. Many developers have a limited budget of AI credits per week or per month. When they run low, they switch to a cheaper, lower-end model so they can keep working until the budget resets.
A lower-end model still writes code, as well as it can. But it is not as careful as a stronger model, and not as careful as a good human developer. Nobody on the team gets a warning when this happens. The code still looks fine and the tests it writes still pass. The quality just quietly drops for those weeks.
What AI cannot see
Even a careful AI, on its best model, has limits that come from how it works. These are the gaps a human tester covers almost without thinking.
It never sees the whole project at once
You cannot give AI your whole project’s context in every prompt and every session. AI coding tools work inside a limited context window. They read the files they think are relevant and work from that.
Most of the time that is good enough. But a real product is connected. A user signs up, makes a payment, the system processes some data, adds records, deletes records. Everything is linked. A human on your team knows that flow by heart. AI tries to work it out from the files it opened, and it does not always get it right.
So the feature AI built works fine, and the tests AI wrote for it pass. But something related, in a different feature of the same workflow, breaks.
It does not know who else uses your code
Now take it one step further. Say AI writes a shared library, and that library is used in 10 different projects across your organization.
AI knows the library. It can make sure the library works well on its own. But unless someone tells it, it does not know that it should check how those 10 projects use it. A human tester can import the new version into each project, build them and test them. AI will not think of doing that by default.
As your number of projects grows, this gap grows with it.
It does not notice what feels wrong
Some problems are not in any test at all. A human tester might say, “This page has been loading slower for the last two or three days.” Nobody wrote a test for that. It comes from using the product every day and having a feel for how it should behave.
AI will not stop and think, “The feature I just built made this page slower,” unless someone asks it to check. That kind of judgment comes from human experience and intuition.
To be fair to AI: almost all of this can be handled with AI, if it is set up right. Good context files, clear standards, strict rules. But in most of the codebases we see, it is not set up right. Things are written loosely, tested loosely, and still sent to production.
Warning signs for founders and engineering leaders
If you are a founder, a Director or a VP of Engineering, you will not read every commit. But there are signals you can watch. These are the ones that showed up in our own experiment and that we look for at Regressionly.
- How many existing tests fail when a new feature is written. This is the best early signal. Those failures show you the real impact of the feature on the rest of the system. Every one of them needs a human to look at it, at the very least.
- Tests being edited to pass. When AI changes a test that was failing, ask: did the code change, or did the expectation change? If the expectation changed, a human must approve it.
- Failures called “not related”. If AI or a developer says failing tests are unrelated to the current work, treat that as something to check, not something to accept.
- Skipped checks. Any commit made with
--no-verify, or any branch where CI can be skipped, is a hole in your safety net. - Numbers that move without a reason. Database size, storage, error logs, page load times. If they change and nobody can explain why, something changed underneath.
- The three-to-six-month mark. If your team has been heavily vibe coding for a quarter or more with little human verification, assume some gaps have already piled up.
What we changed after the experiment
That experiment taught us more than any article could. This is what experience looks like. Here is how we work now, and what we recommend to teams building with AI.
- Tests run in CI, and CI cannot be skipped. A local pre-commit hook is useful, but anyone, including AI, can skip it with
--no-verify. So we run the same tests in CI and use branch protection to block merges when they fail. This now applies to every branch, including the dev branch. Before, dev could skip CI. Note that the soft-delete bug never went to production with failing tests. It went with passing tests that had been rewritten. CI alone would not have caught it, which is why the next rules matter. - AI may not change a test without human approval. AI can write code freely. But tests are the thing that tells you everything is right. If AI also rewrites the tests whenever they fail, there is no human in the loop at all. Any change to an existing test’s expectation must be reviewed by a person.
- Proper context for AI. We keep context files and written standards in each project, so AI works from the same rules every time instead of guessing, like deciding on its own that delete means soft delete.
- Guardrails for weird behavior. Every time we saw AI do something strange, we added a rule or a check for it.
- A human approves every feature. Even with unit tests, integration tests and UI tests in place, a human tests the AI-generated feature before it goes to production. It can be two people, one who understands the feature and one who reviews it. But in the end, the understanding and the testing are done by a human.
None of this means using less AI. We still use AI to write code and tests every day. It just means AI does not get to approve its own work.
Frequently asked questions
Why do tests pass but production still breaks?
Tests only check the scenarios someone thought of. When AI writes both the code and the tests, the same blind spots end up in both. So the tests pass, but the missed scenario, a related workflow or another project using the code can still break in production.
Can AI write reliable unit tests?
AI writes useful tests quickly, and we use it for that every day. But “cover all scenarios” is not a guarantee. In one of our runs, AI wrote 128 tests, then 47 more when asked twice. Three of those extra tests covered real scenarios the first set had missed.
Should AI be allowed to fix failing tests?
AI can propose a fix, but a human should approve any change to what a test expects. A failing test is often telling the truth. In our experiment, AI “fixed” a failing delete test by changing it to match a bug, and the bug reached production with all tests green.
Is a pre-commit hook enough to stop bad code?
No. Git lets anyone skip local hooks with --no-verify, including AI agents. Run the same tests in CI and use branch protection to block merges when they fail. Even then, CI only enforces the tests you have. It cannot tell you that a test was rewritten to pass.
How long before problems show up in a vibe-coded codebase?
In our experience, usually after a quarter, around three to six months of heavy AI coding with little human verification. Small misses pile up across features, and old code starts behaving in ways nobody expected.
What is human-in-the-loop testing for AI-generated code?
AI writes the code and the tests, and a human then verifies the feature before release. The human runs real user workflows, checks related features and dependent projects, reviews any test AI changed, and notices things no test covers, like a page getting slower.
What does Regressionly do?
Regressionly provides a testing pod: human testers who work alongside AI-driven development teams. They make sure every feature that reaches production has been verified by a human, not only by AI.
Where Regressionly fits
The fix for this problem is actually simple. Let AI write the feature. Let AI write the tests. Then, before it goes live, have a human cover and test all the expected scenarios.
The hard part is doing that every time, for every feature, while your team keeps shipping fast. That is what we do at Regressionly.
We give you a testing pod, a team of human testers who work alongside your AI-driven development. Their job is to make sure that what goes to production is a human-verified feature, not an AI-verified feature. They:
- Test the real user workflows end to end, not just the function that changed
- Check the related features and the other projects that depend on your code
- Review tests that AI has changed, and ask whether the code or the expectation moved
- Notice the things no test was written for, like a page that got slower this week
Your users trust that their payments, messages and data go through a tested app, not a hacked-together, vibe-coded one. If you have millions of users, that trust is your business.
AI will get you to 95%. It is fast, and it is good. 95% is right. The other 5% is where you need Regressionly.
If your team has been building with AI for a few months and you are starting to see the warning signs above, talk to us. We will help you find the 5% before your users do.
Related reading
- Code Review Is Not Testing, and AI Code Needs Both: real examples of reviewed code failing on its first run, and what good test evidence looks like
- Why AI-Generated Code Needs Human Testers: why the fix is a human in the loop, not more automation
- What Is Vibe Testing? A Practical Definition: what AI vibe testing is good at, and where it fails
- The Timezone Advantage: Why Overnight Regression Testing Works: how a team in India tests your builds while your engineers sleep
- Services · How it works · Pricing
References
- Git documentation: git commit, the
--no-verifyoption that skips the pre-commit hook - GDPR Article 17: Right to erasure
- GitHub Docs: About protected branches
About the authors
Regressionly provides human-in-the-loop testing pods for teams that build with AI: AI speed, human judgment, every release verified by a person.
Sunil Kumar is the founder of Regressionly. He ran the two-sprint experiment described in this article on Regressionly’s own internal applications.
Share Article
Read more by topic:
Related Articles
Why AI-Generated Code Needs Human Testers
AI-generated code can pass every test and still break the business. Six blind spots only humans catch, and the review rules managers and QA leads need.
Code Review Is Not Testing, and AI Code Needs Both
AI code can look right in review and still fail when it runs. Real examples, what review catches, what only testing proves, and how to get both.