The most common reaction to a red build in 2026 is not “let me investigate” - it is “it’s probably just the tests.” The phrase is a reflex now. It is also a confession: engineering teams have quietly stopped believing what their CI tells them, loudly enough that you can measure it on Reddit.
This is a different conversation from the 40% test-maintenance tax. Maintenance is about hours. Trust is about belief. Hours are recoverable. Belief compounds, and once it’s gone, the team behaves the same way whether the suite is right or wrong.
What Reddit is telling us
TestResults.io ran a six-month sentiment analysis across 4,180 r/softwaretesting comments referencing CI reliability between November 2025 and May 2026.
Three of the top four recurring phrases describe the same behaviour from different angles: developers giving up. “Disabled the test to ship” appeared in 22% of negative comments. “Quarantine pile is huge” in 16%. “Stopped trusting the suite” in 13%. These are not complaints. They are actions - what teams did to restore flow when CI got in the way. The cheapest path back to green is to delete the signal, not investigate it.
The math of a 5% flake rate
The Reddit conversation maps onto a number that’s easy to compute. A mid-size webapp suite: 400 tests, twenty-minute runs, twenty runs a day. Apply a flake rate of 5% - on the optimistic side of what most teams measure.
The compute is the visible piece. The invisible piece is what 20 false failures per run does to the human reading them. The first one gets investigated. The fifth gets a retry click. The fifteenth gets disabled. By the twentieth, the reviewer isn’t looking - they’re batching, and the batch reaction is “it’s probably just the tests.”
Google’s internal study put 16% of their CI failures in the flaky bucket. That number is, if anything, conservative for teams without Google’s stabilization infrastructure.
Most CI failures are not what you think
Split a quarter’s failed runs into a 2x2 by determinism (does it reproduce?) and subject (the code, or the apparatus?). Only one quadrant earns the human attention it currently gets.
Roughly 22% of CI failures are the kind of regression CI was built to catch. The remaining 78% are noise: flake (16%), environment (~28%), and maintenance (~34%). The trust collapse on Reddit is a rational response to noise eating signal at this ratio. If three out of four red builds are not about the code, the team’s prior on the next red build is “noise” - and the team is statistically correct.
Why “self-healing” sometimes makes flake worse
A careful note, because we ship a product near this space.
The current wave of AI-assisted testing tools includes a feature called self-healing, auto-repair, or smart selectors. The pitch: when button.submit-btn becomes button.checkout-cta, the AI finds “the submit button” and updates the selector. Flake drops, dashboard goes green.
The risk is structural. The AI matches on intent (“find the submit button”) against a DOM that may have multiple buttons that look like submit buttons. When it picks the wrong one, the test passes against the wrong element. You have converted a noisy false negative into a silent false positive - strictly worse, because false positives don’t generate red builds. They just ship.
We’ve seen this on our own dogfood: a healer “fixed” a broken checkout test by re-binding it to a coupon-modal button. The suite went green for eleven days. A customer reported the checkout bug on day twelve.
Visual diffs sidestep this. A visual diff compares what was rendered to what was rendered last time, not what a selector matched. If the wrong element is on the page, the diff is red. The comparison is deterministic in a way selector-based assertions structurally cannot be. This is not an argument that visual diffs replace functional assertions - they don’t. It is an argument that one category of test can rebuild trust because its failure mode is honest.
The trust curve and the quarantine cliff
Plot developer trust against measured flake rate, and the curve isn’t linear. It bends sharply around 8% - the point where teams start disabling tests faster than they fix them.
The teal dashed line is what the same curve looks like for visual diffs against stable baselines. It doesn’t collapse at the same rate, because the failure mode is bounded: a visual diff is either a real pixel change or a baseline drift, both of which a human can verdict in under a minute. The trust collapse is fundamentally about unbounded investigation cost.
The seam still needs a human
Visual diffs alone do not close the loop. They produce honest failures, which is necessary but not sufficient. The remaining question - pass, miss, or regression - still needs a human verdict, because the AI can compare rendered outputs but cannot yet judge whether the output meets the business goal of the change.
We wrote about the shape of that loop in Human-in-the-Loop. Orchestrator runs the suite, three diff engines sort the queue, human renders one of three verdicts, next ticket files itself. Three minutes per merge. The seam is what makes the rest trustworthy.
What this means tomorrow morning
- Measure your suite’s actual flake rate. If you don’t know it, you are almost certainly past 5%. Failed-then-passed-on-retry divided by total runs, last two weeks.
- Audit your quarantine list. Every quarantined test is a paid-for assertion you are no longer running. Longer than a sprint to fix? You are at the cliff.
- Ask the team to grade trust on a 0-100 scale. Anonymous, one question. If the median is below 50, you have a trust tax, and no amount of new tests fixes it - the existing ones have to start failing honestly first.
Lastest is open source, runs in CI in about three minutes per hundred specs, and ships with the three-engine diff (pixel, structural, perceptual) the seam needs.
More for builders like you
- Visual testing for solo founders
- Visual testing for SaaS MVPs
- Visual testing for devs doing QA
- Free selector checker for visual tests