How We Built Lastest to Be Self-Testing

We built Lastest to be self-testing: the same visual regression product our users run, pointed at our own staging cluster, after every deploy. We build it the same way our users build their products too, half plan, half vibe, mostly in a hurry. That worked for shipping the first version. It absolutely did not work for shipping the tenth version.

Every time we upgraded a base image, swapped a Helm chart, or let the agent refactor "just one little thing," our Kubernetes deploys would find a new and creative way to break. Sometimes the dashboard rendered fine but the diff worker silently dropped jobs. Sometimes a chart rolled out green and the API gateway 504'd under real traffic. Vibe-coding got us most of the way to a feature; it routinely got us nowhere near a stable release.

So we did the obvious thing in retrospect, and the unobvious thing at the time: we made Lastest self-testing by pointing Lastest at Lastest. This post is how that loop actually works, and what it caught that nothing else did. If you want the mechanics first, the whole thing runs on the same visual regression engine we ship.

The problem: vibe-coding is a feature factory, not a release engine

The pattern was always the same. Claude Code or Cursor would happily write the feature. The kubectl apply would happily roll. The pods would happily come up. And then, four hours later, a real user would happily file the bug we should have caught.

The breakages clustered into three buckets:

• Version-upgrade rot. Bumping Postgres, Playwright, or the Next.js minor would silently change a behavior that no unit test covered - but a real screenshot of a real page would have caught instantly.
• Cross-service drift. The frontend assumed v3 of a payload while the worker had been quietly upgraded to v4 by an agent two PRs ago.
• Load-only failures. Everything green at 10% utilization. At 80%+ CPU and queue saturation, races appeared, sockets dropped, and the diff worker would deadlock against itself.

Happy-path E2E tests caught roughly none of those. We had to build a test layer that ran on the same substrate the product runs on, against real load, after every deploy. And the cheapest way to do that turned out to be the product itself.

Step 1: expose Lastest to itself via MCP

Lastest already ships an MCP server with roughly 20 tools so coding agents like Cursor or Claude Code can drive the whole platform: run a build, read a diff, approve a baseline, ask which tests a change touches. We exposed that same surface to our internal coding agents and to a dedicated CI agent that has nothing to do but watch our own staging cluster.

The MCP server is the seam. It means the agent that writes a Lastest feature is talking to the same tool surface that tests Lastest. No translation layer, no flaky shell scripts wrapping kubectl exec. Just typed tool calls that succeed or fail loudly. (We wrote up the pattern in general terms in the MCP testing loop for solo founders.)

1. Agent edits services/diff-worker/src/butteraugli.rs
2. Agent asks Lastest which areas that file touches
   -> Smart Run returns: ["diff-engine", "review", "baseline-approval"]
3. Agent runs only those areas against the "main" baseline,
   under a deliberately squeezed load profile (~80% CPU)
4. Lastest replays 200+ recorded journeys against the staging cluster
   while a load generator holds CPU at >= 80%
5. Agent reads diffs, classifies, escalates only what it can't explain

Two details matter here. First, those replays are zero-token: the AI only runs when we create or fix a test, so a replay is plain Playwright execution we can fire thousands of times a day for nothing. Second, step 3 uses Lastest's Smart Run, which reads the git diff and runs only the tests the change actually touches. And we do not run our self-tests against an idle cluster. We run them against a cluster being deliberately squeezed.

Step 2: stop pretending happy-path tests are testing

The single biggest unlock was admitting that "click the button, see the page render" is not a test of a production system. It is a test of a demo.

Our self-test suite now stratifies into four layers, each with its own failure budget:

Layer	What it catches	Runs when
Happy-path replay	"Did we break the login page?"	Every PR
Visual diff at p50 load	Layout shifts, render regressions	Every PR
Diff at p80+ load	Race conditions, queue starvation, slow-network rendering	Pre-deploy + nightly
Chaos replay	Pod evictions mid-test, dropped Redis connections, partial 5xx	Nightly

The third row is where the interesting bugs live. A diff worker that handles 50 jobs/min cleanly will, at 500 jobs/min, sometimes hand back a screenshot from another tenant's run. We have caught that exact bug class several times now. None of those would have been visible at p50.

Step 3: catch the niche stuff - bugs that only appear above 80% load

This is the part that surprised us most. Once we were running visual regression against a deliberately overloaded staging cluster, we started finding bugs that no unit test, no integration test, and no LLM code review would ever flag:

• A WebSocket reconnect storm that only triggered when the API pod was at >85% CPU. The dashboard looked fine; the live-update indicator silently stopped updating. A visual diff caught the missing "live" badge.
• A baseline-approval race where two reviewers approving simultaneously under load produced two divergent baselines stored under the same hash. Caught by a structural diff against a known-good post-approval state.
• A Helm chart upgrade that flipped a default terminationGracePeriodSeconds from 30 to 5. At low load, nobody noticed. Under sustained load, in-flight diff jobs were killed mid-render and re-queued, producing flickery half-screenshots that our perceptual diff engine flagged as "novel."
• A Postgres minor-version bump that changed the default sort order on a query without an explicit ORDER BY. Tests passed. The dashboard's "recent runs" list silently rearranged. A pixel diff lit up like a Christmas tree.

None of these are happy-path bugs. None of these are bugs an agent writing a Playwright test from scratch would think to cover. All of them shipped to staging at some point, and all of them were caught before production because Lastest was watching itself the whole time.

Step 4: make version upgrades boring

The biggest cultural change was that version upgrades stopped being scary.

Before: bump Playwright minor, hold breath, deploy, watch Sentry, get paged at 2am, revert. After: bump Playwright minor, let the self-test suite replay 200 journeys at p80 load against a staging cluster running the new version, read the diff report, ship if green.

The agent does the boring part. It calls get_impact, runs the affected areas, classifies the diffs, and only escalates the ones where it can't decide. Most upgrades land with a one-line PR comment: "All 200 baselines stable under p80 load. Approving." The ones that don't are the ones where you actually want a human to look.

What it took to actually do this

We did not build any of this from scratch. The pieces were:

• The MCP server - already shipping in Lastest. We just pointed our internal agents at it.
• A load generator - k6 holding the staging cluster at a configurable utilization band while replays run.
• Three diff engines - pixel (Pixelmatch), structural (SSIM), and perceptual (Butteraugli) - because under load, "different" means different things at different layers. Pixelmatch lights up on raw byte changes, SSIM is layout-aware, and Butteraugli is human-eye aligned so it catches real rendering bugs while ignoring anti-aliasing and font noise.
• Branch-isolated baselines - so a feature branch can't corrupt main's known-good state, even if the agent is wrong about whether a change was intentional.
• One CI agent with nothing to do but watch staging, run the suite, and post diffs to the PR.

That's the entire stack. None of it is novel; the leverage came from connecting the pieces with MCP so an agent could drive them. The same self-hosted dashboard runs the whole loop, and screenshots never leave our network.

The shape of a self-testing product

The thing we keep coming back to is that this only works because the product's own tool surface is rich enough for an agent to use it without inventing anything. If our test loop required the agent to write Playwright from scratch each time, we'd be back to flaky selectors and 6-attempt token burns. Instead, the agent picks an area, fires a deterministic replay, and reads a structured diff. The agent is the conductor; Lastest is the orchestra. Including when the orchestra is playing a piece about itself.

We are not done. The next thing we want is replay-under-chaos as a first-class profile, where we deliberately drop a pod mid-replay and see whether the visual state recovers cleanly. That's a level of testing nobody asked us for, and it's the level we now expect from ourselves before we let anything roll to production.

If you want to make your own product self-testing the same way, start where we did. Self-host Lastest for free under FSL-1.1, point its MCP server at your coding agent, and run unlimited zero-token replays on your own infra. Prefer to skip the ops? Lastest Cloud is a flat $299/month with no per-seat or per-screenshot fees, so cranking your replay volume costs nothing extra. The full source, the MCP server, and the diff engines are open at github.com/las-team/lastest.

The point is not that you have to use Lastest specifically. The point is that "your product tests itself, on the same substrate it runs on, under real load, with an agent driving the loop" is achievable with off-the-shelf parts. We just needed a few bad deploys to figure that out. If you want the bigger picture of why this pattern is the difference between a pilot and a rollout, read the 2026 AI testing scale gap.