Back to Blog

Vibe Testing: The Missing Half of Vibe Coding

Vibe Testing: The Missing Half of Vibe Coding

You shipped it over the weekend. A side project, a landing page, a little SaaS thing. You barely wrote any code - you described what you wanted, the model wrote it, you nudged, it nudged back, and by Sunday night it was live.

Monday morning, a tiny feeling creeps in. It works. You know it works because you clicked around. But you have no idea what is going to break next week when you ask Claude to add one more thing.

That feeling is the vibe coder's moment of reckoning. You trusted vibes to build it. And now you need something more than vibes to keep it working. Vibe testing is the missing half: the same plain-language, click-through, AI-assisted loop that built the app, pointed at keeping it working instead of shipping it.

Two-column comparison. Left column (vibe coding, amber): describe in English, AI generates code, validate by vibes, every run is new. Right column (vibe testing, teal): click through once, describe in English, AI generates a Playwright test, replay deterministically forever. The takeaway is that vibe testing reuses the exact same loop but freezes the output instead of regenerating it.
Vibe testing is vibe coding's loop run backwards: same plain-language input, but the output is a frozen, deterministic test instead of fresh code.

What vibe coding actually is

Andrej Karpathy coined the term. You describe what you want in plain English, the AI writes the code, you validate by vibes. You run it, it looks right, you ship. The describe-and-generate loop is so fast it feels like cheating.

It feels magical because it is magical. A single developer can now do what a team did five years ago. This is not a fad, it is not going away, and the people dunking on it in 2024 are going to look the way people dunking on Stack Overflow looked in 2010.

But there is a part nobody talks about.

The part nobody talks about

Once the code exists, how do you know it still works tomorrow? Early studies of AI co-authored codebases keep landing on the same pattern: not necessarily more bugs in absolute terms, but more of the significant kind, the regressions that quietly ship to users. We dug into the mechanics in testing AI-written code and the edge-case problem. The short version: generated code is confident, plausible, and untested exactly where it matters.

The obvious answer is "ask the model to write tests." And here is the dirty secret: LLMs write tests where they are easiest, not where regressions hide. They test the happy path. They mock the interesting parts. They assert on things that cannot fail. They generate 400 lines of test code that proves almost nothing.

You do not want to un-vibe-code your way out of this. You do not want to sit down at 11pm and hand-write Playwright selectors and snapshot assertions for an app you barely touched the source of. That is the whole reason you vibe coded in the first place.

You want to vibe test.

What vibe testing is

Vibe testing is the same AI-assisted, plain-language, click-through approach that built the app, applied to validation instead of creation. You click through the flow once. You describe it in a sentence. The AI turns that into a durable test. Done.

The contrast matters. Vibe coding is generative - every run produces something new. Vibe testing is regenerative - it captures the current vibe and replays it, pixel for pixel, forever. One is a spark. The other is a freeze frame.

If vibe coding is "describe it and the model makes it," vibe testing is "do it once and the machine remembers what right looks like."

Why it has to be deterministic

Here is the key technical point. LLMs are non-deterministic. Tests have to be. You cannot build a safety net out of a material that changes shape every time you touch it.

The naive approach - "just ask the model to test it every run" - fails for three reasons. First, the model grades its own homework. Second, it takes a different path every run, so a green result on Monday and a green result on Tuesday are not the same green. Third, tokens compound: every CI run, every branch, every PR, forever. A test suite that costs money per execution is a test suite you will disable.

The answer is simple. Generate once with AI. Store as plain Playwright code. Replay forever for $0. That is the whole thesis behind Lastest for vibe coders: AI runs only when you create or fix a test, and every replay after that is ordinary Playwright execution with zero tokens. Run the suite a thousand times a day on every PR and the token bill stays flat at nothing. Self-hosted, the screenshots are unlimited too.

Left-to-right pipeline. Step 1: click through the flow once. Step 2: AI generates a resilient Playwright test (tokens spent here, once). Step 3: store as code in git. Step 4: replay in CI deterministically, marked zero tokens. A loop arrow returns from replay back to step 4 labelled 'forever, $0'. The takeaway is that AI cost is paid once at authoring time, never at replay time.
The cost shape of vibe testing: AI spends tokens once at authoring, then deterministic replays run forever for $0.

Why it has to be visual

DOM assertions pass when the UI is visibly broken. You have seen it. The button exists, has the right aria-label, the right text content, the right click handler - and it is rendered behind a modal, or two pixels off screen, or the wrong color on a background the designer swapped last night. Green tests, broken product. (And selectors break the other way too: Lastest's 7-layer self-healing selector fallback walks from data-testid down through role, text, and even OCR so a refactor doesn't turn your whole suite red overnight.)

Vibe coders care about the vibe - the look, the feel, the flow. That is what you were optimizing for when you prompted the model. So that is what your tests have to watch. Visual regression testing is vibe testing. Anything else is asserting on ghosts.

Lastest ships three diff engines - pixelmatch, SSIM, and Butteraugli - because "did this pixel change" and "would a human notice" are different questions, and you want the machine to answer both.

Why it has to have a human in the loop

The trap at the other end is full autonomy. An agent that records, runs, judges, and self-approves is just a more expensive version of the LLM grading its own homework. You end up with a suite that is always green because the thing deciding green is the same thing that drew the picture.

Vibe testing lives in the middle. AI has the speed. You still have the judgment. Lastest gives you exactly one human review seam: a diff shows up, you look at it for two seconds, and you render one of three verdicts - pass, fix, or regression. AI never approves its own output. That single seam is not a compromise; it is the whole point. Neither pure manual review nor pure autonomy is enough. The loop is.

Flow with one decision diamond. AI records and runs the test, a visual diff is produced, then a single human review seam renders one of three verdicts: pass (test stays green, teal), fix (update the baseline, amber), regression (block the PR, red). An annotation notes the AI never approves its own output. The takeaway is that vibe testing keeps exactly one human decision point between AI speed and a shippable verdict.
Vibe testing keeps one human seam: AI produces the diff, a person renders pass / fix / regression, and the AI never grades its own homework.

The practical setup

It is three steps. Self-host the stack on your own infra (open source, free forever). Open the recorder, click through one flow, say what it does in plain English. Wire the resulting test into your CI so every PR gets a visual diff before it merges. If you want one less thing to operate, Lastest Cloud is a flat $299/month with no per-seat and no per-screenshot fees.

That is it. No tokens burning on every test run, three diff engines (pixelmatch, SSIM, and Butteraugli) deciding what a human would actually notice, and a coding-agent-friendly MCP server with around 20 tools so Cursor or Claude Code can drive the whole thing. The code is on GitHub at las-team/lastest, and the vibe-coder-specific walkthrough lives at /for/vibe-coders.

The new way to keep it working

If vibe coding is the new way to write code, vibe testing is the new way to keep it working. You are not going back to hand-writing selectors any more than you are going back to hand-writing boilerplate. The tools caught up. The workflow caught up. The safety net had to catch up too.

Build it on vibes. Lock it in with vibe tests. Ship on Friday without the Monday dread.

Start vibe testing today. Self-host Lastest for free, forever (FSL-1.1, your infra, unlimited screenshots and replays), or skip the ops and try Lastest Cloud at a flat $299/month. The full source is on GitHub at las-team/lastest - star it, fork it, point it at the thing you shipped this weekend.