Documentation / Wiki / Running Tests

Running Tests

Lastest supports two execution modes. Local Playwright execution on the host is no longer supported -- every test runs inside an Embedded Browser pod, even in local development.

Execution Modes

Mode How When
Embedded Browser (default) Browser runs in a containerized pod with CDP live streaming back to the UI. Provisioned dynamically into k3d locally, or into your cluster in production -- one EB per test. Default for all dev and prod runs
Remote Runner Tests dispatched to remote machines via the @lastest/runner npm package over HTTP polling Distributed execution, different OS/browsers, CI/CD

Both modes support both running and recording tests. EB pods are short-lived (one job, one EB) -- the provisioner pre-warms a small pool to keep latency low.

Triggering Builds

Builds can be triggered in multiple ways:

Manual

Click Run in the UI to trigger a build immediately. You can:

  • Run all tests in a repository
  • Run a specific test suite
  • Run individual tests
  • Use the Compose page to cherry-pick specific tests and versions

Webhook

When a PR is opened or updated:

  • GitHub webhooks trigger a build automatically
  • GitLab MR webhooks trigger a build automatically
  • Results are posted back as PR/MR comments

CI/CD

  • GitHub Action -- use the reusable action (see CI/CD Integration)
  • CLI Runner -- run pnpm test:visual in any CI pipeline

Schedule

  • Cron-based -- automate builds on preset schedules (daily, weekly, hourly) or custom cron expressions
  • See Scheduled Runs for details

Build Statuses

Status Meaning
passed All tests passed, no visual changes
safe_to_merge Tests passed, only approved/insignificant changes
review_required Visual changes detected that need review
blocked Unapproved meaningful changes present
failed One or more tests failed to execute

Smart Run

Run only tests affected by your code changes:

  1. Select a feature branch (not main/master)
  2. Lastest compares against the default branch via GitHub/GitLab API
  3. Tests are matched to changed files by URL patterns and code references
  4. Only affected tests run, skipping unchanged areas

This dramatically reduces test time for large suites while maintaining coverage.

Debug Mode

Step through test execution with live feedback:

  • See each step as it executes
  • View screenshots at each step
  • Identify exactly where failures occur

Headed Playback

Replay any test (or test version) with a visible browser window for debugging. Useful when a failure isn't reproducible in headless or you want to watch selectors resolve in real time. Available from the test detail page; honours the same setup/teardown sequence as a normal run.

Recording Verification

After a recording finishes, Lastest runs a sanity-check pass that resolves every captured selector on the page before saving the test. Bad selectors are surfaced immediately rather than at the next test run, dramatically reducing brittle-on-day-one failures.

Watermarked Share Links

Any build, run, or visual diff can be shared via a public link without requiring login. The shared view embeds a watermark and read-only controls so external reviewers (designers, PMs, customers) can see screenshots and approve/comment without an account on the instance.

Parallel Execution

Configure the maximum number of tests that run simultaneously:

  • Set in Settings Reference
  • Applies to the embedded-browser pool and remote runners
  • The EB pool size and warm-pool keep-alive are tuned via env (EB_LAUNCH_INTERVAL_MS etc.)
  • Each remote runner has its own parallel limit

Test Composition

Cherry-pick tests and pin specific test versions per build:

  1. Go to the Compose page
  2. Select which tests to include
  3. Optionally pin specific versions (override the latest version)
  4. Run the composed build

Branch Baseline Management

Lastest manages baselines per branch:

  • Fork: When a branch is created, baselines are forked from the parent branch
  • Merge: When a PR is merged, baselines merge back to the target branch
  • Promote: Promote test versions across branches
  • SHA256 carry-forward: Matching hashes mean instant pass without pixel comparison

Setup & Teardown

Setup Scripts

Run preparation steps before tests execute:

  • Playwright scripts -- browser-based setup (login, seed data via UI)
  • API scripts -- HTTP requests to seed data programmatically
  • Test-as-setup -- use another test as a setup step

Teardown Scripts

Clean up after tests complete:

  • Repository-default teardown sequence
  • Per-test overrides (skip default steps or add extra ones)
  • Teardown errors are non-blocking -- they won't fail your build