Most test automation tools are built for one operating mode: a product team that owns its application and treats a flaky verdict as an annoyance. A validation lead running Veeva Vault lives in a different mode. You do not own the application. Veeva ships three general releases a year and you absorb them across every country you run. Your evidence gets read by an inspector years later. And a verdict from a probabilistic model is not evidence at all.
That is why Lastest has a regulated profile, referred to internally as QS mode. It is not a separate product. It is the same self-hosted runner, the same recorder that emits plain Playwright into your git repository, the same three diff engines and nine check layers. What changes is a set of switches that a general-purpose tool gets wrong by default under GxP. This post walks through each switch, why it moves, and what it deliberately leaves alone.
Why the default profile is the wrong default under GxP
A general-purpose visual testing tool optimizes for speed of authoring and low friction. That produces three habits that are fine for a product team and unacceptable in a validated environment.
- The verdict can come from a model. Many tools now let a language model decide whether a screen "looks right" or whether a difference "matters". That is a probabilistic judgment. Run it twice and you can get two answers. Evidence that cannot be reproduced is not evidence.
- Data leaves the tenant. Screenshots of a Vault sandbox contain unapproved promotional claims, draft labeling, clinical documents, or submission content. A tool that uploads them to a vendor cloud for diffing creates a data-residency question your security team must answer before anyone records a single test.
- Visual first, text second. Pixel and perceptual comparison is the right primary layer for a marketing site. For an audit trail row, what matters is the literal string: who, what, when. A perceptual engine will happily pass a row whose signature meaning changed, because nothing "looks" different at a glance.
Where the verdict comes from
The most important switch is also the simplest to state: in the regulated profile, nothing probabilistic produces a verdict.
AI is still present, and still useful. When you record a flow in Vault PromoMats and the recorder emits Playwright, a model can tidy the selectors, name the steps, and propose which check layers to enforce. When a Vault release renames a button and a step breaks, a model can propose a repair. Both happen at authoring time, both produce a diff to a file in your repository, and both are reviewed and committed by a named person with a reason.
Replay is different. Replay is the Playwright code in your repo executing against your sandbox, capturing artifacts, and comparing them to a baseline with a deterministic engine. No model endpoint is called. The same code against the same state gives the same answer, which is the property that makes a pass/fail worth referencing from a change record.
This is also why artifact ownership matters. The tests are Playwright files in your git repository, not records in a vendor database. If Lastest disappeared tomorrow, your PQ evidence would still point at code you can read, run, and diff.
Switch by switch: general profile vs regulated profile
| Switch | General profile | Regulated profile |
|---|---|---|
| Verdict source | Deterministic replay; AI assists authoring and repair | Same engine, plus a hard rule the profile enforces: nothing probabilistic produces a verdict |
| Data egress | Depends on how you deploy | Nothing leaves the tenant without an authenticated identity |
| Text layer | Log | Enforce, so Part 11 manifestations are checked literally |
| DOM layer | Log | Enforce, so audit-trail rows and signature blocks are checked structurally |
| Gamification | Leaderboards, Beat the Bot, achievements | Removed |
| Edit history | Versioned | Versioned with a required reason on every edit, AI fix, and restore |
| Confirmed regressions | Reviewer decides what to file | File full-evidence issues |
| Run output | Diff report per run | Timestamped, machine-generated artifacts per step |
| Onboarding | Start from your own app | Segment fork seeds Veeva Vault and Salesforce release-regression suites and turns the profile on |
Everything else is unchanged: the three diff engines, the per-repo enforce / log / disable setting for each layer, per-layer approve, reject, or snooze on every step, and the docker compose deployment inside your network. The regulated profile changes defaults and removes options; it does not add a parallel engine you would have to assess separately.
Why text and DOM outrank pixels under Part 11
21 CFR Part 11 asks, among other things, for secure, computer-generated, time-stamped audit trails that record operator entries and actions, and for signed electronic records to show the printed name of the signer, the date and time of signing, and the meaning of the signature, such as review, approval, responsibility, or authorship. In Vault these manifestations appear on screen as text: an audit trail row, a signature block, a line stating who signed, when, and what the signature means.
A visual layer tells you the region looks the same. A text layer tells you the string is the same. Those are not the same claim. A font change in a Vault release can make a pixel diff fire on every signature block while the text is identical; an SSIM or perceptual engine tuned to ignore that font change can then miss a meaning label that changed from "Approved" to "Reviewed". In the regulated profile, text and DOM default to enforce so these manifestations are checked literally, and visual remains available for the layout question it answers well.
Practically, a reviewer can accept a visual drift caused by a Vault restyle while the text layer independently confirms the signature meaning did not move. The auth-bridge and seeded-data groundwork in the Vault CRM testing recipe carries over unchanged.
The switches that remove things
Gamification disappears. Leaderboards, Beat the Bot, and achievements help a product team learn a new tool. In a validated environment they are a liability: an inspector reading a leaderboard next to a PQ record will reasonably ask whether testers were rewarded for volume over care. So the profile removes them rather than hiding them behind a setting.
Anonymous access disappears. Nothing leaves the tenant without an authenticated identity. Share links, exports, and integrations all resolve to a named account. This is what gives the versioning switch its meaning: every edit, every AI-proposed fix that was accepted, and every restore of an earlier version carries who did it, when, and the reason they entered. A confirmed regression files an issue carrying the full evidence for the step.
What Lastest is not
Be precise about scope, because your assessor will be.
- Lastest is not an e-signature system. It does not capture signatures and makes no Part 11 claims about them.
- Lastest is not a Part 11 records system. Its artifacts are test evidence. The controlled record is whatever your QMS says it is, and the artifact is referenced from that record.
- Lastest is not a validation or QMS system. It does not hold your URS, your risk assessment, your traceability matrix, or your approvals.
What it is: a test runner that executes Playwright you own, inside your network, and produces timestamped, machine-generated evidence per step for change control you already have. That is a narrow claim on purpose. Narrow claims survive inspections.
What stays yours
GAMP 5 second edition and FDA's Computer Software Assurance thinking point in the same direction: apply critical thinking, scale assurance effort to the risk a function carries for patient safety, product quality, and data integrity, and let tools generate evidence rather than transcribing it by hand. Both also make clear that the regulated company keeps accountability. A tool can lower the cost of producing evidence. It cannot decide what needs assuring.
- URS and risk assessment. You decide which Vault functions are high-risk (signature meaning, audit trail, document state transitions, controlled-copy distribution) and which are not. That decision selects which flows get enforce layers and which get log.
- PQ scripts and acceptance criteria. A recorded flow becomes a candidate PQ step only once you have written down the expected result and why it is the expected result.
- SOPs. Who may approve a diff, how snooze is used, how long artifacts are retained, and how a filed issue enters your deviation process are procedural questions the profile cannot answer for you.
- Approvals. The regulated profile records who approved a comparison. Whether that person is qualified to approve it is your training record, not ours.
- Tool assessment. Lastest is itself a tool in your environment and you assess it like any other. The source is available, the deployment is yours, and the replay path has no model in it, which keeps that assessment short.
Three releases a year, many countries
Why a separate profile instead of simply being careful? Volume. A Vault general release lands three times a year. Multiply by the applications you run (CRM, PromoMats, Clinical, RIM), by country configurations, and by the flows your risk assessment marks as high. That is hundreds of comparisons per release, and the manual alternative is a room of people re-executing scripts against a go-live date.
The regulated profile makes each of those comparisons produce the same kind of object: a timestamped, per-step, per-layer result with artifacts you can reference from the release's change record. When the same suite runs against your EU, US, and Japan configurations, the differences between countries show up as diffs rather than as three separate spreadsheets. The Veeva Vault release-testing page covers how the seeded suites are structured; the full layer and engine reference is on the features page; the pharma and life sciences deployment itself is described at pharma.lastest.cloud.
Frequently asked
Does turning on the regulated profile make Lastest Part 11 compliant? No, and be wary of any tool that says yes about itself. Part 11 applies to your electronic records and signatures and to the controls around them. Lastest produces test evidence for a change record you control. The profile removes behaviors that would undermine that evidence; compliance remains a property of your systems and your procedures.
Can we use AI at all in the regulated profile? Yes, at authoring and repair time. A model can draft a test from a recording or propose a fix when a Vault release breaks a selector. Its output is a change to a Playwright file that a named person reviews and commits with a reason. It never grades a run, and replay calls no model endpoint.
Does the regulated profile run air-gapped? Yes. Lastest runs via docker compose inside your network and can operate with no outbound connection. If you want AI assistance at authoring time you point it at a model endpoint you control; without one, authoring is manual and replay is unchanged.
What does the onboarding fork actually seed? Choosing the pharma and life sciences segment seeds Veeva Vault and Salesforce release-regression suites and turns the regulated profile on. The seeds are Playwright in your repository, so you edit them like any other code. They are a starting structure for your own risk-based suite, not a PQ script, and they do not replace your URS or acceptance criteria.
Start here
If your evidence today is screenshots pasted into a spreadsheet, the useful first step is a short review of what your next Vault release actually has to prove. Book a release-readiness review and we will walk through your high-risk Vault flows, show how the regulated profile handles each one, and leave you with a suite structure you own.
The one-line version: the same runner, with the probabilistic parts kept out of the verdict, the data kept inside your tenant, the text checked literally, and the decisions kept where your SOPs already say they belong.