Visual Regression Testing

Visual regression testing catches the visual bugs your assertion-based tests will miss — a button that's still there but two pixels off, a font that loaded weirdly, a hidden CSS rule that broke layout. AegisRunner does pixel-level screenshot comparisons using pixelmatch and surfaces a 3-panel diff for every change.

Turning it on

Visual regression isn't on by default — it adds noise and storage to runs that don't need it. To enable it:

1. Open the suite or run modal.
2. Toggle Visual regression on before starting the run.
3. Click Start Run. Screenshots are captured at every step.

To make it the default for a project, enable it under Project Config → AI Config → Default visual regression.

How comparison works

1. First run with visual regression on — screenshots are saved as the baseline for each step.
2. Subsequent runs — new screenshots are diffed against the baseline pixel by pixel.
3. Differences are highlighted — changed pixels show in red on the diff view.
4. You decide — accept the new screenshot (becomes the new baseline), or reject (it's a regression).

The 3-panel diff

When differences are detected, the run page shows them side by side:

Panel	What's shown
Baseline	The accepted reference screenshot.
Current	The new screenshot from this run.
Diff	Pixels that changed, highlighted in red.

Each step in a test gets its own diff, so a multi-step flow shows you exactly where the change is — not just "something broke".

Accept or reject

Action	What happens
Accept	The new screenshot becomes the baseline. Future runs diff against this.
Reject	The diff is logged as a visual regression. The baseline is unchanged. The run is marked failed.
Accept all	If you intentionally redesigned the whole page, accept all diffs from one run in a single click.

Accept/reject is captured in the audit log (Pro and above) so you can see who approved which baseline.

Per-step screenshots

Baselines aren't taken once per page — they're captured at every test step. A login flow has separate baselines for:

• Initial page load
• Email field filled
• Password field filled
• Submit button clicked, success state visible

Each step's screenshot is diffed independently. You can accept the post-fill screenshots while rejecting the success-state screenshot, for example.

Threshold and noise control

Tiny rendering differences (anti-aliasing, font hinting, sub-pixel placement) would otherwise produce false positives on every run. AegisRunner ignores differences below 10% of total pixels by default — generous, but cheaper than chasing meaningless diffs. Tune lower if your suite is stable enough to demand a tighter check.

Mobile and device-specific baselines

Baselines are scoped to the browser and the viewport. A test running on mobile Chrome has a different baseline than the same test on desktop Chrome. This avoids false-positive failures when a test runs on a viewport you didn't seed.

Tests generated from a mobile-device scan automatically use mobile viewport, so their first run with visual regression on creates the mobile baseline.

Combining with baseline replay

Visual regression is most reliable when paired with Baseline Replays. Why:

• Free-form scans aren't deterministic. Different runs reach different states, so screenshots diverge for reasons that aren't bugs.
• Baseline replay walks the exact same path every time, so the only thing that changes between runs is your code.

Recommended setup for CI: run baseline replay with visual regression on. The combination gives you a deterministic fail signal — the test path is fixed, and any pixel diff is a real change.

Storage and retention

Screenshots take space. AegisRunner keeps:

• Current baselines — kept indefinitely (they're the reference).
• Run-level screenshots — retained according to your plan: Free 7 days, Starter 30, Pro 90, Business 1 year, Enterprise unlimited.
• Diff images — same retention as run-level screenshots.

If you need longer retention for legal or audit reasons, upgrade to a higher plan or use the Playwright export to keep your own copies.

Common questions

Why is every step showing as a diff?

You're probably running on a different browser or viewport than the baseline was captured on. Check the run's browser and viewport, and if it's intentional, accept the diffs to seed the new baselines.

Can I view diffs after the run finishes?

Yes — go to the run page, open any test, click on a step. Diffs are available for the full retention window of your plan.

What about font rendering on Linux vs Mac?

Visual regression runs in our infrastructure (consistent Linux fonts), so this isn't an issue between two AegisRunner runs. If you export to Playwright and run locally on Mac, the screenshots won't match — that's expected.

Can I exclude visual regression from one specific test?

Yes. Open the test, toggle off Visual regression on the test detail page. The suite still runs visual regression for everything else.

Related

• Baseline Replays — pair with visual regression for deterministic CI.
• Running Tests — where the toggle lives.
• Test Baseline Review — review AI tests before running visual regression on them.