Visual Regression Testing

Visual regression testing detects unintended visual changes by comparing screenshots between test runs at the pixel level using pixelmatch.

How It Works

1. First run — screenshots are captured as the visual baseline for each test step
2. Subsequent runs — new screenshots are compared against the baseline pixel-by-pixel
3. Differences detected — pixels that differ are highlighted in the diff view
4. Review — accept the new screenshot as the new baseline, or reject it as a regression

The 3-Panel Diff View

When differences are found, AegisRunner shows a 3-panel comparison:

Accept / Reject

• Accept — the new screenshot becomes the baseline for future comparisons (intentional change)
• Reject — marks the difference as a visual regression that needs fixing

Threshold Settings

Visual comparison uses a configurable pixel difference threshold to handle minor anti-aliasing and rendering variations. By default, differences below 0.1% are ignored.

Works with Regression Manifests

Visual regression testing is most powerful when combined with regression manifests:

1. Set a crawl as baseline — captures screenshots for every page state
2. Run a regression crawl — replays the same pages and interactions deterministically
3. Compare screenshots — pixel-level diff between baseline and current run

This ensures visual comparisons are between the same page states (same button clicked, same modal opened), not random exploration results.

Per-Step Screenshots

Visual baselines are captured at the test step level, not just per page. This means:

• A login form test has baselines for: page load, email filled, password filled, submit clicked
• Each step's screenshot is compared independently
• You can accept/reject each step's visual change separately

Related Documentation

• Regression Manifests — Deterministic crawl replay
• Test Baseline Review — Approve/reject test cases
• Running Tests — Test execution with visual comparison