Test Generation
AI-Powered Test Generation
Learn how AegisRunner uses AI to automatically generate comprehensive test suites from your crawl data.
AI-Powered Test Generation
AegisRunner uses a 3-pass pipeline to generate Playwright test suites from crawl data.
The 3-Pass Pipeline
- Pass 1: ScenarioPlanner (deterministic code) — Analyzes page context (forms, buttons, links, states) and produces scenario plans. Decides WHAT to test.
- Pass 2: AI Materializer (LLM) — Implements scenarios as concrete Playwright steps using real page selectors. Multiple providers in parallel.
- Pass 3: Selector Validation (deterministic) — filterByKnownSelectors ensures tests use real selectors from crawl data. Prevents fabricated selectors.
7 Scenario Families
| Family | Priority | What it tests |
|---|---|---|
| Form Tests | Highest | Happy path (e2e), empty submit (negative), field limits (boundary) |
| Interactive Buttons | High | Modal/dropdown triggers, add-to-cart flows, multi-step journeys |
| Navigation | Medium | Header nav links, content links — verifies destination content loads |
| Multi-step E2E | High | Chains crawler-discovered state transitions into journey tests |
| Form Validation | High | Tests from crawler-captured validation states |
| API Endpoints | Medium | Smoke tests for discovered API endpoints |
| Fallback | Low | Every page gets at least one test |
Test Types
| Type | Example |
|---|---|
| e2e | Fill login form → submit → assert dashboard loads |
| negative | Submit empty form → assert validation errors |
| boundary | Enter 1 char in minLength=8 field → assert error |
| regression | Click Add to Cart → assert cart badge updates |
| smoke | Click nav links → verify destinations load |
Quality Rules
The AI is instructed to:
- NEVER assert the same element it just clicked
- ALWAYS verify outcomes on a DIFFERENT element
- Use only selectors from the VALID SELECTORS list
Custom prompts: Add project-specific instructions to guide test generation toward your critical paths.
AI Providers
Requests distributed across: Z.AI GLM-5.1, DeepSeek Reasoner, MiniMax M2.7, MiMo V2 Flash, OpenRouter Devstral 2, Chutes, with automatic failover.