Control plane outside. Execution plane close to the product.
TestPilot Works separates project orchestration, run state, reporting and audit metadata from browser/API execution. The recommended deployment is hybrid-first: TestPilot Works hosts the control plane while a private runner executes tests inside the customer environment.
Deployment modes
- Private Runner: execution stays inside the customer network.
- Hybrid Runner: managed orchestration plus customer-local execution; this is the preferred default.
- Cloud Runner: suitable only for approved public or allow-listed test systems.
Target URLs and credentials stay runner-local.
The control plane does not need to store the real customer staging or internal endpoint. It stores an environment label such as staging or uat; the real BASE_URL, API_BASE_URL, credentials and allowlist remain on the private runner.
A runner may claim a job only when tenant, project, supported suite and environment match. Teams with staging and UAT normally deploy separate runner instances with separate local endpoint configuration.
Customer-specific test packs
Private Playwright suites can live outside the platform core in a customer-controlled repository or filesystem. The runner chooses the local test root; the control plane cannot override it.
Trigger tests without exposing a human dashboard credential.
CI credentials are project-scoped, revocable, expiring and limited by explicit suite and environment allowlists. Raw credentials are shown once and stored only as hashes.
Supported CI-triggered suites are smoke, api and regression. Performance smoke is intentionally excluded from CI credentials and remains a customer-controlled runner action.
Pipeline helper
A small release gate built from fresh QA evidence.
The gate evaluates the latest completed Smoke, API and Regression result for a project and environment. Evidence older than 24 hours is treated as stale.
| STATE | MEANING |
|---|---|
| READY | All required suites passed and the evidence is fresh. |
| BLOCKED | At least one latest required suite failed. |
| STALE | Required suites passed, but at least one result is older than 24 hours. |
| NO_DATA | At least one required suite has no completed result. |
Flaky tests do not automatically block a release, but they raise a READY release from LOW to MEDIUM risk.
Useful QA signal without shipping test-level customer data.
The runner aggregates Playwright results locally and uploads only five counters for this reporting path:
Test names, page URLs, assertion messages, screenshots, traces, request bodies and response bodies are not part of the aggregate reporting contract. CI also validates the summary schema so new telemetry cannot be added silently.
Least privilege is the default operating model.
- Use staging and test environments by default.
- Use least-privilege test identities and customer-controlled secret stores.
- Never hard-code credentials or write them into logs.
- Keep screenshots, video and traces disabled unless explicitly required.
- Redact authorization headers, cookies, tokens, passwords, API keys and session data before logs leave the runner.
- Use short retention and failure-only artifacts when artifacts are enabled.