Using KwikQA
Checking outside changes
Coding agents and Playwright's healer change tests too. kwikqa check holds those changes to the same gates KwikQA's own repairs pass, so a test can't be made green by weakening it. It reads your files and runs your tests on your machine or runner. It sends nothing to KwikQA and counts nothing toward your plan.
$ npx kwikqa check [--base <ref>] [--analysis <path>] [--yes]
The gates
A change passes only when every gate passes. A finding about a file names that file, and a scope or policy finding also names its line.
- Scope and policy. Only the test paths your committed
kwikqa.config.jsonmakes editable may change, within the same repair rules: no skips,fixme, fixed waits, added retries, forced actions, swallowed errors, or removed, swapped, or loosened assertions. The rules read JavaScript, TypeScript, Java, and Python test files. A change to any other file, such as a Cucumber feature file, can't pass this gate: a file outside the editable paths fails it, and a file the rules can't read is not checked. - Ownership. The failing tests the change touches must have been diagnosed as the test's fault, by an analysis of the code before the change. Run
npx kwikqa analyzeon the failing run before the agent edits the test. The analysis must also have read each changed file as the base has it. A change to a test whose failure was the application's fault fails the check, because the test may have caught a real bug. A change that touches no failing test of the analyzed run isn't judged, so the verdict is incomplete. A change to a Java, Python, or Cucumber-JVM test that KwikQA can't confirm reaches a browser library isn't judged either: the gate reports it as not checked, with the reason, and the verdict is incomplete unless another gate fails. See Supported frameworks. - Validation. The static checks must pass, and the failing tests the change touches are re-run after you approve, except a test the ownership gate doesn't judge. A change to a test file runs each touched failing test three times in a row, with retries off. A change to shared test support, such as a page object, fixture, helper, or configuration, runs every test that uses it, and compares each with how it stood in the analyzed run: a test that passed there must still pass, and a test that failed there and still fails fails the check too. For Playwright, Cypress, and WebdriverIO, those tests run three times with retries off. For Maven, Gradle, and pytest runs, they run once. For Cucumber-JVM, a change to any Java file runs the runner once instead. Java and pytest changes use the project's own Maven, Gradle, or pytest, from the build root nearest the changed files. A shared change also needs a current analysis: one from an older KwikQA doesn't record how the other tests stood, so run
analyzeagain first. Without a terminal, validation runs only with--yes; without either, the gate is not checked and the verdict is incomplete. On Windows, Maven and Gradle validation isn't available, so those changes are not checked.
Verdicts and exit codes
| Verdict | Meaning | Exit code |
|---|---|---|
passed | Every gate passed. | 0 |
failed | A gate found a problem. The findings say which, and where. | 1 |
incomplete | A gate couldn't decide, for example because there was no analysis of the failing run, a change touches a test that KwikQA can't confirm reaches a browser library, or validation couldn't run. The output names what was missing. | 1 |
Options
| Option | Effect |
|---|---|
--base <ref> | Check a branch's changes since it left <ref>, such as origin/main. Without it, KwikQA checks your uncommitted changes. |
--analysis <path> | The failing run's analysis record (.kwikqa/runs/<id>/result.json). Without it, KwikQA uses the newest analysis in this checkout that has a failing test the change touches. |
--yes | Approve running the touched tests and the static checks without asking. Without it and without a terminal, validation isn't run and the verdict is incomplete. Needed in CI. |
Putting a failed change back
After a failed check of uncommitted changes, at a terminal, KwikQA lists the modified files it can put back as HEAD has them, and asks once. Only a yes at that prompt restores anything; --yes doesn't answer it. KwikQA keeps a copy of the change in .kwikqa/adopted/ first, so nothing is lost. New and deleted files are listed but left alone, and so are files outside the workspace or larger than 1,000,000 bytes. A file that changed after the check is left alone too.
It never offers a restore in CI, with --base, or after a check that passed or was incomplete.
As a required check on pull requests
In CI, kwikqa check publishes its verdict where kwikqa ci publishes its summary: the job summary and one pull-request comment of its own (on Azure Pipelines, a build summary and a pull-request thread; on GitLab, the log and a merge-request note; on Jenkins, the log and a pull-request comment). A pass only updates an earlier comment. Because failed and incomplete exit 1, branch protection can require the job.
The example passes --yes, since CI has no terminal to approve validation.
on:
pull_request:
paths: ["tests/**"]
permissions:
contents: read
pull-requests: write # only for the comment
jobs:
kwikqa-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0 # check compares with the base branch
- run: npm ci && npx playwright install --with-deps
- run: npx kwikqa check --base origin/${{ github.base_ref }} --analysis failing-run/result.json --yes
env:
GITHUB_TOKEN: ${{ github.token }}
Ownership is judged by the analysis of the run that failed. In the job that ran kwikqa ci on your default branch, upload .kwikqa/runs/*/result.json as an artifact, then download it into failing-run/ in this job. Don't take it from the pull request's own runs, or from a fork's: a record the pull request could produce proves nothing about who is at fault. Without it, the check is incomplete and the job fails.