How it works
The workflow
KwikQA is a fixed pipeline with a clear stop at every step. Each command ends in one outcome and one reason code, written to a record in .kwikqa/runs/.
1. Read the run
KwikQA reads your framework's report: Playwright's JSON report; KwikQA's own Cypress or WebdriverIO reporter; or mochawesome, @wdio/json-reporter, or JUnit. It also reads the reports of other frameworks, such as JUnit and TestNG through Maven or Gradle, pytest, Jest, Vitest, Cucumber, Allure, and CTRF, and diagnoses their runs. Repairs cover only the supported frameworks. For Playwright it also reads each failing test's trace and page snapshot when they exist. Cypress and WebdriverIO keep no trace, so npx kwikqa analyze --capture re-runs the failing specs with KwikQA's failure capture, which records each failed attempt's DOM, command log, console, and network; for Playwright it re-runs the failing tests with full tracing. Capture covers only those three frameworks. Either way it asks before running anything. It checks that the run is complete: an interrupted run, a report with errors, or a run with no failures stops here with a reason code instead of a guess. Failures that share a symptom are grouped, so one root cause isn't diagnosed twice.
2. Build the evidence packet
From the report, the traces, and short excerpts of the test source, KwikQA builds a bounded evidence packet on your machine. It follows the action that failed, from the trace or the Cypress or WebdriverIO command log: the element it targeted or, when a locator or selector matched nothing or too much, the elements most like the one it wanted. Evidence closest to the failure is kept first, and if the packet would be too large, what explains the failure least is left out and listed. Secrets and sensitive fields are redacted, items the policy doesn't allow are withheld, and images are described by size only. Every redaction and omission is counted in the preview.
3. Ask, once, with your approval
The preview shows the destination, model, evidence, cost estimate, caps, and a hash of the exact payload. Approving sends one request with that payload; the request has a fixed output cap and a dollar cap. Nothing is retried behind your back.
4. Grade the diagnosis
The answer is checked against the evidence before you see it. Each cause must cite evidence that exists in the packet. It gets an evidence grade (strong, moderate, weak, or insufficient) and a probable owner (test-support, application, environment, or undetermined). Confidence alone never authorizes a repair: a repair needs positive, cited evidence that the fault is in test code.
5. Propose a repair
A Java, Python, or Cucumber-JVM failure is repaired only when its failing tests are end-to-end browser tests, so KwikQA checks that before it sends a repair request. A test it can't confirm reaches a browser library is diagnosed but not repaired, and fix sends no repair request for its group. See Supported frameworks.
fix sends a separate repair request. The model must say what the test intends, what behavior the repair preserves, and which evidence each edit relies on. KwikQA resolves every edit against your files locally and checks the result against the repair rules. A proposal that breaks a rule is rejected with its reasons and never shown as a patch.
6. Apply with a journal
You approve the exact diff and the exact validation commands. KwikQA writes a journal before it touches a file, then applies the patch. Files with your own uncommitted changes are never edited. In CI, a ci.fix policy you commit approves instead, for the runner's own checkout only, and the validated repair arrives as a draft pull request.
7. Validate or roll back
validate runs the original test, and the tests the change could affect, in fresh processes with your own test runner (Playwright, Cypress, WebdriverIO, Maven, Gradle, or pytest), writing new artifacts without touching the original evidence. Cypress can't run a single test by name, so KwikQA runs its spec with every other test skipped except those that passed before, which must still pass. WebdriverIO runs the one test through its grep option. A pass of every required check is repaired. A failure, or any check that is skipped, interrupted, stale, or incomplete, rolls the patch back. Rollback restores exactly the bytes KwikQA changed and stops with rollback-conflicted rather than overwrite edits you made afterwards.
Java and Python repairs are validated from the build root nearest the changed files: the nearest directory above them that holds a Maven or Gradle build, or a pytest configuration (a conftest.py marks one only where no configuration is above it). A Gradle subproject's build runs from its root project, and a Maven module runs from the outermost aggregator that lists it. Maven and Gradle select each test by its class and method (a Cucumber scenario by its runner and feature line), and pytest by its node id, from that directory. If the changed files lie in two builds, validation is unavailable.
Shared test code
A repair to shared test code, such as a page object, fixture, helper, or framework configuration, affects every test that uses it, so KwikQA validates it over all of them:
- Find the affected tests. For Playwright, Cypress, and WebdriverIO, they are the failing test and each test that imports the changed file, directly or through other modules. A change to a Playwright or framework configuration file, or an import graph that can't be bounded completely, validates every test of the projects the change belongs to instead. For Java and pytest, they are the tests that reach the changed file through imports, fixtures, the
conftest.pyfiles above each test, apytest_pluginslist, or pytest'spythonpath. Gradle'stestFixturesandintegrationTestsource sets are read like the test sources, so a change there reaches the tests the same way. A module that apytest_pluginslist names affects every test of the run. If KwikQA can't name them completely, validation is unavailable, and nothing is applied. For Cucumber-JVM, a change to any Java file reaches every scenario, since a step definition can be in any Java file. The failing test must be among the affected tests. - Run them before the patch. After you approve, the affected tests run once on the current code, and KwikQA records each one's outcome. Failing tests are expected at this point. If the failing test passes in this baseline, KwikQA doesn't patch it. If the baseline doesn't finish with complete results, or doesn't report every affected test, the repair stops before anything is applied.
- Run them after the patch. For Playwright, Cypress, and WebdriverIO, each affected test runs three times, with retries off. For Maven, Gradle, and pytest runs, the affected tests run once.
KwikQA keeps the patch only if every required check passes:
- the failing test passes after the patch;
- no other affected test fails after the patch. One that already failed in the baseline and still fails counts as a failure;
- every required run finishes with fresh, complete results, and each affected test appears in them.
Any other result rolls the patch back: a failed check, a check that is unavailable, or results that are stale or incomplete.
8. One retry, then stop
After a failed validation, fix --retry rebuilds the evidence from the validation run and asks for one revised patch, with fresh approvals. If that fails too, KwikQA rolls it back and stops. It never loosens its rules or loops until the build is green.