Overview: Automation testing using AI generally ends with "test rot". Kaizen Fix gets around test rot by switching the test source from the analysed system requirements to the code source as the logic to test.
Why: Gathering requirements in Agile software is continuously changing, so building tests based on the ask coupled with AI updating code realistically means the behaviour will change, DOM/Shaow DOM updating per build, and the full code base needs to be retested to ensure operational behaviour.
Hypothesis: Break the E2E testing into two distinct parts:
1) Does what the user asked for match what is delivered? Ask AI to document the requirement from the code and compare it to the requirement the stakeholder signed off on.
2) When code is updated, does this break any existing logic? Regression tests to check existing logic work; if not, this requires a man in the loop to validate the change.
Proposed Solution:
The four-step loop: 1. Code with AI → 2. Generate the code specification → 3. Tests generated by AI and MCP → 4. Test Baselines (run the dynamic Playwright test suites). As the release occurs, the developer can automatically run the old tests and identify what is no longer working and why. This can also be run during development. I've been working on this different approach: an AI-assisted testing framework that treats the source code as the single source of truth — and turns it into a living, executable specification.
Key Point: reruns against the original/previous baseline catch genuine behaviour changes, not test rot. I deliberately left it unbranded so you can drop in KaizenFix (or keep it vendor-neutral) depending on where you're posting it.
How it works, in four steps:
1. Read the AI-generated code
AI (Claude or GitHub Copilot, working inside the IDE) analyses the application's source — components, routes, validation, business logic — and generates a full behavioural specification: what the system does today, fully documented, straight from the code.
2. Turn behaviour into programmed behaviour requirements
That specification becomes a structured requirements document. Not aspirational requirements — actual behaviour. If the app rounds a value, enforces a limit, or hides a button under a condition, it's captured.
3. Generate the regression suite
From the actual requirements into Playwright tests. AI generates detailed Playwright end-to-end tests — using platform-specific best practices for selectors and patterns, and environment configuration so the same suite runs against dev, UAT, or production. Each web app lives as its own isolated project with its own context and rules.
4. Lock in the baseline
At this point, the tests and the app agree by construction — the suite documents and validates the working behaviour. The tests part of the documentation.
The payoff comes when things change:
When the app or its logic evolves, rerun the original suite. Anything that breaks is a genuine, intentional-or-not change in behaviour — surfaced immediately, with the old expected values as evidence. Regenerate the spec, diff it against the last one, and you can see exactly what changed and whether you meant it.
