Testing for Coded Apps (automated E2E testing to get high quality)

Overview: Automation testing using AI generally ends with "test rot".  Kaizen Fix gets around test rot by switching the test source from the analysed system requirements to the code source as the logic to test.

Why: Gathering requirements in Agile software is continuously changing, so building tests based on the ask coupled with AI updating code realistically means the behaviour will change, DOM/Shaow DOM updating per build, and the full code base needs to be retested to ensure operational behaviour.

Hypothesis: Break the E2E testing into two distinct parts:
1) Does what the user asked for match what is delivered?  Ask AI to document the requirement from the code and compare it to the requirement the stakeholder signed off on. 
2) When code is updated, does this break any existing logic?  Regression tests to check existing logic work; if not, this requires a man in the loop to validate the change.

Proposed Solution: 
The four-step loop: 1. Code with AI → 2. Generate the code specification → 3. Tests generated by AI and MCP → 4. Test Baselines (run the dynamic Playwright test suites).  As the release occurs, the developer can automatically run the old tests and identify what is no longer working and why.  This can also be run during development.  I've been working on this different approach: an AI-assisted testing framework that treats the source code as the single source of truth — and turns it into a living, executable specification.

Key Point: reruns against the original/previous baseline catch genuine behaviour changes, not test rot. I deliberately left it unbranded so you can drop in KaizenFix (or keep it vendor-neutral) depending on where you're posting it.

How it works, in four steps:

1. Read the AI-generated code

AI (Claude or GitHub Copilot, working inside the IDE) analyses the application's source — components, routes, validation, business logic — and generates a full behavioural specification: what the system does today, fully documented, straight from the code.

2. Turn behaviour into programmed behaviour requirements

That specification becomes a structured requirements document. Not aspirational requirements — actual behaviour. If the app rounds a value, enforces a limit, or hides a button under a condition, it's captured.

3. Generate the regression suite

From the actual requirements into Playwright tests. AI generates detailed Playwright end-to-end tests — using platform-specific best practices for selectors and patterns, and environment configuration so the same suite runs against dev, UAT, or production. Each web app lives as its own isolated project with its own context and rules.

4. Lock in the baseline

At this point, the tests and the app agree by construction — the suite documents and validates the working behaviour. The tests part of the documentation.

The payoff comes when things change:

When the app or its logic evolves, rerun the original suite. Anything that breaks is a genuine, intentional-or-not change in behaviour — surfaced immediately, with the old expected values as evidence. Regenerate the spec, diff it against the last one, and you can see exactly what changed and whether you meant it. 

Amended +- June 2026 with Picture below: Source unknown

Looking for a cheap quick UI testing and monitoring Tool - end test and Ghost Inspector Review

Problem:  My client is looking for a simple tool to monitor a website is up and running and can run a small set of UI tests and asserts to verify it is working as expected.

Initial Hypothesis:  There are a lot of monitoring sites like uptime that meet this requirement, but I reviewed Ghost Inspector and endtest.  I am not looking to do full CI as I would look at Selenium WebDriver for an enterprise solution for UI testing.

Resolution:  Trial endtest and Ghost inspector on my O365 subscription to validate it monitors and alerts, can perform advanced logins and it can validate custom pages after JavaScript injection.  Price and feature wise both tools are pretty similar.

Ghost Inspector Initial Thoughts
Easy to use and there is a recording function for Chrome.  This review has put me off Ghost Inspector to some degree but definitely an excellent product to evaluate.
Bad review for Ghost Inspector but it does assume enterprise level UI testing more suited to tooling like Selenium.

endtest Initial Thoughts
Easy to use, setup testing in a matter of minutes, recorded actions and assertions.  The trial is limited as I could not check the scheduling mechanism, but end test looks like the ideal tool for my requirement.  Would need to go for the pro licence at $79 per month.  A simpler smaller option would be more attractive but let's see what the client thinks.

Other Tools for UI Testing:
Selenium
Selenium IDE is an excellent tool for UI testing and UI automation testing.  Here is a post on Selenium I did a few years back.

qTest Explorer
This is a Manual recording and documentation tool that records browser and desktop interaction.  It's straightforward to use and great for Manual UI testing.  It is not for automation or re-running test but great if the project requires manual testing and proof.

Smoke Testing SharePoint using Selenium IDE

Overview:  In SharePoint we need to retest code often as we make incremental changes.  Basic smoke testing is useful in that is allows you a certain degree of confidence that a bug has not crept into your latest deployment.

Most projects have varying degrees of control to ensure bugs do not cause unexpected behaviour and on the more advanced practices is unit testing and coded UI tests.  The unit tests are tricky with all the new SharePoint development methods.  Jasmine is a JavaScript testing framework and check out the @SPDoctor (Bill) for basic testing information.  Unit testing SP is difficult as the revs end up test SP and not the changes.  A lot of the code is UI driven which is hard to unit test.  I have previously written about code UI testing, MTM as part of the MS test and continuous testing.  And part of this is Selenium WebDriver.  I've used it once on a large project and it was awesome.  Now as you go into production you normally will do some manual smoke testing to check the deployment.

This post looks at automating smoke test.

There are various tools for recording smoke test or you can do the manual eyeball approach favoured my most SP project still.  I have used Powershell (both to gen http requests and to control IE).  MTM is good but it requires buy into the whole MTM process.  Personally I like Selenium IDE for Firefox.  It offers recording, and the capturing functionality is miles ahead of anything else.  These Selenium generate test can be export and used with Visual Studio or build into TFS or on a project I used TeamCity to run automated continuous build and integration.

Note:  Selenium IDE is the recording piece and Selenium WebDriver is the heavy duty real testing integration part.

Get Started:
1. Download Selenium IDE (make sure you get the Selenium IDE, I'm using version 2.9.0) 
2. Understand the UI and capabilities (YouTube basic Selenium IDE videos are great)
3. Install Selenium IDE on Firefox as shown below:

4. Launch Selenium IDE using FireFox

5. Record and run Selenium test (2 minute short video providing a Selenium IDE recorded test against a SharePoint Online Team Site).

Another option is to control IE using PowerShell
PS> $ie=New-Object -com internetexplorer.application  #open ie
PS> $ie.Navigate("http://www.radimaging.co.uk/Pages/default.aspx")

Non Functional Testing for SharePoint

Overview:  Functional Requirements are the business requirements that the business define for the application being built.  Non-functional testing is concerned with performance, reliability, scalability, recovery, load,  security and usability testing.  For SharePoint it is a good idea to test this at a platform level and then verify the individual application non functional testing is appropriate.

SharePoint Non Functional Testing:
All of these test should be performed against your various SharePoint platforms and will dictate the SLA's offered to the business using SharePoint as a service.  Baseline testing is a good idea as the differences can be used to determine the efficiency of the individual application being created.

Proxies:
Fiddler is my favourite (other tools for capturing web traffic Charles, BurpSuite, WireShark and you can use the developer tools shipped with the browsers). tcpdump, goPacket are awesome for network monitoring.

Use Fiddler to:

• Observe traffic (http/https requests, headers,..)
• Replay sessions, 
• Evaluate performance,
• Set break point

A common misconception is the function of performance vs load testing.  

Performance is primarily concerns with looking at typical user usage scenarios and see how long each page takes to load.  So a recorded script with wait time between recorded calls is useful  It's also worth looking at the same page with minimal data and also with large amounts of data.  

Load Test involves recording the standard users interaction (ensure some users query heavy and some light amounts of data), there are no wait times.  The norm is to multiple this concurrent number of users by 100 to know the amount of users the farm business application can support.  e.g. a concurrent user load of 100 users where the performance is acceptable means the farm should handle 10,000 users (100 users times 100).  You run the 100 users in a steady state for a few hours.

Stress testing is similar to load testing but you keep stepping up the number of concurrent users until the SharePoint farm starts running out of resources and throttling requests.  Once the system degrades and performance is not acceptable is pretty much how many users the application on the SharePoint farm are supported.  So if the performance throttles at 170 users, the farm can handle 17, 000 normal usage users.

References:
http://www.guru99.com/non-functional-testing.html