Playwright series - Post 2 - Refactored TS code for Consciously verify Apps in Production

Overview: Create a function with Playwright tests that loops through all my production apps, logs in, and validates the Page title load on each app's home page.

Steps:

1. Create the spec.ts code that reads app.json to loop thru and validate sites

2. Record and Store the session state for all future test runs (Used for MFA in the tests runs)

3. Create an apps.json file containing URLs to open and validate

4. After running the test, you'll see that the 3 tests were completed successfully. In my case, there were 2 Power Apps with MFA enabled and an anonymous public website that had been checked.

Optional: 

Create short cuts to run your tests using PowerShell

PS C:\repos\PW> npx playwright test -g "Prod-CanvasApps" --project=chromium --headed

Next Steps:

Run continuously using the Azure Playwright Service.

=========================

Playwright Series

Playwright Post 1 - Overview of E2E testing using Playwright

Playwright Post 2 - Continuously Test/Monitor Canvas apps and website with MFA enabled (this post)

Playwright Post 3 - Add App Insights logging inside your Playwright tests 

Playwright Post 4 - 6 Min walkthru of Playwright testing with Azure Monitor

Low Code testing with Playwright - 1. Intro Exercise (15 min)

Exercise 1.  Install and setup you first Playwright Test

1. Verify VS Code is installed 

2. Ensure the Microsoft Playwright Extension is installed (use the default language TypeScript)

 

3. Create a folder using Windows Explorer as shown here: C:\Users\paulb\source\repos\Playwright\Mendix

4. Open the folder in VS Code and ensure the project is initialised C:\Users\paulb\source\repos\Playwright\Mendix> npm init playwright@latest --yes -- --quiet --browser=chromium --browser=webkit

5. Verify your screen looks similar to this...

Exercise 2.  Record and run your first Playwright test

1. On the "Testing" area, select "Record new"

2. The recorder opens, type in a url in my case I used "https://www.paulbeck.co.uk"

3. Click and assert text exists on the page

4. Stop the recorder and close the browser.

5. Run the test, as shown below, validate the result

Mendix tips & thoughts

Mendix Series

1.  Overview of Mendix 

2. Mendix - Part 2 - Diving deeper

3. Extensibility for Mendix Studio 

4. Building a Mendix Widget for the Mendix marketplace (Convert text to Audio using Azure AI)

5. Mendix Tips & Thoughts (this post)

6. Competitors

Mendix Charting/dashboarding options:

• E-Charts (Community supported) are simple and nice,
• Anycharts (very common),

Grafana can be integrated, but I haven't tried it with Mendix.

Power BI can show reports using an iFrame widget.  I like this approach.

Module/Widget Support Note: Check whether libraries are supported by Mendix/Platform or Partner.

Community-supported obviously can be changed by the community. Partner-supported is also an option if you have an agreement with the partner or trust them.

Performance Testing Tool options:

A colleague has used JMeter and feels it was not ideal. 

I'm considering using the Microsoft Playwright Testing service and Playwright testing.

Enterprise Logging/SIEM SaaS integrations supported by Mendix:

1. App Dynamics.
2. Datadog,
3. New Relic,
4. Dynatrace, and
5. Splunk

Watch out for:

Logging

Each system logs to loc files on the local machine; these can be pushed into the Postgres local instance.  This can result in a massive storage load being added for auditing and logging.  

Monitoring

Logs can be shipped out using backup and restored or by calling the REST Open API.

Maintenance

Mendix builds a database per app per environment, so the recommendation is to use at least 3 per app as you need dev, Test, and Prod. Each instance, by default, uses PostgreSQL (you can only use PostgreSQL if you use the Mendix-provided images deployed on AWS).  

Global Variables

Mendix doesn't have a concept of a global variable at the start or per session. You can load lookup data, which is often held centrally in your enterprise. This can get heavy quickly, but you can copy Open API results into the local PostgreSQL database so it only gets local data.  

Costs

With Mendix, the cost can escalate rather quickly. Reduce cost by scaling back to the most minor instance in dev and test, especially post-go-live. Each app has a separate database, so management and connectivity can become hard to control. 

Performance

Number of Controls & Size of each application.

Mendix Series

1.  Overview of Mendix 

2. Mendix - Part 2 - Diving deeper

3. Extensibility for Mendix Studio 

4. Building a Mendix Widget for the Mendix marketplace (Convert text to Audio using Azure AI)

5. Mendix Tips & Thoughts (this post)

6. Competitors

MS Build Conference 2024 Online Notes

A slick keynote, but I'm summarising the items i liked from the online sessions I watched:

Low code CoE thoughts

Key areas to address for a Low Code CoE:

1. Portfolio of Products
2. People/Skills/Experts
3. Process/ALM
4. Platform/Governance/Cleanup/Architecture/NFR's (scalability, SLAs)
5. Promotion/delivery/training

Consider putting the project through a Business case validation, particularly for larger projects.  The business case is how your business visualises and communicates a proposed project, which should continuously be refined.  Points to cover:

1. Business Case (Idea)
2. Customer Segment >
3. Value Proposition (what do they get) > 
4. Channels (API, website, AI Services, .. ) Customer Achievement (get and retain) >
5. Revenue (how to make money)
6. Key Resources (patches, maintenance) covers technology and data.  What tech, people need to build and continue delivering.  Data: where does it come from, quality, restrictions, relevancy.
7. Partners (dependency e.g. AWS, Azure, Mendix, OutSystems, UiPath, Microsoft, and backup options)
8. Cost breakdown

CoE needs to clearly understand its boundaries and responsibilities.  

I am a big fan of Total Ownership, clear requirements and goals, using RACI, and stand-ups. For the rest, I try to be agile and malleable. Big projects require more oversight and planning, so I allow teams to change and focus on delivery.  

Low code ROI/TCO and observability - Monitoring Low code Platforms

Overview: With Low code gaining tremendous traction, the rate at which apps are being built is increasing quickly.   More citizens can build and pro developers can build more.

Governance helps maintain the quality of the apps and their code, but as an app needs to be updated, it becomes significantly expensive to maintain and update the low code solutions.  The P-F curve has been around for awhile for regular assets (I first saw this 20 years ago with Mining equipment in Southern Africa, Machine maintenance on the mines help significantly reduce the cost of the assets/equipment and reduce catastrophic outages, anyway I've digressed...).  

Microsoft Low Code/Power Platform:

All low-code platforms have the same issues, governance improves the situation but it does not address the P-F interval, and monitoring comes into play.  Microsoft's Canvas Apps has the ability to ship logs to Azure Monitor (App Insights) and there are internal analytics available on the platform.  These are great and should be implemented.  Now most enterprises have multiple environments and tenants in Power Platform, ALM is used to improve the quality of the application being  released.

Canvas apps has an internal testing tool called Test Studio which is fairly limited and yet still underused.  Tests can be recorded, on deployment releases the recorded rules can be run and reported on in ADO or any CI/CD pipeline.  

You can also make web calls from CI/CD or a dedicated service to continually run advanced availability tests, you should do this.  It ensure we pick up when part or all of the process changes.  And you can setup alerts and respond.

Tools:

You can also log to SIEM (enterprise logging tools) and use this for Monitoring.  For instance, you can ship you App Insights/Log Analytics/Azure Monitor logs to Dynatrace so it is part of your overall monitoring and response strategy.

You can continuously monitor API using CI/CD tools like ADO but I prefer to use Postman's Enterprise Infrastructure service (it's awesome).  Similarity, I am a full convert to Microsoft Playwright (I was a Selenium and Cypress fan, not i'm only Playwright).

Grafana, Power BI, Azure Dashboards all have different pros and cones for monitoring.  There is also some great AI stuff coming out at the moment around UI testing.  Specifically, I've seen Dynatrace, Playwright and BrowserStack use AI to compare screenshots for test validation.

I'll be expanding on automation, continuous testing in this series of posts.  These are my initial thoughts and there is a lot of good results coming out.

Mendix - Part 2 - Diving deeper (E2E automation testing of Mendix using Playwright)

Mendix Series

1.  Overview of Mendix 

2. Mendix - Part 2 - Diving deeper (this post)

3. Extensibility for Mendix Studio 

4. Building a Mendix Widget for the Mendix Market Place (Convert text to Audio using Azure AI)

5. Mendix Tips & Thoughts

6. Competitors

AI with Mendix (current version Mendix 10.5.x):

1. Logic bot - recommends what you are likely to do, like a copilot as you go along building the app
2. Performance bot - shows redundancies, recommends performance improvements 
3. Chatbot in beta

Playwright is a good UI testing tool for Mendix:

For more advanced applications, Playwright is a good testing framework that can help developers know their code is running end-to-end, useful for monitoring applications and behaviour, and also can be used as part of the CI process to validate Mendix end user accessibility as shown in this mp4 (7 minutes - good video).

Thoughts:

• Deployment can take time when the project grows.
• Domain modelling: can use existing models and add to Mendix.
• Playwright is perfect for Mendix testing, here is a basic dev example.
• The Mendix Community is a good place to post questions.

I needed to change from US format to UK date time format:

The community has the answer: Mendix Forum - Question Details

Mendix Series

1.  Overview of Mendix 

2. Mendix - Part 2 - Diving deeper (this post)

3. Extensibility for Mendix Studio 

4. Building a Mendix Widget for the Mendix marketplace (Convert text to Audio using Azure AI)

5. Mendix Tips & Thoughts

6. Competitors

Review of Browser Stacks Low Code Automated test tool

Overview: Low code testing relies heavily on complete UI end-to-end testing.  It needs to be fast, flexible, quick to correct, scale-able, highly configurable and BrowserStack's low code test tool is in beta and definitely on the right path - for me, it needs a few features.  I ran my testing against customised apps created on three platforms:

1. Mendix low code,
2. Microsoft's Blazer hosted on Azure Web App, and
3. Canvas app within Power Platform. 

Tip: I've looked and use BrowserStack for many years and it has moved from being an device emulators infrastructure testing provider to a full ALM testing platform.  The low code Browser stack has a recorder to capture steps.

Where does Low Code fit into Browser Stack:

Image1. Low code automation works well as part of the full BrowserStack Platform or just using the product by itself.

Pros of the Low Code BrowserStack Product:

• The local recording feature is easy to set up and use
• Seamless integration with the cloud version running on BrowserStack's infrastructure
• Logical layout of UI, little to no training required
• UI validation using the DOM or, more importantly, screenshots using BrowserStack's AI verification (required further review) has the potential to self-heal as in the screen changes, but the validation can be smart enough to understand it is just an updated screen (example a single colour in the page and the position of the name is moved).
• SDK is available to work with the full BrowserStack platform.
• Not Low code specific but BrowserStack generally has the new phones included in their offering within days of being release.

Cons:

• Provide a webhook or allow for a REST client call as a step (I'd want to log directly from the test run into Azure Monitor)
• More run options, I'm sure it's already on the road map, but the ability to run every hour for continuous monitoring.
• Refresh tokens on a schedule (allows you to not use MFA such as SMS codes or Authenticator).
• Make it clear if the run is from the local or the browser, and keep the historical logs for both together.
• Export results - I could not find this, but it would help compare step performance.
• I use DevOps, I'm unlikely to take the whole BrowserStack platform unless i need the emulators which is what BrowserStack was originally famous for.

Summary: This is an excellent tool for testing; the low code product was still in beta when I reviewed it.  It is a nice tool and has the potential to disrupt the market.  I feel Playwright is a better point solution and integrates to CI/ALM platforms.

References: https://www.browserstack.com/low-code-automation/features?utm_source=google&utm_medium=cpc&utm_platform=paidads&utm_content=668760067900&utm_campaign=Search-Brand-EMEA-Navigational&utm_campaigncode=Core+9045914&utm_term=e+browserstack

Other: 

Image 2. Emulate a Samsung Galaxy phone on Android using the Chrome browser.

Thought: I like BrowserStack's reporting, clean and simple on tests and easy to get the history.

Mendix Overview

Overview: Mendix is a market leader in low-code app builders. While I predominantly use the Power Platform, Mendix can be a good option.  

The ALM has Version Control. This is intuitive and follows a local checkout version commit back to a main branch (simple version control). It allows you to use branches so comprehensively and flexibly. Checking in small and often is a good idea, or you run the risk of large, complex competing merges. I believe it is git, but from the Mendix Studio Pro IDE, it is seamless.  

Build a local Version using Mendix Studio Pro and deploy it to the cloud. There are several options, including on-prem. The free version is basic and has limitations, but it has proven helpful for learning.

Note: There is also a Mendix Studio (Not Mendix Studio Pro) but it is hardly used by anyone.

Mendix supports sprints and boards, so you can work with User Stories in the Developer Portal for ALM.

An App Package can be stored, and it is a good idea to use this as the base for all projects in your company so that basic branding and naming conventions are consistent.

Deployment anywhere such as on-prem. via Kubernetes deployment, as well as the major cloud platforms i.e. AWS, Azure, GCP, Oracle.

Marketplace - templates, connectors, and components to reuse. 

Domain Modelling is excellent. When creating an app, you can choose your database (not the canned version). Modelling and exposing via OpenAPI contract and generating CRUD screens are easy. 

Get Data Options

Types of Data Sources available in Mendix.

Logic/Programing: In Mendix, we use flows for logic; there is no programming language.

• Microsflows - Server-side logic.
• Nanoflows - Smaller basically client side (think JavaScript logic)

Tip: The variable state is typically stored within the local Postgres database for later use.

Publishing to cloud production versions is very easy, and the local version, which is being developed, is seen on localhost. Mendix has simple, great built-in source control.

Image 1. A high-level overview of the logical components making up Mendix.

Pros:

1. Easy to use.
2. The basics for Low code are included, such as version control, project management, and deployment/publishing.
3. Build native mobile apps.
4. Improve business processes efficiently.
5. Supporting multiple languages is unbelievably simple and easy.

Image 2. Add multiple Languages to your app

Simple exercise: Call a key-secured API and display it on a Mendix page after watching this 7-minute video on API Calls.

The running example has:

1. Various pages and forms show and persist database information. 
2. A REST Call to a 3rd party using the OAuth key.  
3. Publishing a REST API based on a table and an associated entity.
4. Displays an Azure Chatbot

Me playing around with a Mendix App:

1. Get a REST endpoint and verify using postman (using a key for secure access)

Image3. Postman showing the REST call to be used

2. Create a new "microflow" as shown below:

3. Add a new "Action" of type "REST Call"

4. Add a JSON Structure file

5. Decide which attributes to pull out

6. Create an "Entity" in the Domain model to hold the retrieved data.

7. Map Model to the Import as shown below

...

Mendix Series

1.  Overview of Mendix (this post)

2. Mendix - Part 2 - Diving deeper

3. Extensibility for Mendix Studio 

4. Building a Mendix Widget for the Mendix marketplace (Convert text to Audio using Azure AI)

5. Mendix Tips & Thoughts

6. Competitors

Power Platform Competitors

Mendix Series

1.  Overview of Mendix (this post)

2. Mendix - Part 2 - Diving deeper

3. Extensibility for Mendix Studio 

4. Building a Mendix Widget for the Mendix marketplace (Convert text to Audio using Azure AI)

5. Mendix Tips & Thoughts

6. Competitors

Overview: I like Power Platform, but other options exist. Most competitors' products cover a piece of what the Power Platform covers.

Originally Generated with Thanks to Gareth R.

Mendix Vs PowerApps: 

• Mendix and PowerApps are compared across various criteria. Mendix is suitable for complex, enterprise-grade applications with extensive customisation options but is generally more costly.
•  PowerApps is ideal for rapid app development, integrates seamlessly with Microsoft services, and is more cost-effective for smaller projects. Larger projects require premium licenses per user, so PowerApps can become extremely expensive. Once you build many apps and your user base has premium licencing, PowerApps becomes extremely economically viable.  
• Mendix scores higher in integration capabilities, scalability, performance, innovation, and accessibility, while PowerApps scores higher in ease of use, development speed, customization, and support and community.  
• Many factors affect which low-code tool to use, including what the developers are trained in, how many apps you build and support over the next 3-5 years, and what licences and technology stacks your business currently has. Getting Canvas app developers is far easier and cheaper than getting Mendix developers and experts.
• A common scenario for businesses that use Mendix is having Office 365 E5 plans; then, it is about picking the right technology for the right project.  
• Mendix excels in Factory and manufacturing applications.

Mendix Studio Pro version 10.x

Mendix: I have used Mendix a fair amount and like the product. Mendix is straightforward, logical and easy to learn. 

Cons: 

1. The publishing is very slow.
2. For the free version, you have to use the Mendix subdomain. The basic plan is $50/month for my small demo. The cost of a Mendix licence and hosting is high for a one-off project; therefore, it is more suitable for building medium-sized projects.
3. Each app has a separate database, which needs to be managed or, in the cloud-hosted scenario, does not allow direct access except via backups or REST OData API's. You can build multiple front ends, but this is more of a workaround than a good idea.
4. For decent ALM, follow Mendix guidance as it is tightly coupled to the IDE/dev process.
5. Building many apps/gap-filling apps, as well as large complex apps, can overwhelm Mendix.

Outsystems is the other recognised leader in the Low-code space. Its focus, to me, is on building large, complex solutions faster using low code.

Outsystems focuses on Low code for large enterprise systems. It is big on flows like Mendix to build logic, but unlike Mendix, which only allows custom Java code in widgets, Outsystems allows custom code in multiple languages to be integrated.

Outsystems can use either SQL Server or Oracle for its main data store. Similar to Mendix, Outsystems can expose REST APIs via automatic generation. Both products are good at consuming APIs.

Both systems allow for MFA IDP SSO authentication. Data is encrypted at rest and in transit by default. Both have many compliance standards, such as ISO and GDPR.

I like the way Outsystems can use AAD Security Groups; Mendix can be awesome, but it's not my favourite.

I'd choose Outsystems if I were building a large, complex system using low code over Mendix, assuming no in-house skills in either.

Airtable:  Has lots of templates and easily connects to various data sources.  Easy to extend or build apps on the platform. Provides storage and low-code apps.  It's like having Dataverse and Power Apps.  There are pre-built templates to get the team off to a start.

Nintex: Bought K2 and have a long history in workflows (workflows for SharePoint and O365), screen/form generation and form building.  If your company uses Nintex, it's worthwhile, but I wouldn't use it for new projects or if the team does not have significant experience.

UiPath is an RPA tool that is strong for automation. UiPath is ahead of Power Automate Desktop (PAD) from a desktop automation and recording perspective. Both UiPath and Power Automate allow for attended and unattended runs. PAD is part of Power Automation Premium or Power Automate Process licences. Automation Anywhere and Blue Prism are other prominent players in the RPA space.

Postman Workflows: It's a dark horse, but people love Postman, and this may become an interesting option for Rapid development.

Amazon: AWS has various services that allow for Low-code solutions. "Amazon Honeycode" is model-driven, and Canvas apps can call code running as a service, i.e., Lambda's, which is the same service as Azure Functions on AWS. AWS Honeycode has predefined templates as starting points, which is more like Salesforce's low-code approach. This allows the developer to break out and write complex logic or persist the database in S3 storage. "Amazon QuickSight" works like Power BI to report on solution data.

Salesforce Lightening: Allows for the building of custom apps and the utilisation of Salesforce CRM and its data. 

Retool: Good set of connectors for APIs.

Appian: This drag-and-drop UI is for rapid development. It has RPA built in. I hear it's not super easy for non-tech staff, and customisation can be difficult. It is aimed at enterprises, so cost and performance need to be watched. There is no native database; probably only if a business is already into Appian would I stick with it.

Further out competitors:

Windsuft.io

Mendix Series

1.  Overview of Mendix (this post)

2. Mendix - Part 2 - Diving deeper

3. Extensibility for Mendix Studio 

4. Building a Mendix Widget for the Mendix marketplace (Convert text to Audio using Azure AI)

5. Mendix Tips & Thoughts

6. Competitors