Artificial Intelligence as a mega trend

Overview of AI

The world has gone mad for AI.  Constantly, I see overhype and poor messaging leading to misunderstanding of potential.  

AI is not new, the ability to use it for commercial gain is new at the scale we now have.  AI is great at helping us find patterns, get information and is primarily a decision support system (DSS).

AI is not smart, good at making complex decisions, and has bias.

This means AI is useful for specialization not generalization of "smartness", now as ChatGPT et al. is wide ranging, people are assuming it is a general area tool.  Actually, ChatGPT is specialist in breaking up and grouping language on top of a data source.  For those in the technology industry we pretty much know that ChatGPT is a good google (search engine).  

So, what is AI going to be successful at?  Well this is my predation:

AI will impact massively, in aspects of each industry:

1. Healthcare - guess what more surgeons and people will be needed not less.  Here I focus on Healthcare examples.  People need people to interact with avatars are a joke, I can talk to alexia already.  There is very little to nothing in this space except for snake oil sales men.  Please prove me wrong! More skilled people needed.

2. Software development/IT - here is a big one.  Programmers roles will change significantly, people with good understanding and knowledge will thrive and people with superficial knowledge and lack of ability to truly understand and work thru challenges will disappear.  Technologist will focus on difficult problems and add unbelievable improvements to all business processes.  The amount will continue to grow.  There is not a lot for agentic, "smart AI" int he space and we are 50 years away from this imo.

3. Manufacturing - it won't make the impact where the media says it will.  We are good at manufacturing.  The sub functions that will benefit will be things like machine maintenance, using sensors, performance/behavior will change.  This will allows you to improve Machine Maintenance (MM) and scheduling.  Think railway lines, they need to be shut down and it costs millions to trim hedges, imaging now you know the level crossing "lifty uppy-doowny" box/bar is showing signs of fatigue.  Shift the fix left and save un scheduled breakdown, the train line and it's know on-effects result in a massive improvement.  We are already good at manufacturing and lean and automation, the improvement, order of magnitude better is in machine maintenance not product improvement.  More skilled people need.  More skilled people needed.

Things like defect detection are already well established using Visual AI down the mm or less.  Rubbish detection, using AI will be better - sure it will get cheaper and easier to buy these system but AI is merely the enabler and its been available for well over a decade.  More skilled people needed.

4. Service Industry - Robots serving people, please, its mad, except at MacyD's (MacDonalds) and honestly, min wage workers are pretty efficient there and it will be too sterile.  Pushing out patties, well if you need AI for this, you don't know what AI tries to do.  automation yes, but in processing, packaging it is already there.  Big stuff with AI will be in social media and advertising (and don't get me started there, automated advertising will absolutely fail, we need to invent a missile to destroy non-human posts).  More people will be needed in services.  

Analogy:
1. Old technology: Hand weaving material was big profitable business in Britain, along came looms, these workers got upset and broke the looms, and ended up in prison or broke, these were the luddites (refused to embrace technology).  The luddites, ended up broke and all could have been avoided by embracing technology as they know the most about material and production they are the natural experts.   

2. Trend jumpers on: Too many companies wanted to build looms and a handful of players did brilliantly and still exist today.  Think Microsoft, AWS, they are transitioning from being programming technology companies to AI technology companies.  They still solve the same problem of process improvement.  The weavers that decided to go into building, repairing looms did exceptionally well but ultimately ran out of requirement and their price was driven down as there was enough supply.  Still a good change.  A lot of people also got smashed here, be careful inventing the technology in processes, you get it right, you are a hero, get it wring, go find a new job.  Lots of sales silver bullets are being produced.  Lot's of AI experts, absolute rubbish.  With rare exception, you are not an AI expert unless AI was in you job description more than 5 years ago.  Beware the snake oil sales men, nowadays they come in many forms, sizes and shapes :)

3. Embrace change:  Normal common-sense (smart) people, realized they actually had 4 options:

1. Learn how to use a loom.  Use the technology available and use it to build garments faster;
2. Build looms and support the loom business;
3. Do nothing, continue to offer hand weaving labor to the market.  So take your pension and hope like hell you win the lottery (i'm still backing this option for myself); or
4. Expert hand craftsmen or women :) become the best hand weaver int he world and people pay you for your expertise, these people's descendants/business still exist.  But big surprise: it's hard, it takes a long time, it's unlike to make you rich.,, so sure go do this if you are a genius in your field and love it, but don't die of surprise when you go broke or don't get the return you deserve for all that hard work.

Summary: Embrace technology and AI, it is only a decision support system.  More skilled people are needed, as you have the background, being skilled and embracing change means you are more in demand.  Sitting on your backside waiting for the lottery means you are like 90% of people and you'll get 2 jet skis and a new husband! yipee.

Healthcare

Good Use case: Diagnostic medicine

Diagnostic medicine has become the center of health care and the ability to use AI, which is better at detecting abnormalities than the best radiologist using a single trained model, results are in near time.  This means consultant Radiologist and specialists can get reports in seconds that are of unbelievable quality.  GP's have best guess within seconds rather than well... we all know this.

AI also give probability so it's easy to prioritize any reporting that is life threatening to a specialist so they are working om the hardest work and get the deep information provided by the AI.  

This is possible because we are talking about a relatively narrow field of data we have taught AI to deal with. Think of x-rays, the results are far superior to an expensive resource that takes at least 12 years to train.

Should we stop training Radiologists and diagnosticians and spend our money on AI?  Absolutely not!!   

Radiologists should be using the AI reports, validating, using the info and extrapolating, when an issue is detected, this must be added back into the learning model resulting in improving the AI.   AI should not act, it must only be used to support.  Acting should be restricted to notifying relying parties such as GP's on.  

Good Use case: Online GP Appointments and triage

My you have an issue, you go onto a NHS app that will ask you for your symptoms and ask a few follow on questions.  It will only give you it's best guess (this is already amazing imo.), this in turn, will triage your call into "go to your emergency department, they know you are coming", "let's book you an emergency appointment", or "this is my recommendation, and why".  Dr Google, actually becomes useful (weird medical insider joke).  Honestly we could do so much more but, care is given to the right people, "shift-left" (sooner you catch it the cheaper and better the solution, 1005 applies to healthcare.

Preventive medicine, and nudge-technology will have profound improvements for peoples lives and lifestyles.  Hooking into ambulance services, and driverless automated vehicles,..  people do the hard stuff and make the decisions AI does the piece efficiently and quickly that we as humans aren't good at. Hopefully you are understanding narrow vs wide industries.

Bad Examples: Robot Surgery or treatment rooms

Robots replace people in operating theatres.  it is insane!!  Potentially, a surgeon could us AI to get better diagnostic data sooner, they could even use tech like AI watching operations, then ping in messages if it thinks the actions are not optimum or there is a clear risk that a priority has changed.  It brilliant for decision support. it's not a good idea to try to 

openBIM for AEC understanding

Within the AEC industry, standards are necessary to ensure that all project stakeholders are speaking the same language, thereby improving collaboration.  We can also process data to automate various processes if the data is standardised.

BIM (Building Information Modelling) is used to improve collaboration on infrastructure projects.  BIM is essentially ISO 19650, and it has various levels.

Building Models contain 3D information that shows how assets fit together.  Each of these assets may contain properties that can be used to look for clash detections.  Think of a CAD diagram, it lays out the plans for a building so all parties can see the proposed plan.  As CAD technology advances, you can add more information about the project.  For example, as an electrician, I only want to see the layers that affect my work.  CAD can be further extrapolated to show products and material information.

closedBIM: These were the original big BIM systems, including AutoCAD, Revit, and Bentley ProjectWise.  These tools feature visual editors and viewers, allowing them to securely store the files needed for a project and ensure that the appropriate people have access.  These all have their own proprietary standards.

openBIM: Read other parties' data, improves collaboration and consensus.  Easier to switch tools to reduce costs or get better features.  Consists of:

1. IFC (common language)
2. bSDD (industry common language)
3. IDS (Requirement specification)
4. BCF (check)
5. openCDE (sharing with APIS)

Industry Foundation Classes (IFC) serve as the basis for standardising how information is handled.  Has standards for location, such as geographic information.  Materials, Geometry, and Spatial Structures are covered by IFC classes.  In each industry, these base IFCs are added to.  The BuildingSmart bSDD is an extension of IFC for specialised industries and sectors, published to provide more specific, agreed-upon standards.  

Project Requirements: These can vary, but having an agreed-upon format, such as an Information Delivery Specification (IDS), is helpful. Although it is not necessary or widely used, it ensures that precise details are provided.  Therefore, collaboration allows all parties to clearly understand what is needed using IDS.

IDS uses bSDD, which is based on IFC, so that the requirement specifications are precisely laid out.

openCDE defines technical interfaces, .....

AI Vendor Management - Formiti

AI is going crazy, and you can build your own but generally you need to look at a supplier, so it's worth understand management of Vendors, you as the controller using their service are at risk of them not making their AI operations transparent.  It's a big business risk to my clients.  

GDPR is closely linked to AI, and if you use a service/vendor, the reputation and fine risk may fall on you as the provider.  Need visibility into each vendor, how they are using AI, in turn they are using vendors so it's a nice complex dependency problem.  You need to be aware of what you are relying on.

Ensure contracts with vendors consider AI, how the process your data and how their sub process vendors do the same.

Track website customer behaviour, we use a vendor to clean up the data.  In turn, I have no idea that they are using AI outside of the UK or EU.  Follow the dependency chains as all this needs to be transparent to the end customer if needed.

Copilot Studio 2025 Notes

Copilot Studio is amazing, the licensing is complex, the AI integration is excellent. Architects really need to understand Licensing and billing or AI will get out of control.  The Purview and governance looks very good.  Copilot Studio Cost Estimator (preview June 2025)

MS Build 2025: 

MCP Server in Preview - possible to collect data from other AI services or write back.

Connector Kit - So can add custom connectors to from Power Platform Connectors including Copilot Studio - sounds great.

Agent Flow - Added functionality added to Power Automate flows (Copilot Studio aware), deployed via solutions.

Note: M365 Agent Toolkit is looking interesting to allow agents to do tasks with Office add-ins done using VS Code.

Licensing

You need to be aware:

• M365 agents - need all end users to have M365 copilot licences, retail $20/user.  Alternatively users can consume the agents using a PAYG model per message (it racks up quickly).  Can add these to MS Teams and it appears then the people with licences can ask the M365 agent and others see the results (some more testing and understanding is need here by me).
• Copilot Studio - Makes need copilot studio AI Studio/maker licence $30/retail, users don't need any licence to use but you pay per msg and this can rack up nice and quickly so watch the usage.  Buy in bulk message credits can help reduce the cost.
• Each prompt generates multiple messages, these are all billable (complex to calculate)
• (If you use Copilot Studio and it calls Azure AI Foundry, also bills Tokens (also complex to estimate)
• Copilot Studio is using AI Foundry connector, it is a Premium connector)

Playwright Post 6 - Automating Canvas App MFA login for Playwright unattended for Canvas apps

Overview:  Modern security makes automating logins requiring MFA rather difficult.  This post looks at possible approaches to automate the login.

Option 1. Turn off MFA—not really, but you can set a conditional rule in EntraId to not perform MFA. This is not an option in many enterprises.

Option 2. Time-based One-Time Password (TOTP)—Microsoft Authenticator makes this pretty difficult. At least I can't do it, as the APIS are relatively limited. This is kind of expected, as it's a security measure.

Option 3. Programmatically acquire an access token without browser automation, use MSAL with a client secret or certificate (for confidential clients). 

Option 4.  Use Playwright to record the login and intercept the access token once logged in.  Then you can store it and use it.  There are a few easy options to get the session:

4.1. Retrieve the access token from the response once logged in

4.2. Retrieve from your local storage:

  const token = await page.evaluate(() => {

    return window.localStorage.getItem('adal.idtoken') || window.sessionStorage.getItem('adal.idtoken');

  });

4.3. Retrieve the token using Playwrite at the command run level

Note: This adds the token to my repository. Don't save the token to your repository if you don't realise that the Access/Bearer token will expire depending on what your EntraId sets. The default is 1 hour.

Option 4.3.1. Like option 4.3, use the refresh token to silently generate a new Access token. You store the refresh token during the recorded login (by default, it lasts for 90 days) to generate a new access token when you need it.

Option 4.3.2.  Take it further back to generate the refresh token using the access code you get at the original login, renew the "refresh token", and generate a new access token to run your tests.

If you decide to store your access token, refresh token or code, don't store them in your code repo.  You know why if you got this far.

Thought: as a refresh token works for 90 days on a sliding scale, I've never used the option 4.3.2, as by storing the refresh token, all I need to do is to extend the refresh token by using it to get an access token and the refresh token has 90 days from that point. 

This is the plan I'm thinking of using:

1. Store the refresh token in Azure Key Vault (AKV).
2. Your app retrieves the refresh token from AKV.
3. Uses the refresh token to get a new access token.
4. Store the new access token to run my ALM/CI/CD playwright tests

Playwright Series

Playwright Post 1 - Overview of E2E testing using Playwright

Playwright Post 2 - Continuously Test/Monitor Canvas apps and website with MFA enabled

Playwright Post 3 - Add App Insights logging inside your Playwright tests 

Playwright Post 4 - 6 Min walkthru of Playwright testing with Azure Monitor

Playwright Post 5 - Understanding how Playwright works

Playwright Post 6 - Unattended testing when secured with MFA (this post)

Entra AAD Security Groups - Remember

Overview: I have lost count of the number of poor Active Directory and Azure Active Directories I have seen.  I don't think I've ever seen a good Active Directory actually.  Certainly nothing large over 5K users. 

I'm working with a multinational, and we need to improve the security.  Things are a little all over the place, oddly named and inconsistent, basically the normal for an 300k internal user enterprise with history and multiple aquations.

I identify a coupe of properties that will really create a nice hierarchy, issue is I'm using more than the allowed 5k Dynamic AAD Security Groups.  

Group Types to be aware of relating to Entra

1. Static AAD Security Groups

Got to add the users manually, or at least automate the process for anything but the smallest of Entra users.

Static AAD Security groups can be nested.

3. Dynamic AAD Security Groups

Up to 5,000 dynamic groups.

You can inherit Security groups or be inherited (no nesting).

3. Distribution AAD Groups

Used for email and calendars, not security.

4. O365 Groups/Teams Groups

They can inherit O365 groups or AAD Security groups.  They are managed within the org so not the best idea to place heavy security on manually managed teams. 

Resolution:

I have a full hierarchy of users within divisions and subdivisions.  By adding users statically via automation to there lowest level AAD Security Group.  Then I can add the child groups.  This gives me multiple groups that have more and more users in as we go up the hierarchy.  Additive groups with positive security gives me the best options.  

Future Wishes:

If only Entra supported more dynamic AAD Groups per tenant or allowed Dynamic groups to be nested in static AAD groups

Playwright Post 5 - Understanding how Playwright Works

Playwright as a tool consists of two main parts.

Part 1: Playwright Library: This is the automation of a browser using the Page Object Model (POM). It provides a uniform API to run against the 3 main browser engines, automating tasks like navigating, clicking, filling in form data, and validating content on a web page. Classes include APIRequest, APIResponse, and BrowserContext. The worker process runs the API calls sequentially. Unified library API calls are sent to the browser context, which runs unaware of the calling context.  

Top link runs in Node.js and makes API library calls, there is no timing between the Node.js (Controller) and the browser instance (running Chromium instance)

Part 2: Test Runner: This part runs the Playwright tests.

Playwright Series

Playwright Post 1 - Overview of E2E testing using Playwright (this post)

Playwright Post 2 - Continuously Test/Monitor Canvas apps and website with MFA enabled 

Playwright Post 3 - Add App Insights logging inside your Playwright tests 

Playwright Post 4 - 6 Min walkthru of Playwright testing with Azure Monitor

Playwright Post 5 - Understanding how playwright works (this post)

Playwright Post 6 - Unattended testing when secured with MFA 

Other Posts

Upgrading two C# Blazor web applications and verifying using Playwright - super fast 

Mendix - Part 2 - Diving deeper (E2E automation testing of Mendix using Playwright)

Low-code testing with playwright walkthru

Continiously Monitor Apps using Playwright with TS

Testing Canvasa apps with Playwright using C# (rather use TS, it's better)