Sunday, 30 November 2025

SharePoint for BIM

Overview: Building Information Modelling (BIM) Enterprise Document Management Systems are expensive.  I decided to use SharePoint instead, as we already have SharePoint licencing, security is approved, and it works with Entra MFA.  SharePoint offers great API's and is well-known and easy to manage.  Lastly, I got much better performance from SharePoint than the other two dedicated Doc Mgm Systems I evaluated.

Concerns:

  • BIM version, numbering, and workflows: No issues seen; merely need a simple workflow
  • Web Viewer for CAD files

Advantages:

  • Part of the Microsoft ecosystem
  • Free, as we all have E3 licences
  • Users know how to use and manage SharePoint, so no training is required
  • SaaS is running and is part of our business SLA, with 99.9% uptime.  SharePoint has an active-active architecture, built-in backups and data is stored in multiple locations
  • Reduced setup and no external 3rd party requiring contracts and approvals.  No supplier has nearly as many compliance assertions including ISO27001, SOC1, SOC2, SOC3, GDPR, 
  • Security is already ready with the client Entra userbase with MFA.  DLP and sensitivity labels.   Great built-in data residency, audit logs and retention policies.  File Sync is super helpful in working with large CAD files, especially in remote locations.  All data is encrypted at rest and in transit.
  • SharePoint is widely used in construction projects.  Customers and third parties can securely access SharePoint Online for central collaboration.
  • Mobile-friendly, tool-friendly, management-friendly approach to BIM.
<ProjectCode>-<CreatedDivision/Partnercode>-<DocType>-<Discipline/Category>-<IncrementNo>
BLD123-MACE-CAD-ELE-00001

HLD Architecture Designs for Finding SharePoint File data

 SharePoint data source using Microsoft Foundry to build an Agent



Sunday, 2 November 2025

Edge Computing Notes (IoT)

WIP

Edge Computing is where computing is done close to IoT devices and machines/PLCs.  Basically, if it happens on the "edge" of your network.  The processing occurs on local devices, gateways, or edge servers near IoT sensors, cameras, or machines.

  • Low Latency: Ideal for applications like autonomous vehicles, industrial automation, and AR/VR.
  • Bandwidth Efficiency: Only relevant data is sent to the cloud, reducing costs.
  • Reliability: Systems can continue functioning even with intermittent connectivity.
  • Over the past few weeks, I ordered a Raspberry Pi, which I intend to use for processing data from remote IoT sensors, namely Cameras, LiDAR, and temperature.

    MQTT is the programming language to ...

    Node-RED provides a web browser-based flow editor, which can be used to create JavaScript functions (Wikipedia)

    Azure IoT Edge is a cloud-to-edge computing platform that extends Azure services and custom logic to IoT devices. It allows you to deploy containerised workloads (such as AI models, analytics, and business logic) directly on edge devices, enabling local data processing and reducing reliance on continuous cloud connectivity. This improves latency, bandwidth efficiency, and resilience in offline scenarios.

    IoT Edge Modules are Docker-compatible containers running Azure services, third-party apps, or custom code. Examples: Azure Stream Analytics, Azure Machine Learning models. [learn.microsoft.com]

    IoT Edge Runtime must be installed on each edge device. Handles module deployment, security, and communication between modules, devices, and the cloud.

    Includes:

    IoT Edge Agent: Deploys and monitors modules.

    IoT Edge Hub: Manages messaging between modules and the cloud. [learn.microsoft.com]


    Azure IoT Hub is a cloud service for managing IoT Edge devices, configurations, and monitoring.


    A Raspberry Pi has an OS, whereas an Arduino device has no OS:

    • Arduino is an Application + Microcontroller (MCU) - Compile code in C++ or C
    • Pi is Application + Library & System Functions (OS) + MCU



    Why Use Ada on MCUs?

    • High reliability (ideal for avionics, medical devices).
    • Built-in concurrency and timing features.
    • Safer than C/C++ in many cases due to strong typing and runtime checks.

    • If you need maximum safety and reliability, Ada is superior.
    • If you need performance, simplicity, and broad support, C dominates.
    Ada is a strongly typed, multi-paradigm programming language originally developed for the U.S. Department of Defense to ensure reliability in mission-critical systems.
    Setup:
    • Install GNAT (GNU Ada Toolchain)
    • GNAT Studio or VS Code with Ada extensions.
    • Run gnat --version in your terminal
    • Write the code:

    Compile & Run> gnatmake hello_mobility.adb

    Friday, 31 October 2025

    Playwright Agents in VS Code

    I started looking at the latest version of Playwright late last night. The Agents add-in for VS Code is amazing.  I can't stop improving my code, my tests, and my automation.  It is highly addictive.

    Playwright 1.56.1 includes the new Playwright CLI, which has the test agents as shown in VS Code above:

    Node: v22.21.0
    npx: 11.6.2
    Playwright test package: ^1.56.1

    Sunday, 12 October 2025

    Federated and Event Driven Architecture

    Event Driven Architecture (EDA): System components interact with each other by producing, detecting and reacting to events.

    Event-driven APIs differ from conventional REST APIs, offering improved scalability, strong service decoupling, reduced network traffic, and greater flexibility. Even-driven APIs need to address the challenges of monitoring, distributed tracing, security, and versioning.

    Distributed, event-driven architectures are a powerful approach to building decouple long-running, high-performance, scalable systems and can use Conflict-Free Replicated Data Types (CRDTs) to provide eventual consistency.


    An Event Mesh is a dynamic, interconnected network of event brokers that allows events to flow seamlessly across distributed applications, regardless of where they are deployed—on-premises, in the cloud, or at the edge.

    Federated architecture allows each system to flexibly interact with other systems while remaining independent, so it can be easily extended, and individual pieces (a system) can be replaced relatively quickly.

    Thoughts: Like Cross cutting concerns where you build IT runway to perform specific tasks and then call them, Federated Architecture, each system does a job so for instance, there is a standalone system that can be replaced or extended for Requesting a Room (1st system), this allows the user to reserve a room using the booking system (2nd system), this in turn calls the communication system that handles email, teams meeting with reminders (3rd system) and then calls the communication systems (n services/systems)

    Events are facts; they are loosely coupled to the booking system.  This approach allows for the reuse and easy creation of highly customised business processes.

    Thought: Choosing between an Event Mesh and a Federated Architecture...

    Thursday, 9 October 2025

    Medallion Architecture in Fabric High Level Organisation Design Pattern

    Microsoft Fabric is excellent!  We do still need to follow good practices we have been using for years, such as making data accessible and secure.   Possibly the most used architecture for Big Data is the Medallion Architecture pattern, where data is ingested normally in a fairly raw format into the bronze layer, then transformed into more meaningful and usable information. Lastly, the gold layer exposes data relationally using semantic models to reporting tools.

    Overview: This document outlines my attempt to organise enterprise data into MS Fabric using a Medallion Architecture based on Fabric Workspaces.  Shortcuts are better than imported data, but it does depend on factors such as what the data source is, what data we need, how up-to-date the data is and performance requirements from the systems involved.

    The reports and semantic models can get data from other workspaces at any of the medallion layers.  This architecture lends itself well to using the new Direct Lake Query mode.

    Summary of a Design used by a Large Enterprise:

    Medallion architecture using Fabric Workspaces.

    Friday, 26 September 2025

    Microsoft Fabric High-level architecture

    Overview: Microsoft Fabric is an end-to-end analytics platform that unifies data movement, storage, processing, and visualisation. It integrates multiple services into a single SaaS experience, enabling organisations to manage their entire data lifecycle in one place.  One Lake is at the core of MS Fabric.

    Image 1. One page High-Level Architecture of MS Fabric. 

    European Fabric Conference in Vienna Sept 2025 takeways

    FabConEurope25 was terrific in Vienna last week.  Great opportunity to meet Fabric and data experts, speak to the product teams and experts, and the presentations were fantastic.  The hardest part was deciding which session to attend as there are so many competing at the same time.  

    My big takeaways:
    • Fabric SQL is excellent.  The HA, managed service, redundancy, and shipping logs ensure that OneLake is in near real-time.  Fabric SQL supports new native geospatial types.  SQL has temporal tables (old news), but row, column and object-level (incl. table) security is part of OneLake.   There are a couple of things security reviewers will query, but they are addressed.
    • Fabric Data Agent is interesting.  Connect to your SQL relational data and work with it.
    • User-defined functions (UDF), including Translytical (write-back), HTTP in or out, wrap stored procedures, notebooks,.... - amazing.
    • OneLake security is complex but can be understood, especially with containers/layers, such as Tenant, Workspace, Item, and Data.  There is more needed, but it's miles ahead of anything else, and Graph is the magic, so it will only continue to improve. - amazing, but understand security.  Embrace Entra and OAuth; use keys only as a last resort.
    • Snowflake is our friend.  Parquet is fantastic, and Snowflake, including Iceberg, play well together with MS Fabric.  There are new versions of Delta Parquet on the way (and this will even make Fabric stronger, supporting both existing and the latest formats).
    • Mirroring and shortcuts - don't ETL unless you need to shortcut, then mirror, then ETL.
    • Use workspaces to build out simple medallion architectures.
    • AI Search/Vector Search and SQL are crazy powerful.
    • New Map functionality has arrived and is arriving on Fabric.  Org Apps for Maps is going to be helpful in the map space.  pmtiles are native... (if you know you know)
    • Dataverse is great with Fabric and shortcuts, as I learned from Scott Sewell at an earlier conference.  Onelake coupled with Dataverse, is massively underutilised by most orgs, 
    • Power BI also features new Mapping and reporting capabilities related to geospatial data.
    • Other storageCosmosDB (it has its place, but suddenly, with shortcuts, the biggest issue of cost can be massively reduced with the right design decisions).  Postgres is becoming a 1st class citizen, which is excellent on multiple levels. The CDC stuff is fantastic already.
    • RTI on Fabric is going to revolutionise Open Telemetry and AI, networking through the OSI model, application testing, digital twins, and live monitoring,....  I already knew this, but it keeps getting better.  EventHub and notebooks are my new best friends.  IoT is the future; we all knew this, but now with Fabric, it will be much easier to implement safely and get early value.
    • Direct Lake is a game changer for Power BI - not new, but it just keeps getting better and better thanks to MS Graph.
    • Manage Private Endpoint as improved and should be part of all companies' governance.
    • Purview... It's excellent and solves/simplifies DLP, governance and permissions.  I'm out of my depth on Fabric Purview and governance, and I know way more than most people on DLP and governance. Hire one of those key folks from Microsoft here.  
    • Warehouse lineage of data is so helpful.  
    • We need to understand Fabric Digital Twins, as it is likely to be a competitor or a solution we offer and integrate. 
    • Parquet is brilliant and fundamentally is why AI is so successful.
    • Powerful stuff in RDF for modelling domains - this is going to be a business in itself.  I'm clueless here, but I won't be in a few weeks.
    Now the arr..
    • Pricing and capacity are not transparent.  Watch out for the unexpected monster bill!  Saying that the monitoring and controls are in place, but switching off my tenant doesn't sit well with me if workloads aren't correctly set out. Resource governance at the workspace level will help fix the situation or design around it, but it will be more expensive.
    • Workspace resource reservation does not exist yet; however, it can be managed using multiple fabric tenants. Distribution will be significantly improved for cost control with Workspace resource management.
    • Licensing needs proper thought for an enterprise, including ours.  Reserve Fabric is 40% cheaper, and it cannot be suspended, so use the reserved fabric just as you would for most Azure Services.  Good design results in much lower cost with Workloads.  Once again, those who genuinely understand know my pain with the workload costs.
    • Vendors and partners are too far behind (probably due to the pace of innovation)
    Microsoft Fabric is brilliant; it is all under one simple managed autoscaling umbrella.  It integrates and plays nicely with other solutions, has excellent access to Microsoft storage, and is compatible with most of the others.  Many companies will move onto Fabric or increase their usage in the short term, as it is clearly the leader in multiple Gartner segments, all under one hood.  AI will continue to help drive its adoption by enterprises.

    Saturday, 13 September 2025

    Railway Infrastructure - Signalling

    Understanding Railways Series:

    UK Railway Industry for Dummies, focusing on Rail Infrastructure

    Railway Infrastructure - Signalling (this post)

      Signalling for Railways 

    Historically, trains were operated by a driver who knew a lot about the train, the route, and even the train itself.  Speed, signalling, and braking distance were controlled by the driver, and they had to manage the train, the driving, and the route information affecting them.  Signalling was used to help the driver determine whether the rail line was clear or approaching, when to slow down, and to provide reminders, thereby allowing the driver to control the train safely by managing multiple factors.

    Advancements in technology have built on existing infrastructure and approaches, enabling more trains, safer travel, and better information.  For example, wheel counters allowed signalling to ensure that a train and all its wheels have cleared a section.

    Telecommunications allow a central control centre (ROC) to communicate with the drivers.  GSM-R...

    The train should run regardless of temporary telecommunications issues or unavailability (assuming it is safe).  The onboard computer on every train can provide the driver with information or act on it, such as automatically stopping the train if a red light is detected while the train is on the section of track the driver is entering.  

    ETCS is for modern signalling; the fundamental principle is that all trains, routes and tracks can be managed centrally safely (see SIL4).  Everything on the railway has to be safe.  Super safe, if unsure, stop is the principle.

    Common Equipment

    Balise

    A balise is a device mounted on the track between the two rails that sends messages to a train as it passes overhead, primarily conveying the train's position.


    Understanding Railways Series:

    UK Railway Industry for Dummies, focusing on Rail Infrastructure

    Railway Infrastructure - Signalling (this post)


    Sunday, 17 August 2025

    GIS Notes

    What is GIS?

    GIS stands for Geographic Information Systems, which are tools and techniques for capturing, managing, storing, processing, and analysing spatial data. It is part of the broader geospatial technology ecosystem, which also includes drones, remote sensing, and GPS.

    Geospatial data (Raw)

    Definition: Any data that includes a geographic component, describing the location and attributes of features on Earth, contains raw information, like points, lines, and polygons, that has a real-world location associated with it.
    Examples: a car's GPS location or a customer's address.

    GIS data (Organised)

    Definition: Geospatial data that is structured, stored, and analysed using Geographic Information System software.
    Examples include a digital map of roads created from GPS data or layers of data showing flood-risk areas.

    Summary: Geospatial data is the foundation: the raw material for all things spatial. GIS is a toolset that may include tools like ArcGIS from Esri.

    Other:
    In the AEC space, building and Asset management rely heavily on GIS within BIM.
    ArcGIS is the industry leader in GIS tooling, and comes in three versions: 
    • Desktop (ArcPro, Arc Toolbox, ArcCatelog),
    • Server (), 
    • SaaS ArcGIS Online (AGOL).
     GIS data comes in various formats such as Shapefiles, GeoJSON, KML, or geodatabase feature layers.

    What WGS84 and GeoJSON Mean?  

    These are the most common formats for storing position (WGS84) and shape data with coordinates (GeoJSON) 

    WGS84 (World Geodetic System 1984) is the global standard geographic coordinate reference system. It represents positions on Earth using latitude and longitude in decimal degrees.

    GeoJSON is a widely used format for encoding geographic data structures in JSON. According to RFC 7946, all GeoJSON coordinates must use WGS84 (EPSG:4326).

    The OSGB/OSGB36 coordinate system, also known as the Ordnance Survey National Grid or British National Grid (BNG), is a geographic grid reference used in Great Britain.  

    The European Terrestrial Reference System 1989 (ETRS89).

    Standards and formats Notes

  • ISO 19115: International standard for geographic information metadata.
  • Shapefiles: Upload as a ZIP archive containing .shp, .shx, and .prj files.
  • File Geodatabases: Must be compressed into a ZIP file before uploading.
  • WMS/WMTS Services: Add Web Map Service (WMS) or Web Map Tile Service (WMTS) layers by providing the service URL.
  • Prefix Conventions

    • Points: Use prefix pt_
    • Polygons: Use prefix py_
    • Lines: Use prefix ln_
    • Raster: Use prefix rs_

    ESRI Competitors

    Esri have ArcGIS, which is the gold standard in GIS. It's pretty expensive if you don't need the feature set offered by AGOL/Esri.  
    Google offers easy-to-set-up and use mapping, geo services and APIs Google Maps Platform Pricing   . I'd usually choose this when hardcore analytics are not needed

    Common source of data used to mash onto maps

    DEFRA,
    Environmental Agency, 
    Natural England,
    Esri Living Atlas,



    Thursday, 7 August 2025

    GitHub Copilot with Claude Sonnet 4 is amazing, and GPT 5 is even better

    I couldn't sleep, so I decided to build a Pulumi C# application that uses an existing MCP Server. My forms will utilise the client to allow me to access my Azure subscriptions and resources - wow.  Build a really cool tool quickly - Claude Sonnet 4 is once again significantly better than GPT-4.1 for programming with GitHub Copilot.

    Update Sept 2025: I'm now using GPT-5 over Claude Sonnet with GitHub Copilot when programming in VS Code.  Both feel about the same quality to me.

    GitHub have this for comparing AI models for GHCP, which is very useful.

    I am using GPT-5-Codex, which "is a version of GPT-5 optimised for agentic coding in Codex".

    I am also really liking GitHub Copilot code review

    Anthropic's Claud 4.5 is also excellent..

    Wednesday, 30 July 2025

    AI for developers and Architects

    The cost of prototypes is unbelievably low using AI. 

    Rapidly creating a prototype, especially with new or less well-known technology, is where I derive significant benefits from AI.

    How to build application prototypes?

    1. Write /reverse prompt/Adjust instructions into md file
    2. Agentic AI (specialising in Doc Extraction) to extract and refine from md file
    3. Run using IDE-based copilot (VS Code with GitHub Copilot) (AmazonQ) (Cursor, Windsurf, Steamlit) 
    Thoughts: Developers are adjusting to using Ai to support software solutions.  The developer role will continue the trend of making technical implementation more accessible, allowing knowledgeable IT engineers or domain experts to build faster and better than citizen/amateur developers.  Ai assists in complex decisions!  

    What needs to improve?
    • Knowledge is key.  AI needs to have narrow expertise at the right time. i.e. only domain knowledge, not influenced by other data.  Quality of input data used to train.  Allows for dynamic reasoning.
    • Session/long-term contact agreement/understanding to improve the understanding between your IDE and me.  Remember how I prompt and provide feedback on how I digest information.  Context between the human developer and Ai is Paramount.
    • Control of IDE integration with coding copilots, clear return to the user developer to make better decisions.  Context is Paramount.
    • Governance & Data (Connectors, API's, code complex processes (MCP maybe), quality of data).

    Retrieval Augmentation Generate (RAG)


    AI needs to be able to connect to my existing Tool Landscape:
    I use Azure, C#, Playwright, and GitLab.  I want my IDE to work with these tools and many more.  MCP servers publish their functionality, and I can connect my Copilot/Agent to use multiple MCP servers.  This is what GHCP does for VS Code, allowing you to add MCP clients dynamically to use existing MCP Servers. 

    Model Context Protocol (MCP)

    MCP is a protocol (created by Anthropic) that allows an MCP client to connect to an MCP server, which in turn provides specialist knowledge. Authentication should use OAuth to secure access.

    My Applications/Agents use the MCP to ask the MCP Server, 'What can you do?' so they know how to use it.  The MCP server allows you to interact with a system.  It is often referred to as the "Arms and Legs" of AI.

    The MCP Server, when built, informs the client of its capabilities and then performs actions, such as updates, via an API.  

    Summary: Use MCP to enable the client to communicate with other resources/tools.

    NB: An Agent can utilise multiple MCP Servers.

    Agents-to-agent (A2A) 

    A2A allows agents to work together.  So two agents can leverage each other; the other agent solves the issue and returns the answer for the first agent to use.  Whereas MCP allows any agent to speak to a source.  Agents complete a task and give it back to the calling agent. 
     
    Summary: Use A2A to talk to specialised Agents, and the agent returns the calling agent's answers.


    Wednesday, 2 July 2025

    Artificial Intelligence as a mega trend

    Overview of AI

    The world has gone mad for AI.  Constantly, I see overhype and poor messaging leading to a misunderstanding of potential.  

    AI is not new; the ability to use it for commercial gain is new at the scale we now have.  AI excels at helping us identify patterns, gather information, and is primarily a decision support system (DSS).

    AI is not intelligent, but rather good at making complex decisions, and it has biases that we teach it.

    This means AI is useful for specialisation, not a generalisation of "smartness", now that ChatGPT et al. are wide-ranging, people are assuming it is a general-purpose tool.  Actually, ChatGPT is a specialist in breaking down and grouping language based on a data source.  For those in the technology industry, we pretty much know that ChatGPT is a good Google (search engine).  

    So, what is AI going to be successful at?  Well, this is my prediction:

    AI will have a massive impact on many industries:

    1. Healthcare guess what? More surgeons and people will be needed, not fewer.  Here, I focus on Healthcare examples.  People need to interact with others; avatars are a joke. I can talk to Alexa already.  There is very little to nothing in this space except for snake oil salesmen.  Please prove me wrong! More skilled people are needed.

    2. Software Development/IT - This is a significant one.  Programmers' roles will change significantly; those with a good understanding and knowledge will thrive, while those with superficial knowledge and a lack of ability to truly understand and work through challenges will likely disappear.  Technologists will focus on challenging problems and introduce significant improvements to all business processes.  The amount will continue to grow.  There is not a lot of agentic, "smart AI" in the space, and we are 50 years away from this, imo.

    3. Manufacturing - it won't make the impact that the media says it will.  We are good at manufacturing.  The sub-functions that will benefit include machine maintenance, sensor usage, and performance/behaviour will change.  This will allow you to improve Machine Maintenance (MM) and scheduling.  Think of railway lines; they need to be shut down, and it costs millions to trim hedges. Imagine now that you know the level crossing "lifty uppy-doowny" box/bar is showing signs of fatigue.  Shift the fix left and save the unscheduled breakdown; the train line and knock-on effects shall result in massive improvement.  We are already proficient in manufacturing and, to some extent, automation. If the AI is not significantly better, it is not worthwhile. More skilled people are needed.  

    Machine Maintenance in Manufacturing.  AI is needed to mature MM. 

    Techniques such as defect detection are already well-established using Visual AI at the micron level.  Rubbish detection.  Using AI will be beneficial - sure, it will become cheaper and easier to acquire this system capability for less, but AI is merely an enabler, and it has been available for well over a decade.  More skilled people are needed.

    4. Service Industry - Robots serving people, please, it's mad, except at MacyD's (McDonald's) and honestly, minimum wage workers are pretty efficient there, and it will be too sterile.  Pushing out patties, well, if you need AI for this, you don't know what AI tries to do.  AI & automation are already in the processing and packaging processes.  The big stuff with AI will be in social media and advertising (and don't get me started there, automated advertising will absolutely fail. We need to invent a missile to destroy non-human posts).  More people will be required in these new and changed services.  

    Analogy:
    1. Old technology: Hand weaving material was a big profitable business in Britain; along came looms; these workers got upset and broke the looms and ended up in prison or broken; these were the Luddites (who refused to embrace technology).  The Luddites ended up broke, and all this could have been avoided by embracing technology, as they were the most knowledgeable about materials and production. They are the natural experts.   

    2. Trend jumpers on: Too many companies wanted to build looms, and a handful of players did brilliantly and still exist today.  Think Microsoft and AWS; they are transitioning from being programming technology companies to AI technology.  They still solve the same problem of process improvement.  The weavers who decided to go into building and repairing looms did exceptionally well, but ultimately ran out of demand, and their prices were driven down as there was an excess of supply.  Still a good change.  Many people also got hurt here. Be careful inventing new technology in processes; you get it right, you are a hero; get it wrong, go find a new job.  Lots of sales silver bullets are being produced.  There are tons of "AI experts", but mostly this is absolute rubbish.  With rare exceptions, you are not an AI expert unless AI was in your job description more than 5 years ago.  Beware the snake oil salesmen, nowadays they come in many forms, sizes and shapes :)

    3. Embrace change:  Normal, common-sense (smart) people realised they actually had 4 options:

    1. Learn how to use a loom.  Use the technology available and use it to build garments faster;
    2. Build looms and support the loom business;
    3. Do nothing, continue to offer hand-weaving labour to the market.  So take your pension and hope like hell you win the lottery (I'm still backing this option for myself); or
    4. Expert hand craftsmen or women :) Become the best hand weaver in the world, and people pay you for your expertise; these people's descendants/business still exist.  But big surprise: it's hard, it takes a long time, it's unlikely to make you rich.,, So, sure, go do this if you are a genius in your field and love it, but don't die of surprise when you go broke or don't get the return you deserve for all that hard work.

    Summary: Embrace technology and AI, it is only a decision support system.  More skilled people are needed, as you have the background, being professional and embracing change means you are more in demand.  Sitting on your backside waiting for the lottery means you are like 90% of people, and you'll get 2 jet skis and a new husband! yipee.

    Healthcare

    Good Use Case: Diagnostic medicine

    Diagnostic medicine has become the centre of healthcare, and the ability to use AI, which is better at detecting abnormalities than the best radiologist using a single trained model, yields results in near real-time.  This means that consultant radiologists and specialists can receive reports in seconds that are of unparalleled quality.  GPs have the best guess within seconds, rather than well... we all know this.

    AI also provides probability, so it's easy to prioritise any reporting that's life-threatening to a specialist, allowing them to focus on the most challenging work and receive the in-depth information provided by the AI.  

    This is possible because we are dealing with a relatively narrow field of data that we have taught AI to handle. Think of X-rays; the results are far superior to an expensive resource (a Radiologist) that takes at least 12 years to train.  And more to get brilliant.

    Should we stop training Radiologists and diagnosticians and allocate our resources to AI?  Absolutely not!!   

    Radiologists should be using the AI reports, validating, using the info and extrapolating, when an issue is detected, this must be added back into the learning model resulting in improving the AI.   AI should not act. It must only be used to support.  Acting should be restricted to notifying relying parties such as GPs.  

    Good Use case: Online GP Appointments and triage

    If you have an issue, you go onto an NHS app that will ask you for your symptoms and ask a few follow-up questions.  It will only give you its best guess (this is already amazing imo.), this in turn, will triage your call into "go to your emergency department, they know you are coming", "let's book you an emergency appointment", or "this is my recommendation, and why".  Dr Google actually becomes useful (weird medical insider joke).  Honestly, we could do so much more, but care is given to the right people, "shift-left" (the sooner you catch it, the cheaper and better the solution, 100% applies to healthcare).

    Preventive medicine and nudging technology will have profound improvements in people's lives and lifestyles.  Hooking into ambulance services and driverless automated vehicles,.. people do the hard stuff and make the decisions. AI does the piece efficiently and quickly that we as humans aren't good at. Hopefully, you are understanding the difference between narrow and wide industries.

    Bad Examples: Robot Surgery or treatment rooms

    Robots replace people in operating theatres. It is insane!! A surgeon could use AI to obtain better diagnostic data more quickly; they could even utilise technology like AI-assisted operations, then send messages if it detects that the actions are not optimal or there is a clear risk that a priority has changed.  It is brilliant for decision support. It's not a good idea to try to automate complex decision-making.


    This post does not look at Strategy and Purpose

    All AI solutions give themselves an exponentially better chance of success, regardless of industry, if they have a strategy, purpose, and FAIR data (Findable, Accessible, Interchangeable/exchangeable, and Reusable).

    Saturday, 28 June 2025

    UK Railway Industry for Dummies focusing on Rail Infrastructure

    Understanding Railways Series:

    UK Railway Industry for Dummies, focusing on Rail Infrastructure (this post)

    Railway Infrastructure - Signalling

    Rail assets are organised into large, hierarchical asset classes that are interdependent, forming a rail system. These rail assets are organised using detailed, lower-level assets built from taxonomies and ontologies tailored to each jurisdiction within the rail industry.  Railway interaction and asset operations must comply with stringent rail regulations.  Safety has a massive focus.

    Taxonomy organises data hierarchically, while ontology models both hierarchies and complex relationships between entities and their properties. In the rail industry, ontologies are crucial for successful asset modelling.

    The picture shows examples of significant assets (high-level)

    Main Railway Infrastructure Assets high-level overview.

    An AI-generated image to explain commonly used railway terms.



    The railways consist of "rolling stock, rail infrastructure, and environment"; these components have multiple relationships with one another.
    1. Rolling stock is the trains.
    2. Rail Infrastructure relates to: 
        2.1. Electrification/power/energy, generally used for power supply for signalling, train power, and telecoms.  Key protocols for using devices are MQTT, MODBUS and IEC
    • UK network supplies at 400kV, and train propulsion uses 25kV (AC) for Mainlines and 1-3kV (DC) for requiring step-down.  Train propulsion power is referred to as Traction Power Supply.
    • Non-Traction Power Supply, used for signalling, station power, and lighting
    • Overhead Line Equipment (OLE) is critical in railway electrification.  OLE span - Longer spans reduce the number of masts, lowering installation costs—but only if mechanical and electrical limits are respected.
    • Overhead Contact System (OCS) or Third Rail: Transfers power to trains.
        2.2. Telecommunication, track-to-control, and train-to-control are used to communicate, including sensors and IoT devices.
        2.3. Signalling relates to ensuring train safety so the train knows there is a train ahead of it, and issues when to slow down.
        2.4. Track Engineering, also known as Rail Engineering and The Permanent Way, involves the rails, connectivity, support, extensive physics and geometry, steel rail installation and joining, ballast (the ground on which the track is laid), drainage, substructure, and sleepers. It gets detailed with rail joins (Fishplated) and even the welding process used.  Fastening types, baseplates, sleepers, off-track maintenance such as hedge trimming (you won't believe the rules unless you work in the rail industry) ...

    SCADA systems integrate sensors, Remote Terminal Units (RTUs), Programmable Logic Controllers (PLCs), and Human-Machine Interfaces (HMIs) to collect real-time data, automate processes, and ensure operational safety and efficiency. In rail, SCADA typically manages traction power, station systems, and communications infrastructure.

    3. The environment refers to the existing conditions before the railway, including the topography and type of terrain, bridges, and rivers.

    The interdependencies with the rail industry are perfect for numerous AI scenarios.  With any AI, you need high-quality data and must secure it appropriately.  Bring information from across business functions together to enable automation, ML, and AI, and support better decision-making.

    Each country or jurisdiction has different rules for trains, and operators must comply with Health, Safety, and Environment (HSE) regulations.  There are industry rules adapted to each jurisdiction, and standards vary by region.  For example, most jurisdictions have a gauge width requirement; in the UK, the standard gauge is 4 feet 8 1/2 inches (1435mm).  There are exceptions, such as heritage railways in the UK.  There are manufacturing standards for everything.  EN13674 is the British Rail specification for the actual pure steel used to manufacture the track to be installed.

    ISO 55000/1/2 addresses Physical Asset Management.  Building Information Modelling (BIM) enhances the design and construction process, and both apply to Rail Infrastructure.  There is generally a disconnect between Asset Management and BIM, and the International Foundation Modelling (IFC) aims to establish a standardised asset model for the railway industry; we are on v4.3.

    SIL4 - Used for critical safety systems such as railway interlocking control systems. Safety Integrity Level is defined in functional safety standards, such as IEC 61508. SIL1 is the lowest level, and SIL4 is the level at which a system has the lowest likelihood of dangerous failure and the highest reliability. 


    References used: 

    Permanent Way Institution (2023) Understanding Track Engineering. London: The PWI. Available at: https://www.thepwi.org/product/understanding-track-engineering/ (Accessed: 4 July 2025)

    Camarazo, D., Roxin, A. and Lalou, M. (2024) Railway systems’ ontologies: A literature review and an alignment proposal. 26th International Conference on Information Integration and Web Intelligence (iiWAS2024), Bratislava, Slovakia, December 2024. Available at: https://ube.hal.science/hal-04797679/document (Accessed: 4 July 2025).

    Network Rail (2021) Asset Management: Weather Resilience and Climate Change Adaptation Plan. London: Network Rail. Available at: https://www.networkrail.co.uk/wp-content/uploads/2021/11/Asset-Management-WRCCA-Plan.pdf (Accessed: 4 July 2025).


    Thursday, 26 June 2025

    openBIM for AEC understanding

    Within the AEC industry, standards are necessary to ensure that all project stakeholders are speaking the same language, thereby improving collaboration.  We can also process data to automate various processes if the data is standardised.

    BIM (Building Information Modelling) is used to improve collaboration on infrastructure projects.  BIM is essentially ISO 19650, and it has various levels.  BIM is a Process! Not a technology vendor, the technology merely helps you with your BIM process.

    Building Models contain 3D information that shows how assets fit together.  Each of these assets may contain properties that can be used to look for clash detections.  Think of a CAD diagram, it lays out the plans for a building so all parties can see the proposed plan.  As CAD technology advances, you can add more information about the project.  For example, as an electrician, I only want to see the layers that affect my work.  CAD can be further utilised to display product and material information.

    closedBIM: These were the original big BIM systems, including AutoCAD, Revit, and Bentley ProjectWise.  These tools feature visual editors and viewers, enabling them to securely store the files required for a project and ensure that the appropriate individuals have access.  These all have their own proprietary standards.

    openBIM: Read other parties' data, improves collaboration and consensus.  Easier to switch tools to reduce costs or get better features.  Consists of:

    1. IFC (common language)
    2. bSDD (industry common language)
    3. IDS (Requirement specification)
    4. BCF (check)
    5. openCDE (sharing with APIS)

    Industry Foundation Classes (IFC) serve as the basis for standardising how information is handled.  Has standards for location, such as geographic information.  Materials, Geometry, and Spatial Structures are covered by IFC classes.  In each industry, these base IFCs are added to.  The BuildingSmart bSDD is an extension of IFC for specialised industries and sectors, published to provide more specific, agreed-upon standards.  

    Project Requirements: These can vary, but having an agreed-upon format, such as an Information Delivery Specification (IDS), is helpful. Although it is not necessary or widely used, it ensures that precise details are provided.  Therefore, collaboration allows all parties to clearly understand what is needed using IDS.

    IDS uses bSDD, which is based on IFC, so that the requirement specifications are precisely laid out.

    openCDE defines technical interfaces, .....

    The History of BIM

    I learnt a lot about BIM from a great BIM leader, Steph Neath.  Here is a diagram of the evolution of BIM from Steph:

    Evolution of BIM (Building Information Modelling) as explained to me.

    Thursday, 5 June 2025

    AI Vendor Management - Formiti

    AI is going crazy, and you can build your own but generally you need to look at a supplier, so it's worth understand management of Vendors, you as the controller using their service are at risk of them not making their AI operations transparent.  It's a big business risk to my clients.  

    GDPR is closely linked to AI, and if you use a service/vendor, the reputation and fine risk may fall on you as the provider.  Need visibility into each vendor, how they are using AI, in turn they are using vendors so it's a nice complex dependency problem.  You need to be aware of what you are relying on.

    Ensure contracts with vendors consider AI, how the process your data and how their sub process vendors do the same.

    Track website customer behaviour, we use a vendor to clean up the data.  In turn, I have no idea that they are using AI outside of the UK or EU.  Follow the dependency chains as all this needs to be transparent to the end customer if needed.

    Monday, 2 June 2025

    Copilot Studio 2025 Notes

    Copilot Studio is fantastic, the licensing is complex, and the AI integration is excellent. Architects really need to understand Licensing and billing, or AI will get out of control.  The Purview and governance look very good.  Copilot Studio Cost Estimator (preview June 2025)

    MS Build 2025: 

    MCP Server in Preview - possible to collect data from other AI services or write back.

    Connector Kit - So, you can add custom connectors from Power Platform Connectors, including Copilot Studio - great stuff.

    Agent Flow - Added functionality to Power Automate flows (Copilot Studio aware), deployed via solutions.

    NoteThe M365 Agent Toolkit appears to be an interesting tool that allows agents to perform tasks using Office add-ins with VS Code.

    Licensing

    You need to be aware:

    • M365 agents - require all end users to have M365 Copilot licences, retailing at $20/user.  Alternatively, users can consume the agents using a PAYG model per message (it racks up quickly).  I can add these to MS Teams, and it appears that people with licences can ask the M365 agent, while others can view the results (some more testing and understanding are needed here by me).
    • Copilot Studio - Requires a Copilot Studio AI Studio/maker licence at $30/retail. Users don't need a licence to use it, but you pay per message, and this can rack up quickly, so watch your usage. Buying bulk message credits can help reduce costs.
    • Each prompt generates multiple messages, which are all billable (complex to calculate)
    • (If you use Copilot Studio and it calls Azure AI Foundry, it also bills Tokens (also complex to estimate)
    • Copilot Studio utilises the AI Foundry connector through its Premium connector.

    Monday, 26 May 2025

    Playwright Post 6 - Automating Canvas App MFA login for Playwright unattended for Canvas apps

    Overview:  Modern security makes automating logins requiring MFA rather difficult.  This post looks at possible approaches to automate the login.

    Option 1. Turn off MFA—not really, but you can set a conditional rule in EntraId to not perform MFA. This is not an option in many enterprises.

    Option 2. Time-based One-Time Password (TOTP)—Microsoft Authenticator makes this pretty difficult. At least I can't do it, as the APIS are relatively limited. This is kind of expected, as it's a security measure.

    Option 3. Programmatically acquire an access token without browser automation, use MSAL with a client secret or certificate (for confidential clients). 

    Option 4.  Use Playwright to record the login and intercept the access token once logged in.  Then you can store it and use it.  There are a few easy options to get the session:

    4.1. Retrieve the access token from the response once logged in

    4.2. Retrieve from your local storage:

      const token = await page.evaluate(() => {
        return window.localStorage.getItem('adal.idtoken') || window.sessionStorage.getItem('adal.idtoken');
      });
    4.3. Retrieve the token using Playwright at the command run level

    Note: This adds the token to my repository. Don't save the token to your repository if you don't realise that the Access/Bearer token will expire depending on what your EntraId sets. The default is 1 hour.

    Option 4.3.1. Like option 4.3, use the refresh token to silently generate a new Access token. You store the refresh token during the recorded login (by default, it lasts for 90 days) to generate a new access token when needed.

    Option 4.3.2.  Take it further back to generate the refresh token using the access code you get at the original login, renew the "refresh token", and generate a new access token to run your tests.

    If you decide to store your access token, refresh token or code, don't store them in your code repo.  You know why, if you've made it this far.

    Thought: as a refresh token works for 90 days on a sliding scale, I've never used the option 4.3.2, as by storing the refresh token, all I need to do is to extend the refresh token by using it to get an access token, and the refresh token has 90 days from that point. 

    This is the plan I'm thinking of using:

    Tuesday, 20 May 2025

    Entra AAD Security Groups - Remember

    Overview: I have lost count of the number of poor Active Directory and Azure Active Directory environments I have encountered.  I don't think I've ever seen a good Active Directory, actually.  Certainly nothing significant over 5K 15k users. 

    I'm working with a multinational, and we need to enhance our security measures.  Things are a bit all over the place, with oddly named and inconsistent elements, which is basically the norm for a 300k internal user enterprise with a history of multiple acquisitions.

    I've identified a couple of properties that will create a nice hierarchy, but the issue is that I'm using more than the allowed 5k Dynamic AAD Security Groups.  

    Summary of Entra, Microsoft 365 groups, and Distribution Lists work

    Group Types to be aware of relating to Entra

    1. Static AAD Security Groups

    • Need to add users manually or automate the process for anything but the smallest Entra users.
    • Static AAD Security groups can be nested.

    2. Dynamic AAD Security Groups

    • Up to 5,000 dynamic groups. Updated Oct 2025: 15,000 
    • You can inherit Security groups or be inherited (no nesting).

    3. Distribution AAD Groups

    • Used for email and calendars, not security.  
    • Only use distribution lists if the user does not have a P1 Entra licence.  
    • Simpler and better to email-enable Static Security Groups.

    4. O365 Groups/Teams Groups called Microsoft 365 Group

    • A Microsoft 365 Group can inherit from O365 groups or AAD Security groups.  
    • They are managed within the org, so it's not the best idea to place heavy security on manually managed teams. 

    Resolution:

    I have a whole hierarchy of users within divisions and subdivisions, and I add users statically via automation to their lowest-level AAD Security Group.  Then I can add the child groups.  This gives me multiple groups that have an increasing number of users as we go up the hierarchy.  Additive groups with positive security provide the best options for me.  Stop using Distribution lists and make the AAD Security Groups email-enabled.

    Alternative option on Email:

    Create an O365/Teams group, then add the Security groups to the team, and this will now be email-enabled.  This is more for a collaboration-type approach or if you want to use dynamic groups.

    Future Wishes:

    If only Entra supported more dynamic AAD Groups per tenant or allowed Dynamic groups to be nested in static AAD groups.

    Monday, 12 May 2025

    Playwright Post 5 - Understanding how Playwright Works

    Playwright as a tool consists of two main parts.

    Part 1: Playwright Library: This is the automation of a browser using the Page Object Model (POM). It provides a uniform API to run against the 3 main browser engines, automating tasks like navigating, clicking, filling in form data, and validating content on a web page. Classes include APIRequest, APIResponse, and BrowserContext. The worker process runs the API calls sequentially. Unified library API calls are sent to the browser context, which runs unaware of the calling context.  

    Top link runs in Node.js and makes API library calls, there is no timing between the Node.js (Controller) and the browser instance (running Chromium instance)

    Part 2: Test Runner: This part runs the Playwright tests.


    Playwright Series

    Sunday, 11 May 2025

    Serverless & FaaS

    Serverless eliminates the need for Infrastructure provisioning.  It is done for you; you just don't need to provision it.  Serverless should have built-in scaling so you don't even need to monitor it.

    Function-as-a-Service (FaaS) enables us to develop functions that perform specific tasks in response to events.  It is nearly always stateless and should be stateless.

    Important: FaaS is the code/logic that sits on serverless infrastructure.

    The big three public cloud providers provide serverless thru:

    • Azure Functions
    • AWS Lamdas
    • Google Cloud Functions

    You merely deploy your Functions onto the public cloud offerings.