Edge Computing Notes (IoT)

WIP

Edge Computing is where computing is done close to IoT devices and machines/PLCs.  Basically, if it happens on the "edge" of your network.  The processing occurs on local devices, gateways, or edge servers near IoT sensors, cameras, or machines.

Low Latency: Ideal for applications like autonomous vehicles, industrial automation, and AR/VR.

Bandwidth Efficiency: Only relevant data is sent to the cloud, reducing costs.

Reliability: Systems can continue functioning even with intermittent connectivity.

Over the past few weeks, I ordered a Raspberry Pi, which I intend to use for processing data from remote IoT sensors, namely Cameras, LiDAR, and temperature.

MQTT is the programming language to ...

Node-RED provides a web browser-based flow editor, which can be used to create JavaScript functions (Wikipedia)

Azure IoT Edge is a cloud-to-edge computing platform that extends Azure services and custom logic to IoT devices. It allows you to deploy containerised workloads (such as AI models, analytics, and business logic) directly on edge devices, enabling local data processing and reducing reliance on continuous cloud connectivity. This improves latency, bandwidth efficiency, and resilience in offline scenarios.

IoT Edge Modules are Docker-compatible containers running Azure services, third-party apps, or custom code. Examples: Azure Stream Analytics, Azure Machine Learning models. [learn.microsoft.com]

IoT Edge Runtime must be installed on each edge device. Handles module deployment, security, and communication between modules, devices, and the cloud.

Includes:

IoT Edge Agent: Deploys and monitors modules.

IoT Edge Hub: Manages messaging between modules and the cloud. [learn.microsoft.com]

Azure IoT Hub is a cloud service for managing IoT Edge devices, configurations, and monitoring.


A Raspberry Pi has an OS, whereas an Arduino device has no OS:

• Arduino is an Application + Microcontroller (MCU) - Compile code in C++ or C
• Pi is Application + Library & System Functions (OS) + MCU

Why Use Ada on MCUs?

• High reliability (ideal for avionics, medical devices).
• Built-in concurrency and timing features.
• Safer than C/C++ in many cases due to strong typing and runtime checks.
  
  
• If you need maximum safety and reliability, Ada is superior.
• If you need performance, simplicity, and broad support, C dominates.

Ada is a strongly typed, multi-paradigm programming language originally developed for the U.S. Department of Defense to ensure reliability in mission-critical systems.

Setup:

• Install GNAT (GNU Ada Toolchain)
• GNAT Studio or VS Code with Ada extensions.
• Run gnat --version in your terminal
• Write the code:

Compile & Run> gnatmake hello_mobility.adb

Playwright Agents in VS Code

I started looking at the latest version of Playwright late last night. The Agents add-in for VS Code is amazing.  I can't stop improving my code, my tests, and my automation.  It is highly addictive.

Playwright 1.56.1 includes the new Playwright CLI, which has the test agents as shown in VS Code above:

Node: v22.21.0
npx: 11.6.2
Playwright test package: ^1.56.1

Federated and Event Driven Architecture

Event Driven Architecture (EDA): System components interact with each other by producing, detecting and reacting to events.

Event-driven APIs differ from conventional REST APIs, offering improved scalability, strong service decoupling, reduced network traffic, and greater flexibility. Even-driven APIs need to address the challenges of monitoring, distributed tracing, security, and versioning.

Distributed, event-driven architectures are a powerful approach to building decouple long-running, high-performance, scalable systems and can use Conflict-Free Replicated Data Types (CRDTs) to provide eventual consistency.

An Event Mesh is a dynamic, interconnected network of event brokers that allows events to flow seamlessly across distributed applications, regardless of where they are deployed—on-premises, in the cloud, or at the edge.

Federated architecture allows each system to flexibly interact with other systems while remaining independent, so it can be easily extended, and individual pieces (a system) can be replaced relatively quickly.

Thoughts: Like Cross cutting concerns where you build IT runway to perform specific tasks and then call them, Federated Architecture, each system does a job so for instance, there is a standalone system that can be replaced or extended for Requesting a Room (1st system), this allows the user to reserve a room using the booking system (2nd system), this in turn calls the communication system that handles email, teams meeting with reminders (3rd system) and then calls the communication systems (n services/systems)

Events are facts; they are loosely coupled to the booking system.  This approach allows for the reuse and easy creation of highly customised business processes.

Thought: Choosing between an Event Mesh and a Federated Architecture...

Medallion Architecture in Fabric High Level Organisation Design Pattern

Microsoft Fabric is excellent!  We do still need to follow good practices we have been using for years, such as making data accessible and secure.   Possibly the most used architecture for Big Data is the Medallion Architecture pattern, where data is ingested normally in a fairly raw format into the bronze layer, then transformed into more meaningful and usable information. Lastly, the gold layer exposes data relationally using semantic models to reporting tools.

Overview: This document outlines my attempt to organise enterprise data into MS Fabric using a Medallion Architecture based on Fabric Workspaces.  Shortcuts are better than imported data, but it does depend on factors such as what the data source is, what data we need, how up-to-date the data is and performance requirements from the systems involved.

The reports and semantic models can get data from other workspaces at any of the medallion layers.  This architecture lends itself well to using the new Direct Lake Query mode.

Summary of a Design used by a Large Enterprise:

Medallion architecture using Fabric Workspaces.

Microsoft Fabric High-level architecture

Overview: Microsoft Fabric is an end-to-end analytics platform that unifies data movement, storage, processing, and visualisation. It integrates multiple services into a single SaaS experience, enabling organisations to manage their entire data lifecycle in one place.  One Lake is at the core of MS Fabric.

Image 1. One page High-Level Architecture of MS Fabric. 

European Fabric Conference in Vienna Sept 2025 takeways

FabConEurope25 was terrific in Vienna last week.  Great opportunity to meet Fabric and data experts, speak to the product teams and experts, and the presentations were fantastic.  The hardest part was deciding which session to attend as there are so many competing at the same time.  

My big takeaways:

• Fabric SQL is excellent.  The HA, managed service, redundancy, and shipping logs ensure that OneLake is in near real-time.  Fabric SQL supports new native geospatial types.  SQL has temporal tables (old news), but row, column and object-level (incl. table) security is part of OneLake.   There are a couple of things security reviewers will query, but they are addressed.
• Fabric Data Agent is interesting.  Connect to your SQL relational data and work with it.
• User-defined functions (UDF), including Translytical (write-back), HTTP in or out, wrap stored procedures, notebooks,.... - amazing.
• OneLake security is complex but can be understood, especially with containers/layers, such as Tenant, Workspace, Item, and Data.  There is more needed, but it's miles ahead of anything else, and Graph is the magic, so it will only continue to improve. - amazing, but understand security.  Embrace Entra and OAuth; use keys only as a last resort.
• Snowflake is our friend.  Parquet is fantastic, and Snowflake, including Iceberg, play well together with MS Fabric.  There are new versions of Delta Parquet on the way (and this will even make Fabric stronger, supporting both existing and the latest formats).
• Mirroring and shortcuts - don't ETL unless you need to shortcut, then mirror, then ETL.
• Use workspaces to build out simple medallion architectures.
• AI Search/Vector Search and SQL are crazy powerful.
• New Map functionality has arrived and is arriving on Fabric.  Org Apps for Maps is going to be helpful in the map space.  pmtiles are native... (if you know you know)
• Dataverse is great with Fabric and shortcuts, as I learned from Scott Sewell at an earlier conference.  Onelake coupled with Dataverse, is massively underutilised by most orgs, 
• Power BI also features new Mapping and reporting capabilities related to geospatial data.
• Other storage: CosmosDB (it has its place, but suddenly, with shortcuts, the biggest issue of cost can be massively reduced with the right design decisions).  Postgres is becoming a 1st class citizen, which is excellent on multiple levels. The CDC stuff is fantastic already.
• RTI on Fabric is going to revolutionise Open Telemetry and AI, networking through the OSI model, application testing, digital twins, and live monitoring,....  I already knew this, but it keeps getting better.  EventHub and notebooks are my new best friends.  IoT is the future; we all knew this, but now with Fabric, it will be much easier to implement safely and get early value.
• Direct Lake is a game changer for Power BI - not new, but it just keeps getting better and better thanks to MS Graph.
• Manage Private Endpoint as improved and should be part of all companies' governance.
• Purview... It's excellent and solves/simplifies DLP, governance and permissions.  I'm out of my depth on Fabric Purview and governance, and I know way more than most people on DLP and governance. Hire one of those key folks from Microsoft here.  
• Warehouse lineage of data is so helpful.  
• We need to understand Fabric Digital Twins, as it is likely to be a competitor or a solution we offer and integrate. 
• Parquet is brilliant and fundamentally is why AI is so successful.
• Powerful stuff in RDF for modelling domains - this is going to be a business in itself.  I'm clueless here, but I won't be in a few weeks.

Now the arr..

• Pricing and capacity are not transparent.  Watch out for the unexpected monster bill!  Saying that the monitoring and controls are in place, but switching off my tenant doesn't sit well with me if workloads aren't correctly set out. Resource governance at the workspace level will help fix the situation or design around it, but it will be more expensive.
• Workspace resource reservation does not exist yet; however, it can be managed using multiple fabric tenants. Distribution will be significantly improved for cost control with Workspace resource management.
• Licensing needs proper thought for an enterprise, including ours.  Reserve Fabric is 40% cheaper, and it cannot be suspended, so use the reserved fabric just as you would for most Azure Services.  Good design results in much lower cost with Workloads.  Once again, those who genuinely understand know my pain with the workload costs.
• Vendors and partners are too far behind (probably due to the pace of innovation)

Microsoft Fabric is brilliant; it is all under one simple managed autoscaling umbrella.  It integrates and plays nicely with other solutions, has excellent access to Microsoft storage, and is compatible with most of the others.  Many companies will move onto Fabric or increase their usage in the short term, as it is clearly the leader in multiple Gartner segments, all under one hood.  AI will continue to help drive its adoption by enterprises.

Ai worth a look

 Googles - Agent Development Kit

Enable AI assistance with Azure DevOps MCP Server

Microsoft Foundry