Sunday, 27 December 2020

AIP and Sensitivity Labels

Overview:  AIP has had many names and twists over the past few years.  The functionality has been improving, but the naming and changes made it difficult to implement well.  Finally, I feel Microsoft Azure Information Protection is implementable at scale.

Summary: Sensitivity labels have have the ability to allow documents and email to be classified to protect email and files.  One can track, and encrypt documents/email.  You can also use sensitivity labels to protect SharePoint sites, Teams sites and Microsoft 365 Groups.  Within AAD (B2C) I can assign sensitivity labels to Microsoft 365 Groups.

Sunday, 13 December 2020

ISO 27001 Certification

Overview:  I have been thru several ISO and security audits over the years for various companies offering SaaS products.  This post outlines a some of my note around the latest ISO 27001 audit I touched on.

ISO 27001 covers Information Security Management (ISMS) which is about protecting and managing your businesses information assets to reduce your business risks.

Parts to an ISO 27001 audit:

  • Part 1 - Check you have the correct documentation.  
            Output is a go ahead and get a visit plan from the auditor.
  • Part 2 - Checks you as a business are complying/working to the documentation.  Basically evidence based reporting based on visual confirmations and discussing with the staff using interviews to verify compliance (sample based auditing).  Findings normally grouped into 3 types of findings: 1)   Opportunity for improvement = suggestions, need to review before next audit to see if this is worth implementing 2) Non conformance - Minor = can have a few of these, look to fix 3) Non conformance - Major - won't get certification with a major.  There is a period to address/fix major issue/issues.  Always complete the phase 2 audit as they may discover other majors.
            Output Findings report and several weeks latter the certification.
  • Certification
  • Yearly: Need to repeat and show you are improving based on the findings and the audit will generally go into specific areas in more detail.
More Info:
Data Protection and Regulation note - see bottom of post for ISO27001

Tuesday, 1 December 2020

Testing your home Internet Speed using your IPhone

Problem:  Broadband offers various speeds options when purchasing, the actual speeds you get are usually well below and depend on you specific instance. 

Initial Hypothesis:  iOS has multiple apps to monitor speed to your iPhone.  

Resolution:  Download "Speedtest" using the app store an any Apple device.   5G performance is fantastic.

Below are my Results, I live in South West London (Zone 4)

Sky broadband - SW London

Broadband Download (Mbps)  Up Speed (Mbps)  Location             
Sky phone 34.80 5.72 SW London 
EE 4G - LTE 13.00 0.13 SW London
O2 4G - LTE 16.90 10.20SW London
EE - 5G 372.00 19.80 Newcastle

EE 4G - Mobile
EE 4G - SW London

O2 4G - SW London

5G - Newcastle

Speed tests vary greatly, so worth doing at least 3 to get an average. 70 Mbps download on EE4G is very possible so the download speeds can be as good as my Sky broadband.  5G performance is fantastic - the MIFI/5G routers are going to be awesome when 5G rolls out to my area.  Using O2 and EE at my home, O2 is way faster down but interesting the upload speed is amazing using O2 (The O2 tower is way better positioned).

Sunday, 29 November 2020

Azure SQL Basic Options Summary

OverviewAzure SQL is incredible.  There are a lot of options when choosing how to host database and performance good.  "handles patching, backups, replication, failure detection, underlying potential hardware, software or network failures, deploying bug fixes, failovers, database upgrades, and other maintenance tasks", from Microsoft Docs and Azure SQL.

Azure SQL is the PaaS database service that does the same functions as SQL Server did for us for many years as the workhorse for many organisations.  Microsoft initially only offered creating VM's and then installing SQL Server on-prem.   Azure SQL is Microsoft's PaaS SQL database as a Service offering on the cloud.  Azure SQL is a fully managed platform for SQL databases that Microsoft patches managed backups and high availability.  All the features that are available in the on-prem. Edition are also built into Azure SQL with minor exceptions.  I also like that the minimum SLA provide by Azure SQL is 99.99%.

Three SQL Azure PaaS Basic Options:

  1. Single Database - This is a single isolate database with it's own guaranteed CPU, memory and storage.
  2. Elastic Pool - Collection of single isolate databases that share DTUs (CPU, Memory & I/O) or Virtual Cores.
  3. Manage Instance - You mange a set of databases, with guaranteed resources.  Similar to IaaS with SQL installed but Microsoft manage more parts for me.  Can only purchase using Virtual Core model (No DTU option).
Thoughts: Managed Instances recommend up to 100TB but can go higher.  Individual databases under elastic pools or single databases are limited to a respectable 4 TB.

Two Purchasing Options:

  1. DTU - A single metric that Microsoft use to calculate CPU, memory and I/O.  
  2. Virtual Cores - Allows you to choose you hardware/infrastructure.  One can optimise more memory than CPU ratio over the generalist DTU option.
Thoughts:  I prefer the DTU approach for SaaS and greenfield projects.  I generally only consider Virtual Cores, if I a have migrated on-prem. SQL onto a Managed Instance or for big workloads virtual cores can work out cheaper if the load is consistent.  There are exceptions but that is my general rule for choosing the best purchasing option.

Three Tiers:

  1. General Business/Standard (There is also a lower Basic Level)
  2. Business Critical/Premium
  3. Hyperscale


Point in time backups are automatically stored for 7 to 35 days (default is 7 days), protected using TDE, full, differential and transaction log backups are used to point in time recovery.  The backups are stored in blob storage RA-GRS (meaning in the primary region and all the read-only backups are stored in a secondary Azure region).  £ copies of the data in the active Azure Zone and 3 read only copies of the data.

Long Term Retention backups can be kept for 10 years, these are only full backups.  The smallest retention is full backups retained for each weeks full backup.  LTR is in preview available for Managed Instances.

Azure Defender for SQL 

Monitors SQL database servers checking vulnerability assessments (best practice recommendations) and Advance Threat Protection which monitors traffic for abnormal behavior.


  1. Only valid IP's can directly access the database, Deny public Access,
  2. AAD security credentials, use service principals
  3. Advanced Threat Protection has real time monitoring of logs and configuration (it also scans for vulnerabilities), 
  4. Default is to have encryption in transit (TLS 1.2) and encryption at rest (TDE) - don't change,
  5. Use Dynamic data masking inside the db instance for sensitive data e.g. credit cards
  6. Turn on SQL auditing,

Note: Elastic Database Jobs (same as SQL Agent Jobs).

Azure offers MySQL, Postgre and MariaDB as hosted PaaS offerings. 

Note: The Azure SQL PaaS Service does not support the filestream datatype : use varbinary or references to blobs. 

Monday, 26 October 2020

Identity Server - OAuth and OIDC

Overview:  The current version of Identity Server is 4.  Identity server is basically a .NET Core 3.1 application that is an Identify Provider (IdP) similar in role to PingId, SiteMinder, AAD b2C.  Identity server allows application (native mobile, web sites and servers) to securely authenticate users.

OAuth2 Grant Types:

Flow Description Client Grant Type
Authorization with PK Authorization Code Grant Type.  Default choice for authorization. Native mobile Apps, Windows app, Browser Apps Code
Client Credential Server-to-server (S2S) communication also refereed to as Machine-to-machine (M2M). Server,Consoles,Services ClientCredentials
Implicit Rather use the Authorization Code Flow with PKCE Native Apps & SPA's often use Implicit Flow Implicit
Resource Owner Pswd

Scopes: The authorisation Server specifies the "scope" that the user has consented too.  So for an API you can limit the actions the user can perform.  Always name your scopes by the API and the Verb e.g. "" is better than "read".

Monday, 19 October 2020

APIM High Availability and Performance across Regions

Overview:  APIM can be setup in multiple regions and incoming request will be routed to the closest APIM endpoint.  If there is only 1 APIM region, it is best to ensure the API/App Service/Function is hosted in the same region.  With multiple APIM's you can also host a API in the same region.  The routing is either done automatically using Azure Front Door or via policy on the APIM.

More Info

Friday, 9 October 2020

App Insights - Website and API Monitoring

Overview:  App Insights has functionality to run scheduled web requests and log the output in App Insights.  There are multiple advantages to this including end to end active monitoring of web sites and API's, and keeping the application warm.

Below I show a simple request to my blog (public website) and the results, Azure refers to this test type as a URL Ping test which is basically a URL HTTP GET request.  

Wait a few minutes and Refresh to see the results:

Very easy way to include a constant check that your API or Website is running.  There is also the options to create "Multi-step web test" using Visual Studio.  You can record the authentication and assert for known response content to build advanced constant monitoring.

Tip: The URL does need to be publicly available.

The content I used to test out the functionality comes from the Microsoft Docs site.
Also see Live Metric Stream that is part of App Insights.

More Info: 
App Insights MultiStep Tests  Replacement Option for MultiStep Test based on Azure Functions

Thursday, 1 October 2020

App Insights - Basic Introduction

OverviewAzure App Insights is a great platform for collecting logs and monitoring cloud based applications on Azure.  All Azure Services can push logging information into App Insight instances.  This can be errors, usages, performances logging that in turn is easy to query.  There are SDKs for developers that can be used to add custom logging to applications.  I am a big fan of AppDynamics for logging and monitoring but for SaaS and on a new project I'd go with App Insights.

Retension:  App Insights can keep 730 days worth of logs.  For long term storage, "Continuous Export" can be used to push data into storage accounts as soon as it arrives in AppInsights.  Retaining the App Insight logs for 90 days has no additional cost, so the default to store logs should be set to 90 days at least in most situations.

What is logged and what can be logged:  
  • All Azure Services can be configured to send service logs to a specific App Insight instance.
  • Instrument packages can be added to services to capture logs such as IIS, or background services.  You can pull in telemetry from infrastructure into App insights e.g. Docker logs, system events.
  • Custom code can also call the App Insight instance to add logging and hook into exceptions handling.  There are .NET, Node.JS, Python and other SDK's that should e used to add logging, exception capturing, performance and usage statistics.

App Insights has a REST API to query the logs.  The "API Explorer" tool is awesome for querying App Insights online.  

The data below comes from Microsoft Docs.

"What kinds of data are collected?

The main categories are:

  • Web server telemetry - HTTP requests. Uri, time taken to process the request, response code, client IP address. Session id.
  • Web pages - Page, user and session counts. Page load times. Exceptions. Ajax calls.
  • Performance counters - Memory, CPU, IO, Network occupancy.
  • Client and server context - OS, locale, device type, browser, screen resolution.
  • Exceptions and crashes - stack dumps, build id, CPU type.
  • Dependencies - calls to external services such as REST, SQL, AJAX. URI or connection string, duration, success, command.
  • Availability tests - duration of test and steps, responses.
  • Trace logs and custom telemetry - anything you code into your logs or telemetry."

Azure Dashboards

Tuesday, 29 September 2020

Sunday, 13 September 2020

Building better Software Thoughts

Overview:  I see a lot of development teams, and they always seem to have areas they are good at and capabilities teams need improvement on.  Key is culture and building a happy team where team members trust and help one another.

Building a culture where teams enjoy code reviews is also key for successful Software projects.   To improve software, reviewing various areas not only code reviews are essential.  For me, clear requirements are the number 1 factor in improving teams performance.  

Companies are getting better at building software; I aim to work on these topics to improve the delivery of software within scrum teams:

  1. Code Reviews & Peer Reviews (Daily reviews are awesome, should be pretty short and enjoyable not someone trying to show off or hours long)
  2. Collaboration (Standups, Slack/Teams, Code tools have collaboration built in)
  3. Documentation & Requirements Reviews
  4. Better tooling including better CI/CD tooling including static analysis tools
  5. Unit Testing, automate coding standards, Integration testing, UI Testing, and API testing 
  6. Requirements (Use Stories are clear and Acceptance Criteria)
  7. Cadence is improving thanks predominately to Agile practices; I like short release cycles (2-3 weeks depending on the team and industry).  Changing requirements, indecision kills software projects.  Agile helps, but decisive knowledgeable product owners increase the likelihood of the project succeeding.

Benefits of Code, Documentation and Requirement Reviews:

  1. Improved software quality & product delivery
  2. Share domain knowledge
  3. Training team members (useful for onboarding new team members)
  4. Reduce support and fix costs
  5. Lower cost & faster development

Options Layering API's on Data Sources - Micrososervices kind of

Hasura takes data sources such as SQL, Postgress & MySQL and converts it into GraphQL API's.  SQL Server is in preview.  Service is available on Azure and hooks into AAD and AAD B2C.  Hasuru looks extremely interesting and useful.  Potentially a great time saver.

CDS/DataFelx/Oakdale - Allows for Entity creation and provides REST API's.

SharePoint lists provide HTTP API's for CRUD operations.

REST API's vs GraphQL

OpenAPI specification (previously known as the Swagger specification) is my default for an API, this allows for a known RESTful API that anyone with access can use.   Open API has set contracts that returned defined objects which is great, you can work with the API like a database with simple CRUD operations as defined by the specification.  The issue is that the returned objects are fixed in structure so you may need 2 or more queries to get the data you are looking for.  Alternatively, GrapghQL allows the developer to ask for the data exactly as the want it.
Open API example:
/api/user/{2} returns the user object  // Get the user object for user 2
/api/users/{2}/orders/10  // Returns the last 10 orders for the user
GraphQL example:
Post a single HTTP request.
query {
 User(id: "") {
    orders(last: 10 {

Database Option Notes:
PostgreSQL - Hybrid relation database (Communitity edition free, standard and enterprise higher loads) sits on Linex but can also use Windows OS.  Replicas are used for additional read-only nodes (I think up to 5 geo relicas allowed) 
Can be hosted on Azure or AWS IaaS.  PostgreSQL is also offered as a fully managed service (PaaS) on Azure (either single server or hyperscale).
MariaDB community fork off MySQL - Open source relation database used in the LAMP app stack.  Azure MySQL manage service can have up to 5 replicas for read heavy workloads.

Sunday, 6 September 2020

Working with CDS data structures for non CRM types

Overview:  I am working on a Power Platform solution and I need to use CDS.  Basically, I need to be able to see and edit values within CDS.

Option 1: Microsoft SQL Server Management Studio (SSMS) version 18.6 allows connectivity and read-only access.  Here are the instructions.

Option 2XrmToolBox has fantastic tools for Dynamics and Power Apps.  There are a lot of individual tools from various contributors.

Here I am using "Entity Relationship Diagram Creator" to look at the relationships between the CDS entities.

Saturday, 5 September 2020

Reducing Power Apps Dynamic calls and where to store Power Apps data.

Overview:  Power Apps is driven by data and generally that data comes from Connectors.  So the great news is there are a lot of different connectors and if in trouble I always find the custom connector can be relied upon.  When working with Power Apps, it is not as simple as just having a data source and consuming it, one needs to consider all the data sources, do we need live data, performance.  Basically, going and dynamically pulling lots of dynamic sets of data repeatedly leads to poor performance.

Identify data sources:  CDS, Azure SQL, SQL on-prem, CosmosDB, RSS, Open API's...

Understand the security, and the amount of data being pulled.  For example, if we need all the airport codes in the world for a drop down so the user can choose their closest airport e.g. JFK is for New York John F Kennedy airport.  There are roughly 4000 commercial airports in the world. 

Options: Call an Open API service.  Power App by default returns sets of 500, Power Apps max return count is 2000.  You still need to perform 2 calls with paging to get the full data set.  You could use a type ahead if the API supports it, but their will be a lag after each keystroke when Power Apps runs out to the service.  And there will be a lot of calls.  More suitable would be to do 2 calls with 2,000 record for each call and bind the control to the returned data.

A further improvement would be to store the airport lookup on data load or on first request, then subsequent requests would use the table/collection.  In effect, you are locally caching all the airport codes using 2 calls for each Power Apps user session.

For the airport example, one could also store the data using a Excel import, but beware the data is imported into Power Apps and store locally.  Big issue is the data is static in Power Apps, to update it, you need to re-import the Excel table.  So brilliant for sat days of the week or Months of the year as these never change.  Fairly static data like airport codes work well, but require a publisher level overwrite to update the list.  Also, storing extreme amounts of static data leads to bloat of the app and that data still needs to be loaded.  So I would not consider it for 100k+ items as a general rule.  

More Info:

Todd Baginski has a great video on using Excel import and creating language variations/multi-lingual Power Apps using Excel imports.

Alagunila Meganathan on C# Corner has a good post on Excel Imports for Power Apps.

Monday, 24 August 2020

AWS vs Azure services offering comparison for Solution Architects

Overview: Microsoft provides a useful list that allows me to know AWS services aligned to Azure Services.  This is pretty useful if you know 1 platform considerably better than another to quickly figure out your options on either AWS or Azure.

My Service comparison notes:

Amazon Relational Database Service (RDS) – SQL Server, Oracle, MySQL, PostGress and Aurora (Amazon’s proprietary database).  

Azure SQL lines up with Amazon's RDS SQL Server Service.  Although Aurora is probably also worth the comparison as it's AWS's native DB option.

Amazon DynomoDB – Same as CosmosDB – NoSQL database.

Amazon Redshift is the data warehouse.  Can be encrypted and isolate.  Support Petabytes of data.

Amazon ElastiCache run Redis cache and MemCached (simple cache).

AWS Lamda – Azure Fundtions.  Serverless.

AWS Elastic Beanstalk – Platform for deploying and scaling web apps & Services.  Same as Azure App services.

Amazon SNS – Pub/Sub model – Azure Event Grid.

Amazon SQS – Message queue.  Same as Azure Storage Queues and Azure Service Bus.

Amazon Step Functions – Workflow. Same as logic apps

AWS Snowball – Same as Azure Box.  Physically copy and transport to data centre for upload.

Virtual Private Cloud (VPC) – Azure virtual network 

Tip: I am glad that I did the AWS Certified Cloud Practitioner exam as it helped my understand of the AWS offering which has been very useful in large integration projects.  I have worked with AWS IaaS (EC2, API gateway and S3 historically).  Like Azure, there are a lot of Services and features.  Basically, there are equivilant services for Azure & AWS.  It may be a 2-to-1 service offering or it is not something offered by the cloud provider.

Sunday, 23 August 2020

AWS vs Azure vs GCP Comparison

Overview:  I predominately use Azure & Microsoft for all my cloud services.  

I have installed multiple SharePoint farms and setups on AWS EC2 instances and I'm currently preparing for the Cloud Practitioner AWS exam.  I have used Google for authentication, SaaS nut not as a IaaS offering.  I'm also a huge fan of Heroku which is great for PaaS and I used this to host my game built for Facebook games.  I've also seen IBM's cloud offering a few years ago.  For me it is too niche and not as feature rich.  So basically I understand Azure's offering well so I found this comparison pretty useful.

My Thoughts:  The contenders:  I really like Heroku for it's simplicity.  I feel for a small Indie developer or company, Heroku has a good free and cheap simple billing options.  GCP, I really can't comment from a good position of knowledge but from what I've used, I like GCP.  IBM's offering, well if you are a partner, you cloud go with this option but it is aimed more a large business partners.  IBM's cloud is IaaS focused, with some PaaS offerings but once again I'm not an expert.

AWS, has always been really easy to use.  It is big and complex like Azure with many offerings.  Basically, I'd choose AWS if the organisation was already using it and the people in the org know have experience with AWS.

Azure, so in my world Azure and O365 feel like the dominant player but the diagram below provides a great insight into the relative size of the Cloud infrastructure market.  Azure SaaS offering O365/M365 is also huge and hosted on Azure.  

There is good resource that goes into looking at the various cloud providers.  My current take away is Amazon is the biggest player in the IaaS field.  Azure has IaaS, a large PaaS offering and a massive SaaS (including Dynamics and O365) offering (Amazon has not equivalent).  

Sunday, 16 August 2020

Onboarding and New Starters for SaaS

Overview:  We large SaaS projects, various stakeholders tend to start at different times.  For me it takes awhile to get all team members up to speed with: domain knowledge, technical knowledge, our design and big picture, company specifics. 


In person and Teams/Skype meetings are great for building relationships and getting the most information to the new starter.

Record presentations, then shorten and annotate the presentation to make focused high impact onboarding material that can be reused across various roles.

For each starter, ensure they have an introduction/onboarding plan:  Book meetings, get the people the background, have resources e.g. websites, mp4. glossary of terms ready:


List out useful resource specific to the role.  I tend to keep these in SharePoint but as long as content if searchable and consistent and of value, I am in the right place.

Saturday, 1 August 2020

Possible Technical Roadmap and thoughts on a startup

Overview:  A friend of mine's son has recently built a web site, and it looks impressive, and we started discussing his project, this turned into a detailed technical conversation to everyone's horror at a BBQ.  This chap is 14, and all I can say is wow.  All hosted and built without spending any money.  I interview many developers, and his knowledge is outstanding.  

I've drawn his architectural explanation below, added a few items to check and what my next piece would be.

Clarification of High Level Design
  1. Build the Native mobile apps, getting users buy-in with a Native App will be considerably higher.  As you have used React, use React Native (React is separate to React Native, a completely new codebase).
  2. Cordova/PhoneGap is a wrapper that would allow you to keep a single code base and merely inject the existing code into Native wrappers for iPhone and Android.  As your app is HTML 5, and already looks like a modern mobile app, I'd use PhoneGap.  At least try it out and see if it fits.  You'd then only have to deal with a single code base to update all the platforms.  You could always use Google's Flutter if you wanted a single code base for the web site and Mobile native apps. 
  3. Ensure your API's are OpenAPI/Swagger
  4. Security - the API already has some protection.  Ensure the database is protected/secured.
Revenue model:  Ads (Gambling Ads pay well). Make a premium-paid for service (keep end-user usage/sign-up free).

Thursday, 23 July 2020

Shopify - Add optional installation on the cart for shoppers

Problem:  On my Shopify shopping cart, if a buyer is checking out and they have plants/flowers in their cart, I need to offer them the ability to have someone plant for them.

This could have various options such as in furniture, you could offer an assembly service.

Initial Hypothesis: Shopify use Liquid as it's scripting language.  Liquid allows us to combine Liquid with HTML, CSS and JavaScript to get the desired page behavior.  Below is my User Story:

As a shopper I want to be able to add labor to allow me to have my flowers/plants installed/planted so that I know an expert has got my flowers in correctly.

I also need to add the appropriate amount of labor, so if less than 3 plants charge for 1 hr  otherwise charge for 2 hrs.  Shopify has variants allowing me to have a labor product for 1 or 2 hrs.

Resolution:  Amend the cart.liquid cart summary page to allow for Labor to plant for the Shopper to easily be added.

Desired Behavior: 
Add a button to add installation

The optional installation cost for 1 hr is added to the store
Steps to Implement: 
1. Create a new Product in Shopify "Plant for me please".  Add it to a unique collection, mine was "Last Minute Checkout Items".  Add a couple of variants for time/cost as shown below:
2. Go to the Product page and append .xml to the end of the page e.g., and get the Variant_Id's, we need these to add the correct amount of labor cost later.
3. Open the cart.liquid file and add the  following logic:

Shopify thoughts

Overview:  Recently, I had two requests for some work around shopping carts/auctions.  I dismissed the first request as it is not what I specialize in.  On the second request, I decided to take the project on at a hugely discounted rate as I felt I did not have expertise in the field.

Shopify: Shopify is an amazing product and eco system, and I have build a great shop on the SaaS Shopify platform in record time.  There are great plug-ins to add functionality such as gift registry's, and Mobile add integration.  I have now been playing with the product, deliver solutions, worked with app vendors, reviewed competitor's, had help from Shopify support and I can honestly say that it is awesome.


  • There are simple API's that are well documents.  
  • The internal language is Liquid which is basically a UI scripting language.
  • Shopify SaaS has add ins and templates, so look before developing

In the code snippet above I am building custom functionality int he cart checkout to add specific products to allow more sales on the store depending on custom rules provided by the customer.

The templates OOTB or that shop admins can purchase follow responsive design as one would expect.  There are add-ons for iOS and Droid to make native apps, very well priced with good functionality changed on a monthly reoccurring basis (circ $40-100/month).  I need a custom mobile app for both platforms so I'm between choosing to build with Flutter or Blazor.

Friday, 10 July 2020

Power Apps Tracing to App Insights Not Working in edit mode

Overview: Power Apps integrates to directly log to App insights.  This post looks at the issues around Tracing in App insights.

Setup & Verify:
App Insights instance Instrument Key Required, Config you Power App to Trace to App Insights, and create a button to test.
The Monitor Tool is fantastic for tracing all outbound traffic, you no longer need to go to the service and check if Power Apps is reaching.  For example I use to have to look at the APIM Azure App logs for custom Connectors.
I can see all my interactions in Power Apps and travelling out.  This is a massive win for Power Apps so giving Tracing from the front end.

Greg Lindhorst, wrote a post on using the Power Apps Monitor Tool for debugging and performance improvement.

Problem: It normally takes a few minutes (like 2 minutes) to show up in App Insights, it is not showing up after 15 minutes.  Let's see, I've drop the Power Apps team a message and i think it's a bug that has crept in recently.  Today is 10 July 2020.  I've also notice that none of my Session page tracing is showing up in App Insights.

Update: 11 July 2020, the Power Apps behaviour has been on my mind.  I'm in edit mode, and when I publish the app and use it, App Insights logs perfectly.  When in edit mode running the app, even direct traces, no logging into Power Apps.  I did not realise this, but it kind of makes sense.

Resolution: Publish & Run the app, and the Tracing and Power App session tracing shows up in App Insights.

Only when I Run the Published App to my Page Views and Custom Traces get logged in App Insights.

Tuesday, 30 June 2020

Multi-Geo for MS Teams

O365 offers multi-geo tenants to meet data residency rules for 13 countries and regions (as of 30 June 2020):
  1. Australia
  2. Asia Pacific 
  3. Canada
  4. European Union
  5. France
  6. India
  7. Japan
  8. Korea
  9. United Kingdom
  10. United States
  11. United Arab Emirates
  12. South Africa
  13. Switzerland.
Teams data resides in SharePoint Online, OneDrive for Business and Exchange Online.  With Multi-Geo enabled, a company can specify where data will reside.  There are 2 parts to multi-geo:
  • User specific data.  This data is stored in various satellite Geo for each user e.g. email, OneDrive
  • Company/Project/division specific data e.g. file shares, Document libraries
For more info on Multi-Geo on O365

Microsoft "Multi-Geo is currently available to Enterprise Agreement customers with a minimum of 250 Microsoft 365 Services subscriptions."

The UK South Azure Region has 3 data centres/zones, and it's geo-paired paired with UK West there is over 150 mile between the regions. 

MS Teams Background Info:

Wednesday, 24 June 2020

Postman API Builder Intro

Overview: Tools for building and mocking API's.  Swagger has good tooling and my original preferred choice.  APIM - Great tooling, part of Azure and easy to replace mocks as you go along with the live implementation.  Postman is offering a great set of functionality to rival Swagger and APIM.  This post looks at Postman's new functionality around building API's.

Postman API Builder:
Not only a test rig, it now offers the ability to build API's and mock:
  • Mock - so you can test supports key and OAuth authentication
  • Assert Tests - You can specify asserts in postman
  • Test suite - generate collections/Collection Runner - Allows a set of related tests to run sequentially.
  • Document the API
  • Monitor
  • Version control for changes e.g. GITHub
  • API Versions supported
  • Note: Free plan has all of this, limited on the number of API's but all the features are on the free plan.  The main notation formats are support including:  Open API specification (OAS) & GraphQL
I like Swagger tooling, I have done a few projects find APIM fantastic for building API's quickly.  Postman historically was merely my test rig but looking at the functionality, Postman API Builder is a great option for designing and building API's.  Postman is a good tool for building into CI/CD pipelines to validate API's.

Thursday, 28 May 2020

Microsoft Teams Power Apps Integration

Overview: Teams are amazing, I was a complete Slack fan, but I'm 100% now a teams supporter.  It's part of O365, replaces Skype (which was great but only a chat app like zoom), you get your email, and can add all your apps and websites to your Team.

Adding your custom Power Apps to Teams:

Adding A Power App to MS Teams:

  1. MS Teams uses the Chrome engine (Chromium) as it's browser.
  2. A feature I don't like about Teams is that when i switch focus to say a chat window and come back to my Power app within MS Teams, I loose my place in my power app and the app is loaded from scratch.
  3. I believe the problem of apps maintaining session state will be solved shortly with pop out Windows in Teams around July/Aug 2020.

Friday, 8 May 2020

cURL for Windows 10 & Azure Cognitive Service Primer

In this example I am using Azure Cognitive service to provide a jpeg using curl on my Windows 10 Surface laptop.

Sunday, 3 May 2020

Common Software Architectural Patterns

The goal of Solution Architecture is to:
Have a common understanding of how a technical solution shall be reached, the diagrams are useful too have a communicable roadmap and ensure it is complete.  Later the diagrams are used to ensure all relevant parties have a clear unambiguous shared understanding of the IT solution.

The main tools to communicate the architectural solution design are diagrams & documents that utilise common previously used and understood patterns to ensure a safe, scalable, stable, performant, and maintainable solution.  "4+1 view, which includes the scenario, logical, physical, process, and development views of the architecture", source.

Below are patterns and thoughts that I have come across and used to solve building high quality solutions.

3/N Tier Architecture/Layered:
1) Presentation/UI layer
2) Business Logic
3) Data Layer/Data source
Here are a couple of possible example over the years you could of used
ASP > C++ Com > SQL Server 2000
ASP.NET (Web Forms) > C# Web Service (XML/SOAP) > SQL Server 2008
ASP.NET C# > C# Business Object Layer > SQL Server 2008
KO > MVC > SQL 2012
Angular 3 > C# Web API (swagger contract) > SQL 2016
REACT.JS > Node.JS > Amazon Redshift
UI > Azure Functions/Serverless > SQL Azure
Flutter > C# Web API .NET Core 3 (swagger/OpenAPI) published on Azure App Service > SQL Azure/Cosmos

API's:  Over the years we have seen many different API's at a high level:  
  • Proprietary formatted API's > 
  • XML with SOAP coming out of XML based API's > 
  • REST/JSON (other popular formats are: RAML, GraphQL > 
  • Event Driven API's may be the next big jump.

Thoughts:  As time has progressed, scaling each of these layers has become easier.  For instance Azure SQL has replication and high Availability and scalability automatically built in.  No need to think about load balancing in depth.  Plug and play and ask for more if you need it.
Microsoft SQL Server use to be a single server, then came replication, clustering, Always-on-availability, scaling greatly improved performance.
Middle Tier or Business layer use to be a singleton pattern - go thru a single server for business logic, slowly load balancing improved and caching become better.  Nowadays merely ramp on on you cloud provider.

Sharded Architecture: Application is broken into many distinct units/shards.  Each shard lives in total isolation from the other shards.  Think SOA or Microservice architectures often use this approach.  "SOA is focused on application service reusability while Microservices are more focused on decoupling".  

The problem with tight coupling multiple services are:
  1. Complexity - Difficult to change code and know the effects.  Also services need to be deployed together to test changes.
  2. Resilience - Service goes down, the whole suite goes down.
  3. Scalability - Can be an issue as the slowest component becomes the bottleneck.  
For instance build a complete application to handle ordering and a separate system that handles inventory.  So both could be in different data stores so let's say orders are on CosmosDB and Inventory is on Azure SQL.  Some of inventory data is static in nature so I decide to use App Caching (Redis).  Both the data sources site on independent server-less infrastructure.  So if you see inventory has an issue, merely scale it.  The front end store would seamless connect to both the separate.  "Sharding" databases/horizontal partitioning is a similar concept but only at the database level.  Sharding can be highly scaleable, allow for leveraging and reusing existing services, can be flexible as it grows.  Watch out for 2 Phase Commit (2PC/Segas/Distribute transactions)

Thoughts Pros: 
Great to reuse existing services instead of creating yourself. e.g. App Insights on Azure.
Great for high availability.
Increased latency - you may need to go to various systems in sequential order.
Need keys to manage e.g. clientId for this decouple architecture type, this architecture can also become complex especially if you need to expand a shard to do something it doesn't do today.
Data aggregation and ETL can become complex and have time delays.

Event-driven architecture: Client sends a request that includes a response for the server to contact when the event happens.  So if asking a server to do a complex calculation, the client could keep polling a long running operation until the server has the answer or use an Event-drive architecture to can you pls calculate and when you are done, send the response to me at...  Types of Event-driven architectures are: WebHooks, WekSockets, ESB (pub-sub), Server Sent Events (SSE).
Only run when an event happens.  They are loosely coupled.  In Azure it generally covers: Functions, Logic Apps, Event Grid (event broker) and APIM.  Easy to connect using Power Platform Connectors.
  • Client/Service sends a broadcast event
  • Consumers listens for events to see if they want to use the event
Hexagonal Architecture,
Command Query Responsibility Segregation (CQRS) - pattern/method for querying and inserting data are different./seperated.  This is a performance and scaling pattern.
Domain Driven Design (DDD) - Design software inline with business requirements.  The structure and language of the code must match the business domain.  DDD Diagrams help create a share understanding of the problem space/domain to aid with conversation and further understanding within the team. 
RACI Diagram - visual diagram showing the functional role of each person on a team or service.  Useful for seeing who is responsible for what part of a service or their role within a team.
Event Sourcing Pattern or AMQP
Competing Consumer Pattern – Multiple consumers are ready to process messages off the queue.
Priority Queue pattern -Messages have a priority and are ordered for processing based on priority.
Queue-based load leveling
Throttling pattern
Retry pattern
Circuit breaker pattern
The Twelve-Factor App methodology is a methodology for building software-as-a-service (SaaS) applications.

Key Design Decision (KDD) Document helps outline why decisions where made.  This is also often called Architectural Decisions Document or Template.
RAID Log - 

Streaming/MessageBus: Kafka, IoT,
Azure Messaging Service is made of of 6 products:
1. Service Bus - Normal ESB.  Messages are put into the queue and 1 or more apps can dirrectly connect or subscripbe to topics.
2. Relay Service - Useful for SOA when you have infra on prem.  Exposes cloud based endpoints to your on-prem. data sources.
3. Event Grid - HTTP event routing for real time notifications.
4. Event Hub - IoT ingestion, highly scalable.
5. Storage Queues - point-to-point messaging, very cheap and simple but very little functionality.
6. Notification Hub - 

Azure Durable Functions - Azure Functions are easy to create logic but are not good at long running or varying length duration functions.  To get around the timeout limits there are a couple of patterns for Functions making them better at handling long running operations.  The most common patterns are: Asyn HTTP API's (Trigger a a function using HTTP, set off other functions and the client waits for an answer by polling a separate function for the result), Function Chaining (Execute functions sequentially once the last function completes), and Fan out/Fan-in (first function call multiple functions that run in parallel) 

Lambda: great for large data architectures.  Has a batch vs streaming concept.  Each transaction pushed into a queue/stream (Kafka/Azure Queues/Azure Event Grid) and large data can be stored for later batch processing.

"Onion Architecture is based on the inversion of control principle. Onion Architecture is comprised of multiple concentric layers interfacing each other towards the core that represents the domain. The architecture does not depend on the data layer as in classic multi-tier architectures, but on the actual domain models."

Distributed Application Runtime, Dapr: Video

SAST/DAST: are application security testing methodologies used to find vulnerabilities in applications.  Another threat modeling approach is STRIDE.

OpenAPI vs GraphQL
OpenAPI specification (previously known as the Swagger specification) is my default for an API, this allows for a known RESTful API that anyone with access can use.   Open API has set contracts that returned defined objects which is great, you can work with the API like a database with simple CRUD operations as defined by the specification.  The issue is that the returned objects are fixed in structure so you may need 2 or more queries to get the data you are looking for.  Alternatively, GrapghQL allows the developer to ask for the data exactly as the want it.
Open API example:
/api/user/{2} returns the user object  // Get the user object for user 2
/api/users/{2}/orders/10  // Returns the last 10 orders for the user
GraphQL example:
Post a single HTTP request.
query {
 User(id: "") {
    orders(last: 10 {
You can see that for complex changing systems, GraphQL is potentially a better choice.  I also like the idea of using HASURA for ORM using GraphQL against PostgreSQL (hopefully SQL Server and others).

More Info: