Observability in Azure SaaS Solutions

Problem:  Software has many places where errors and tracing is logged to.   Support get an incident, they need to investigate figure out how widespread the issue is and then try patch together various logs to figure out the problem.

Thoughts:

Observability is not a new concept, we need to be able to: 1) view and connect logs & 2) tracing and view metrics & notifications.

Implementing Observability must cover:

CI/CD allows devOps teams to find issues early using Unit testing.   Automated testing on UI.  API automation testing is also great.

Azure offers continious monitoring by performing various API calls to ensure your servie is running and any failures are picked up hopefully before any customers are aware.  You can also be notified of performance slow down, check performance speed between releases.  Which is great for identifying bottlenecks and with the Azure PaaS world, it is easy to increase the processing causing the bottleneck.  

Performance metrics built into the CI/CD and developers work allows us to identify issues early and costs miles less to correct early. 

Security and LINTing in CI/CD also allows us to pickup issues early and correct at a way lower cost.

Instrument you hardware and software, well on Azure you can use App Insights and you have a fantastic instrumenting platform that captures events.  A big reason to use Azure Services for as many of the function pieces in your solution.

Work In Progress ...

TBC Azure App Insights detail, ParentOperationId, Linking operations with a ServiceBus or work process call.

Creating a Power Platform Custom Connector from a Postman Collection

Problem:  I like to discover an API and play with it using Postman.  I save the collection and theoretically I should be able to upload the postman collection into the custom connector to generate the Power Platform connector.  Until a few days ago, the custom connectors only supported Postman v1.0 collections.  And Postman has not save using the v1 format for a long time.  

I was happy to see the Custom connectors support v2.0 and 2.1 of postman collections.  Only problem is while it says it does if is still broken.  So I have to go the old school way.

I wanted to generate custom connectors for HMRC's REST API so that I can connect with the Power Platform.  Option 2 below gives me a way to build my collections using HMRC's API's and generate the appropriate version 2 Swagger definition.

Resolution: 

Option 1: 

1. Export the postman collection in v2.1
2. Upload the postman file to https://apitransform.com/ and generate the OpenAPI specification (version 3.0.0) gets generated.
3. Upload the OpenAPI 3.0.0 specification file to Transformer | APIMatic and generate the JSON Open API 2.0 version.
4. In Power Apps add a new custom connector based off the Open API file (version 2.0) to generate the connector.

The process is a pain, but this approach does not need any software installed.  I do feel Power Platform are about to improve by accepting the Postman v2 collections and it would be great if it could read Open API file in YAML and JSON in both V2.0 & 3.0 but any of these permutations would be a game changer.

Warning: Large collections end up with information being lost in the translations.

Option 2: 

1. Export the postman collection in v2.1
2. Upload the Postman 2.1 Collection file into Transformer | APIMatic and generate the JSON Open API 2.0 version.
3. In Power Apps add a new custom connector based off the Open API file (version 2.0) to generate the connector.

Distribute tracing using Azure Application Insights across Azure SaaS product

Overview:  Building SaaS products using multiple underlying Azure PaaS, and IaaS services with multiple Microservices supporting and calling each other is great.  The issue is we need to be able to trace, debug, and observe the logic flow through multiple Microservice calls.  Distribute Tracing on Azure supports technical players such as devOps teams, developers, support, technical leads and/or architects to find and trace the entire execution to figure our what is/has happened.  Application Insights provides rich functionality on Azure PaaS services.

Distribute Tracing:  A good option for providing consistent tracable logging is to use Application Insights with the Distributed Tracing to trace the flow of each transaction.  The original request generates an Id which is set as the operation_parentId.  Now we can easily follow the execution of a specific operation.

Thoughts:  Distributed tracing generally catches exceptions from the underlying infrastructure services such as SQL, Azure Functions, App Service on Windows but in code you can add additional tracing information.  The tracing info can be exception based but most of this is picked up anyway or trace base (when an even happens you want to record).

It is a good idea to instantiate the Telemetry client once per service e.g. webAPI and merely call using the same instantiate telemetry object instance throughout each application.

On exception in both client and Server side code write to App Insights telemetry.  Below is the C# server side code snippet:

try

{

    ...

    telemetry.TrackTrace(message, SeverityLevel.Warning, properties);

}

catch (Exception ex)

{

    telemetry.TrackException(ex);

}

More Info:

Distributed Tracing in Azure Application Insights - Azure Monitor | Microsoft Docs

Application Insights API for custom events and metrics - Azure Monitor | Microsoft Docs

JMeter - The basics

JMeter is an easy to use open source load testing tool by simulating network requests.  JMeter is good for figuring out how well the server side responses are working under different test conditions.  JMeter is built with Java and can run on Linux, Mac or Windows using a Java Virtual Machine (JVM). 

JMeter is Single Agent:

• JMeter runs from the machine it is installed on so it does not have multiple agents.  Saying that it can simulate hundreds of users on fairly low spec machines.  
• To avoid network latency, test on the same subnet or data center.  A simple VM in Azure (with 2 vCPU's and 8 GB RAM) can mimic over a thousand requests per second.
• You can run tests off multiple machines to generate extreme loads (first I would use 8 cores and 64GB ram until the network traffic is maxed).
• Install the Windows JDK 11 before installing installing JMeter.

JMeter GUI

Open /bin/jmeter.bat

Azure DevOps User Stories Tips

 Quick Point on User Stories and Acceptance Criteria.

1.      User Stories description must follow the format:

As the <role> , I want to <feature> so that <benefit>

Note: Always follow the exact same format and bold up the standard/fixed parts for user stories.  Pls keep consistency across your teams user stories.  Under the user story in the description, feel free to add more description, annotated images (very useful) and links to Figma, Axure, UI mocks or Miro.

2.      Acceptance Criteria (Use Gherkin Language) under the user story (ensure it goes into the User Story section and not comments or the description)

Scenario:
  Given
  When
  Then

Example

Scenario: Employee requests leave
  Given an employee has sufficient leave available in the year
  When the employee schedules leave (holiday)
  Then the employee is informed his request is valid and his manager is informed of the request.

Note: Always follow the exact same format and bold up the standard/fixed parts for user stories.  Pls keep consistency across your acceptance criteria.  I bold and use the four parts as shown above in the example.

3.      Other:  Order is Tasks belong to User Stories, User Stories belong to a Feature.  Features belong to Epics  These items must be related within Azure DevOps. 

Scrum - Part 1
Scrum - Part 2 
Scrum - Part 3 - Scaling Scrum
Scrum - Part 4 - Kanban and Scrum
Scrum - Part 5 - Certification PSM 1

Open Banking & Crypto currency - Capital Gains Tax

In the 2019/2020 & 2020/2021 UK tax years, I bought and sold crypto currencies; I happened to make a little money out of it thru no skill of my own.  "A rising tide lifts all boats".  It has been bothering me so I looked up how taxing crypto works and it falls under Capital Gains. 

UK individuals get £12,300 I am well below the threshold for owing HMRC additional tax.  I still need to report my gains on my SA100/Self Assessment with HMRC using SA108 for reporting Capital Gains.  There is also a great initiative Payment Services Directive 2 (PSD2)/Open Banking offered by companies like TrueLayer that provide API's to get current account statements/transaction using OAuth2 permissions.  PDS2 is pretty useful for anti-money laundering (AML), Know Your Customer (KYC) and fair amount more that I'm not an expert in working with.

It is worth using this capital gains allowance if you can get solid returns off applicable assets such as shares or cryptocurrency.

"When you dispose of cryptoasset exchange tokens (known as cryptocurrency), you may need to pay Capital Gains Tax.

You only have to pay Capital Gains Tax on your overall gains above your tax-free allowance. The Capital Gains tax-free allowance is: £12,300".

TrueLayer and Token is a service that connects to all UK commercial banks instead of you having to individually connect to the banks Open Banking API's.

References: 

Payment Services Directive 2 and Open Banking | UK Finance

https://www.gov.uk/capital-gains-tax/allowances

Elastic Database Client Library for client database segregation on Azure PaaS for SaaS

Overview:  Provide a logically separated database instance for each client on my SaaS solution.  Using the Elastic Database client library from Microsoft on Azure PaaS services provides logical security separation of data, performance is on a per customer, and easy scalability.  Use Azure SQL Elastic Pools (HA redundant secondary database, built in DR).  Also add temporal tables for a full history of all transactions.

PoC:

1. Provision 3 databases - A Shard Map Manager (Catalogue) database and 2 client databases (tenants/shards).
2. Add shard related metadata to the Catelogue database for each of these databases.
3. Create below Three service principals in Azure AD: 

• Management Service Principal: for creating shard metadata structure.  A database contained user in Shard Map Manager db and each tenant db.
• Access Service Principal: to load shard mapping at application side.  A database contained user in Shard Map Manager db.
• Connection Service Principal: to connect tenant database.  Database contained user in each tenant db.

                        Management Service Principal: for creating shard metadata structure

CREATE USER [shard-map-admin-sp] FROM EXTERNAL PROVIDER

EXEC sp_addrolemember N'db_ddladmin', N'shard-map-admin-sp'

EXEC sp_addrolemember N'db_datareader', N'shard-map-admin-sp'

EXEC sp_addrolemember N'db_datawriter', N'shard-map-admin-sp'

GRANT EXECUTE TO [shard-map-admin-sp]

 

Access Service Principal: to load shard mapping at application side

CREATE USER [shard-map-access-sp] FROM EXTERNAL PROVIDER

EXEC sp_addrolemember N'db_datareader', N'shard-map-access-sp'

GRANT EXECUTE TO [shard-map-access-sp]

                                                         

Connection Service Principal: to connect client/tenant database

CREATE USER [tenant-connection-sp] FROM EXTERNAL PROVIDER

EXEC sp_addrolemember N'db_datareader', N'tenant-connection-sp'

EXEC sp_addrolemember N'db_datawriter', N'tenant-connection-sp'

EXEC sp_addrolemember N'db_ddladmin', N'tenant-connection-sp'

GRANT EXECUTE TO [tenant-connection-sp]

References:

https://docs.microsoft.com/en-us/azure/azure-sql/database/elastic-database-client-library