Showing posts with label monitoring. Show all posts
Showing posts with label monitoring. Show all posts

Wednesday 31 January 2024

Low code ROI/TCO and observability - Monitoring Low code Platforms

Overview: With Low code gaining tremendous traction, the rate at which apps are being built is increasing quickly.   More citizens can build and pro developers can build more.

Governance helps maintain the quality of the apps and their code, but as an app needs to be updated, it becomes significantly expensive to maintain and update the low code solutions.  The P-F curve has been around for awhile for regular assets (I first saw this 20 years ago with Mining equipment in Southern Africa, Machine maintenance on the mines help significantly reduce the cost of the assets/equipment and reduce catastrophic outages, anyway I've digressed...).  

Microsoft Low Code/Power Platform:
All low-code platforms have the same issues, governance improves the situation but it does not address the P-F interval, and monitoring comes into play.  Microsoft's Canvas Apps has the ability to ship logs to Azure Monitor (App Insights) and there are internal analytics available on the platform.  These are great and should be implemented.  Now most enterprises have multiple environments and tenants in Power Platform, ALM is used to improve the quality of the application being  released.
Canvas apps has an internal testing tool called Test Studio which is fairly limited and yet still underused.  Tests can be recorded, on deployment releases the recorded rules can be run and reported on in ADO or any CI/CD pipeline.  

You can also make web calls from CI/CD or a dedicated service to continually run advanced availability tests, you should do this.  It ensure we pick up when part or all of the process changes.  And you can setup alerts and respond.

Tools:
You can also log to SIEM (enterprise logging tools) and use this for Monitoring.  For instance, you can ship you App Insights/Log Analytics/Azure Monitor logs to Dynatrace so it is part of your overall monitoring and response strategy.

You can continuously monitor API using CI/CD tools like ADO but I prefer to use Postman's Enterprise Infrastructure service (it's awesome).  Similarity, I am a full convert to Microsoft Playwright (I was a Selenium and Cypress fan, not i'm only Playwright).

Grafana, Power BI, Azure Dashboards all have different pros and cones for monitoring.  There is also some great AI stuff coming out at the moment around UI testing.  Specifically, I've seen Dynatrace, Playwright and BrowserStack use AI to compare screenshots for test validation.

I'll be expanding on automation, continuous testing in this series of posts.  These are my initial thoughts and there is a lot of good results coming out.

Sunday 30 July 2023

Latency Metrics for API's

Overview:  More and more software is built on API's, we often need to know what are our slowest performing API's and how important are they.  Monitoring latency is how we determine performance and performance issues.  We need to know the fastest, most used, slowest, and average time to complete.  You need to look at all to get a full picture.  Latency metric let us know what percentage of monitoring metrics fall into a range.  

For instance, if you API end averages 1 seconds for all requests (10k) over an hour that sound okay, but if the majority of requests don't have data say 90%, and the slowest 10% of requests could be averaging 5 seconds.  Monitoring metric percentage take out the slowest % of performance requests and show the  faster performers, so in this scenario the slowest 10% of requests are excluded .  Often referrer to by the percentage as P90 i.e. 90%.  I use 95%/P95 normally but it's becoming more common to us 99%/P99 or even P99.9.

Tuesday 20 June 2023

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards

Series

App Insights for Power Platform - Part 1 - Series Overview 

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics 

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM 

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards (this post)

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power Automate Licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern

Overview: Azure Dashboards are excellent but, if you want beautiful dashboards use the Azure Grafana service or Power BI dashboards.

It's always difficult to identify KPI's and graphs to allow support and stakeholders to quickly digest monitoring information.  An option is to have an overview for the PM, PO, Business owners,,, and a separate set of dashboards for support.

Identify What is important?  End-to-end testing is always nice especially if running CI to detect abnormalities.

For instance, this Azure Dashboard, fires of a Canvas App recorded Test (done using test studio) and shows the speed (performance) of the run, i then warn the user if the performance is too slow.  I also grab the oldest successful run from 7 days ago to check if performance is considerably different.  

Power automate has it's own logging, but integrating log entries when a error occurs via Log analytics, allows me to see if any of my workflows have a problem.  This is discussed in part 6 of the logging series on Flows.

Azure services are often used when building solutions using the Power Platform.  The most common are Functions, App Services, APIM, Service Bus, maybe sendgrid.  So we need to know that they are working, and help the user see performance or issues.  Here are a couple of examples.





Series

App Insights for Power Platform - Part 1 - Series Overview 

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics 

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM 

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards (this post)

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power Automate Licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern


Friday 9 June 2023

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics

Series

App Insights for Power Platform - Part 1 - Series Overview 

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics  (this post)

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM 

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards 

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power Automate Licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern

 There are two ways to setup App Insights:

  1. Classical approach (soon to be removed), and
  2. The Version 2 approach also refereed to as the workspace based app Insights approach.

Image1. The Version 2/Workspace-based App Insights approach stores all new logs in Log Analytics storage.

More: Using App Insights using "version 2".  The original app insights stored it's logs within itself, this is sometimes refereed to as "classic app insights".  Classic App Insights is being deprecated so version 2 is compulsory from early 2024.  "Version 2" stores App Insights Logs in a workspace (Azure Log Analytics).

We need all our service i.e. Canvas apps, Dataverse, Power Automate, APIM, ESB, Key Vault, Azure Functions to store operation logs in App Insights workspace based approach.  We shall discuss Canvas App Logging in <App Insights for Power Platform - Part 3 - Canvas App Logging>.

Note: Operations logged to app insights store under the hood consist of 3 parts: 

1. App Id to log to

2. Content to put into the Log analytics for full logging

3. Metric data.

Setup App Insights

Open The Azure Portal.

In your subscription, you need a resource group and storage, go and add the log analytics and the app insights.

Setup the Log Analytics instance to connect the App Insights instance too.  The free tier is normally sufficient for demo purposes.

Setup App Insights using a Workspace/Log Analytics, and pls name your resources properly.

View your logs

App Insights is integrated well with all Azure services and are easily accessible.  We will go into the AppInsights Blade and look at the Logs, I added this query that will look for all logs and ordered them to show the latest first.

Note: The logs are stored in Log analytics.  To view the logs you can either use App Insights or Log analytics and the syntax is slightly different, see the image below:

Terminology Worth Understanding:

App Insights stores data in Log Analytics, you can read/write thru App Insights or Log analytics.  There is also Azure Metrics.  All of these services fall under the umbrella term of Azure Monitor.  When writing to the logs, the data is made up of 3 parts. 1, identifier for the log 2, Log analytics data that can be queried and 3, metric data. 

APIM Monitoring & Logging via Portal:

Sample Kusto Queries:

// Function used to call APIM

dependencies 

| where cloud_RoleName == "azure-func-name-01"

| where type  == "HTTP"

| where target !contains "login"

| order by timestamp desc 

// Check Outbound APIM 

requests 

| where cloud_RoleName == "devapim North Europe"

| order by timestamp desc 

// Backend data is in the customDimensions logged by APIM

dependencies   

| where type == "Backend"

| order by timestamp desc 

| extend req = tostring(customDimensions["Request-Body"])

//| project  timestamp, id, req

| where req contains "BJ69 TFF"

// Retrieve Canvas app data based on customDimensions logged 

pageViews

| extend 

    AppName = tostring(customDimensions["ms-appName"]),

       Env = tostring(customDimensions["ms-environmentId"]),

    LastSuccess = datetime_diff('minute', now(), timestamp)

| where AppName == "Bus Revenue Inspection"

| summarize by Env 

//| summarize arg_max(timestamp, *), Count = count() by AppName

//| order by LastSuccess desc

//| project LastSuccess, NoOfPageViews = Count


Example querying Azure Log Analytics for Traces I raised from a Canvas App

// KQL syntax varies slightly when querying the Log analtics rather than App Insights. 

AppTraces

| where Message contains "App Loaded with Events issue - Compliance Subject"

| extend 

    AppName = tostring(Properties["ms-appName"]),

    Env = tostring(Properties["myappEnvironment"]) // Properties is used instead of customDimensions

| order by TimeGenerated desc


Series

App Insights for Power Platform - Part 1 - Series Overview 

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics (this post)

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM 

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards 

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power Automate Licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern

App Insights for Power Platform - Part 1 - Series Overview

Overview: Microsoft have great capabilities for logging and monitoring.  In this series of posts I will be examining the various parts of logging that may be useful in building solutions that are well monitored, provide alerting, easy tracing, and identifies issues or potential issues as soon as possible.

I am looking at App Insights for Power Platform monitoring.  So this includes: 

  • Power Apps (Canvas, and model apps),
  • Power Automate,
  • APIM, 
  • Azure Functions, 
  • Azure Service Bus, and
  • App Insights.

I shall be setting up a demo environment and these are the logical components being covered.


All the components making up the solution shall log into Log Analytics (left-hand side of the diagram).

For Continuous Integration, my clients will be Postman monitor (it's awesome and so easy to use all those postman collections), DevOps is great and I'll use it to run smoke tests after new releases.  I also use flows, to report on flows (sounds nuts but i love it).  These are at the bottom of the diagram. 

Lastly on the right of the diagram, I look at extracting logs for reporting (Power BI), and Monitoring using Azure DevOps (p.s. think about Grafana instead of DevOps Dashboards, it so nice).

Couple of extras are: Availability Logging, alerting, automating Canvas app testing, Playwright.  

From the diagram, you can see the data is now held in Log analytics and it can be queried via Log Analytics or App Insights using Kusto.  Note: the syntax is slightly different.

Series

App Insights for Power Platform - Part 1 - Series Overview (this post)

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics 

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM (this post)

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards 

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power automate licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern

Tip: The Power Platform Admin Centre has a good overview of the Power Platform, but to make logging and monitoring better push data into Azure Log analytics and monitor and alert centrally.

Also seeView and download Dataverse analytics - Power Platform | Microsoft Learn

Sunday 14 May 2023

Dynatrace Product Play

 Dynatrace is pretty similar to Azure Monitor.

  • Dynatrace (really good if you use multi cloud) Dynatrace - Saas offering is on AWS.  Can be on-prem.  
  • Making workloads observable is using Logs, Traces, Events, and metrics into Dynatrace.  From these ingested events we analyse and can automate behavior. 
  • OneAgent is deployed on the Compute i.e. VM, Kubernetes.  Can import logs from other SIEMs or Azure Monitor, so you can eventually get Azure service logs such as App Service or Service Bus.  
  • Does Full stack and includes code-level and applications and infrastructure monitoring, also can show User monitoring.  
  • Dynatrace offers scalable API's that are sitting on Kubernetes.  
  • "Davis" is the AI engine used to help figure out the problems.  
  • Alerting is solid.  
  • Dynatrace can log against 1) network/Infra 2) SDK 3) DEM (User monitoring,..)  logs, traces, metrics are ingested either using OneAgent or OpenTelemetry.
  • Management Zones - user only see's information they have access to and need.
  • Define a Site Reliability Guardian (SRG) to each program/project, this allows you to identify thru RAG boards the current and recent state of the various pieces.  There are Guardian templates to use as a starting point.
  • W3C Trace Context is used - it allows for end-to-end tracing.  OpenTelemetry or Dynatrace keep the trace and provide in headers (traceparent.
  • Create documentation and tutorials for Dynatrace.  Dynatrace has a playground tenant for playing on.
High-level Architecture hosted on AWS.

High-level architecture for capturing logs et al. and then using the data.

Product Screen Shots:






Azure & Dynatrace
  • Abnormality detection using AI. shall greatly improve observability and security. 
  • End-to-end visibility is what makes it so amazing.
  • Enterprises often use Dynatrace as there central SIEM solution, shipping from Azure in Dynamics takes planning but works well, categorise and ensure the right into is pushed into Dynatrace.  
  • Dynatrace is the leader in Gartner and Forrester in it's space.
  • Grail - lake house, schema-less, allows for easy fast query.  Massive scale.  Bring all data together and query at hyperscale.  Grail is in 15 regions either on AWS, Azure, or GCP for customers to use.  UK looks like AWS only. 
  • Grail: Record level protection, masking data, support access controls (elevate privileges).  
Dynatrace architecture for Grail from Barcelona conference 5 Oct 2023.

Collect all events in Grail, automate the process of identify suspicious activity relating to security.  Faster reaction time.

Azure offers Dynatrace as a SaaS service
Updated 16 Feb 20224