Monday 28 August 2023

App Insights for Power Platform - Part 10 - Custom Connector Logging Thoughts

Overview: One of our developers was asking about a log he was struggling to trace, and it took me awhile and a lot of help from the community to truly understand the issue.  My scenario is shown below:


Scenario: I have a Dataverse change triggering a flow, the flow calls a Custom Connector, that in turn calls an Azure Function (that I control).  So the flow fails, and I have used a pattern in the flow to catch the error and log it into Log Analytics.  All good, then I don't see the event where the action calls the function, my function has logging enabled.  I can see I am getting a 401 unauthorized error.  

Initial Hypothesis: The Power Platform use APIM internally to implement Custom Connectors, and there is no access for clients/tenants to see the internal logging/traffic.  Microsoft have provide the ability to use iLogger on the custom connector to log the traffic.  

We have flows that intermittently get a 401, when the flow is manually rerun, the flow works and I can see the traffic coming into the Function.

The failure rate is extremely low and retries almost always fix the issue, and a third try always ensures the transaction goes thru.

Resolution:  Add logging to the custom connector so we can speak with MS support about the issue.  Add alerting to notify support, they can contact the user or chose to rerun the flow. 

Alternative: If I  enable the code, I can override the behaviour and inject C# code to work with the backend, or handle logic such as replacing text,... 

1. In step "4. Code" tab of the custom connector, add the code below:

You can do any C# logic, I'm sending the original request thru and if it doesn't return me a 200, I'm logging it as critical. 

2. Update the connector, go to the next step "5. Test" > "Update Connector" (Tip: follow the steps)

3. Run the "Test operation", open the Response and validate the response body is correct, then open the "Code logs" tab.  If it is blank, re-run the "Update Connector" (irritating but true). 

304 return from the API, which is cached and not a problem, but 400, 500 would be an issue, could also look out for 429s.

Full C# Code:

public class Script : ScriptBase

{

    public override async Task<HttpResponseMessage> ExecuteAsync()

    {       

        this.Context.Request.Method = HttpMethod.Get;

        HttpResponseMessage response = await this.Context.SendAsync(this.Context.Request, this.CancellationToken).ConfigureAwait(continueOnCapturedContext: false);

        Context.Logger.LogTrace("Custom Connector ListBooks called "); 

        if (response.StatusCode == HttpStatusCode.OK)

        {    Context.Logger.LogTrace("Success");      }   

        {    Context.Logger.LogCritical("Critical | " + response);     }

        return response;

    }

}

More Info:

https://learn.microsoft.com/en-us/connectors/custom-connectors/write-code (NB)

https://never-stop-learning.de/logging-in-custom-connector-code/ (NB) The 2nd part of this post on the Alternative , is a rehash of this amazing post - I amended the logic and now I'm wondering is I could write to App Insights using the SDK?

Series

App Insights for Power Platform - Part 1 - Series Overview 

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics 

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM 

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards 

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power Automate Licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging (this post)

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern


Sunday 20 August 2023

Common Scrum Team improvements

 WIP

Problem: Bugs must not change scope - common to see a bug and it morphs into improvements, other changes.  

Fix: Raise new bug. Don't keep amending and adding new features.

Problem: Team testers/automation testers do not understand the requirement.  Cause by the User Story (US) or bug being ambiguous and Acceptance Criteria (AC)

Fix: Ensure User Stories are in a standard format i.e. "As a <>, I want to <>, so that <>.".  Ensure Gherkin is used for AC (Scenario, Given, When, Then, and preferably more than 1 AC per user story.  Could have: Annotate diagrams to clearly convey artefact information.  Also a narrated recording for bugs.  This could be the person recording the bug, the person that fixed the bug as proof to increase understanding. 

Sunday 13 August 2023

App Insights for Power Platform - Part 9 - Power Automate Licencing

Overview:  Licencing is extremely complicated, but there are threshold limits that are being reduced at the moment, August 2023.  

O365 users get get the lowest priority profile, can only run the standard connectors, and have a "request" limit of 6,000 requests per day.

What is a Request?

Each flow consists of a combination of triggers, actions, and responses when cloud flow is run, the instance walks thru the actions such as Create a SharePoint list item, setting variables, 

What counts as a Power Platform Request

"Here are some guidelines to estimate the request usage of a flow.

  • One or more actions run as part of a flow run. A simple flow with one trigger and one action results in two "actions" each time the flow runs, consuming 2 requests.

  • Power Automate Flows, by default, run in the context of the Flow Owner.  The "actions" are worked out against the Flow Owner.

  • Every trigger/action in the flow generates Power Platform requests. All kinds of actions like connector actions, HTTP actions, built-in actions (from initializing variables, creating scopes to a simple compose action) generate Power Platform requests. For example, a flow that connects SharePoint, Exchange, Twitter, and Dataverse, all those actions are counted towards Power Platform request limits.

  • Both succeeded and failed actions count towards these limits. Skipped actions aren't counted towards these limits.

  • Each action generates one request. If the action is in an apply to each loop, it generates more Power Platform requests as the loop executes.

  • An action can have multiple expressions but it's counted as one API request.

  • Retries and additional requests from pagination count as action executions as well."

Here are my thoughts which seem to differ from the MS notes provide above: Not all Actions count as a request, If i look at the Power Automate Analytics it gives me a break down on the API calls to understand the "Request" counting.  Basically any action that does an API call when run adds to the request count.  

Guide for planning for limitations:

  • O365 users get 6k request per days
  • Dynamics and most per user plans get 40k requests per day.
  • As a rough guide, I count simple workflows as 3 requests average, medium as 7 requests, large can be over 100 so it is better to build the workflow and from the analytics you can get the number of requests per day.
  • For each flow multiply by the estimated number of calls
  • Understand who the quest is attributed to (either the user or the owner of the flow, the requests are counted against the flow owner unless the flow use a pay per flow model.)


Example: to calculate billable actions/billable requests

I have a single Flow running against my O365, the flows has a Power Apps trigger, then creates a new list item and lastly responds to Power Apps.

1 Cloud flows that has 3 billable actions run 5 times will result in 15 billable actions.

I have 6k per 24 hrs on an O365 licence, most of the other licences such as Power Automate premium, an account has 40k requests per 24 hours.
I could run the flow 1,200 times in 24 hrs under an O365 licence.

Series

App Insights for Power Platform - Part 1 - Series Overview 

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics 

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM 

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards 

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power Automate Licencing (this post)

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern

Friday 11 August 2023

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 1 - Series Overview 

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics 

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key) (this post)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM 

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards 

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power Automate Licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern

Overview: Logging & monitoring for Canvas apps is done in two parts: App Insights, and using the Canvas app Monitor.  This post focuses on logging via App Insights.


Note: Once a solution that contains a Instrumentation key, they app logging key cannot be alter unless you make the environment have unmanaged layers.  You can use PowerCli and compose a new managed solution for each DTAP environment but it's a new compile for each environment.

Example:

In the annotated diagram below including a log snippet.  

1. Canvas App has an instrumentation key, the log captures the front end action

2. Calls to Dataverse & Power automate Flows are logged (relies on step 1)

3. Custom connector is calling an Azure Function (Function is logging to Log Analytics or app Insights),

4. the function logs into APIM and sends APIM a request (APIM logging is setup on the end points), and

5. APIM sends an outbound API request and captures the response (relies on step 4)

Note in this example I have Correlation tracking enabled on the Canvas App to get the full timeline, as shown below, it has been an experimental feature for a few years now.


When I turn off the Correlation, it is not as easy to trace items from start to finish.  All I get by default is the steps 3&4 data in my transaction search timeline.

All 5 pieces are still captured but the timeline has to be pieced together for tracing.


I would also enable the preview feature for logging as well as the experimental if the clients governance allows experimental features to be turned on.

Summary: Always add as many logging features as possible in Canvas Apps, think about where your logs go and also setup logging on Azure services to transaction can be traced.

Sunday 6 August 2023

Microsoft Purview DLP Policies

Overview: Microsoft Purview allows a company to create end device DLP policies and push out to all client devices.

Create policies, and takes up to 2 hours to update end client devices and show on the "Device Monitoring" dashboard.  Client Analyser tool - check endpoint client devices are getting the latest policies, and can speed it up.  Device obviously has to be online to get the latest update.

WIP

Thursday 3 August 2023

Visual Studio App Center

Visual Studio App Center is an Azure/Microsoft Service for packaging and deploying mobile apps. "Visual Studio App Center lets you automate and manage the lifecycle of your iOS, Android, Windows, and macOS apps. Ship apps more frequently, at higher-quality, and with greater confidence." Microsoft Learning

Useful for Canvas apps that will be deployed via the stores or InTune to mobile user devices.

To login to the App Center go to https://appcentee.ms/apps (I logged in with an Azure Tenant admin account)


First create an Organisation, in the organisation you can create your mobile Apps for the Organisation.

Note: Max 2 new orgs per day.  I delete all my Orgs and was messing around to find i could not add anymore organisations.

Problem: I cannot create new organisation in the App Center and i get the error "You can not create more organizations at this time. Please try again later."


Hypothesis: I thought I had broken my tenant and tried repairing it with the AppCentre Cli, no joy.  Went to ms support and they immediately told me that a max of 2 new orgsanizations can be created per day.

"The limit for creating organizations is indeed 2 organization's per day, to prevent abuse by malicious actors. Now you have to wait 24 hours to create other 2 more organizations."

The error message could provide a clearer expalination.

Resolution: I'll wait until tonight as my 24 hour 2 org limit should be available in a few hours.


Tuesday 1 August 2023

Wrap Feature of Canvas Apps for Android store deployment

Overview:  I am going to use the wrap feature for Canvas Apps to make a separate deployable package (apk) for Android devices.   Ensure you have setup the Wrap solution so the feature is available in each environment you deploy to.  

Tip: Wrap deployment has many nuances, I try take them away by have Admin on the Power Platform tenant and Azure.  Not ideal for enterprise client deployment.

Power Platform Setup:


There are a few ways to get to the wrap screens, here I click the vertical ellipses, and select "Wrap"


Walk thru the wizard as shown below, step 1, select the apps 



Step 2 - choose mobile platform
I need to disable signing to get Android/apk to build.  i.e. Under "Sign my app", turn it off.

Step 3 - Configure Branding

Step 4 Register a new app


Complete step 4 Register your app (ensure all boxes are ticked and green)

Step 5.  Manage Output - you need to get a Auth token by selecting the "Get App Center token" button and signing in (I'm a tenant admin and have AAD Admin rights)
Step 6. Build the App and wait to ensure it completes

Step 7. Verify Build


Step 8. Check you App Center

Note: On the Power Platform tenant, you may need to change configuration to allow third party apps.  I add this once per tenant i.e. not per mobile app on a tenant.  So i just need to to this once per DTAP env e.g. Production.

PS> Install-Module -Name Microsoft.PowerApps.Administration.PowerShell

PS> Install-Module -Name Microsoft.PowerApps.PowerShell -AllowClobber

PS> Add-AdminAllowedThirdPartyApps -ApplicationId 84xx-xxx-xxx-xxx


Update: 1 Aug 2023: I can't get the build to work for pkgfiles and the wizard is returning multiple js errors during the process.  I delete my organisation, create new organisations and Apps.

Image. Cannot build apk package in app centre