App Insights for Power Platform - Part 11 - Custom Connector

Overview:  Power Automate can set retry policies on custom connectors, Canvas apps using a Custom Connector, does not have any retry configuration.  FYI is the Custom Connector gets a 5.x.x error it shall retry 4 times.  Proven using 500 and 502 errors.  408 (timeout) and 429 (to busy) errors appear to throw 4 times (retry driven by canvas app; using the Custom Connector trigger shall only try once).  

My example:  My Canvas app uses a custom connector, that calls my Azure Function, in turn this calls my APIM, and APIM calls the 3rd party.

My Azure function returns a 500 or 502 or 408 (response timeout) or 429 if the response has not been received in 10 seconds, and I push the http code back to the Custom connector.  I can see the response from the 3rd party is taking +-35 seconds.  I can see from network traces 4 invoked calls that all fail with 408 http response codes.

Result:  The custom connector retries 4 times resulting in my Power App being locked for 40+ seconds.

Possible remediation: 

1. Return a 418 HTTP response code (I am a teapot), my app calls the function using the custom connector once.  So using 418 (also tested with 400 - but 400 is not the right response) errors behaves differently from the 5xx, 408, 429 errors from the API.  

Note: Originally I was caught out as I trigger using the custom connector test rig, this only tries once.  But when called from Power Apps, it shall try four times.  Returning 408 is not a fix.  Returning 418 ensures I only try once, get a better user experience and now I have 418 logs that I add error details to.

2. 3rd party API should not take 35 seconds, improve it.  

3. I could set my timeout on the specific function to 40 seconds, however if the call starts going to 41 seconds, my canvas app will be locked for over 160 seconds.  

4. Go to all API's and if they are fast set the timeout as short as possible so the app does not get locked out while waiting for the 4 responses.

Summary: Examine the 3rd party API's, get them stable and performant and per the agreed SLA's.  If you only want to try once ensure that time outs on the 3rd arty are set to the SLA or if you intercept the request, you can choose the timeout, by examining the API's you can see the optimum time to avoid timeouts and using the 418 response code, the call only happens once.

Series

App Insights for Power Platform - Part 1 - Series Overview 

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics 

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM 

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards 

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power Automate Licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern (this post)