Friday 6 March 2020

Power BI Notes

Overview:  Power BI is for reporting  and analytics of your data.  There are basically 2 ways to show Power BI Reports: User specific and app specific.

Power BI Embedding Models:

  1. User Specific/User Owned Data - Call the Power BI services as yourself/the current user using delegate permissions. 
  2. App Specific/App Owned Data - Call the Power BI service using a generic app permissions.  For example a public website, no Power BI licence required and every user of the site has the same access to view Power BI data.

Sunday 1 March 2020

Power Automate Notes

What is Power Automate?
Power Automate previously called Flow.  Power Automate contains "Flows".  Power Automate is workflow including RPA options, refered to a Power Automate Desktop (PAD).  Power Automate is a workflow engine that is based on Azure Logic Apps.  Powerful extendable workflow solution for low code automation.  Allows workflows to be easily automated with 3rd part systems e.g. SAP.

Used for:
  1. Personal Workflows e.g. I send an email to all people that have not update the DevOps Scrum board on a in the last day as a scrum master.
  2. Business Process e.g. Holiday request form.  If more than 10 days, need senior manager approval.  Generate RFP based on an event.  Historically, used K2 or Nintex or WCF workflows for business processes.
  3. Integration: e.g., move twitter posts into my warehouse for data-mining later.
A key concept is who shall the flow run as, a user or a service principal.  Setup a Service Principal in Power Automate » Benedikt's Power Platform Blog (benediktbergmann.eu)

Flows run against the owners account for calculating allowable usage (default), it is common practice to use a service account (normal AAD account with user and Pswd) to own specific flows.  A service account is a normal user account from AAD/Entra's perspective.  A dedicate account ensures that when a specific user leaves your business, the flow continues to run.
Img 1. Usage limits in Flows are counted against the flow owner.



Agree Power Automate Standards:
  1. Flow Names: start with a verb  format: Verb + What the Flow does + (trigger) e.g. "Get Tax Rates (Instant)".  I like to also prefix with the Company and Project, but feel free to have a standard to suit your business.  e.g. EY-USTax Get State Taxes (Instant) or EY-USTax Get All US State Tax Rates (Scheduled) or Get SalesForce Data.  Optional, for project specific workflows I also prefix witht he project name e.g. USTax-GetTaxRate. 
  2. Description: Short description to help readers understand what the flow does.
  3. Actions: Leave the Action desc and add info e.g. Compose: Tax Note.
  4. Variables: prefix with "v" e.g. vTaxTotal in camelCase.  e.g. vUserAge.
  5. Error handling & Logging: Catch errors and log into to App Insights via an Azure Function or Log using the built in Azure Log Analytics Action.  More logging means better traceability.
  6. Scope: Add scope actions for try catch logic.  Add multiple actions inside a "Scope" Action
  7. Terminate the flow with the Terminate Action if the flow has failed.
  8. Environment Variables: Great for logging as I go thru DTAP.  Also see.
  9. Connection Reference Name: Agree a format, does this flow run as  user or as a specified user.
  10. Loop Sparingly, use First() for performance.
  11. Owner: I like to use a service account in dev, it's a good idea to add tech owners as when it needs updating to support and easily find who they should talk too.  Understand who you are running the flow as, this ties to licencing but is critical.  You need to know you Actions and licencing limits on a project.
  12. Comments:  Im not a huge fan as the naming should make most flows make sense/self documented, but for tricky logic, comments are great.  Agree a standard.
  13. Retry policy: What happens if an action fails, do you want to try again?  
I borrowed a lot of the standards from "Matthew Devaney"
I found this document later - it is excellent. by Prasad Athalye - Best Practices for Power Automate(Microsoft Flow) Development

Licencing:
  1. Seeded licence is part of O365.  Use standard functionality such as standard connectors without needing to pay more for advance.  The advance/premium connectors are not part of the O365 licence.
  2. Per User licence -  Allows the  user $15 retail, can get discount with bulk and can use the advanced connectors & on-prem. gateway.  Many users need multiple workflows, normally personal workflows.
  3. Per User RPA licence - same as above but also has amazing RPA capabilities.
  4. Per Flow/Process - $100 per process per month, min 5 flows per month licences.  Anyone can use as part of the process.  Use for few people but process does a lot of workflows.  Can add a process one at a time after the first 5.
Licencing MS page
Power Automate has some licence add-ons available: AI builder and an unattended RPA add-on.
"Power Apps licenses will continue to include Power Automate capabilities", I don't know what is included but I assume it means any connector I can use in Power Apps, assuming I'm in Power Apps I can make Flows for.

Build workflows:
  • Can get a dedicated IDE tool for Power Apps or use the browser (which i always use).
  • There are over 350 connectors (in both standard and premium) and you can always use a custom connector to any OpenAPI (Swagger) endpoint.
  • Templates have some examples and often help as a starting point to make your own custom Flows in Power Automate.
  • Easy clear tracing so you  can see what part of the workflow is working and where you fail, and you can drive into the issue.  Super easy to use.
  • Example of an Instant Cloud flow triggered by a canvas Power App...

Query a Dataverse table in a Flow using OData

ParseJSON is fantastic for converting Open API/OData/JSON into an object

Extending - break out for programmatic control, I use C# Functions from my Flows and call them via HTTP triggers.
Retrieving  a row from the Dataverse custom "Subject" table.

Robotic Process Automation (RPA):
  • Also known as UI Flows within Power Automate.  Microsoft have purchase and integrated Softomotive for UI flows to add WinAutomation.
  • Attend (user logged in) and unattended version (complete tasks without manual intervention)
  • Can have multiple instances
  • API is generally better than using RPA as it is versioned and generally not changeable, whereas using a website, they website can be changed causing the RPA flow to fail.  Useful for instance when the RESP API is incomplete.
  • Recording tool for creating UI flows - Web use Selenium to record.
  • 3 Types: 1) Windows /Desktop/Screen reader and 2) web/website (Selenium) and 3) WinAutomation (covers both Windows and Web, easy to use but not as full featured yet).
  • WinAutomation has a drag and drop IDE, has error handling.
  • UI flows are well priced.  Also get AI builder credits with UI flow licences.
  • "Power Automate Per user plan with attended RPA to use UI flows and WinAutomation" Microsoft.
AI Builder:
Cognitive builder e.g recognize forms and extract data. E.g. receive invoices and add to accounting SaaS software.

Other: Zapier is a good tool for end user automation.  Easier than Power Automate but not as structured.  I'd use Zapier to automate in small businesses without O365 or flow licencing and allow end users to do it themselves.

Problem Solving:

Problem:  Another developer created a Flow I need to use from a Canvas page inside a model app.  The Flow is showing up when i say add but i get the error "Unable to add flow"
Initial Hypothesis: The flow is owned by another developer having ownership and the connection is done thru their account, take ownership and change the Connection (Dataverse connection in my case)
ResolutionTo use an existing flow, you need: 1) flow in same solution as the canvas app, 2) Ownership and the connection string needs to be switched to the new developer

Problem: Migrating solutions between environments, all the workflows fail when they use the Dataverse connector with 403 errors. Tracing the flows, I can see the error  "Flow client error returned with status code 'BadRequest' and details {error: code: 'XrmApplyUserFailed... UserNotinActiveDirector ... does not exist in user tenantId"
Problem: Using the service Principal account needs to be registered in the new Dataverse environment.
The connector issue was fixed, we had to recreate the connection from scratch making sure it was set up with a/the service principal. Then we added the registered app into the environment as an application. user.


Friday 28 February 2020

I love Power Apps

No self respecting technologies/programmer/ex-programmer likes simple solutions.  I still chose C# as my go to problem solver (I learnt to use a hammer 18 years ago and now all my problem look like a nail).  Don't tell anyone under 30 I still love C#, they think you are weird if you don't use Javascript for everything.  And 15 years ago I was arguing the same with the C++ golden oldies.  So the circle of life goes on.  Enter Simba, the Lion King...  Well not really but Power Apps is just making programmers look ridiculous.

Power Apps can do a lot...

MS have announce that components and PCF’s are part of Power Apps.  I've been using this for awhile and it's awesome.

This diagram outlines extending Power Apps using Components:  Developers often say you can’t extend Power Apps so it is not a real language, my answer is really… 
The other argument for PowerApps is if you need to do any complex programming.....  Use an Azure Function (this allows C#, PowerShell, JS, …).  Simple to build in your own language, expose using an http endpoint and you can call the end point using a custom connector.  
So if you need: 
  • Complex UI = go PCF, 
  • Reusable UI e.g. menu's = use Components, 
  • Complex processing or secure code logic = Azure Functions.
Here are some PCF examples that we can use that are already built:

So what is stopping Power Apps:
  1. Licencing - the licencing is expensive so you need to choose selectively when it is appropriate.  If MS change it's going to just go crazy the usage. IMHO. 
  2. Functionality:  MS have introduced automated testing to the Power Apps platform but there are improvements scheduled.  For me the biggest issue is code reuse.  I wish there was a function library that you could write functions to and call in a single line of code, my individual field logic can be insane and tough to amend on complex large Power App UI's.   
Excellent blog on Power Apps

Saturday 22 February 2020

Catch Error in Power Apps and App Insight Logging

Error Handling:
App Insights logging: https://sharepains.com/2019/01/24/powerapps-experimenting-with-error-handling/  Replaced as Microsoft have built in telemetry as of 3 Feb 2020.
https://powerapps.microsoft.com/en-us/blog/log-telemetry-for-your-apps-using-azure-application-insights/

Example Error capturing and tracing to Azure AppInsights:
IfError( // Perform API Call , // Fallback so log here! ,
    Trace("Pauls Unique PowerApp",TraceSeverity.Error, {UserName:User().Email,         Role:gblRole, ErrorMsg:ErrorInfo.Message, ErrorControl:ErrorInfo.Control,         ErrorProperty:ErrorInfo.Property});     Notify("Err message ..." & ErrorInfo.Message); // Display the error on the UI
More detail..

Possible Canvas Apps Error Handling Pattern:
  1. Ensure AppInsights key is added to each canvas app
  2. Use IfError() to check calls and logic
  3. Use the Trace method to write info to App Insights
  4. Do I want to enable the Experimental error handling features (great to trace by correlationId)
  5. Consider all Power Automate that use Power Apps (ensure you use the V2 Connector)
  6. Never use IfError to handle business logic
To Review your App Insights Logging:
Open you Azure Portal > Open your App Insights blade >
Click the "Search" navigation option > Free text entry e.g. "Loyalty PowerApp"
App Insights, finding Traces generated in Power Apps

Monitoring Tool within Power Apps

The Monitor tool in Power Apps is great for debugging and tracing.
Start a monitor on the open Power App.

Monitor Tool - Showing a GET via a custom Connector and the returned response

Function/Code Logging:
Server-side code should log to App Insights or you logging framework.
It is ideal with the Trace within Power Apps explained above to be used in conjunction with 3rd party API calls.

Overview: C# code needs to have logging. If an error occurs an appropriate response must be bubbled up for the next lay

Possible C# Error Handling Pattern:

  1. All catch write exception to Log analytics or App insights 
  2. Calls to data sources, Azure Services and third party API's and complex logic ideally should be wrapped in a try catch and log the error to App insights using the C# App Insights SDK 
  3. The catch blocks ideally return the failed information so the caller code can deal with the logic using the output.  If you don't deal with the returned message, simply log the exception and rethrowing the error (this needs to be a conscious decision on each catch) 
  4. Catch specific errors: log, if you don't pass info to caller rethrow the error if applicable (bubble), respond accordingly i.e. catch the specific error and lastly use a catch all. - Heavy, but only add to existing code where this happens often or we are having problems, i.e. be specific
  5. Don't use Try, Catch to deal with business logic

Thought: Bubble up means: Code must log exceptions and returns appropriate reply to the caller, if you don't send the appropriate reply rethrow the exception after logging it so the caller has to deal with it.


Tuesday 18 February 2020

Power Apps Component bug fixed


Problem:  Creating a new component with Input parameters.  When using the component and setting the custom input parameter with a global variable, the value passes thru, as the variable is reset, the control does not update with the variable.  Replication Steps are recorded  in the 2 min 20 sec recording showing the issue.


Initial Hypothesis: I have a large Power App project that uses this technique a lot.  The global variable were working until Fri 14 Feb 2020.  I believe the Power Apps version has been updated and is causing the issue.  I have tried the work on multiple environments and tenants.  It is happening across them all.

https://powerusers.microsoft.com/t5/Building-Power-Apps/Power-App-Components-not-working-with-Global-variables-used-with/m-p/473414

Warning: This affects Power Apps release 3.200221.27 (rolled out to me on Fri 14 Feb 2020), I think 3.20014.23 (latest release as of 13 Feb 2020) was still working correctly.  Don't change your app if it published on the lower versions until Microsoft release a new version.  This only affects Power Apps using Components and using Global variables on the component's input parameters.

Resolution: Power Apps release 3.20022.17 as of 20 Feb 2020 (UK tenant) has been fixed.  Quickly identified and corrected.

Wednesday 12 February 2020

Power Apps - DTAP Azure diagram

Overview:  A sample architecture for DTAP in a highly controlled environment.  There are a lot of variations and the ability to use Power Apps publishing.


There are a few ways to manage environment deployments.
  1. Simple: Create in unmanaged solutions and Export and Deploy Managed solutions manually using the Power Platform Ux.
  2. Enterprise: Build Azure DevOps pipelines to add unpacked code to Github and deploy solutions using Azure pipelines.
  3. 3rd party ALM tooling, some is specific to Dynamics.
  4. Nov 2022 - Microsoft announce Power Platform Pipelines 

Custom Connectors:  As you step thru the DTAP environments using ALM to deploy you need to point the custom connector to the appropriate APIM (API):
Environment Variables are extremely useful for ALM


Saturday 8 February 2020

Lean UX for SaaS - Consider Removing

I am not an expert in UI design ( I am a big fan on keep it simple and consistent) - these are my notes and LEAN UX from workshops, and an expert I have worked with.  They have help me understand LEAN UX as a concept.  A lot of Agile pragmatic teams are already using a lot of LEAN UX without realising. 

Ux is how our users interact with our products.  Using LEAN UX to provide your users interaction design is a good approach. LEAN UX needs to meet their expectations, and help them meet there needs.  User Experience Design over-arches the Interactive design that we normally identify as key when developing SaaS software.  LEAN UX's goal is to reduce the UI development time with a great overall experience.

This book is Gospel for Lean UX.

LEAN UX Definition: Lean UX is focused on the experience under design and is less focused on deliverables than traditional UX.  The core objective is to focus on obtaining feedback as early as possible so that it can be used to make quick decisions. The nature of Agile development is to work in rapid, iterative cycles and Lean UX mimics these cycles to ensure that data generated can be used in each iteration.  from the Interaction Design Org.

Miro is commonly used for Design in LEANUX.

Low-Fidelity Tooling such as Balsamic is great for the actual design phase.  Anyone should be able to use, there is no over intricate detail.  Allows for validation from all people on the team immediately.  To gain common understanding and convergence on the target.
Key Principles to follow

  1. Collaboration - no hero's, we all work together
  2. Focus on Outcomes
  3. Share Understanding
  4. Permission to fail - Fail fast, learn, and it accepted
  5. Talk to you customers, show people early PoC
  6. Avoid analysis paralysis

Concepts:

  • ME-NTERFACE - Personalised & Customised user Experience
  • GAMIFICATION - Rewards to intensify a users experience (includes gratifications, such as badges and celebrations)
  • PLACE-ONAS - User Voice interactions
  • Problem Statements - Help frame what we are try to solve.
  • Story Maps - Useful to get a flow and feature understanding.

References:

https://www.interaction-design.org/literature/article/a-simple-introduction-to-lean-ux

Notes: Miro is great for drawing, sharing, it's useful for big enterprises.  It's pretty expensive unless you use it heavily and their is a free edition.  Good integration with Jira and Confluence.  A lot of templates and can share your own templates easily, good for brainstorming, similar to Lucidchart with more collaboration.