Testing Power Apps using Microsoft Playwright with C#

Overview: I am using a C# Console app to run my power Apps to verify that it is working correctly.

Approach:  I'm not using NodeJs but I'll use C# and add it into my devOps pipelines.  I assume you have VS Code installed (you need the C# extensions, I have kept my notes at the bottom of the post).  I'm using .NET 6 core SDK and the Microsoft.Playwright package.  I'm not using any of the .NET testing frameworks to keep it simple.

Tips: Playwright by default use headless browser testing, I want the test to use my local JWT that I have stored against my Chrome browser.  If I tell Playwright to use show the browser i.e., headless=false, it logs me in using my cached credentials.  You should parameterise the login and store the username/pswd somewhere secure like a key vault. 

Code Example: 

Replace the C# code in the console as I have done above, and hit F5 (Run>Debug).  The test logs me into the Canvas app and returns the Title of the page. 

Note:  I like the Canvas app testing feature but this is better as I can easily add testing to my CI/CD pipelines, tooling and recording is better and there are more ways I can use the Playwright approach.  

Notes for installing Playwright on VSCode to use C#

1. Install VS Code and add the following two extensions

2. Open the VS Code terminal, create a C# Console app using .NET6, add the reference to the Playwright package, install the headless browsers against you profile (only needed once per machine). 

3. Write and run the tests.  Tip: It's easier to record the code and then add it to the C# console.

Note: It's worthwhile using NUnit as it's much easier to test than doing it the way I've done it here.

Playwright using the NUnit framework:

Using the OpenAI PowerFx language helper leading into building a Tax chatbot using Power Apps (No model training)

I had some time today, and I've been hearing a lot about OpenAI/GPT-3, so I thought I'd have a look around and wow it's good.  There are so many uses and it's just easy to use.

For instance, Power Apps uses PowerFx which is a simple language that Microsoft had added the "Ideas" are to that helps you build PowerFx code.  Honestly I found it extremely irritating when it was released recently but after reading more on Open AI I to take a closer look and it definitely is a great idea.

Thoughts:  I don't memories code so I feel this is a great tool for citizen and professional developers.  I think it will get a lot of attention and improve over time and become a key part of PowerFX and app development.

More Info:

https://learn.microsoft.com/en-us/power-apps/maker/canvas-apps/power-apps-ideas

OpenAI - Took less than an hour to setup the demo

The Power Platform has a Premium Connector to use the OpenAI API's.  Robin Rosengrun has a demo and built and published a connector. 

I spent some time playing around with the OpenAI GPT-3 API using the "davinci-002" engine in a Power App connector.    

First search data result is correct

This is completely wrong, although HMRC have amended the go live date many times.  I was hoping for "6 April 2024"

The third question result is good - but could be better.

ARM Templates vs Bicep vs Terraform vs Pulumi

ARM templates, they are instructions that run against Azure to create resources.  

ARM templates will update settings such as tags when added, deleted or updated using json.

If a dependent resource is called e.g. storage, the ARM template creates the storage and if variables are supplied it shall use them.  Removing or changing the dependency won't update the dependent resource, it will create a new storage resource and point to the new storage. 

Note: Templates don't remove dependent resources if they are pulled from the ARM template. It will not upgrade to the latest instructions. I.e. it does not take the infrastructure to desired state config (DSC).

There is a DSC modules for ARM templates, but i believe this is more for VM's.

Bicep is a domain specific language which is DSC, it's json and way easier to build up your infrastructure.

Bicep will run ARM templates "under the hood" and provide the full desired state.  Bicep is a Microsoft Azure only concept and can be used for private cloud infrastructure.  

Terraform is an open source DSL to build IaC, and it works on AWS, Azure and GCP (and Oracle ...).

Terraform is created by HashiCorp, HashiCorp have an amazing tooling set that uses the Terraform language to build, and changes Infrastructure.  Uses the HCL Language.

AWS CloudFormation is the native way to build IaC on AWS.  Supports json and YAML.

Pulumi is an exciting competitor to Terraform.  Can us multiple languages and easy to write new modules.  Code is compiled and uses appropriate cloud API's to create IaC.   Compare Terraform to Pulumi.  Uses TypeScript/JS, Python or C#.  Pulumi is Open Source, and there is a free tier SaaS offering on Pulumi Cloud.

Common Dev setup could be: TypeScript with Pulumi created using VS code npm for packages.  

Pulumi needs a program yaml file, tells it the language to use and other setup.  2 types of providers: Terraform bridge providers and native provides.  There are over 150 connectors most are Terraform .  Support multiple languages e.g. TypeScript, C#, Python Java, YAML.  Easy to integrate into CI/CD/DevOps.  Pulumi is basically a command line tool that can be called from DevOps including Pulumi Cloud.

Pulumi (Any) vs Terraform (HCL lang) vs AWS: CDK (TS, C#, Py, Go, Java) or CloudFormations (YAML)  vs Azure: ARM/Bicep (DSL).

Nutanix - new to me but looks like VMWare using IaC for multi-cloud basically Terraform but manage and monitor.  Very likely to be wrong here.

Summary:

Use what the organisation uses, Bicep is great if you only are using Azure, and for smaller work/specific DTAP IAC using ARM templates is a well trodden route.  If the organisation wants multi-cloud (do not take multi-cloud lightly) then Terraform and even Hashicorp using Terraform is the way to go.  Terraform and Pulumi are for multi-cloud/run at a higher level than CloudFormation, ARM/Bicep.

Note: Resist multi-cloud at all costs, I've heard ridiculous reasoning like we don't want to be tied to only AWS or Azure.  That's cool but costs go up exponentially with multi-cloud, it's not even the infra that costs, it's the management overhead and testing/verification.  Which parts can be changed.  A business would need a really good reason to go to multiple vendors.

Power Apps issue with OptionSets Syntax irregularity

Problems: OptionSets in Power Apps (Canvas apps can be a little nasty to work with, today I took a couple of hours  trying to use an OptionSet to filter results.  This should have taken 5 minutes but there is a weird behavior when using switch or if statements to dynamically filter using options sets.

Hypothesis:  using if/switch logic when using an OptionSet in Dataverse to filter results in the code being change/interpreted incorrectly at run time. 

Resolution:  Use a variable to hold the dynamic OptionSet choice as shown above.  I hope this post can help anyone having issues relating to Option sets for filtering and updating.

In the example below I show how to Insert a new Row using a Patch where the row contains an OptionSet.  It's ugly but, it works...

On the Save...

Set(orgType, Blank());

If(

    <ddlOrgType.Value> = "Personal", Set(orgType, 'Org Type'.'Sole Trader'),

    <ddlOrgType.Value> = "Incorporated", Set(orgType, 'Org Type'.'Limited Company'),

    <ddlOrgType.Value> = "Partnership", Set(orgType, 'Org Type'.'Partnership')

    );

Insert the new row for Subject

Deploying a Canvas App direct to mobile devices

Overview:  Canvas apps have a feature called "Wrap" that can take a canvas app via a solution and allow it to be access naively on iOS and Droid.  This allow users to open the canvas app without going thru the Power Apps client on their mobile device.  The apps can be shipped using the app stores/marketplace or Microsoft Intune for enterprise deployment.

Enable the feature in your dev/un-managed environment using the "Power Platform Admin centre":

There is a great wizard that takes care of a lot of the underlying work such as registering the app in Azure Portal AAD.

Use the Wrap wizard (you'll need a lot of access rights on the Power Platform and Azure Portal)

Pick your package types, this post outlines the full process for Android (apk). Below shows the iOs file needed for the Apple store but you can distribute using InTune.

 

ALM Power Platform - Problem Solving

 Use the logical name of the embedded canvas app

Problem:  I have embedded canvas apps inside a Model app that I deploy via a solution.  When I move from Development to Test, the users reported that they don't have access "This app isn't opening correctly" "It looks like you don't have access to this app."  And when I dive deeper, I see the Model app in Test is point to the Development Canvas App; not the test canvas app.

Initial Hypothesis: My thinking is that the Model App in Test is pointing to the Canvas App in Dev.  In dev, I point to the test canvas app and it works.  The workaround is to Open the managed solution in the Test environment and change the AppId  to the App Id of the canvas App in TEST.   It works but it's an awful work around.  This tells me that my App Name is not working, as the Static App name is used before the AppId (looks for the app name in the current environment) and then the AppId when the app Name doesn't resolve, I believe the app name is not working.  

I changed the AppId (to a non-existing guid) and the app wouldn't load in dev.  The app name should be the logical name and not the display name.    

"The App Name property specifies the name of the canvas app to embed. The model-driven form looks for the canvas app with the specified name in the current environment. If a canvas app with that name can't be found in the environment, the canvas app control will use the App ID to load the canvas app. Enter the App Name for an existing canvas app."  Add an embedded canvas app on a model-driven form - Power Apps | Microsoft Learn

Resolution:  Open the table > Forms > Form containing the canvas app

Or, another alternative is to use Environment variables for all the canvas AppID's and change them per DTAP environment on the deployment.

More Info:

Share an embedded canvas app - Power Apps | Microsoft Learn

Guidelines and troubleshooting when working with embedded canvas apps - Power Apps | Microsoft Learn - scroll to "Known issues and limitations with embedded canvas apps", "The App Name property value is missing or is incorrectly defined"

Embed a canvas app on a model-driven form - Power Apps | Microsoft Learn

Reporting from Dataverse HLD options

Overview:  Recently I was looking at reporting from the Dataverse/CDS and I drew up these options that give the business various options but the overhead grew as the solution improves.  Client is on the MS stack so I have looked at the reporting options, There are a ton of variations but this is a good start for my options.

Reporting options from the Dataverse

Synapse, can be replace by any Data warehouse or Data lake solution and as Dataverse is not massive could also just use regular SQL Server for reporting.

Power BI can be replaced with other tools such as Tablau but for embedding and the MS stack, Power BI makes the most sense.

Continuous Validation with Azure

I listened to Jen Perrin from Microsoft today with Scott Hasselmann on testing for Power Apps from the Ignite 2022 conference.  Chaos Studio is a tool to faking issues such as network faults, and coming backup after say 10 minutes, and it helps determine the behavior that happens.  

I have been thinking and doing continuous monitoring for awhile in SaaS products, thus can be as simple as checking you API's and SPA's are working and checking every few hours and notifying key people as early as possible.  More advanced options are running postman and/or selenium tests that run thru various tests such as login in add and cleaning up some data.  Continuous Validation is fantastic as you can perform scenarios from a specific target.

The tooling and Playwright (which i played with a few years ago) on Azure looks amazing and well suited for the Power Platform.  This is a massive area to improve software, and reliability. 

Power Apps Portal and Power Automate licencing thoughts

Update: 13 August 2023 - Power Apps Licencing has change considerably since this post, for example here is updated information on Power Automate licencing.

My Technical Working Notes for Microsoft Technology: App Insights for Power Platform - Part 9 - Power Automate Licencing (pbeck.co.uk)  A major change in naming and cost has been release August 2023.

Overview:  The Total Cost of Ownership (TCO) is key to any project.  I recently was asked about a project that wanted to use Power Apps for external user access, the data is in the dataverse/CDS) and their are tons of workflows.  The cost of buying a per app licence for external users is a non-started (without volume discount $5 or £3.80 per user per month  and the only viable options left are:

Design by Cost:

Write the front end using free to distribute front end application (i.e. ReactJs, Angular, C#, Flutter, Blazor,...) and all the flows using the per-flow licencing model.  Per-flow licencing is crazy expensive as it is per flow and my users have lots of flows when the login once a year.  So per flow is also not an option with a custom front-end.  At Ignite 2022, Microsft announce a new Power Automate Embedded SDK with pay as you go pricing, this may be a good option, but's its untested.

The $5/month/app plan doesn't really work as users login over the last few months of the year and all users would need to be licenced all year around so $60 is way too high.  The Power Apps Pay-as-you-go Plan allows costs $10/£7.45 per user that logs in during a month.  So in my case an average user logs in 2 times in a year with the monthly consideration, so i'm still looking at $20 per year.

User/Per App/month: £3.80

User/Month (multiple apps): £15.10

Active User/app/month Pas-as-you-go  £7.45

Power Apps Portals/Power Pages/Dynamics Portals is my last hope.  The licencing is based on per login per 24 hrs.  So a user that logs in three times in a 24 hr period is considered 1 licence.  If the user logs in on 4 different dates at any point in the year, the client is changed for 4 logins.  Need to buy the licences in advance each month.  Sold in blocks, the smallest being 100 logins in a month.  At the lowest levels, cost is $2 per 24 hour login, but this reduces to a much lower cost relatively quickly.   With our expected numbers, we'd get onto Tier 3, and the cost per login is $0.70 per login.  If an average user logs in 4 times a year on separate dates, our cost is basically $2.80 per user per year.

Note there is also a Page Portals Capacity licence required.  We are expecting an average of 175k page views, which cost a further $200.

13 Oct 2022: Licencing for Power Pages change yesterday, 1) monthly anonymous active users 2) monthly authenticate active users.  Prepaid is cheaper than pay as you go, assuming you get you numbers right.  https://www.microsoft.com/en-us/licensing/news/power_pages_general_availability#:~:text=Today we are announcing licensing and pricing details,to purchase through prepaid subscription and pay-as-you-go plans.

Summary: In this case, Power Pages is the best option.  

Model Apps - Controlling the UI with JavaScript

Problem:  I have a form of view in a Model Driven App (Power Apps) that needs to show 1 of 2 tabs depending on a control in the form having a value.

js> If the "Status"="Active show the "Agent" tab else show the "General" tab

Hypothesis: inject JS into the specific form that will check the status display control, and if active, get the Agent tab via the DOM, and hide it.

Solution:

1. Ensure you have two tabs on your form, my form is called "Information", see below

2. Create a JavaScript file to manipulate the DOM, my example is below:

Create a .js file to upload

Reference Blog for most common JS CRM work

3. Upload the file and publish.

4. Bind the JS function on the form or page being adjusted.

5. Result when the Status is active to hide the "Agents" tab.

Generating a Canvas App from a Custom Connector (Open API)

Overview: The Power Platform CLI in August was updated and 1 of the new features is the ability to use the CLI command line to generate a Canvas app using an Open API as the data source.

The Power CLI supports the GET and POST endpoints.  Which is normally +85% of the endpoints anyway.  

In this post I walk thru the steps to add books using an Open API and retrieve all the books as well as the new book added.  

Solution: 

1. I used the free postman API endpoints (2 gets and a post) > Exported the collection

2. In Power Apps > using a Solution > Add a new Custom Connector (tip: I used Automate)> Upload from a Postman Collection > Pass in the Postman collection generated in the last step.   Save the Custom Connection as shown below.  Lastly test the Custom Connector.

3. I used Visual Studio code with the Power Platform Extension

Using the Power Platform CLI, in the terminal run ps> pac to verify the Power Platform extension is available.

// Get the environment you want to work on

ps> pac admin list

// Get the Custom Connector Name or Id you are going to base the Canvas app on

ps> pac connector list --environment a9adbbba-c45d-eac1

// Generate the msapp package

ps> pac canvas create --msapp "C:\Radimaging\BookDemo.msapp" --connector-display-name "BookDemoCN" --environment "a9adbbba-c45d"

4. Import the export Power App.  Select your environment, and create a new Canvas driven Power App.  "Open" has recently changed in the UI.

The app will get created, I had to add the Custom Connector to the App and you are ready to customise

Beware: Custom Connectors and Connection References in Managed Solutions.  It's a good idea to check they work in the deployment pipelines as the rework can be a pain.  

Connection references in solutions - Power Apps | Microsoft Learn

Summary: This is a great way to build an application quickly using the pac create.  It is a bit rough, and needs amending after import on the 3 endpoints I've used but a great start to a new feature. 

More Info:

https://learn.microsoft.com/en-us/power-platform/developer/cli/reference/canvas

Power Platform Added Features

 August/September 2022 has been a busy month for the Power Platform

Here are two experimental features I think are massive: Named formulas & ParseJSON

New upcoming feature

ParseJSON

Coauthoring improvement will also be important for enterprise development.

October 2022: 

Power Pages are now in GA (Genral Availability)

Nearly 700 Connectors

A few new features/functionality I like: 

Generating a Word Document from a Row change in Dataverse without SharePoint

Problem:  Dataverse/Dynamics has a great way to generate Word or Excel documents in Model apps using the UI.  I want to generate a generate letter from a CDS entity/Table when the status changes.  And I'm not allowed to use OneDrive/SharePoint (governance restriction).

High-Level Steps:

1. Create Dataverse Table - ensure "Notes" & "Business process flows" are enabled

2. Generate the Word Template & add it to the "Template View"

Generate Word Template with dynamic column vals

Add Template

3. Create a Process to generate the output/docx from the Word template i.e. generate when status of the row changes

Activate the Workflow

4. Use Power Automate to call the Dynamics Process

Custom HTTP connector in Power automate to POST x-www-form-urlencoded data

Problem:  As part of my OAuth process I need to swap and authorisation code for an Access Token using an API, the issue is that I need to get the token into PowerApps.  

Initial Hypothesis:  Initially I created an Azure Function that does the API post as it was the easiest.  But I reverted and thought it must be easier for me to have fewer working parts and as my solution used Power Automate and I've previously used Power Automate with the HTTP custom connector I'd use the same approach.  It proved fairly tricky to get the HTTP connector to work but with some playing around and clarification of my thinking it became rather straight forward.

Firstly, I need to ensure the API is working, so I use Postman:

Postman POST request using form-urlencoded.

When I get the 200 response, I wanted to see the underlying HTTP traffic so I opened Fiddler:

Solution: From the raw HTTP trace, I realised I needed to post the body in Power Automate in the correct format.  Key value pairs for parameters and separate with an Ampersand.  Also, Url's need to be escaped/encodes, in C# there are functions to encode and decode.  I needed to do the encoding using Find() and Replace() methods (sic).  From Power Automate, I return the Access token, refresh token and other info back to Power Apps.

More Info:

Uri.UnescapeDataString(String) Method (System) | Microsoft Docs

Understanding HTML Form Encoding: URL Encoded and Multipart Forms - DEV Community

Enterprise Canvas App Development - Overview

Overview:   It is a good idea to have an agreed set of standards in an organisation and within the development teams.  Microsoft has Power App Canvas Coding standards and guidelines, use this as a base.

At a high level, the design and implementation needs to consider:

• Security,
• Usability,
• Performance,
• Maintainability & support, and
• Accessibility.

Governance:

• What connectors can the business use, 
• Groups of users (who can do what),
• Licencing,
• Enforcement of coding standards
• Monitoring & support

Also see: App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

WIP

<Add coding standards for Power Apps and Power Automate>

Dynamics 365 CE Intro

Overview: Dynamics 365 CE (Customer Engagement) is a subset of the Power Platform and Dataverse.  It provides commonly used entities such as Accounts and Contacts.  There are five apps that can add specific CRM functionality.  It is Microsoft's CRM solution.

There are 4 core subject (model) apps in Dynamics 365 CE

1. Sales (MB-210) - client management including leads and turning:
1.  leads (Qualify step) into an 
2.  opportunity (Develop step), goes to the 
3.  Proposal step consists of generating quote, revising quote, marking acceptance, and invoice, 
4. Order created/lost (Closed step).
2. Marketing (MB-220) - Lead generation & qualification, turn prospects in customers.  Build campaigns to market your business.  Email, social media, content creation to convert leads.  Ensure you target the right audience using the correct channels and breaking leads into segments.  I have only used this lightly and it is powerful.  Events are useful for planning such as speakers and managing leads/event analytics. Referred to as Dynamics 365 Marketing.  Customer Voice integrates with Marketing, e.g., embed survey sent our via email.
3. Customer Service (MB-230)/Customer support - Dynamics 365 Customer Service - Low tough Customer Experience(Cx) and it also provides for high tough as needs demand, provides omnichannel options, chatbot, email, SMS, service requests, wiki's, phone in.  Case management, raise record, route to where can be solved, and get to answer.
4. Field Service (MB-240) - Dynamics 365 Field Service.  Send people out to customer sites, schedule, ensure technician has work order and materials to complete the job.  

Common Data Model - Agreed between various companies including Microsoft so there is a set schema that we use as a base for our businesses.  Example entities:

• Account - Company or individual (I think of this as a legal entity).  Accounts have a primary contact, can have multiple contacts per account.  Accounts can have parent accounts.
• Contact - is a person, can belong to multiple account.

LinkedIn Sales Navigator - Sync LinkedIn, so a contact changes, can be pulled into dynamics.  Messages from Linked in are tract in dynamics.

Dynamics 365 Sales Insights - understand your relationships, and evaluate activities.

Dynamics 365 Customer Service / Customer Service Hub - 

    1) Raise a case via most suitable formats (align with SLA), 

    2) route case (align with SLA) e.g.,  

    3) Enter research such as calling customer, spoke to colleague against, knowledge base against the case.  Lastly, can record fix e.g., called customer and walked them thru steps, and problem resolved

    4) Close Case (includes reason)

Customer Service Notes

• Allows to integrate with other tools, ....
• Knowledge management - write articles in knowledge base (KB), submit for review, all staff can search to find fixes/info.  Can also import from SharePoint, EpiServer, Wordpress and others into KB.
• Omnichannel - communication with customer on multiple channels e.g., Teams, SMS, virtual agent, social media. Hook into external channels also.
• Analytics - measure customer service activity.  Show kpi's to show emerging topics, agent performance, how quickly cases are being resolved.
• Also has remote device management.

Dynamics 365 Field Service - Deliver on site service & manage workforce example is plumbing business or mining machinery maintenance.  Importantly deals with scheduling & work orders.  Deploy, fix issues/perform work from work order, the audit completion.

Note: MB-910 exam covers Dynamics 365 CE solutions.  It is a simple exam that does not require annual renewal and gives the hold CRM Dynamics Fundamentals Status.  The MB-210 specialises in the Dynamics 365 Sales and provides a Associate status which needs to be renewed annually with Microsoft. 

There are specific solutions focusing on ERP (Think SAP (Enterprise), Oracle/JD Edwards (enterprise), Baan now Infor, Sage(small-medium), Dynamics 365 (small to medium), Workday (seems to be everywhere lately) ) focus on areas like Finance, Operations, Supply chain management, Machine maintenance.  MB-920 covers the ERP fundamentals certification.

Dealing with JSON strings in Power Apps

Overview: I get a JSON string representing an object into Power Apps.  I want to extract the values so i can use them in PowerApps.  Since this post, the ParseJson function was release in Power Apps, use that instead.

Tip: The is a great website for looking at JSON.  Otherwise I tend to use VS code.

Possible Solutions:  

1. In Power Automate, split up the JSON (using ParseJSON) and return multiple Power Apps variables - only useful is you need a small specific subset of the JSON data.
2. In Power Apps, use string manipulations to extract data as shown here:

Thoughts: I feel this is a big challenge for NoCode/LowCode solutions like Power Platform, it can be overcome with a little jiggery-pokery but I feel the PowerFx language should have an ability to cast JSON into an object and in turn allow object arrays in collections.  A lot of people have been asking for this on UserVoice.

Update: Sept 2022

The ParseJSON function is now in Power Canvas Apps, a huge improvement.  There is still an issue, the ParseJSON in not strongly typed, so it's still hard to deal with complex objects containing arrays in Canvas Apps.  Ideally I'd like to be able to add a schema to run against the untyped object to have a strongly type object to work with.

What is technical debt? and how to handle it

Overview: Technical Debt generally refers to a buildup of deficiencies that makes changing code or optimizing systems difficult.  The key is to identify what in you organisation/program/project makes up technical debt.  

Technical Debt generally refers to poor or missing NFRs such as Performance, Security, Maintainability, Reliability, Scalability,  Testability, or Resiliency.  But it also can go further into future architecture, so if this part of our system is popular can we easily adjust and keep releasing features.  So as you can see, technical debt can be very wide and it's far better to focus a subset otherwise PO and PM's tend to scope everything under technical debt and wit gets nasty telling them about "additional technical debt".

I find the easier way to go about defining what is technical debt to avoid long discussions to to list out what cannot be considered technical debt.  This would be my minimum starting point:

1. Bugs (Functional defects);
2. Technical Skill Debt;
3. Process Defects (Lack of process or poor process, such as Configuration Management);
4. Feature Debt (Wrong or delayed features or missing functionality (recent favorite example is "how can a system not have customer off boarding it's obviously technical debt", this is feature debt, make sure stakeholders know or it falls into the old IT/Dev are weak and missed things description.); and
5. UI/UX Defects (Inconsistent or poor or changing user experience).

Another items is spaghetti code that falls under the NFR of code maintainability, with old systems you have to be pragmatic, if the product brings in $100k per year it's not a good idea to spend $120k a year making the code more readable but not improving the technology as a general rule.  On old systems, I try to keep code maintainability out of the technical debt,.  You should put it to another more detailed section, just don't lump everything especially when it is huge changes all under technical debt.  Dev teams loose focus and it causes problems don the line.  All too often, over exercised bundling debt pushed into technical debt results in "even more interest to pay later".

Azure Application Gateway Basics

Azure Application Gateway is a http traffic load balancer that allows me to manage my web traffic.  So based on urls, paths, the traffic can be pushed on accordingly.  It also provides for sticky sessions.  In my head I see Azure Application Gateway as a layer 7 HTTP Load Balancer and a Web Application Firewall (WAF).  There are 3 core concepts: Front-ends, rules and backends that are used to route traffic from the front-end to the back-ends.

Azure Load Balancer works at level 4 where Azure application gateway is for web traffic at layer 7 of the OSI model.

Azure Front Door is basically for world wide Application gateway functionality.

Checklist on the Application Gateway: 

• You should enable the WAF unless you already use another WAF like Imperva enterprise wide, do this under the "Web application firewall" option, setup OWASP rules.  
• Config session affinity, and HTTPS termination using the "HTTP settings".  
• "Listeners"  are what listens for incoming http requests.
• "Rules" bind the listeners to the backend pools.
• "Backend pools" are used to point traffic to the end points.  Also remember to setup "Health Probes", this allows the App pool to verify the backends are working.
•  Ensure you set the diagnostics logging to send logs to "Log Analytics" so you can examine the logs.

Features: 

1. SSL/TLS termination - Terminate SSL using certificate
2. Autoscaling - increase the size or instance count based on traffic requirements
3. Can be setup to be zone redundant
4. Static IP address that doesn't change
5. WAF and DDoS (pretty new 2022, applied using Azure DDoS Sentinel service) capabilities - The WAF allows you to apply OWASP rules and add additional custom rules.  Bot protection is also built in, rules need to be applied to get it to work.  
6. URL-based routing
7. Multiple domains/site hosting
8. Redirection
9. Sticky sessions/session affinity
10. HTTP/2 support
11. Custom error pages (so useful)
12. Rewrite headers and URL

References:

How an application gateway works | Microsoft Docs

WAF options

Personal Access Tokens (PAT)

Personal Access Token (PAT) are used a lot for automation in DevOps, once again need to lookup the informal on PATs.  I use Postman and I use PATs to automate connecting to Azure resources.

https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows

Store the PAT as it won't be retrievable again.

Microsoft Sentinel - Introduction

Overview: Microsoft Sentinel is Microsoft's cloud hosted SIEM solution.  It can take data from Azure, on-prem., and other cloud infrastructure such as AWS.  Sentinel gathers data from various sources, and evaluates to generate alerts.  

SIEM allows software products and services to bring security information/events into a management tool to provide real-time analysis of security from all applications and networks across the enterprise.  SIEM stands for security, information, and event management and is commonly used by the COS (Security Operations Teams of a business), the SOC is often outsourced or partially outsourced to Service companies e.g. Avanade are good at this.  Other SIEM products are: SolarWinds security and monitoring (great product), McAfee ESM, Rapid7, Splunk, and Datadog.  There on installable and SaaS solutions but I prefer SaaS solution like Microsoft Sentinel.  AWS and Google also provide SIEM services.

Microsoft Sentinel (previously called Azure Sentinel) is cloud-native and collects SIEM data from systems, applications and networking to provide an overview of the organizations security alerts, and allows identification for remediation.  It accepts several common formats for capturing events and offers a REST - API.  Sentinel also has a ton of native connectors.

"Microsoft Sentinel is a scalable, cloud-native, security information and event management (SIEM) and security orchestration, automation, and response (SOAR) solution." Microsoft Docs

Simple Password protection got GTM - Starting Page

Overview:  I am use the AuthPro service to check a user has a code to access the website.  This is useful for Go to Market (GTM) where you only want a subset of customers to work on your software.  Think of a beta where a user can sign up, you give people with an NDA a code to be able to go thru the sign-up process.

Simple solution:  

To get to the sign-up/register page, use an external service e.g. AuthPro to manage a passcode.  AuthPro injects JS and allows the admin to configure the behavior.  It was pretty simple to setup so only specified users could go thru our sign up process.

Demo:  This page allows the user to login, if, successful the user can proceed and use the website.

Type in the username "password" and the password "password" for access to the hidden page.  

Another Option: Imperva has a lot of features including "Login protection" that can easily achieve the same outcome.

NIST/RMF - Cyber risk control

Overview: NIST National Institute of Standards and Technology - Provides Risk Management Framework (RMF) - Is a framework to reduces security risk to systems and data.

Goals of NIST RMF:

• Consistent and cost effective set of security controls
• Repeatable assessment approach
• Technology neutral
• Implement an efficient risk-based security and privacy program.

Notes:

• Each of these six steps have Special Publications (SP) that are applicable to the area.
• The core document to for RMF is NIST SP 800-37 Revision 2. 
• Used to identify security/pricacy risks at both the operation and system level