Continuous Validation with Azure

I listened to Jen Perrin from Microsoft today with Scott Hasselmann on testing for Power Apps from the Ignite 2022 conference.  Chaos Studio is a tool to faking issues such as network faults, and coming backup after say 10 minutes, and it helps determine the behavior that happens.  

I have been thinking and doing continuous monitoring for awhile in SaaS products, thus can be as simple as checking you API's and SPA's are working and checking every few hours and notifying key people as early as possible.  More advanced options are running postman and/or selenium tests that run thru various tests such as login in add and cleaning up some data.  Continuous Validation is fantastic as you can perform scenarios from a specific target.

The tooling and Playwright (which i played with a few years ago) on Azure looks amazing and well suited for the Power Platform.  This is a massive area to improve software, and reliability. 

Azure DevOps Series - Azure Pipelines

Azure Pipelines are good at deploying solutions by setting up the infrastructure (I prefer to use PaaS and get out of the Infrastructure world, using ARM templates) and deploying code with the appropriate DTAP environment configuration.  Azure Test Plans are used to verify builds. 

Overview: DevOps allows for shorter duration periods to deploy code into production.  Some industries still require a high amount controlled environment code changes, think medical systems.  In the PaaS world with DevOps pipelines automation of code and verification is drastically reduced.

It's key to figure out your DTAP deployment strategy.  I outline a mixture of an old school deployment strategy with a PaaS DevOps model that is fairly risk adverse below:

This PaaS approach requires three types of pipelines:

1. PaaS Infrastructure - One-offs and updates
2. Build/App Deployments
3. Database data or Schema (Bounded Context) updates 

Build agents

The instructions from Azure pipelines requires agents to run the instructions.  The easiest is to use MS host pipelines/agents.  If you need more power or software, use self hosted agents, these can be on VM's or hosted in docker containers.  It's a good idea to ensure the build agents are running the latest version as it doesn't change often. Self host if you need to run software that is not part of the MS hosted agent set.  V3 of the agent is excellent, try use this first until you outgrow it, or have specific timing concerns.

Azure DevOps Series Posts:

1. Azure DevOps Series - Overview 
2. Azure DevOps Series - Azure Boards 
3. Missing
4. Azure DevOps Series - Azure Pipelines (This Post)
5. Azure DevOps Series - DevSecOps

Observability in Azure SaaS Solutions

Problem:  Software has many places where errors and tracing is logged to.   Support get an incident, they need to investigate figure out how widespread the issue is and then try patch together various logs to figure out the problem.

Thoughts:

Observability is not a new concept, we need to be able to: 1) view and connect logs & 2) tracing and view metrics & notifications.

Implementing Observability must cover:

CI/CD allows devOps teams to find issues early using Unit testing.   Automated testing on UI.  API automation testing is also great.

Azure offers continious monitoring by performing various API calls to ensure your servie is running and any failures are picked up hopefully before any customers are aware.  You can also be notified of performance slow down, check performance speed between releases.  Which is great for identifying bottlenecks and with the Azure PaaS world, it is easy to increase the processing causing the bottleneck.  

Performance metrics built into the CI/CD and developers work allows us to identify issues early and costs miles less to correct early. 

Security and LINTing in CI/CD also allows us to pickup issues early and correct at a way lower cost.

Instrument you hardware and software, well on Azure you can use App Insights and you have a fantastic instrumenting platform that captures events.  A big reason to use Azure Services for as many of the function pieces in your solution.

Work In Progress ...

TBC Azure App Insights detail, ParentOperationId, Linking operations with a ServiceBus or work process call.

DevOps Tooling

DevOps Tooling Notes

DevOps Tooling is broken down into the following areas, note the tools often overlap in function.  The list is not exhaustive but these are the more common tools I have come across.

1. Version Control: TFS, Git, SVN, ...
2. Bug Tracking: ServiceNow, Jira, ZenDesk
3. Continuous Testing: Selenium, Jasmin or Mocha or Unit.js (JavaScript testing), NUnit, Web Tests (Visual Studio), SpecFlow
4. Continuous Integration (CI): TeamCity, Jenkins, Azure DevOps (bigger) 
5. Configuration Management and Deployment:  Puppet, Chef, ANSIBLE, SALT  (all installed on Linux, obviously work on Windows environments)
6. Containers: Docker, Kubernetes, Microsoft Containers. I think the Azure AKS is pretty much containers for Azure now.
7. Other:  PowerShell, VMWare, HyperV

RESTful API Tooling

1. Swagger - awesome.  Swagger is a set of tools that help document, build and test your API  (Your API conforms to the OpenAPI specification or Swagger specification).  Great way to get a contract for users of the API early on.  Updated 2019/11/25:  Link to Swagger post
2. Swagger UI, Swagger Integrator,...
3. Apiary - UI to create an API and publish with mocks.  I prefer Swagger or on simple projects APIM.
4. API Management (APIM) - flexible Azure service for bring together multiple API securely.  Same as MuleSoft.  Can import OpenAPI's v2 or v3 to create a hosted API.  Can mock and built in test tool.
5. RAML is an alternative to Swagger and Apiary (never used)
6. Blueprint - API documentation tool.  Pretty simple and nice results.
7. Postman - send http requests to the API.  Postman is a REST client useful to check your API.  This is my main tool for testing, exploring REST based API's.  
8. SoapUI - if working with SOAP/XML.
9. Slate - API documentation - I always use OAS/OpenAPI/Swagger.
10. Fiddler - I'm old school and still love Fiddler and it's capabilities.  Fiddler is a great HTTP debugger.  
11. BURP - an HTTP debugger to review traffic.  I've used BURP for security testing and it is great for API debugging.  
12. Charles is another HTTP debugger (never used).
13. cURL - Cmd line to test API's using HTTP, separate exe to run on Windows, Windows 10 has cURL built in.
14. Visual Studio
15. Wireshark - Over the years I have needed packet sniffing to fix issues and always go to Wireshark, I used the tool in the 90's but it had a different name.  Extremely useful for issues relating to firewalls, especially when an environment reacts differently to another working DTAP environment.
16. Tcpdump is another packet sniffer

Testing:
http://www.incyclesoftware.com/2014/02/executing-selenium-ui-tests-release-management/

More Info:
http://blog.sharepointsite.co.uk/2014/02/devops-and-sharepoint.html
http://www.networkworld.com/article/2172097/virtualization/puppet-vs--chef-vs--ansible-vs--salt.html
http://blog.sharepointsite.co.uk/2013/11/iac-presentation-for-sharepoint.html