Showing posts with label CI. Show all posts
Showing posts with label CI. Show all posts

Friday 17 February 2023

Postman Monitor for Continuous Monitoring and Alerting in MS Teams

Overview: Pretty much every tester and developer loves postman. And that is because it makes our lives easier and it just plain awesome.  Postman is bringing out tons of new features and I was playing around today looking how I could do continuous monitoring with my postman collections.

Thoughts & Playing:

I have a postman collection that runs 8 requests and does 14 asserts.  The first request gets a new OAuth token using AAD login.  Then I do a series of requests and I do an assert to ensure I am getting a 200 response code and that the response time is less than 3 seconds on each call.  I can run the collection locally.  Level 100 API verification looks good.

In the past, I have taken this collection and run it as a shortcut on my desktop using Powershell with the Postman CLI to display me the results.  Makes my life easier.

I then added Elgato stream deck so I can run the monitor with a single button push (more me playing than real value).  I'd say I'm at level 200 in continuous monitoring capability.

Next, I setup a monitor on the collection, and this allows me to login and view the dashboard and trace, great stuff, and I get an email if anything goes wrong as an alert.  So now I'm getting serious about monitoring and alerting on my API's.  Level 300 is approaching.




Postman monitoring has integration for MS Teams, and Slack.  It also can send logs to Data Dog and New Relic but now Application Insights (recon this will come soon).  I setup a channel in teams to have a webhook, and I can send in the results using Postman but it's way easier to use the integration on the Monitor to push the result of each run or automatically after 3 failures.

Summary:  This Postman monitoring allows me to send detailed API requests at different intervals so I'm thinking for production: 
  • 5 min for health and basic check (look for performance and service slowdown or failure; add alerts but don't over alert so use teams except if service breaks then Teams groups),
  • Hourly, check key functionality/API's including CRUD operations and clean up (ensure the service is operating for most key endpoints), and
  • Daily, in the early hours run a full regression API set of tests, and clean up afterwards (Support/help desk need to review each day).
Don't over alert, let me say that again don't over alert.  Alerting is like water, you definitely a little but floods are not great.  So with Teams & Slack, it's easy to push results and issues into a channel so key people are aware, and it gives a much better experience than email alerting.

I like the idea of using Postman as it's infrastructure is separate as I generally use the Azure/MS stack including Application insights. 

What Next:  I'd like to figure out how to push results into my logs for reporting off a single source.  I could embed the postman monitoring into iFrames but I'd probably use an Azure logic apps Azure function to listen for the Postman POST, then I can format adaptive cards for Teams, and outlook, easily integrate Twilio for SMS or maybe What's app.  From the logic app i can use a Application Insights SDK to add Tracing.  

Combining with Correlation Id's and App Insights, I can see issues, have them summarised, get the right level of alerting, trace specific issues quickly.  Ideally we capture issues before customers report them. and if a customer reports and issue it can be 100% traced, remediated and fixed for all customers quickly.  Changes to API's and compatibility is also a nice benefit of this approach.

  


Sunday 23 October 2022

Continuous Validation with Azure

I listened to Jen Perrin from Microsoft today with Scott Hasselmann on testing for Power Apps from the Ignite 2022 conference.  Chaos Studio is a tool to faking issues such as network faults, and coming backup after say 10 minutes, and it helps determine the behavior that happens.  

I have been thinking and doing continuous monitoring for awhile in SaaS products, thus can be as simple as checking you API's and SPA's are working and checking every few hours and notifying key people as early as possible.  More advanced options are running postman and/or selenium tests that run thru various tests such as login in add and cleaning up some data.  Continuous Validation is fantastic as you can perform scenarios from a specific target.

The tooling and Playwright (which i played with a few years ago) on Azure looks amazing and well suited for the Power Platform.  This is a massive area to improve software, and reliability. 

Sunday 3 July 2022

Personal Access Tokens (PAT)

Personal Access Token (PAT) are used a lot for automation in DevOps, once again need to lookup the informal on PATs.  I use Postman and I use PATs to automate connecting to Azure resources.

https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows

Store the PAT as it won't be retrievable again.





Sunday 9 January 2022

Azure DevOps Series - Azure Pipelines

Azure Pipelines are good at deploying solutions by setting up the infrastructure (I prefer to use PaaS and get out of the Infrastructure world, using ARM templates) and deploying code with the appropriate DTAP environment configuration.  Azure Test Plans are used to verify builds. 

Overview: DevOps allows for shorter duration periods to deploy code into production.  Some industries still require a high amount controlled environment code changes, think medical systems.  In the PaaS world with DevOps pipelines automation of code and verification is drastically reduced.

It's key to figure out your DTAP deployment strategy.  I outline a mixture of an old school deployment strategy with a PaaS DevOps model that is fairly risk adverse below:

This PaaS approach requires three types of pipelines:

  1. PaaS Infrastructure - One-offs and updates
  2. Build/App Deployments
  3. Database data or Schema (Bounded Context) updates 
Build agents
The instructions from Azure pipelines requires agents to run the instructions.  The easiest is to use MS host pipelines/agents.  If you need more power or software, use self hosted agents, these can be on VM's or hosted in docker containers.  It's a good idea to ensure the build agents are running the latest version as it doesn't change often. Self host if you need to run software that is not part of the MS hosted agent set.  V3 of the agent is excellent, try use this first until you outgrow it, or have specific timing concerns.

Azure DevOps Series Posts:

  1. Azure DevOps Series - Overview 
  2. Azure DevOps Series - Azure Boards 
  3. Missing
  4. Azure DevOps Series - Azure Pipelines (This Post)
  5. Azure DevOps Series - DevSecOps



Wednesday 12 February 2020

Power Apps - DTAP Azure diagram

Overview:  A sample architecture for DTAP in a highly controlled environment.  There are a lot of variations and the ability to use Power Apps publishing.


There are a few ways to manage environment deployments.
  1. Simple: Create in unmanaged solutions and Export and Deploy Managed solutions manually using the Power Platform Ux.
  2. Enterprise: Build Azure DevOps pipelines to add unpacked code to Github and deploy solutions using Azure pipelines.
  3. 3rd party ALM tooling, some is specific to Dynamics.
  4. Nov 2022 - Microsoft announce Power Platform Pipelines 

Custom Connectors:  As you step thru the DTAP environments using ALM to deploy you need to point the custom connector to the appropriate APIM (API):
Environment Variables are extremely useful for ALM


Saturday 25 April 2015

DevOps Tooling

DevOps Tooling Notes

DevOps Tooling is broken down into the following areas, note the tools often overlap in function.  The list is not exhaustive but these are the more common tools I have come across.
  1. Version Control: TFS, Git, SVN, ...
  2. Bug Tracking: ServiceNow, Jira, ZenDesk
  3. Continuous Testing: Selenium, Jasmin or Mocha or Unit.js (JavaScript testing), NUnit, Web Tests (Visual Studio), SpecFlow
  4. Continuous Integration (CI)TeamCity, Jenkins, Azure DevOps (bigger) 
  5. Configuration Management and Deployment:  Puppet, Chef, ANSIBLE, SALT  (all installed on Linux, obviously work on Windows environments)
  6. Containers: Docker, Kubernetes, Microsoft Containers. I think the Azure AKS is pretty much containers for Azure now.
  7. Other:  PowerShell, VMWare, HyperV
RESTful API Tooling
  1. Swagger - awesome.  Swagger is a set of tools that help document, build and test your API  (Your API conforms to the OpenAPI specification or Swagger specification).  Great way to get a contract for users of the API early on.  Updated 2019/11/25Link to Swagger post
  2. Swagger UI, Swagger Integrator,...
  3. Apiary - UI to create an API and publish with mocks.  I prefer Swagger or on simple projects APIM.
  4. API Management (APIM) - flexible Azure service for bring together multiple API securely.  Same as MuleSoft.  Can import OpenAPI's v2 or v3 to create a hosted API.  Can mock and built in test tool.
  5. RAML is an alternative to Swagger and Apiary (never used)
  6. Blueprint - API documentation tool.  Pretty simple and nice results.
  7. Postman - send http requests to the API.  Postman is a REST client useful to check your API.  This is my main tool for testing, exploring REST based API's.  
  8. SoapUI - if working with SOAP/XML.
  9. Slate - API documentation - I always use OAS/OpenAPI/Swagger.
  10. Fiddler - I'm old school and still love Fiddler and it's capabilities.  Fiddler is a great HTTP debugger.  
  11. BURP - an HTTP debugger to review traffic.  I've used BURP for security testing and it is great for API debugging.  
  12. Charles is another HTTP debugger (never used).
  13. cURL - Cmd line to test API's using HTTP, separate exe to run on Windows, Windows 10 has cURL built in.
  14. Visual Studio
  15. Wireshark - Over the years I have needed packet sniffing to fix issues and always go to Wireshark, I used the tool in the 90's but it had a different name.  Extremely useful for issues relating to firewalls, especially when an environment reacts differently to another working DTAP environment.
  16. Tcpdump is another packet sniffer
Testing:
http://www.incyclesoftware.com/2014/02/executing-selenium-ui-tests-release-management/

More Info:
http://blog.sharepointsite.co.uk/2014/02/devops-and-sharepoint.html
http://www.networkworld.com/article/2172097/virtualization/puppet-vs--chef-vs--ansible-vs--salt.html
http://blog.sharepointsite.co.uk/2013/11/iac-presentation-for-sharepoint.html


Wednesday 30 October 2013

Issues Generating wsp's using MSBuild in TFS2012

Problem: I am using a Build Definition within TFS2012 to try create a WSP package.  I am building a solution that contains multiple projects.  1 project needs to generate the wsp from it.  I have made minor customisations such as:
My Custom Build Definition
My environment is TFS 2012, VS2012 & SharEPoint 2013.

 Tip:  To clone Build Definitions you need to add the VSIX "Team Foundation Server PowerShell Tools 2012".

When I run the build I get the following error:

C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\SharePointTools\Microsoft.VisualStudio.SharePoint.targets (387): The "ValidatePackage" task failed unexpectedly.System.InvalidOperationException: The project service does not contain the specified project: 17939c4e-xxec-xxxx-xxxx-e18209d495db. at Microsoft.VisualStudio.SharePoint.Tasks.ValidatePackage.OnCheckParameters() at Microsoft.VisualStudio.SharePoint.Tasks.BuildTask.Execute() at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() at Microsoft.Build.BackEnd.TaskBuilder.d__20.MoveNext()
Exception Message: MSBuild error 1 has ended this build. You can find more specific information about the cause of this error in above messages. (type BuildProcessTerminateException)Exception Stack Trace: at System.Activities.Statements.Throw.Execute(CodeActivityContext context) at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager) at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocal


Resolution:
Install Microsoft Office Developer Tools for Visual Studio 2012 - RTM on the build controller (In my case I have TFS and I have a seperate Build Controller on my CI box so I install MS Developer Tools for VS 2012 - RTM on the CI box.

More Info:
http://social.msdn.microsoft.com/Forums/vstudio/en-US/dc6e39e6-331d-42b2-ac16-b530ccb74d46/tfs-build-and-creating-a-sharepoint-2013-app-file



Tip: Another Point Worth Noting is the build definition can have problems if files names/locations are tool long created by the build controller.
   Tip: When setting up Visual Studio, shorten the length/location where the project/workspace is held locally as shopwn below:
 
 

Generating WSP's in CI for SP2013

There are some comprehensive blog posts on setting up Build automation.  This post is a short summary of creating SharePoint wsp files from a Visual Studio 2012 solution within TFS2012.

Prep Steps:
  1. TFS2012 containing source files.
  2. Need a build controller and a build agent, this can be on the TFS server or a separate server, in my case it is separated out.
  3. On the Server holding the build controller, you need the SP binaries.
  4. Install "Microsoft Office Developer Tools for Visual Studio 2012 - RTM"
  5. Create a new template or clone the default template and adjust it as shown below (You will need the VS extension (VSIX) "Team Foundation Server PowerShell Tools 2012" to clone an existing Build Template):
 
Add the Packaging switch under the advanced Tab in the Process tab

More Info:
http://msdn.microsoft.com/en-us/library/ff622991.aspx