Showing posts with label DevOps. Show all posts
Showing posts with label DevOps. Show all posts

Tuesday 20 June 2023

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards

Series

App Insights for Power Platform - Part 1 - Series Overview 

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics 

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM 

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards (this post)

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power Automate Licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern

Overview: Azure Dashboards are excellent but, if you want beautiful dashboards use the Azure Grafana service or Power BI dashboards.

It's always difficult to identify KPI's and graphs to allow support and stakeholders to quickly digest monitoring information.  An option is to have an overview for the PM, PO, Business owners,,, and a separate set of dashboards for support.

Identify What is important?  End-to-end testing is always nice especially if running CI to detect abnormalities.

For instance, this Azure Dashboard, fires of a Canvas App recorded Test (done using test studio) and shows the speed (performance) of the run, i then warn the user if the performance is too slow.  I also grab the oldest successful run from 7 days ago to check if performance is considerably different.  

Power automate has it's own logging, but integrating log entries when a error occurs via Log analytics, allows me to see if any of my workflows have a problem.  This is discussed in part 6 of the logging series on Flows.

Azure services are often used when building solutions using the Power Platform.  The most common are Functions, App Services, APIM, Service Bus, maybe sendgrid.  So we need to know that they are working, and help the user see performance or issues.  Here are a couple of examples.





Series

App Insights for Power Platform - Part 1 - Series Overview 

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics 

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM 

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards (this post)

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power Automate Licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern


Sunday 6 February 2022

Azure DevOps Series - Integrating Security into Azure DevOps

Overview: Integrate automate security testing into your CI/CD  Azure Pipelines, this area is of expertise is sometimes refered to as DevSecOps.  Azure DevOps provides build and automation servers.  In the OSAWP 2021, number 8 is Software and Data integrity failures.  This covers securing CI/CD pipelines.

CI/CD Pipeline hardening - Code is written and committed to source code repository, Linting (SonarQube), build , test, and deploy.  Can also include infrastructure and networking setup.  YAML & JSON are common for building pipelines.  All these steps need to be hardened.  

Ensure only authorized intended actors can run/use the pipeline or part within the pipeline.  Eg. ensure only developers can check in code, they must have permissions.  

  • Harden but you have to be pragmatic so developer can do there work but also don't over allow access.
  • Ensure logging is running.  
  • Keep plugins and reference frameworks up to date to avoid weaknesses being exploited.  Ensure OS and containers are up to date.
  • Use dedicate build/service accounts.  
  • Using Azure DevOps (ADO) does a lot of hardening automatically.  
  • Don't expose sensitive information in you logs like pswds as if the logs are hacked you have a problem.
  • Ensure artifacts are correctly locked down.  To get artifacts the pipeline only needs read access.
  • Verify SaaS service you use are secure.  Integrate external security SaaS software.
  • For security there are native tools for security, plugins or external service as mentioned above.  Mend/WhiteSource Bolt is a tool used for scanning packages for vulnerabilities.  AzureDevOps has Mend Bolt as a add-in.  There is a free service but it is fairly limited.  Can also run these scans from Developer level and not just in the pipeline.

Azure DevOps Series Posts:

  1. Azure DevOps Series - Overview 
  2. Azure DevOps Series - Azure Boards 
  3. Missing
  4. Azure DevOps Series - Azure Pipelines 
  5. Azure DevOps Series - DevSecOps (This Post)
Note: AWS Pipelines refered to as CodePipelines is the same as Azure DevOps in AWS world.  

Sunday 9 January 2022

Azure DevOps Series - Azure Pipelines

Azure Pipelines are good at deploying solutions by setting up the infrastructure (I prefer to use PaaS and get out of the Infrastructure world, using ARM templates) and deploying code with the appropriate DTAP environment configuration.  Azure Test Plans are used to verify builds. 

Overview: DevOps allows for shorter duration periods to deploy code into production.  Some industries still require a high amount controlled environment code changes, think medical systems.  In the PaaS world with DevOps pipelines automation of code and verification is drastically reduced.

It's key to figure out your DTAP deployment strategy.  I outline a mixture of an old school deployment strategy with a PaaS DevOps model that is fairly risk adverse below:

This PaaS approach requires three types of pipelines:

  1. PaaS Infrastructure - One-offs and updates
  2. Build/App Deployments
  3. Database data or Schema (Bounded Context) updates 
Build agents
The instructions from Azure pipelines requires agents to run the instructions.  The easiest is to use MS host pipelines/agents.  If you need more power or software, use self hosted agents, these can be on VM's or hosted in docker containers.  It's a good idea to ensure the build agents are running the latest version as it doesn't change often. Self host if you need to run software that is not part of the MS hosted agent set.  V3 of the agent is excellent, try use this first until you outgrow it, or have specific timing concerns.

Azure DevOps Series Posts:

  1. Azure DevOps Series - Overview 
  2. Azure DevOps Series - Azure Boards 
  3. Missing
  4. Azure DevOps Series - Azure Pipelines (This Post)
  5. Azure DevOps Series - DevSecOps



Wednesday 28 July 2021

Azure DevOps User Stories Tips

Quick Point on User Stories and Acceptance Criteria.

1.      User Stories description must follow the format:

As the <role> , I want to <feature> so that <benefit>

Note: Always follow the exact same format and bold up the standard/fixed parts for user stories.  Pls keep consistency across your teams user stories.  Under the user story in the description, feel free to add more description, annotated images (very useful) and links to Figma, Axure, UI mocks or Miro.  User Stories should also follow the INVEST (Independent, Negotiable, Valuable, Estimatable, Small, and Testable) breakdown.

2.      Acceptance Criteria (Use Gherkin Language) under the user story (ensure it goes into the User Story section and not comments or the description)

Scenario:
  Given
  When
  Then

Example

Scenario: Employee requests leave
  Given an employee has sufficient leave available in the year
  When the employee schedules leave (holiday)
  Then the employee is informed his request is valid and his manager is informed of the request.

Note: Always follow the exact same format and bold up the standard/fixed parts for user stories.  Pls keep consistency across your acceptance criteria.  I bold and use the four parts as shown above in the example.  You can use "and" to extend the story, just try keep them within the idea of INVEST.

3.      Other:  Order is Tasks belong to User Stories, User Stories belong to a Feature.  Features belong to Epics  These items must be related within Azure DevOps.   Epics > Features > User Stories (Acceptance criteria) > Tasks.

Scrum - Part 1
Scrum - Part 2 
Scrum - Part 3 - Scaling Scrum
Scrum - Part 4 - Kanban and Scrum
Scrum - Part 5 - Certification PSM 1

Azure DevOps - Introduction

Tuesday 1 June 2021

SaaS Azure Testing Thoughts

 Tooling:

  1. API Automation - Postman, Newman
  2. UI Automation - Selenium
  3. IDE - Visual Studio 2019
  4. Test Organization - Azure DevOps Test Plan
  5. CI/CD - Azure DevOps

Code reviews:

Code review is used as a verification technique to ensure that each unit is coded as per standards and expected business logic and inline with coding standards and best practices.  Automate code review built into Azure Pipelines should include:  

  • WhiteSource Bolt - Scan packages for vulnerabilities.
  • SonarQube - Static Analysis, 
  • Blackduck - Open-Source Scanning (OSS) tool.  Used to look for license risks and unused references.
  • Checkmarx - Static Application Security Testing (SAST) tool benefits include: Detect security vulnerabilities, Improve developer practices, and reports on code ownership.  Static code anaylsis.  VeraCode is a competitor product.
  • BugSuite
Code should pass OWASP (Open Web Application Security Project) shows the most common code vulnerabilities.  OWASP ASVS (Application Security Verification Standard) - framework for controls when building applications to cover functional and NFR's for web applications.

Unit testing:

Unit tests are written to ensure every unit of code is working as expected, and to prevent a defect from going to the next level on all C# code.  Xunit and Moq are the tools to be used for unit testing using the standard Arrange > Act > Assert pattern.

As long as Unit test coverage is high and of a good standard, I don't mind if the tests are written before the code (TDD) or as most developers tend to do the tests after the code is written.

API testing:

All API must use Postman collections and Environments for local testing.  The tests need to cover all API's dealing with authentication, authorisation, checking status codes, body responses, headers, data persistence, and post test clean-up.  Use Newman to integrate postman tests into Azure pipelines:

https://www.npmjs.com/package/newman-reporter-htmlextra

Selenium testing:

Code for UI must be automated where possible.

SonarQube: "automatic code review tool to detect bugs, vulnerabilities, and code smells in your code" SonarQube documentation

Code Smells:  Bloaters, OO abusers, ....

Checkmax detects potential security issues

Disposable email addresses: You often need to test login/account creation and it's useful to have temporary disposable email addresses:

Monday 27 April 2020

Azure DevOps/TFS Basics


Overview:  There is a lot you can do with Azure DevOps to monitor your projects.  A couple of simple charts can be used to motivate (or demotivate) your team.  Start simple and build...









Sunday 19 April 2020

Knowledge Transfer/Support Handover

Problem:  Projects that I tend to work on are complete by Scrum teams filled with specialist and specialist contractors who move on after project completion.  Support is generally handled by dedicate people/teams offshore.

Hypothesis: Having high quality support people working alongside you throughout the project is not very common due to costs.  I believe there are key points to cover to ensure that the operational support is effective.  Too many companies merely focus on checklists and the ops team don't get a fundamental understanding of the system.

Resolution:
1. People/Support: Understand the domain - Hard
2. People/Support: Understand the architecture - Easy
3. People/Support: Understand who is responsible for level 1- level 3 support and what that entails.  Easy if done correctly.
4. People/attitude: Hire patient collaborative, eager people in support (most key point) that want to learn and take ownership. Easy if done correctly.
5. Knowledge base - have a wiki or equivalent.  The same issues always present, so document and have an answer that can help your uses.  I also like to record mp4's for different levels of support.  Record the sessions as it is too easy for level 3 people to say they never got a handover or covered something.  This allows people to look back, easily train additional users.  Easy if done correctly.
6. Ensure you have automated tests, they are a great source of how your system works.  And if a fix has to be released, it also easy to validate that the original logic still works.  Hard but it returns great benefits if used.

Saturday 19 October 2019

Evidence Based Management for Scrum

Evidence Base Management (EBM) is an management framework that uses evidence and metrics.  Execs & Middle mgmt don't get insight and normal forecasts.  EBM framework to help management move the company.  The mgmt can use evidence to drive the business forward.  Inspect and adapt using evidence.  I like to keep everything simple and move towards a more complex mature set of measurements as the easiest metrics to implement provide early value to the project.

"Evidence-Based Management is a framework organizations can use to help them measure, manage, and increase the value they derive from their product delivery."  Scrum.org

Examples Types of metrics you may want to measure:
  1. Velocity
  2. Sprint Burn downs
  3. Versioning - versions of product or bugs requiring multiple fixes
  4. Code Coverage & Code Analysis
  5. Missed bugs/escaped bugs - bugs missed in a release
  6. Issues/Failed Deployments
  7. Team happiness - capture metrics in Sprint Retrospective from the team
  8. Financial measure - cost to delivery
  9. Average User Story completion time per team (identify outliers and average for each team.  Do from started status to completed status).
  10. Average story points per team, factoring in the number of team members over multiple sprints (don't compare teams as their unit size is not consistent across teams; if you do try standardize, the teams will give more points to fake outperforming other teams).
Help management nudge the teams and company in the right directions.  To get these metrics, need to explicitly determine what the goals are, so can figure out KPI and we are measuring this right things.  Ensure the management goals are SMART (Specific: Well defined, clear, and unambiguous) or use OKR framework to define goals to figure out the metrics to collect.  Ensure the metrics are important to the goals of the company or team.   Consider using
"Objectives and key results (OKR) is a goal-setting framework that helps organizations define goals — or objectives — and then track the outcome."  cio.com

Common KPI's:  
  1. Escaped Defects
  2. Defects in the last 14 days
  3. In Progress Features and User Stories per team
  4. Team velocity
  5. Stories completed vs stories committed
  6. Tech debt (know issue, items skipped)
  7. Team burndown charts
A couple of columns and dashboard widgets for summarizing progress in DevOps:
  1. Progress by User Story Total minus remaining rolling up to features.
  2. Progress by Work Items in a feature - get user stories, tasks, spikes that make up the feature and shows % of artefacts completed within the feature.
  3. Epic Progress - each epic show the number of features and number of feature completed e.g. 3 of 11 feature completed.
  4. Number of "In Progress" feature or User stories per scrum team - very useful to check a team is using and updating artefacts proactively.
  5. Priority committed and done using priority or WSJF.
Three goals of KPI's in Scrum:
  1. Measure deliverables
  2. Measure team ability to deliver business value
  3. Measure the scrum team itself

Tuesday 13 February 2018

GIT Intro

Overview: Git is is very popular and it is similar to existing version control systems.  The key is to be able to work on multiple branches that you can go to at any point.

Let's get started..

Start a new Git repository (repo) - all folders created under the main git folder are part of the repository. Any changes are kept track of and all change history is recorded.  The new repo has a default main branch created automatically.  To create a new repo use the syntax:

>git init

There are files you don't want to keep track of in the repo, so use the .gitignore files or folders to skip specific files from being tracked int he repo.

Once you are writing code, you get a Git local staging area.  -A means All files and folders

>git add -A

Now you make changes in your staging environment via your IDE, and need to add it back to the default "Main" branch.

Create a repo, add code, change code and commit to the branch.

This is a fantastic simple illustration to quickly understand Git.

GitHub is Git hosted (centrally hosted instances, bought by Microsoft circa 2018 but still open source, https://github.com), the developer clones the GitHub repository and works using a distributed source control.  You can host yourself but has great integration with Azure DevOps.   You can also replace the central GitHub with Azure DevOps (Git).
  • Git ignore files, tells source control not to include certain file types
  • Tags for a specific point in time.  Like labels in TFS/Azure DevOps
  • Pull request (PR) - dev made change, and wants to push the code into main branch, someone else generally approves and the code is pulled into the main (Working Directory) branch (depends on branching strategy)
  • Developer normally branches of Main branch using a cloned copy on local dev env.  The Developer does changes, then does a PR, the PR if approved gets approved it gets merged into Main (automatically or manually per config), Main branch has the latest code and the developer can delete their cloned branch.
  • GitHub Enterprise allows integration with Microsoft Teams (sounds amazing)
  • GitHub Codespaces - Instead of local dev, it allows dev using a browser.  Competes with Microsoft's Dev Box (spins up dev env that is browser accessible).  Microsoft Dev Box iGB VM's to choose from, the 16GB, has 4 vCPU's.  Only bills when DevBox is running but the storage used is continuous.  If left on/max monthly cost, it would cost about £370, if well managed i.e. turned off on weekend and overnight but used for roughly 8 hrs a day cost would be around £85 for a month.  All dev licences are included.  Pls check with Ms this is my understanding.
DevOps has morphed into DevSecOps (Development, Security, Operations) - same team responsible for all the roles.  Continuously ensure security built in, call shifting left so it is not tact on at the end of the project.  Includes monitoring and auditing.  Git like other source control systems assists in DevSecOps.


Sunday 25 September 2016

SharePoint Support Models

Overview:  Large organisations tend to use a tiered support model also called the escalation model.

The Problem with Tier Support Models:
A user finds an issue and explains it to level 1, the Level 1 support guys figures out it's too difficult or not on his easy path and pushes to level 2,  the whole bug needs to be re-explained generally involving the business user that reported the bug and the level 1 support person.  This goes on for 3  to 5 levels in bug organisations and is eventually passed onto engineers or the vendor.  It takes an astronomical amount of time, and provides a poor impression to the business user.  Coupled with a tracking system that the end user don't know how to use and the support people trying to add as much content as possible so as to cover any responsibility as they have not missed anything.  It's just a disaster.  Anyone doing Level 1 and 2 support is generally not happy and poorly remunerated so the turn over is high, end use satisfaction is not good.  A lot of time and focus is wasted.  To me the fundamental problem I have with Tiered support models is a lack of Total Ownership.  Support people pass the problem and tick off the easy fixes.  I have seen multiple support escalation software products and fundamentally the products don't tend to make much difference, it's the implementation of the support process and the quality and ownership of staff that determine good support models (but its far easier for people to blame software).

On the plus side, Level 1 support people are considerably cheaper and if they have good knowledge basis and training they can return 70% plus of incidents at this 1st stage.  Funneling tougher questions to more specific staff.

DevOps: DevOps relies on close collaborative deployment and support, so the upside is you have the developers and they understand the infrastructure and are best position to fix mistakes cleanly and quickly.  In traditional enterprises, we tend to have hundreds or even thousands of applications and you loose economies of scale by having to keep dedicate higher cost people around to do support and it generally affects sprints as people need to be pulled out to fix bugs.  On the plus side, you get fixes done quickly and correctly.

For me the answer is "It depends...", if you are a tech company with 1 main product, think Facebook, AirBnb then devops is clearly the choice approach.  If you are a company that has legacy applications such as a bank, an application supporting mortgage applications has been used for 10 plus years, tier support is much cheaper and effective.  So the tough question would be what about the same bank that is now developing a new complex application, ideally if governance would allow it then I would start developing the product and use DevOps, allowing better deployment and support in a rapidly changing environment.  Once the project starts maturing then the question is when can I move the product to a tier support model.

Summary: DevOps works well in an Agile environment and support is vastly improved, assuming the product is not changing and the number of incidents is low, this the time to transition to a tier support model.

Saturday 25 April 2015

DevOps Tooling

DevOps Tooling Notes

DevOps Tooling is broken down into the following areas, note the tools often overlap in function.  The list is not exhaustive but these are the more common tools I have come across.
  1. Version Control: TFS, Git, SVN, ...
  2. Bug Tracking: ServiceNow, Jira, ZenDesk
  3. Continuous Testing: Selenium, Jasmin or Mocha or Unit.js (JavaScript testing), NUnit, Web Tests (Visual Studio), SpecFlow
  4. Continuous Integration (CI)TeamCity, Jenkins, Azure DevOps (bigger) 
  5. Configuration Management and Deployment:  Puppet, Chef, ANSIBLE, SALT  (all installed on Linux, obviously work on Windows environments)
  6. Containers: Docker, Kubernetes, Microsoft Containers. I think the Azure AKS is pretty much containers for Azure now.
  7. Other:  PowerShell, VMWare, HyperV
RESTful API Tooling
  1. Swagger - awesome.  Swagger is a set of tools that help document, build and test your API  (Your API conforms to the OpenAPI specification or Swagger specification).  Great way to get a contract for users of the API early on.  Updated 2019/11/25Link to Swagger post
  2. Swagger UI, Swagger Integrator,...
  3. Apiary - UI to create an API and publish with mocks.  I prefer Swagger or on simple projects APIM.
  4. API Management (APIM) - flexible Azure service for bring together multiple API securely.  Same as MuleSoft.  Can import OpenAPI's v2 or v3 to create a hosted API.  Can mock and built in test tool.
  5. RAML is an alternative to Swagger and Apiary (never used)
  6. Blueprint - API documentation tool.  Pretty simple and nice results.
  7. Postman - send http requests to the API.  Postman is a REST client useful to check your API.  This is my main tool for testing, exploring REST based API's.  
  8. SoapUI - if working with SOAP/XML.
  9. Slate - API documentation - I always use OAS/OpenAPI/Swagger.
  10. Fiddler - I'm old school and still love Fiddler and it's capabilities.  Fiddler is a great HTTP debugger.  
  11. BURP - an HTTP debugger to review traffic.  I've used BURP for security testing and it is great for API debugging.  
  12. Charles is another HTTP debugger (never used).
  13. cURL - Cmd line to test API's using HTTP, separate exe to run on Windows, Windows 10 has cURL built in.
  14. Visual Studio
  15. Wireshark - Over the years I have needed packet sniffing to fix issues and always go to Wireshark, I used the tool in the 90's but it had a different name.  Extremely useful for issues relating to firewalls, especially when an environment reacts differently to another working DTAP environment.
  16. Tcpdump is another packet sniffer
Testing:
http://www.incyclesoftware.com/2014/02/executing-selenium-ui-tests-release-management/

More Info:
http://blog.sharepointsite.co.uk/2014/02/devops-and-sharepoint.html
http://www.networkworld.com/article/2172097/virtualization/puppet-vs--chef-vs--ansible-vs--salt.html
http://blog.sharepointsite.co.uk/2013/11/iac-presentation-for-sharepoint.html


Saturday 8 February 2014

DevOps and SharePoint

Components of DevOps
  • IT Automation
  • Agile Development
  • Operation and dev teams working together
  • Service vitalisation
  • Continuous releases
  • Automated testing (Recorded Web UI & Unit testings)
  • Performance
Why DevOps
  • Collaboration btwn Dev & Ops teams; reduce unknowns early on, info easily available, better decision making, improved governance.
  • Increase apps faster; faster time to market
  • Improve App quality; environments available and not tedious to deploy
  • Reduce costs; fewer staff required as tasks are automated
DevOps Maturity Model
There are different levels of maturity and your final goal will depend on your company.  Operations departments are very different in a bank as opposed to Facebook.
The 4 main stages to DevOps Maturity are:
  1. Early - Manual traditional approach to software delivery.  Little or no automation.
  2. Scripted - Moving toward more automation.  Some scripting is used to assist the tracked recorded deployment process thru DTAP environments.
  3. Automation - Reusable common automation approach to releasing applications.  Workflow/orchestration using the automation capabilities used over all the DTAP environments.
  4. Continuous Delivery - end-to-end application delivery process from dev to production environments.  Probably includes Continuous Integration, nightly builds

Presentation at SharePoint Saturday 2013 on of Infrastructure as Code (IaC)
http://blog.sharepointsite.co.uk/2013/11/iac-presentation-for-sharepoint.html


Saturday 9 November 2013

IaC Presentation for SharePoint Saturday UK

Overview: I am presenting at SharePoint Saturday UK (9 November 2013).  This post contains my slide deck and will answer any questions that arise from the session.

SharePoint Saturday UK 2013 Web Site

 Infrastructure as Code for SharePoint 2013 PowerPoint Presentation

Outline of the session:
Creating automated farm builds is key to having a stable SharePoint on-prem. service. This session looks at how and why automating infrastructure, SharePoint 2013 builds and assets is a good idea. The session is heavily focused on Infrastructure as Code (IaC). I look at building a large on premise SP2013 stretched farm on stable repeatable infrastructure that includes SSRS, WCA/OWA and Blob storage.

This is a technical overview for IT pro’s & architects, throwing in best practices for looking at large SharePoint 2013 deployments. Concepts such as devops and Continuous Delivery (CD) are examined. PowerShell is extensively used with several takeaways provided to jumpstart SharePoint 2013 service creation via IaC.