Showing posts with label Azure DevOps. Show all posts
Showing posts with label Azure DevOps. Show all posts

Friday 9 June 2023

App Insights for Power Platform - Part 1 - Series Overview

Overview: Microsoft have great capabilities for logging and monitoring.  In this series of posts I will be examining the various parts of logging that may be useful in building solutions that are well monitored, provide alerting, easy tracing, and identifies issues or potential issues as soon as possible.

I am looking at App Insights for Power Platform monitoring.  So this includes: 

  • Power Apps (Canvas, and model apps),
  • Power Automate,
  • APIM, 
  • Azure Functions, 
  • Azure Service Bus, and
  • App Insights.

I shall be setting up a demo environment and these are the logical components being covered.


All the components making up the solution shall log into Log Analytics (left-hand side of the diagram).

For Continuous Integration, my clients will be Postman monitor (it's awesome and so easy to use all those postman collections), DevOps is great and I'll use it to run smoke tests after new releases.  I also use flows, to report on flows (sounds nuts but i love it).  These are at the bottom of the diagram. 

Lastly on the right of the diagram, I look at extracting logs for reporting (Power BI), and Monitoring using Azure DevOps (p.s. think about Grafana instead of DevOps Dashboards, it so nice).

Couple of extras are: Availability Logging, alerting, automating Canvas app testing, Playwright.  

From the diagram, you can see the data is now held in Log analytics and it can be queried via Log Analytics or App Insights using Kusto.  Note: the syntax is slightly different.

Series

App Insights for Power Platform - Part 1 - Series Overview (this post)

App Insights for Power Platform - Part 2 - App Insights and Azure Log Analytics 

App Insights for Power Platform - Part 3 - Canvas App Logging (Instrumentation key)

App Insights for Power Platform - Part 4 - Model App Logging

App Insights for Power Platform - Part 5 - Logging for APIM (this post)

App Insights for Power Platform - Part 6 - Power Automate Logging

App Insights for Power Platform - Part 7 - Monitoring Azure Dashboards 

App Insights for Power Platform - Part 8 - Verify logging is going to the correct Log analytics

App Insights for Power Platform - Part 9 - Power automate licencing

App Insights for Power Platform - Part 10 - Custom Connector enable logging

App Insights for Power Platform - Part 11 - Custom Connector Behaviour from Canvas Apps Concern

Tip: The Power Platform Admin Centre has a good overview of the Power Platform, but to make logging and monitoring better push data into Azure Log analytics and monitor and alert centrally.

Also seeView and download Dataverse analytics - Power Platform | Microsoft Learn

Monday 11 July 2022

What is technical debt? and how to handle it

Overview: Technical Debt generally refers to a buildup of deficiencies that makes changing code or optimizing systems difficult.  The key is to identify what in you organisation/program/project makes up technical debt.  

Technical Debt generally refers to poor or missing NFRs such as Performance, Security, Maintainability, Reliability, Scalability,  Testability, or Resiliency.  But it also can go further into future architecture, so if this part of our system is popular can we easily adjust and keep releasing features.  So as you can see, technical debt can be very wide and it's far better to focus a subset otherwise PO and PM's tend to scope everything under technical debt and wit gets nasty telling them about "additional technical debt".

I find the easier way to go about defining what is technical debt to avoid long discussions to to list out what cannot be considered technical debt.  This would be my minimum starting point:

  1. Bugs (Functional defects);
  2. Technical Skill Debt;
  3. Process Defects (Lack of process or poor process, such as Configuration Management);
  4. Feature Debt (Wrong or delayed features or missing functionality (recent favorite example is "how can a system not have customer off boarding it's obviously technical debt", this is feature debt, make sure stakeholders know or it falls into the old IT/Dev are weak and missed things description.); and
  5. UI/UX Defects (Inconsistent or poor or changing user experience).

Another items is spaghetti code that falls under the NFR of code maintainability, with old systems you have to be pragmatic, if the product brings in $100k per year it's not a good idea to spend $120k a year making the code more readable but not improving the technology as a general rule.  On old systems, I try to keep code maintainability out of the technical debt,.  You should put it to another more detailed section, just don't lump everything especially when it is huge changes all under technical debt.  Dev teams loose focus and it causes problems don the line.  All too often, over exercised bundling debt pushed into technical debt results in "even more interest to pay later".


Sunday 6 February 2022

Azure DevOps Series - Integrating Security into Azure DevOps

Overview: Integrate automate security testing into your CI/CD  Azure Pipelines, this area is of expertise is sometimes refered to as DevSecOps.  Azure DevOps provides build and automation servers.  In the OSAWP 2021, number 8 is Software and Data integrity failures.  This covers securing CI/CD pipelines.

CI/CD Pipeline hardening - Code is written and committed to source code repository, Linting (SonarQube), build , test, and deploy.  Can also include infrastructure and networking setup.  YAML & JSON are common for building pipelines.  All these steps need to be hardened.  

Ensure only authorized intended actors can run/use the pipeline or part within the pipeline.  Eg. ensure only developers can check in code, they must have permissions.  

  • Harden but you have to be pragmatic so developer can do there work but also don't over allow access.
  • Ensure logging is running.  
  • Keep plugins and reference frameworks up to date to avoid weaknesses being exploited.  Ensure OS and containers are up to date.
  • Use dedicate build/service accounts.  
  • Using Azure DevOps (ADO) does a lot of hardening automatically.  
  • Don't expose sensitive information in you logs like pswds as if the logs are hacked you have a problem.
  • Ensure artifacts are correctly locked down.  To get artifacts the pipeline only needs read access.
  • Verify SaaS service you use are secure.  Integrate external security SaaS software.
  • For security there are native tools for security, plugins or external service as mentioned above.  Mend/WhiteSource Bolt is a tool used for scanning packages for vulnerabilities.  AzureDevOps has Mend Bolt as a add-in.  There is a free service but it is fairly limited.  Can also run these scans from Developer level and not just in the pipeline.

Azure DevOps Series Posts:

  1. Azure DevOps Series - Overview 
  2. Azure DevOps Series - Azure Boards 
  3. Missing
  4. Azure DevOps Series - Azure Pipelines 
  5. Azure DevOps Series - DevSecOps (This Post)
Note: AWS Pipelines refered to as CodePipelines is the same as Azure DevOps in AWS world.  

Sunday 9 January 2022

Azure DevOps Series - Azure Pipelines

Azure Pipelines are good at deploying solutions by setting up the infrastructure (I prefer to use PaaS and get out of the Infrastructure world, using ARM templates) and deploying code with the appropriate DTAP environment configuration.  Azure Test Plans are used to verify builds. 

Overview: DevOps allows for shorter duration periods to deploy code into production.  Some industries still require a high amount controlled environment code changes, think medical systems.  In the PaaS world with DevOps pipelines automation of code and verification is drastically reduced.

It's key to figure out your DTAP deployment strategy.  I outline a mixture of an old school deployment strategy with a PaaS DevOps model that is fairly risk adverse below:

This PaaS approach requires three types of pipelines:

  1. PaaS Infrastructure - One-offs and updates
  2. Build/App Deployments
  3. Database data or Schema (Bounded Context) updates 
Build agents
The instructions from Azure pipelines requires agents to run the instructions.  The easiest is to use MS host pipelines/agents.  If you need more power or software, use self hosted agents, these can be on VM's or hosted in docker containers.  It's a good idea to ensure the build agents are running the latest version as it doesn't change often. Self host if you need to run software that is not part of the MS hosted agent set.  V3 of the agent is excellent, try use this first until you outgrow it, or have specific timing concerns.

Azure DevOps Series Posts:

  1. Azure DevOps Series - Overview 
  2. Azure DevOps Series - Azure Boards 
  3. Missing
  4. Azure DevOps Series - Azure Pipelines (This Post)
  5. Azure DevOps Series - DevSecOps



Wednesday 5 January 2022

Azure DevOps Series - Azure Boards

Azure Boards are used for managing software projects.  Regardless of the approach Azure Boards support multiple frameworks and integrate well with tooling such as Visual Studio as you IDE.  Microsoft strap line for Azure Boards is "Plan, track and discuss work across your teams".

You need to login to Azure DevOps and Create an Organisation, in the image below I have two organisation.  

Next I need to create a new project, the image below shows the options.  For Source Control most teams and people choose Git, I came from TFS which is now Team Foundation Version Control and before that Visual Source Safe with various other version control systems but Git is now far more widely used.  I have chosen to make my project "Private".  Lastly, I need to select the Work Item Process.  I prefer to use the "Agile" process templates for Scrum and not the Scrum Templates as I prefer to use User Stories over Product Backlog Items (PBIs) in my Backlog.  I follow the hierarchy Epic, Feature, User Story, Task.  And the "Agile" Template gives the correct naming and board in DevOps.  


Go to "Project Setting" > "Project Configuration" and update the iterations to match your sprint dates as shown below.

Add a new Epic (Top level Item)


There is a little configuration to get your queries and boards setup correctly.

Azure DevOps Series Posts:

  1. Azure DevOps Series - Overview
  2. Azure DevOps Series - Azure Boards (This Post)
  3. Missing
  4. Azure DevOps Series - Azure Pipelines
  5. Azure DevOps Series - DevSecOps

Thursday 30 December 2021

Azure DevOps Series - Overview

Azure DevOps is a SaaS platform that provides tools for deploying software using DevOps techniques generally within Agile software delivery projects.  DevOps is useful for gathering requirements, building the solution, performing daily “integration builds”, and having deliverable end-of-sprint demos.  The key to building software today versus several years ago is that we should automate as much as possible.  Azure DevOps provides excellent tooling to automate and implement automation results in better quality, reduced timelines.  Building software is easy as long as you have great people and precise requirements.  Agile practices and DevOps processes and tooling can help get you to the desired state.

  1. Azure Boards are great for planning and getting the requirements cleanly broken down.
  2. Visual Studio coupled with Azure boards items with Azure Repos (source control options are GitHub & TFVS) is ideal for the development using most languages such as C#, NodesJS, Angular, React, TypeScript.
  3. Azure Pipelines are good at deploying solutions by setting up the infrastructure (I prefer to use PaaS and get out of the Infrastructure world, using ARM templates) and deploying code with the appropriate DTAP environment configuration.  Azure Test Plans are used to verify builds. 
  4. Monitor and Alert – Azure Monitor/App Insights is ideal for monitoring the infrastructure and operating code to detect issues early.
  5. Azure Artifacts help create and share Nuget code packages.

More Info:

Wednesday 28 July 2021

Azure DevOps User Stories Tips

Quick Point on User Stories and Acceptance Criteria.

1.      User Stories description must follow the format:

As the <role> , I want to <feature> so that <benefit>

Note: Always follow the exact same format and bold up the standard/fixed parts for user stories.  Pls keep consistency across your teams user stories.  Under the user story in the description, feel free to add more description, annotated images (very useful) and links to Figma, Axure, UI mocks or Miro.  User Stories should also follow the INVEST (Independent, Negotiable, Valuable, Estimatable, Small, and Testable) breakdown.

2.      Acceptance Criteria (Use Gherkin Language) under the user story (ensure it goes into the User Story section and not comments or the description)

Scenario:
  Given
  When
  Then

Example

Scenario: Employee requests leave
  Given an employee has sufficient leave available in the year
  When the employee schedules leave (holiday)
  Then the employee is informed his request is valid and his manager is informed of the request.

Note: Always follow the exact same format and bold up the standard/fixed parts for user stories.  Pls keep consistency across your acceptance criteria.  I bold and use the four parts as shown above in the example.  You can use "and" to extend the story, just try keep them within the idea of INVEST.

3.      Other:  Order is Tasks belong to User Stories, User Stories belong to a Feature.  Features belong to Epics  These items must be related within Azure DevOps.   Epics > Features > User Stories (Acceptance criteria) > Tasks.

Scrum - Part 1
Scrum - Part 2 
Scrum - Part 3 - Scaling Scrum
Scrum - Part 4 - Kanban and Scrum
Scrum - Part 5 - Certification PSM 1

Azure DevOps - Introduction