Thursday 29 August 2019

SCRUM for SharePoint - Part 3 - Scaling scrum

Introduction:  Scrum teams work best with relatively small teams (3 to 9 people) made up of high-quality individuals.  Scrum teams are made up of: a Scrum master, a product owner and up to 7 team members.  But in large enterprises or Software companies that deliver a large product, there is a need to use many more people to deliver working quality software.  Scaling Scrum is an option, and there are a couple of frameworks to help.  Agile and Scrum struggle to scale, this post outlines possible pathways to scaling scrum.  

PM's vs PO's: In smaller firms and projects, the Product Owner(PO)/Product Manager (PM) are often the same role.  In larger projects these are both key roles but with a very different focus.  PM are outward looking, know where the market is going, what do our customers want.  The PO's focus on what are the piece we will build internally.  For scaling I generally favor PO's have 1 or maybe 2 teams at the most.  I always want a single source of direction from PM.  So you can have 3-4 PO's for a single PM.  And often PM is a function consisting of several PM's. 


Scrum of Scrums (2-3 Teams)
By adding an additional event daily for key people from each Scrum team called a "Scrum of Scrums".  The meeting requires key stakeholders from each team to attend so that the integration between teams picks up dependencies, and ensure cleaner, easier integration.  Scrum of scrums is the simplest form of Scrum scaling and requires a 15-minute meeting that I follow: What we did, what we are going to do, and blockers from all attendees format.

Scrum of Scrums helps reduce the integration unknowns and ensure various teams can solve dependencies in a timely fashion.  The problem is there are diminishing returns as more teams get added, and this works well if there are 2 or 3 teams working on a single project and these teams self-organise.  In my opinion, more than three teams require a more robust Scrum-based framework like Nexus or SAFe.  Keep "Scrum of Scrums" meeting small with a technical scrum team member from the scrum teams.  I do them exactly like a daily stand-up Scrum meeting with technical items moved to the backlog.

Scaled Agile Framework (SAFe) (3 or more Teams)
My experience with Scrum scaling using SAFe while I worked with a big four consultancy firm and with several clients.  SAFe is the most popular approach for scaling Scrum.  SAFe is valuable, and like all these frameworks, it comes down to being implemented well.  SAFe is not easy to get working well.  Within product business with fixed scrum teams, it can work better.  It is a way for an organisation to be ready for Scrum and coordinate large projects to guide direction.  The product managers to product owners are often at sea with what they are trying to achieve.  SAFe focuses on Agile software delivery, Lean product development and system thinking.   The author here outlines that SAFe is for a way of scaling Scrum.  Program Increments (PI) are how direction is agreed for the next period (often a quarter).  Scrum owners get backlog items and then plan 4-5 sprints to get their portion of features/backlog items done.  SAFe only tends to work in my experience if "Organisation Readiness" (scope of work for the PI is reasonably clear and the priority of features is agreed, and the teams are ready to understand their sprint work items) is done before PI planning.  The diagram below shows the centre as Scrum teams and the other SAFe ceremonies' outer layer.  Beware, PI planning is challenging.





Note: Organisations can jump on the Scrum framework and try to make all teams scrum teams.  This can be great and is generally a good sign; however, beware of managers/management comparing teams spring volatility thru story points.  All the teams tend to increase their story points when compared, so they look like they are improving and beating other teams.  Focus on delivering quality in a trusted, rewarding team.

Note:  Using the same Scrum in every team is not ideal.  I like to let each team decide their rules and find their way.  The culture win from team members is where Scrum's value lies.  I like to leverage other frameworks like Kanban and story points.

Note:  I do like to have repeatable CI/DC tasks to be agreed across Scrum teams, even though the team may take longer, it works better from expertise and consistency in software delivery.  The point here is tune to match you requirements.


SAFe Levels
Source Alan Reed (with thanks)

Program Increment (PI) planning - 2 day event.  Day 1, have draft features and User Stories.  Al stakeholders are aware.  Day 2, team breakouts, confirm understanding and what can be achieved, finalist what is in th ePI, what are the risks and have confidence vote.

Nexus (3 or more Scrum Teams)
Nexus is for scaling scrum teams; I have never used it, but I like the thoughts and these guys know their stuff.  I like the concept of an integration team.  I have used this idea with floating team members with high degrees of specialization in CI/CD.  Having a central integration team ensures all the pieces come together for the deliverable.  Mixed with Scrum of Scrum meetings, the number of teams can be increased and for huge products having a central architecture/integration team is a great idea to reduce dependencies and ensure alignment from the various Scrum teams.

LeSS (Large-Scale Scrum) framework LeSS 

Disciplined Agile Delivery (DAD) extends Scrum

Summary:  Scrum works well for small Agile teams; once going over a single team, use daily Scrum of Scrum meetings to scale to two teams.  If the product is large, a dedicate architecture team that is responsible for build releases, integration of workstreams and overall architecture and using daily Scrum of Scrums to deliver allow for larger Scrum-based projects.  
  1. Small - Use Scrum (1 team)
  2. Medium - Use Scrum-of-Scrums (2-3 teams)
  3. Large - Use SAFe or Nexus (3 or more teams)
SCRUM Blog Series:

Sunday 11 August 2019

Send email using O365 in an Azure Function C#

Create an Azure Function and the basics for send an email are below:

MailMessage msg = new MailMessage();
msg.To.Add(new
 MailAddress("someone@somedomain.com", "SomeOne"));
msg.From
 = new MailAddress("you@radimaging.co.uk", "Paul Beck");
msg.Body
 = "This is a test message using <b>O365</b> Exchange online";
msg.Subject = "Test Mail";
msg.IsBodyHtml
 = true;
SmtpClient
 client = new SmtpClient();
client.UseDefaultCredentials
 = false;
client.Credentials
 = new System.Net.NetworkCredential("your user name", "your password");  // Generate a key so pswd does not change or if account uses MFA
client.Port
 = 587; 
client.Host
 = "smtp.office365.com";
client.DeliveryMethod
 = SmtpDeliveryMethod.Network;
client.EnableSsl
 = true;

Tip:  Azure SendGrid is a great service for sending out emails.  Amazon have SES (Simple Email Service).

Sunday 4 August 2019

Scheduled Tasks Options

Problem: Fire off code on a regular interval.

Hypothesis:  Generally code needs to be triggered and that is either thru an action such as an HTTP trigger or time based e.g. every 30 minutes run a job.  Overf the years I have seen many time based trigger mechanisms such as:

Scheduled Windows Tasks that are set to execute on a time base pattern.  Generally ends up calling a command.  Benefits are that nearly any IT department will have a VM or allow scheduled tasks to run.  

Control-M from BMC is  the next step up, basically allows for tasks to be scheduled and the jobs are monitored and can trigger alerts.  So with a scheduled task, you would need to check manually or write code to do the checking and alerting and storage.  JAMS fills a similar gap as Control-M in some organisations.  There are several competitors that have been around for years so always best to check with the client what there policy is for timer jobs.

Hangfire is for .NET and is more advanced and programmable so a good option for background tasks.

Team City - sledge hammer to crack a nut.  Tasks can be scheduled and fired off.  Useful for complex orchestration tasks with dependencies.   Jenkins and other CI/CD tools are in this group.

Old SharePoint had Timer Jobs - can't see this being of any use.

Azure Functions using a Timer Trigger for my is my first choice unless I am restricted at the client for some reason.

Friday 21 June 2019

O365 and AAD using InTune

Overview:  Our company has gone away from traditional on-prem. networking and we use Azure.  We use AAD, Azure Domain Services, Intune and O365 with all laptops and PC's using Windows 10 Pro.  It is so easy and removes so much administration.

Intune: If your users have O365 or E365 licences Intune is included, with E3 accounts you can add on for £7.50 per month.  Intune allows me to deploy a setup that historically would have used GPO to manage the individual machines referred to as "Configuration".  I can verify all my users are compliant with my policies such as Windows 10, ensure they are patched to a certain level.  Defender works brilliantly thru Intune.  I've pulled off our old anti-virus/malware on end-user devices because with Intune it's better with Defender.  I ensure all our PC's and laptops have BitLocker.  Checking all devices my users are using is done thru Intune using "Compliance".

  • I can wipe any PC or device remotely.
  • With the user logins, I can see activity and it provides a great end to end management solution.
  • I haven't used team viewer as we still us LogMeIn for remote support but I'd personally lean to TeamViewer as it's fully integrated with Intune.
  • BYOD devices are also controllable using Intune.
Summary:  Intune is easy to use and roll out and provides good control of end user devices.

Example Policy for Windows 10 devices:

Health Service Setting
  1. Require BitLocker
Required
  1. Require Secure Boot to be enabled on the device
Disabled
Device Properties Setting
  1. Min OS Version
1809
System Security  Setting
  1. Require a password to unlock mobile devices
Required
  1. Simple passwords
Block
  1. Password Type
AlphaNumeric
  1. Min password length
8
  1. Max time of inactivity before password is required
10
  1. Password expiration (days)
45
  1. Number of previous passwords to prevent reuse
12
  1. Require password when device returns from idle state
Required
  1. Encryption of Data storage on device
Required
  1. Device Security - Firewall
Required
  1. Device Security – Antivirus
Required
  1. Device Security – Antispyware
Required
  1. Defender – Windows Defender Antimalware
Required
  1. Defender – Min version
1.295.933.0
  1. Defender – Antimalware intelligent up-to-date
Required
  1. Defender – Real-time protection
Required
Windows defender ATP  Setting
  1. Require the device to be at or under the machine risk score
Medium

Update: 2022-June-20

"BigFix automates discovery, management, and remediation of all endpoints whether on-premises, mobile, virtual, or in the cloud" - product by HCL.  

Competitor is Microsoft Endpoint Manager (MEM).  MEM is useful for patching and monitoring Windows 10/11 devices.  Can setup policy to ensure different notification for the end user to install and cn force if the end user does not install the patch.


Wednesday 22 May 2019

Azure B2C Authentication for SaaS applications

Overview:  This blog post looks at setting up multiple public federation services on an Azure based SaaS web application.  It is worth understanding that a Microsoft account (old passport accounts) is like a google account and not the same as an Organisational Account (Azure AD IT company (e.g. paul@mycompany.com) setup account.

AAD B2C Overview
1. Client using a browser, goes to a website URL
2. User receives a 302 HTTP response and is redirected to Azure's B2C (AAD and Azure B2C are separate services)
3. User is prompted to login (assuming they don't already have a valid token)
4. After the user is authenticated, they get a Valid token.
5. Using a valid token, the users sessions is established on the web site.

The diagram does not show the flow pass B2C, this shall use "Passive-claims base authentication" to select the users Identity Provider e.g. Google account.  Once the user has a google account authenticated, they are redirected back to the B2C service where the Google token is used to issue a B2C user token for the user.  And step 4 continues.

Azure Active Directory (AAD) also sometimes referred to as AAD B2C
Has two types of users, namely:
1. Members - these are internal company users from an organization e.g., paul.beck@mycompany.com
2. Guests - are external users from outside our company e.g., harry@jpmorgan.com
Tip: Native member users passwords are stored in your Azure B2C service.  Whereas, native guest users e.g. harry@jpmorgan.com, actually logs into JP Morgans AAD and our AAD tenant sees him as a guest and issues a SAML token from us based on JPMorgan's assertion that the user is valid on their AAD tenant.
Note:  A guest user can be made a member and a member user can be changed to be a guest user.  There is no good reason that I have come across for switching guests to members (maybe 2 companies merging) but it is possible if you need to do it.
AAD supports the  following protocols: WS-Federation, SAMLP, & OIDC & OAuth2.  WS-Fed and SAMLP are used but go for OIDC as the default.

AAD B2C Instance:
The diagram above show AAD B2C not B2C.  B2B is provisiong on you Azure tenant and is tied to your O365 instance.  B2C is a separate Azure service used fro managing customers identities.  So if I have a website and some mobile applications, offered an API to clients, I would use B2C not B2B for managing security.  You can connect multiple AAD B2C to your single B2C instance.  B2C basically allows you to connect to other Identity providers using SAML, OIDC, OAuth2 and WS-Fed.  B2C also has the option to use it's own local store if the user doesn't want to connect existing accounts.

If a user has a gmail account, B2C can create an object in the service, but the users password is still maintained by Google.   When accessing our applications, the user goes to the B2C service instance, and then is pushed onto their own IdP (Google in this example) , once they authenticate, they are redirected to the B2C servce, get another new B2C token and are redirected back to the app and shall have access.

Billing/Cost of AAD B2C Service:
B2C is base on Monthly Active Users (MAU), you can have 60k users in the B2C but only 20k of the users have actively logged in using the B2C service.  Dormant/unused accounts in a calendar moth are not counted.
Updated 30 Nov 2019: first 50k MAU's are free for single factor authentication.  It's very cheap per user after the initial 50k and get's cheaper the more you have per user i.e. 50k-100k are £0.0041 per user.  So if I have 61,000 users, the first 50,000 are free and I pay £0.0041 per MAU for the next 11,000 users, amounting to £45.10 for my additional 11k users.
Multi Factor Authentication (MFA) is billed at £0.023 per event (think event as each SMS attempt both successful or failed).  So if the users use MFA and each of the 20k MAU users do 3 MFA's per month on average, the first part is free and the MFA part will cost (20,000 users * 3 attempts * 0.023 per SMS) £1380 per month.  It's a bargain.


More Info:
Great Post from my ex colleague Deepak Srinivasan on Guest and Member AAD access
Understanding ADFS Authentication with SharePoint

Sunday 12 May 2019

PowerApps Input Validation

Overview:  Validation can be handled in a multitude of ways in Power Apps, this technique I used for a fairly advanced set of Validation Requirements.


Basic Validation Example
To check that at least 1 of these two textInput box has been filled in.  Add this function to the DisplayMode:
If((IsBlank(srchMemberAccNo.Text)) && (IsBlank(srchMemberEmail.Text)), DisplayMode.Disabled, DisplayMode.Edit)

There are a number of ways to check if data exists in a control, I just try keep them consistent throughout my logic.  Here is a good approach: If(Not(IsBlank(txtBox1.Text)),<true>,<false>)

Designing an API with Web API on Azure

.NET core is great for creating a Web API project to provide an API.
Tip: Swagger is a great tool that has an editor to specify your API.  With the OpenAPI specification for your API, you can generate documentation, or use the code generation feature to generate tests or C# code.
Tip:  Use the Swashbuckle.AspNetCore Nugget package to generate an OpenAPI specification for your API if you use a code first approach.

Do not trust incoming parameters, you must validate.  Always respond with generic header error messages so you don't give away back-end information.  Ensure you log all errors and review them.

Preferred High Level Design I like for Web API projects

The API needs to return Status Codes to allow consumers/clients to know how there operations are doing so I design with these standard HTTP status codes:

CodeCode Status Meaning
200OK (GET)
201Created (POST or PUT)
202Accepted.  Used for long running operations.
204Resource has been deleted (DELETE)
304Not Modified
400Bad Request
401Not Authorized
403Forbidden
404Not Found
422Entity Validation Issue
429Too many Requests (Implement at APIM layer only)
500Internal Error
    



  • The bold codes are the three minimum set of response codes I will use for a simple API.  The next 4 most important are italicized.
  • Name resources/EndPoints with nouns and I always use lowercase e.g. /api/customers is better than verb noun which was common a few years ago, e.g., /api/GetAllCustomers  NB!  No verbs in URL naming.
  • Keep the URL's simple.
  • Always use HTTPS.
  • DateTime data is always in UTC.
  • Sensitive information must be in the header or body, not in the URL as it is not encrypted.

URL/Resource

GET

(read)

POST

(create)

PUT

(Update)

DELETE

(Delete)

/customers

List all customers

Add/created a new customer

Bulk update customers

Bulk Delete or Generally error and don’t implement

/customers/35

Get a specific customer.  Customer 35 is John DoeNAUpdate the John Doe recordDelete the John Doe object/record

HTTP Methods to Support
GETReturn the current value of an object
PUTReplace or update an object
DELETEDelete an object
POSTCreate a new object.  Return 201 for created and 202 for long running operations.
* PATCH - is also used for updating objects but normally a subsets of existing data

Versioning
API's need to change but there can be multiple client applications consuming my OpenAPI.  We use versioning to ensure I don't break existing implementations.  There are different strategies for versioning API's:
  • No versioning:   https://pbeck.co.uk/customers/35
  • URI versioning: https://pbeck.co.uk/v2/customers/35
  • Query string versioning: https://pbeck/customers/35?version=2
  • Header Versioning:  Not in the URL but passed in the header (my preferred option)
APIM has a great implementation for versioning.  Generally, adding optional data is pretty safe for API versioning.  You need to give 3rd parties using your API's time to implement the new version before deprecating and versions.

Tip: Enforce versions are present in the header.  No version return bad request.

JSON
JSON property names SHOULD be camelCased.

Query Params
Allow filtering using query param e.g. https://pbeck.co.uk/customers?firstname='paul'
Allow sorting for the most common properties
DateTime should be in UTC - The pattern for this date and time format is YYYY-MM-DDThh:mm:ss.sTZD. It is recommended to use the ISO-8601 format.  UI does adjustment for local time display.

Paging
Requests must have max return limit e.g. 100 items.  Specifying the limit param if you want to change the size of the dataset to return.

CORS
Cross Origin Request will need to be supported, reply on OAuth for who can use the API.

HATEOAS 
Good way to navigate around your API.  And a principle worth adhering to where possible.  

Authentication and Authorization
API's access should be via OAuth2.0.  It is very easy to hook APIM and underlying App Service (Web API) up to AAD B2C.  Scopes are used for authorization.

Code Review/Quality
"Linting is the automated checking of your source code for programmatic and stylistic errors. This is done by using a lint tool (otherwise known as linter)".   Examples of Linting Tools are Resharper, StyleCopSonarQube, ...

Security

It's a good idea to also have a WAF in front of your API.