Sunday 19 April 2020

Knowledge Transfer/Support Handover

Problem:  Projects that I tend to work on are complete by Scrum teams filled with specialist and specialist contractors who move on after project completion.  Support is generally handled by dedicate people/teams offshore.

Hypothesis: Having high quality support people working alongside you throughout the project is not very common due to costs.  I believe there are key points to cover to ensure that the operational support is effective.  Too many companies merely focus on checklists and the ops team don't get a fundamental understanding of the system.

Resolution:
1. People/Support: Understand the domain - Hard
2. People/Support: Understand the architecture - Easy
3. People/Support: Understand who is responsible for level 1- level 3 support and what that entails.  Easy if done correctly.
4. People/attitude: Hire patient collaborative, eager people in support (most key point) that want to learn and take ownership. Easy if done correctly.
5. Knowledge base - have a wiki or equivalent.  The same issues always present, so document and have an answer that can help your uses.  I also like to record mp4's for different levels of support.  Record the sessions as it is too easy for level 3 people to say they never got a handover or covered something.  This allows people to look back, easily train additional users.  Easy if done correctly.
6. Ensure you have automated tests, they are a great source of how your system works.  And if a fix has to be released, it also easy to validate that the original logic still works.  Hard but it returns great benefits if used.

Sunday 22 March 2020

My Solution Documentation Thoughts

It all depends on the project but this post outlines what I have found to be the best practices for documentation on projects. 

Documentation should not be an after thought but done effectively throughout the development of any project.  It helps clarify thoughts, communicate and should save time.  Documentation is generally poor as it is dumped on people that tend to write it from the wrong point of view.  For example, developers know the products or components but write the code from their point of view not necessarily effective to the enterprises understanding.

Documentation Should Cover

  • Overview & Start-up Documentation - Get the team with a common understanding.  I always like to have a Project Initiation Document (PID) that is kept short and up to date throughout a programme.
  • Architectural Design Decisions (ADD) - Get the technical people on the team with a common understanding.  Software Design Document (SDD)/architecture design document  - Description /overview.  High Level Design (HLD) & Low Level Design (LLD).  Architectural design decisions are stored in a Architectural Design Repository (can be a simple as a file server, I prefer SharePoint and a Wiki index).  
Possible Solution Architecture Information Architecture for holding SA docs 
  • Ensure documentation for: Solution Architecture, Dev processes, Support (wiki's), end user documentation, technical specifications (API's integration points), inline code must be simple and contain appropriate comments, and changelogs
  • Requirements - User Stories/Use Cases.  Get good clear requirements from the business.  This gives the team and architects, developers a clear idea/vision of what is to be built and often helps the product owner/stakeholders have a full clear agreed picture.  User stories are a great way to break apart large piece of functionality.  It's always a good idea to have functional (FR's also often referred to as Business Requirements (BR's)) and no-functional requirements (NFR's).  For me the best way to capture requirements is to use User StoriesFURPS is a way of categorizing requirements, useful to ensure adequate non-functional requirement areas have been covered.  I also like to use the old fashion MoSCoW (Must, Should, Could, Would) for prioritizing.  The most common mistakes I see in projects are requirements are:  1)  "Analysis paralysis" (very common in SDLC but more an issue with usage of SDLC than the methodology. 2) Gall's Law - stakeholders trying to put to much into a system from the start.  KISS/MVP - always opt for Keep it simple and only aim to deliver the minimum viable product.  Acceptance Criteria is a good way to validate when a User Story has been achieved.  Ideally a User Story should have less than 5 or 6 User stories.  If it has more, it is likely that the User story is too big and should be broken up into multiple user stories.  Weighted Shortest Job First (WSJF) is an Agile prioritizing system where you identify the highest priority items to do first.  Weighted matrix is another I have seen.  I also like an informal spend valuation that replies on effort/cost being already assigned.  Propriety Poker is also pretty common with multiple key stakeholders.  Stack ranking is also an easy option.
  • Code Documentation - Code comments & API Documentation/Swagger.  API's are often an architectural constraint in that you as a business may decide to everything needs to be implemented using REST API's.  APIM on Azure is a great tool for documentation and cross cutting concerns.  The developer portal documentation allows 3rd parties or other systems to securely access and documented API.
  • Performance And Testing
  • User/System Documentation - User Guides and knowledge bases. Reduce escalation or time to get end users working.  Support documentation, I use Wiki's, they are easy to use, update, once a problem is solved, it is easy to add a new wiki and all future support is much easier.  Wiki's are quick and easy and should be kept current, don't hold old decisions.  Wiki's are searchable and tag-gable.

Tip: I record a lot of decision and support using Snagit.  It's fast, brilliant for knowledge bases and end user training.  Considerably less effort than written documentation.
Note:  A lot of specific documentation is needed for legal and complaint/regulation, this can be pretty heavy but still best to understand the requirements and do it from day 1.
Thought: Technical Writer (can be a dev, BA, technical architect or a dedicate technical writer) - I believe the BA should also be the test lead on non-scaled Agile products.  They understand the requirement, therefore are best to understand the testing and write clear concise documentation in the form of test cases or acceptance criteria and user stories.
Tip: Use Grammarly and do documentation professionally.  Ensure your documentation is easy to follow, do not have spelling mistakes or grammar issues.  Lastly, consistent layout between different documentation writers must be consisted be this in code comments for full end user documentation.
Thought:  Write in present tense in an active voice, if forces people to look at the now and future.
Note: Companies have guidance and documents, ensure you know the format of documents and comply with company guidelines, this may be as simple as fonts and colours in your documentation to specific document formats such as TOGAF documentation standards.  Make it easy for your project with a little planning.
Thought:  Code comments - Naming should do most of the documentation, but complex logic or implementation decisions should be commented using the KISS principles.  Don't document exactly what the code says e.g. If (status=21)  // Apply logic if status is 21 // Rather us // Update the Customer Web Service if the users email address has change
Comments should not be used to delete code in case the developer needs it.  You have source control, delete the code.

Agile Documentation: Does not mean no or low documentation.  Agile documentation should be clean, concise and save time overall for the team members.  Essential documentation, don't over document or items that are obvious.  Prioritize documentation like we do in backlog evaluation.

Slack/Teams/Email:
I was a Slack evangelist, it is awesome for Agile projects especially for projects with people in different locations.  Well now I am a Teams guy.  It's awesome, simple and let's you remove so many dependencies.  If you haven't used it before and you have office 365, it's a "no brainer".  In 2 weeks everyone will love using teams.  I have had many dysfunctional teams that needed coaching, teams that document everything and in stand-ups you hear "I sent you that in an email".  The first thing I tell these teams is "email is not a defence", go tell or speak to the person.  These teams are To and CC nearly all there email.  I immediately enforce the rule To: means i want a reply CC means it's important to you.  If someone then sends and email that is CC'ed, I ask them why and they generally learn to use email conservatively.  I stopped a team several years back using email for 2 sprints to get them communication and trusting each other again.

Sunday 8 March 2020

Handling Security Incidents

Security Incident: An incident that potentially has compromised a companies systems or data.

Goal:  Focus on restoring confidentiality of systems/data and prevent further attack.  Contain the incident and eradicate the issue.  Full resolution target timeline is met for incidents.  These incidents can take up to 100 days but depends on the complexity.  

Examples:  Virus, Trojan Horse, Stolen data, increased unauthorized permissions, compromised server, copying data, DoS, unauthorized system access, ....

Need to record each event and work through the life-cycle (ISO 27035).  Can be dedicated software or modules such as ServiceNow's Security Incident Response (SIR).

  1. Plan & Prepare
  2. Detection
  3. Assessment and Decision - Get logs, review/analyse, document the findings, notify leadership teams.  Impact/Priority e.g. Critical vs Low business impact.
  4. Response - limit damage plan, decide on approach, notify if needed and remediate.
  5. Lessons Learnt - ensure the threat is removed and potential lessons can help improve the attach surface for similar issues.

https://en.wikipedia.org/wiki/Computer_security_incident_management

Note: Be careful not to delete forensic evidence.

Tip: Organisations must have a Security Incident Plan.  Plan, be ready, know what to do in advance improves the handling of Security incident.


Friday 6 March 2020

Power BI Notes

Overview:  Power BI is for reporting  and analytics of your data.  There are basically 2 ways to show Power BI Reports: User specific and app specific.

Power BI Embedding Models:

  1. User Specific/User Owned Data - Call the Power BI services as yourself/the current user using delegate permissions. 
  2. App Specific/App Owned Data - Call the Power BI service using a generic app permissions.  For example a public website, no Power BI licence required and every user of the site has the same access to view Power BI data.

Sunday 1 March 2020

Power Automate Notes

What is Power Automate?
Power Automate previously called Flow.  Power Automate contains "Flows".  Power Automate is workflow including RPA options, refered to a Power Automate Desktop (PAD).  Power Automate is a workflow engine that is based on Azure Logic Apps.  Powerful extendable workflow solution for low code automation.  Allows workflows to be easily automated with 3rd part systems e.g. SAP.

Used for:
  1. Personal Workflows e.g. I send an email to all people that have not update the DevOps Scrum board on a in the last day as a scrum master.
  2. Business Process e.g. Holiday request form.  If more than 10 days, need senior manager approval.  Generate RFP based on an event.  Historically, used K2 or Nintex or WCF workflows for business processes.
  3. Integration: e.g., move twitter posts into my warehouse for data-mining later.
A key concept is who shall the flow run as, a user or a service principal.  Setup a Service Principal in Power Automate » Benedikt's Power Platform Blog (benediktbergmann.eu)

Flows run against the owners account for calculating allowable usage (default), it is common practice to use a service account (normal AAD account with user and Pswd) to own specific flows.  A service account is a normal user account from AAD/Entra's perspective.  A dedicate account ensures that when a specific user leaves your business, the flow continues to run.
Img 1. Usage limits in Flows are counted against the flow owner.



Agree Power Automate Standards:
  1. Flow Names: start with a verb  format: Verb + What the Flow does + (trigger) e.g. "Get Tax Rates (Instant)".  I like to also prefix with the Company and Project, but feel free to have a standard to suit your business.  e.g. EY-USTax Get State Taxes (Instant) or EY-USTax Get All US State Tax Rates (Scheduled) or Get SalesForce Data.  Optional, for project specific workflows I also prefix witht he project name e.g. USTax-GetTaxRate. 
  2. Description: Short description to help readers understand what the flow does.
  3. Actions: Leave the Action desc and add info e.g. Compose: Tax Note.
  4. Variables: prefix with "v" e.g. vTaxTotal in camelCase.  e.g. vUserAge.
  5. Error handling & Logging: Catch errors and log into to App Insights via an Azure Function or Log using the built in Azure Log Analytics Action.  More logging means better traceability.
  6. Scope: Add scope actions for try catch logic.  Add multiple actions inside a "Scope" Action
  7. Terminate the flow with the Terminate Action if the flow has failed.
  8. Environment Variables: Great for logging as I go thru DTAP.  Also see.
  9. Connection Reference Name: Agree a format, does this flow run as  user or as a specified user.
  10. Loop Sparingly, use First() for performance.
  11. Owner: I like to use a service account in dev, it's a good idea to add tech owners as when it needs updating to support and easily find who they should talk too.  Understand who you are running the flow as, this ties to licencing but is critical.  You need to know you Actions and licencing limits on a project.
  12. Comments:  Im not a huge fan as the naming should make most flows make sense/self documented, but for tricky logic, comments are great.  Agree a standard.
  13. Retry policy: What happens if an action fails, do you want to try again?  
I borrowed a lot of the standards from "Matthew Devaney"
I found this document later - it is excellent. by Prasad Athalye - Best Practices for Power Automate(Microsoft Flow) Development

Licencing:
  1. Seeded licence is part of O365.  Use standard functionality such as standard connectors without needing to pay more for advance.  The advance/premium connectors are not part of the O365 licence.
  2. Per User licence -  Allows the  user $15 retail, can get discount with bulk and can use the advanced connectors & on-prem. gateway.  Many users need multiple workflows, normally personal workflows.
  3. Per User RPA licence - same as above but also has amazing RPA capabilities.
  4. Per Flow/Process - $100 per process per month, min 5 flows per month licences.  Anyone can use as part of the process.  Use for few people but process does a lot of workflows.  Can add a process one at a time after the first 5.
Licencing MS page
Power Automate has some licence add-ons available: AI builder and an unattended RPA add-on.
"Power Apps licenses will continue to include Power Automate capabilities", I don't know what is included but I assume it means any connector I can use in Power Apps, assuming I'm in Power Apps I can make Flows for.

Build workflows:
  • Can get a dedicated IDE tool for Power Apps or use the browser (which i always use).
  • There are over 350 connectors (in both standard and premium) and you can always use a custom connector to any OpenAPI (Swagger) endpoint.
  • Templates have some examples and often help as a starting point to make your own custom Flows in Power Automate.
  • Easy clear tracing so you  can see what part of the workflow is working and where you fail, and you can drive into the issue.  Super easy to use.
  • Example of an Instant Cloud flow triggered by a canvas Power App...

Query a Dataverse table in a Flow using OData

ParseJSON is fantastic for converting Open API/OData/JSON into an object

Extending - break out for programmatic control, I use C# Functions from my Flows and call them via HTTP triggers.
Retrieving  a row from the Dataverse custom "Subject" table.

Robotic Process Automation (RPA):
  • Also known as UI Flows within Power Automate.  Microsoft have purchase and integrated Softomotive for UI flows to add WinAutomation.
  • Attend (user logged in) and unattended version (complete tasks without manual intervention)
  • Can have multiple instances
  • API is generally better than using RPA as it is versioned and generally not changeable, whereas using a website, they website can be changed causing the RPA flow to fail.  Useful for instance when the RESP API is incomplete.
  • Recording tool for creating UI flows - Web use Selenium to record.
  • 3 Types: 1) Windows /Desktop/Screen reader and 2) web/website (Selenium) and 3) WinAutomation (covers both Windows and Web, easy to use but not as full featured yet).
  • WinAutomation has a drag and drop IDE, has error handling.
  • UI flows are well priced.  Also get AI builder credits with UI flow licences.
  • "Power Automate Per user plan with attended RPA to use UI flows and WinAutomation" Microsoft.
AI Builder:
Cognitive builder e.g recognize forms and extract data. E.g. receive invoices and add to accounting SaaS software.

Other: Zapier is a good tool for end user automation.  Easier than Power Automate but not as structured.  I'd use Zapier to automate in small businesses without O365 or flow licencing and allow end users to do it themselves.

Problem Solving:

Problem:  Another developer created a Flow I need to use from a Canvas page inside a model app.  The Flow is showing up when i say add but i get the error "Unable to add flow"
Initial Hypothesis: The flow is owned by another developer having ownership and the connection is done thru their account, take ownership and change the Connection (Dataverse connection in my case)
ResolutionTo use an existing flow, you need: 1) flow in same solution as the canvas app, 2) Ownership and the connection string needs to be switched to the new developer

Problem: Migrating solutions between environments, all the workflows fail when they use the Dataverse connector with 403 errors. Tracing the flows, I can see the error  "Flow client error returned with status code 'BadRequest' and details {error: code: 'XrmApplyUserFailed... UserNotinActiveDirector ... does not exist in user tenantId"
Problem: Using the service Principal account needs to be registered in the new Dataverse environment.
The connector issue was fixed, we had to recreate the connection from scratch making sure it was set up with a/the service principal. Then we added the registered app into the environment as an application. user.


Friday 28 February 2020

I love Power Apps

No self respecting technologies/programmer/ex-programmer likes simple solutions.  I still chose C# as my go to problem solver (I learnt to use a hammer 18 years ago and now all my problem look like a nail).  Don't tell anyone under 30 I still love C#, they think you are weird if you don't use Javascript for everything.  And 15 years ago I was arguing the same with the C++ golden oldies.  So the circle of life goes on.  Enter Simba, the Lion King...  Well not really but Power Apps is just making programmers look ridiculous.

Power Apps can do a lot...

MS have announce that components and PCF’s are part of Power Apps.  I've been using this for awhile and it's awesome.

This diagram outlines extending Power Apps using Components:  Developers often say you can’t extend Power Apps so it is not a real language, my answer is really… 
The other argument for PowerApps is if you need to do any complex programming.....  Use an Azure Function (this allows C#, PowerShell, JS, …).  Simple to build in your own language, expose using an http endpoint and you can call the end point using a custom connector.  
So if you need: 
  • Complex UI = go PCF, 
  • Reusable UI e.g. menu's = use Components, 
  • Complex processing or secure code logic = Azure Functions.
Here are some PCF examples that we can use that are already built:

So what is stopping Power Apps:
  1. Licencing - the licencing is expensive so you need to choose selectively when it is appropriate.  If MS change it's going to just go crazy the usage. IMHO. 
  2. Functionality:  MS have introduced automated testing to the Power Apps platform but there are improvements scheduled.  For me the biggest issue is code reuse.  I wish there was a function library that you could write functions to and call in a single line of code, my individual field logic can be insane and tough to amend on complex large Power App UI's.   
Excellent blog on Power Apps