Showing posts with label ML. Show all posts
Showing posts with label ML. Show all posts

Sunday, 11 February 2024

Basics to setting up an AI Transformation Program - High level

Overview: Many large organisations have begun their Artificial Intelligence (AI) journeys, but some have taken unusual directions, while others are adopting a wait-and-see approach.  As a general rule, I believe most organisations should identify the most important AI use cases and, using a basic scoring system, prioritise the easy, high-value ones first. 

A good option is to hire an AI-focused team to implement the AI program within your business.  Hire internal staff with key domain knowledge and pair them with IT experts, preferably those with strong AI skills.  

Tip: All Digital Transformation Projects must cover People, Process and Technology in that order of importance.

Thought: The two biggest mistakes I have observed as of July 2025 in clients are a lack of clearly defined benefits of AI projects and poor-quality, insecure data.

FAIRS for:

  1. Findable, 
  2. Accessible, 
  3. Interoperable, 
  4. Reusable, and
  5. Secure.

    Tip: If your organisation's data is in a good FAIRS state, AI projects are far more likely to succeed.

1. AI Idea Generation: Collate a detailed list of possible ideas.  I like to use SharePoint, it's a good idea to open up idea generation to the business or make gamification of idea generation to get a good set of ideas.  You'll get lots of overlap, distil it into unique ideas, and bring the team/stakeholders together.

Figure 1. AI idea filter funnel
2. Business Impact: How does this help our customers, generate revenue, or improve our operations?  Can this be quantified?

3. Technological Feasibility: Can technology meet the requirement, and at what cost?  A high-level technical design is strongly recommended, as shown in Figure 2.

4. Implement: If the idea holds up to Business impact, and is possible, select the highest value to lowest effort ideas.  It is a good idea to start with the simpler ideas to get the ball rolling.

Figure 2. HLD example

Note:  This post is looking at building AI to improve your business. AI also affects all internal staff, so train them.  Allow them to use AI tools like ChatGPT, and train them not to expose internal proprietary information.  Give your users the right tools to be as effective as possible.

AI Digitalisation Transformation Checklist:

  1. Data Governance Framework: Do you know what data you have, can you access it, and is it secure?
  2. Enterprise Data Model: Define the rules for quality and integration between the EDM areas (these may be functional or geographic), ensuring we share data and make it available.
  3. Master Data Management: Ensure the data and applications are working off the same sheet of music.  
  4. AI Framework: What are the limitations on what you can do within business governance? How will you identify projects and prioritise them? How do you ensure alignment with the tools?  AI is expensive; don't just start picking our individual solutions.  I'd also call this 'measure twice, cut once.'
  5. Don't eat the whole Elephant: Pick small projects to start with, clearly defined goals that show the wins from AI.
  6. One Process: When the AI solution/project is built, will you still use the old way? It's expensive to create, document, and support multiple processes over the lifetime of solutions that have duplicate steps to accomplish the same task. Make your process simple and the preferred or only approach.
  7. People: Have they checked the process? Is it optimised, or merely converted from the old paper process to digital?  Have you engaged key stakeholders and experts?  We should have asked whether this is the best process and also verified it with non-technical stakeholders.
  8. Cost Optimisation and Value Realisation: AI-based transformations like Software development or engineering projects are about whole-life cost and value; ensure you are not just looking at building something.
  9. Measurements and KPI's: How do we measure value? Are you monitoring usage and costs? They are the easier measurements.  Security is also essential to measure, as well as what constitutes success.  Many benefits are intangible; list them. Can a value be guesstimated for intangibles?
There are many more items, but this is a good check to run repeatedly.

Thursday, 14 September 2023

Microsoft Azure Artificial Intelligence (AI) - AZ-900 Notes

The foundation of all successful AI is high-quality, findable, secure data!

The FAIR principles are guidelines for ensuring that data is 

  1. Findable, 
  2. Accessible, 
  3. Interoperable, and 
  4. Reusable.

    I think it should be FAIRS for:

  1. Findable, 
  2. Accessible, 
  3. Interoperable, 
  4. Reusable, and
  5. Secure.

    Tip: If your organisation's data is in a good FAIRS state, AI projects are far more likely to succeed.

1. Artificial Intelligence (AI) 

  • AI is attempting to make computers use and behave using human intelligence.  
  • Teach PC to do tasks for us - automation.  
  • PC predicts using patterns and can act.  And good at looking for anomalies.
  • PC uses camera/photos to look for patterns.
  • Engage in valuable conversations, use multiple sources of knowledge.

2. Machine Learning (ML)

  • Train PC's to see patterns and see patterns, and look for anomalies.
  • Example. Predict stock prices by looking at factors that affect stock prices.
  • Anomaly Detection - Detects unusual patterns, e.g. CC used in Asia when typically in Europe, but transactions 10 min apart.  Therefore, it is likely to be fraudulent.  Sort rubbish.
  • Predictive models by finding relationships.  Give model data and train the model.  
  • Example: flowers have features/characteristics, e.g. colour, size, no petals, ...
  • Using data to teach a machine
  • Supervise ML - need quality data, including labels.  Avg humidity, hrs sunshine, rainfall, temp, month of year (features), ice creams sold(label/class), so we feed in temp is Regression ML.  Patient has features (weight, sex, age, bmi,...) give value btwn 0-1 of the person developing diabetes.  is Classification ML
  • Unsupervised ML - data is not labelled.  Just the feature provided will group into clusters.  Pulls data out and figures out its own criteria is Clustering ML.  Useful for fraud detection.
  • Training - good data based on a training set and a validation set.  Train model, with most data, then check with the remaining - allows us to see how close to what happened.  Service tries to figure out relationships.  Model is used by test data - see how close/useful the model is.

3. Compute Vision 

  • Self-driving cars, sorting. Sort rubbish.
  • Facial recognition, object recognition,..
  • How do computers see?  The picture is cut up into pixels, data is pulled and used to find possible answers. 
  • Some types on Azure:  object detection, i.e. car, bike, bus.  Image classification, i.e., horse, car.  Semantic segmentation, i.e. Teams blur background.   Image analysis context by bringing various tougher.  Face detection.  OCR - reads the image and converts it to text.

4. Natural Language Processing (NLP) 

  •     interpret, e.g Grammarly, Spam Check, Alexia, 
  •     Knowledge Mining - Extract info from knowledge and gain insights, e.g. social media marketing.

Principals:

  1. Fairness - ensure bias is excluded, e.g., based on gender.
  2. Reliable & Safety - need high confidence and in specific systems cannot fail, e.g. health systems, autonomous cars.
  3. Privacy & Security - Ensure data is protected and not giving away sensitive data.
  4. Inclusiveness - should be fair, i.e. VI users
  5. Transparency - what is the model based on, what could be an issue?
  6. Accountability - who is liable for AI decisions
Azure:
  1. Scalability & Reliability
  2. AI Resources: sit in an Azure Resource Group

AI Services in Azure:
  1. Azure Machine Learning - Developers to train, test and deploy ML models.  Within a subscription, create a Azure ML Workspace (consists of: compute, data, jobs, models) can then publish as a service.  Azure ML Designer, used for creating an ML pipeline, data to train the model.  Automated Machine Learning user only needs to provide the data and select the model to use; the service figures it out. 
  2. Cognitive Services - vision, speak, language, decision.  REST API endpoints have already been trained; choose the model.  Can deploy multiple parts individually or together.

    3. Azure Bot Service - developing and managing intelligent bots like chatbots
    4. Azure Cognitive Search - Data extraction & enrichment for indexing.  Makes data searchable.

Anomaly detector resource - wizard to set up - Add Keys and endpoints to allow access.

Create a new Azure Machine Learning Service, which will create a Workspace.  Use multiple Azure services such as Key Vault, AI, and storage accounts.   
  • Launch Studio
  • Add Compute Cluster
  • Add Data (csv, spreadsheets, nearly any form,...)
  • AutomateML (figure it out without me) and run the job
  • Will show trends
  • Deploy the model (i.e. to a web service)
  • Shows "Endpoints" - get URL and a test rig.