Showing posts with label BI. Show all posts
Showing posts with label BI. Show all posts

Sunday, 30 July 2017

Tech Megatrends in 2017 - The bigger Picture

Problem: What are the significant technology changes that are going to shape the enterprise over the next 5-10 years? 

Initial Hypothesis: Working within technology and making businesses more competitive, I see the speed at which technology changes.  Working with multiple large customers, I know that people are generally looking at a handful of strategic technology trends.  While technology trends are actually all connected and have a hierarchy.  Essentially, I am categorizing the significant tech trends, rather than drilling down to lower levels, such as Mobile application patterns or specific technologies like O365.

While technologies are not so kind as to make themselves easy to segregate, I feel the big trends are:

Analytics - Storage, mining, analysing data and reporting.  Essentially, this is an old industry with new trends, such as larger data sets, new data sources (e.g., social media), and additional reporting formats/media; AI has experienced a surge in recognition.  The trend has been to make analytics available closer to real-time, and it's now moving into predictive.  Robotics is closely tied to data analysis. 

Security - With the increasing amount of data and devices, the old world of protecting one's own assets and monitoring external people in a regulated manner is no longer the primary method of securing.  The key is understanding that identity is king.  Whether it is an individual or a computer, we need to be able to verify the identity of the person or system claiming to be who they say they are (non-repudiation).  Block-chain - While I think it's important and, in some industries, absolutely critical, this, to me, falls under security.  We need to trust and share between machines and transactions. Once again, an old industry is undergoing a shift in focus, with identity and collaboration now central to security, rather than relying on a centralized, castle-like security model.

Cloud Computing - Low-cost computing paid for on-demand is a continuing trend for businesses. Now, with advancements in virtualisation and cloud computing and the emergence of the big three (AWS, Azure, and Google), these services are improving rapidly.  Cloud computing ties to Analytics and security. Virtualisation has progressed, and we are getting better at providing computing safely and at a lower cost.

Sub Trends:

Robotics/Automation - Automation replaces manual steps typically performed by humans, and historically, this has been more prevalent in manufacturing.  Going forward, I envision it replacing rudimentary information worker roles.  Ingesting legislative data is a great example, and coupled with machine learning, we can build up models to identify which laws are applicable in different jurisdictions. Medical diagnoses are often referred to under AI as not intelligence; it's pattern recognition and analytics to determine a likely problem from the data picked up and cleaned.  The medical example will be essential to us thriving on this planet; doctors will only get complex issues, and the rudimentary stuff will disappear and be dealt with better, meaning fewer returning patients.

IoT - is not exactly new; we just have to do it better on a larger scale.  So, while technology is changing, systems communication is not.  I view drones as a subset of IoT, specifically hardware devices that utilize analytics.  Pretty similar to fully automated self-driving cars.


Augmented Reality (AR) and Virtual Reality will be significant, but like blockchain, they will have unique applications across various industries.  It relies on Analytics and security as its underpinning.

Here is some research I gleaned from 5 key firms on current technology mega-trends (this is how I see these firms' views on the tech mega-trends,):

PWC
===
Artificial Intelligence (AI)
Augment Reality (AR)
Blockchain
Drones (UAV)
IoT
Robotics
Virtual Reality
3D printing

EY
===
Artificial Intelligence - Cognitive learning
Robotic Automation (RPA)
Blockchain
Analytics
Internet of Things (IoT)
Cybersecurity

Deloitte
=======
Dark Analytics
Machine Intelligence
Mixed Reality
Inevitable architecture (cloud computing)
Everything as a service (cloud computing)
Blockchain

KPMG
====
Big Data & Analytics/AI
Cloud Computing
Cryptocurrency/Digital Payments
IoT
Robotics
Cybersecurity
Virtual and augmented reality - stock and glasses -Google glasses

McKinsey
=======
AI
Analytics
Robotics

Possible Resolution:  Most people in technology are aware of the megatrends, how relatively important each trend is and the details of how quickly change is happening.  For me, analytics, speed to market, and trust need to underpin everything I deliver.

Friday, 9 June 2017

SharePoint on-premise reporting options

Overview:  As always this really depends on the circumstances but my broad view on reporting for SharePoint is outlined below.

Thoughts:

  • Power BI on-prem. has not been release (at the time of this blog) but will work for SSRS reports and have the Power BI functionality that shall be embedded using an iFrame into SharePoint 1st edition only.
  • The upgrade path from SSRS SharePoint integrated mode is going to be hard, whereas the upgrade path from SSRS native mode will be simple.
  • Power-BI on-prem is not going to support SAML.  Use WAP (Web Application Proxy)/reverse proxy to get NTLM ot Kerbros tickets for authentication.  Only config is ADFS+WAP to access PowerBI on the Extranet.  If you use another Federation service will need to do a passive claim.  For Example if you use SiteMinder, you need to pass onto ADFS + WAP
  • SSRS SharePoint integrated mode shall be available only in SP2013 and SP2016 and not have any continued path going forward.


SharePoint 2013 Options:

SharePoint 2013 BI Options SharePoint 2013 SQL BI Options
Excel Services
PerformancePoint
SharePoint KPI/Filters
SSRS SharePoint Integrated Mode
Power Pivot for SharePoint

The table below shows what you client probably uses historically and the challenge is to map them out to a supportable solution on SharePoint going forward.

BI in SharePoint and the Microsoft stack has drastically changed over the past 2 years.  The next 12 months should keep this pace up with the release on Power BI.  As my knowledge in this area improves I'll update this post.

Other Posts that may be useful:
A good post to read as of EOY 2016 on BI  for SharePoint
SP2013 with SSRS 2012 SharePoint integrated Mode Overview
Installing SSRS on SP2013 Reporting Post

Thursday, 6 April 2017

SharePoint Search to extract SharePoint list Data into SQL

Problem:  I have multiple lists that I need to get data changes updated into SQL on multiple site collections.  Think 1000 site collections with 5 lists in each so 5K lists are being updated (my actual requirement is much larger).

Initial Hypothesis: In SP 2010 or SP 2013 I would use a Full Trust Event Receiver and register each list using a feature but now we are in the new world of no full trust code.  The 1st thought is RER (Remote Event Receivers), I need to register 5,000 of these and they are notoriously unreliable for delivery.  Search has to pick up all changes to list items for indexing and I can break into the crawl pipeline during Content Enrichment.

Proposed Solution:


Points to Consider:
  1. I need to create a Content Enrichment Web Service (CEWS) that I shall call from the crawl component.  I can only register a single web service on a crawl SSA so consider using the toolkit/Microsoft CEWS Framework on the Web Service endpoint this allows multiple Web Servies to be strung together.  Good idea to implement a toolkit so multiple pieces of logic can be applied.  So if BA Insights registered, then can't add another CEWS Web service. CEWS toolkit provides a pipeline of stages to add custom logic later.  So write our own stages and hook into the CEWS framework.
  2. CEWS does not process item deletes so a possibility is using the crawl log to identify deletes
  3. There is no CEWS on SharePoint Online/O365.
Note: Properties returned from CEWS has minimal data input (common custom fields such as Title) and need to augment with registered MP's to get the data in the CEWS web Service.  Do not get a dump of properties/need to register and ask for the properties.

27/04/2017 Note: An idea I had to deal with "Deletes", I could put the item into a status of "About to Delete" and the crawl would pick up the status and the CEWS could delete from he SQL database and then the Web Service could delete the list item from the SharePoint list.  It doesn't work, the CEWS pipeline does not pickup and item updated and then deleted.  So the crawl is smart but I can't enhance using this approach so more work to fix the delete ...