South Africa Compliance & O365

Yesterday, (22 Oct 2016) I presented at SharePoint Saturday Cape Town on securing your Data on O365 and SharePoint. I believe that South Africa is going to have massive requirement around compliance and here and e a few reasons why:

1. POPI
2. FSR bill in parliament at moment, this will enable twin peaks
3. National credit amendment act regulates credit institutions
4. Fic amendment bill also in parliament to govern anti money laundering
5. Banks act governs bank
6. Long term and short term insurance act
7. Consumer protection act

 

Additionally, all of the big 4 are viewing big data and compliance as mega trends.

If you understand O365 security at the authentication level and application level you are well placed for the future.

Note: By application level I am referring to things like DLP, EMS, retention policy, ,,,

SharePoint Support Models

Overview:  Large organisations tend to use a tiered support model also called the escalation model.

The Problem with Tier Support Models:
A user finds an issue and explains it to level 1, the Level 1 support guys figures out it's too difficult or not on his easy path and pushes to level 2,  the whole bug needs to be re-explained generally involving the business user that reported the bug and the level 1 support person.  This goes on for 3  to 5 levels in bug organisations and is eventually passed onto engineers or the vendor.  It takes an astronomical amount of time, and provides a poor impression to the business user.  Coupled with a tracking system that the end user don't know how to use and the support people trying to add as much content as possible so as to cover any responsibility as they have not missed anything.  It's just a disaster.  Anyone doing Level 1 and 2 support is generally not happy and poorly remunerated so the turn over is high, end use satisfaction is not good.  A lot of time and focus is wasted.  To me the fundamental problem I have with Tiered support models is a lack of Total Ownership.  Support people pass the problem and tick off the easy fixes.  I have seen multiple support escalation software products and fundamentally the products don't tend to make much difference, it's the implementation of the support process and the quality and ownership of staff that determine good support models (but its far easier for people to blame software).

On the plus side, Level 1 support people are considerably cheaper and if they have good knowledge basis and training they can return 70% plus of incidents at this 1st stage.  Funneling tougher questions to more specific staff.

DevOps: DevOps relies on close collaborative deployment and support, so the upside is you have the developers and they understand the infrastructure and are best position to fix mistakes cleanly and quickly.  In traditional enterprises, we tend to have hundreds or even thousands of applications and you loose economies of scale by having to keep dedicate higher cost people around to do support and it generally affects sprints as people need to be pulled out to fix bugs.  On the plus side, you get fixes done quickly and correctly.

For me the answer is "It depends...", if you are a tech company with 1 main product, think Facebook, AirBnb then devops is clearly the choice approach.  If you are a company that has legacy applications such as a bank, an application supporting mortgage applications has been used for 10 plus years, tier support is much cheaper and effective.  So the tough question would be what about the same bank that is now developing a new complex application, ideally if governance would allow it then I would start developing the product and use DevOps, allowing better deployment and support in a rapidly changing environment.  Once the project starts maturing then the question is when can I move the product to a tier support model.

Summary: DevOps works well in an Agile environment and support is vastly improved, assuming the product is not changing and the number of incidents is low, this the time to transition to a tier support model.

Mobile Platform Development for SharePoint 2016

Overview:  There are various options for building Mobile applications.  The simplest answer is to have a fully responsive designed web page and mobile users interact with the application using the mobile devices browser. Another option is to write code for each platform so for example write and iOS application what would be loaded in the AppStore, this requires multiple source codes to be maintained.  An improvement on this approach is to use a tool that creates a version from a single source for multiple stores.  So write once deploy to MS, Android and Apples store; PhoneGap and Xamarin are examples.

Xamarin uses C# code and compiles executable applications for each target mobile store such as iOS, Windows and Android.  Xamerin uses it's own IDE or a add-in for Visual Studio.

PhoneGap is also refereed to as Cordova.  PhoneGap creates the web application using HTML5, CSS & JS and wraps this web application in a plaform control/container.  In effect, the html page is hosted in each platforms executable within a web control.  The exe can interact with the web pages:  

Unity is another development platform probably the 3rd biggest and generally favored for gaming.

Summary:  Responsive Web Design works on the mobile device as a native html app using the mobile browser.  If you need to interact with he phones features and you need to write an application for the platforms store, rather use Cordova/PhoneGap where you write once and distribute to each desired platform.

Updated: 29 August 2016
More Info:
http://magenic.com/documents/Xamarin_PhoneGap_eBook.pdf?EBookTest

Excel Services REST API - SP2013 Notes

"Excel Services is a service application that enables you to load, calculate, and display Microsoft Excel workbooks on Microsoft SharePoint 2013. Excel Services was first introduced in Microsoft Office SharePoint Server 2007."  MSDN

There are 4 ways to interact with Excel using Excel Services, this article only looks at utilisting the REST API.  The figure below provide context showing an Excel file with a list of countries with 3 digit ISO codes.

Work in Progress...

Excel Services REST API - SP2013 Notes

"Excel Services is a service application that enables you to load, calculate, and display Microsoft Excel workbooks on Microsoft SharePoint 2013. Excel Services was first introduced in Microsoft Office SharePoint Server 2007."  MSDN

There are 4 ways to interact with Excel using Excel Services, this article only looks at utilisting the REST API.  The figure below provide context showing an Excel file with a list of countries with 3 digit ISO codes.

Work in Progress...

Hybrid SharePoint and Office 365 Authentication Thoughts

Overview: Hybrid scenarios allow enterprise users to seamlessly interact between SP Online and SP on-prem instances, provide search across on-prem and online sites, access data on-prem. while using Office365/SP Online, use Office 365 apps like Flow, Video, Graph and utilise OneDrive.  Picking the right authentication allows users to have a seamless high-value experience bringing together secure access quickly.  Pretty important and to make this happen you need to deal with access.

Organisations have internal authentication mechanisms such as Microsoft's Active Directory.  Large organisations have a tough time migrating to the cloud and with the rapid changes in Security and the cloud this post aims to broadly define paths or options for architects such as myself to follow.

Options 1. Do nothing.
The 1st option is to ignore the cloud but I am going to presume you want to take advantage of Office 365.

Options 2.  Only use the cloud/O365.
Office 365 is huge and for a small or new business, I would strongly look at only using O365 using Azure AD (AAD) credentials.  This means no or little management of Active Directory (AD) and you can pretty much connect to the whole Microsoft SaaS offering quickly.  Most large SaaS offering can work with Microsoft AAD.  Generally, this option is not suitable for large enterprises.

Option 3.  Internal AD and externally use Office 365 Azure AD.
Easy to implement as the internal and external credentials do not link.  Your users do not get a single sign-on (SSO) experience.  Users use the Azure AD credentials when working with Office 365 and your internal credentials when working on the internal network.  The user needs two accounts and to know when to use them.

Option 4.  Internal AD synchronized and creates similar accounts on Azure AD.
Pretty much the same option as option 3 but the usernames appear to be the same to the end users. There are a few variations in this space, you can simply create the accounts with the same name either manually, using a CSV import or using Directory Sync (DirSync).  At this stage, the passwords and accounts are managed separately, the DirSync reduces effort and provisions and removes accounts in Azure AD to match the companies on-prem. AD.  DirSync will reach the end of support in April 2017.

Option 5.  Internal AD automatically syncronises with Azure AD including password sync.
You still have 2 accounts but the accounts on both sides are kept aligned using DirSync and password synchronization.  The same password is stored both in your on-prem. AD and in Microsoft's Azure AD for each user.  The advantage here is that the user name and password for a user is the same if using internal or external applications secured by on-prem. AD or AAD.  This is not SSO enabled, the user needs to login to both AD's separately.

Option 6.  Azure Active Directory Connect.
Similar to option 5 but the Azure AD Connect tooling does all the synchronization of accounts between on-prem. AD and AAD.  This option/method is easier than option 5 and the latest approach but fundamentally it is the same approach with 2 identical accounts for each user.
Note: Only 1 AAD Connect per AAD B2C tenant.  If you have multiple AD's (e.g. AD forest), you use a single instance of AAD connect to grab each AD's objects.
Note: AAD Connect can write back properties to AD on-prem. but it can't create objects/account.
Note: AAD Connect can selectively grab accounts e.g. exclude some OU's.
Note: Default replication is every 30 minutes, but he pswd hash syncs every 2 min, you can config Azure password rest to push back to on-prem. AD if you use password hash sync.

Options 7.  Federate (ADFS)

Active Directory Federation Service (ADFS) provides an Identify Provider and can pass claims based authentication between a trusted Identity Provider.  This post does not explain passive Identity authentication but this is the more advanced option.  There are a lot of federation services but ADFS tends to be the most common (ThinkTexture, Ping, SiteMinder).

Home Realm Discovery (HRD)

When accessing SharePoint Online, you go to AAD as each SPO has it's own AAD.  On AAD you may have ADFS, partners Ping Federation service, other organisations AAD to actually authenticate the user.  So the login experience needs to know where to authenticate the user.  To do this use MS and you can also build custom HRD.  When the user enters the username or email, they are forwarded onto the appropriate ADFS or Federation service to authenticate the user.

Ariel Gordon describes HRD at Microsoft below: 

"How does Home Realm Discovery work?

In the cases above, apps direct users to Azure AD's common endpoint, and Azure AD shows a generic sign-in page. This page waits for users to enter their username then, as soon as the focus moves away from the username field, it makes a server call to look up the configuration for the user's domain. In case of a federated domain, the login page then initiates a redirect to the federation server, such as ADFS. Users then enter their credentials on the federation server's own login page which displays their organization's branding.

When we introduced Company Branding last year, we mimicked this UX behaviour: the Azure AD sign-in page starts off with generic branding then looks up the organization's branding elements after the server call is made. In both cases (federation and company branding) the goal is to ensure that users enter their credentials on a page that reflects their organization's brand.

When does it make sense to bypass Home Realm Discovery?

If your application targets users in a single organization, there's no need to use Azure AD's Home Realm Discovery and you can "accelerate" users to their organization's sign-in page. To do this, your application needs to pass a domain hint to Azure AD, effectively stating "I've already established that the user who's about to sign in is from <this organization>."

When Azure AD receives such a hint, it performs Home Realm Discovery on the domain name hint before rendering a single pixel. If the domain is federated, Azure AD immediately redirects users to the federation server. If the domain is managed, it checks whether Company Branding has been configured for the domain and displays it when found."

How the Recycle bin works

Problem:  A common misunderstanding with technical and business folks is the details of how the recycle bin works.  I missed some of the finer points so I thought I'd record my note.

First stage recycle Bin - User deletes a document and the document goes to the user's recycle bin/1st stage recycle bin.  The 1st stage recycle bin retention period is 30 days by default.  Items in the first stage recycle bin are moved to the second stage recycle bin after the retention period is over or the user chooses to delete the item from their 1st stage recycle bin.

Second stage recycle bin or Site Collection recycle bin - where documents go after they are removed from the 1^st stage recycle bin.  SCA can restore or permanently delete the documents from the 2nd stage recycle bin.

Recycle bin explained for Office 365 - as of May 2016

Note:  The views change between the SharePoint versions but the functionality remains the same.
Note:  An item that I have not understood for multiple years is that the retention period for recycle bins applies to both the 1st and 2nd stage recycle bin.  If the recycle bin retain content for 30 days and the user deletes a document, then 22 days later they delete the document from their own 1st stage recycle bin.  The document only stays in the 2nd stage/site collection recycle bin for 8 days.  From Nik Patel's site "In other words, total time spent by the item in both recycle bins".
Note:  If you do not have access to the farm settings, you can see how long the retention period for the recycle bin is by deleting a document/item from the 2nd stage recycle bin, you get a confirmation box that let's you know the setting.   This approach allows me to see that Office365 SharePoint sites hold recycle bin items for 93 days.

In the Site Collection Admin (2nd Stage) recycle bin delete an item to see how long the retention period is set to.

To Access the Recycle bins: Site Collection administrators page > Site Settings > Site Collection Admin > Site Collection

More Info:
Basics of Recycle bins from Microsoft
Nik Patel has a good article on how the recycle bin works