Enforcing Authentication using the Windows IdP

Overview: SharePoint farms often have multiple Identity Providers (IdP) such as Ping or SiteMinder and you always need to have Windows for the Search and other reasons.  Often you need to force the browser or CSOM code to use a specific IdP, in my case, the Windows IdP needs to be used.

In Fiddler, I have added a header as shown below to enforce the browser uses the Windows IdP:

CSOM HTTP Header added to force usage of the Windows IdP:

Simple Code for a CSOM SharePoint Test Rig

Xamarin is fantastic-

Overview:  I am enjoying working with Xamarin, easy for any C# developer with a little experience and forethought to build decent applications.  Microsoft/Xamarin really reduces the barriers and makes building native mobile applications very achievable.  Nothing to do with SharePoint but maybe my next out of hours project will encompass O365.  I saw a demo at an Azure conference 2 months ago and thought it looks pretty straight forward.  MVVM with C# makes iOS and Android development of native apps easy.

Proposed Solution:  Scratch card game

• Build a Web  Service for holding game and user details
• Build UI to reveal scratch card for Android and iOS
• Sign in using Facebook
• Retain users winnings and games played information 

Summary: PoC is going well, deployed the Web Service and SQL back-end, registered with Facebook and can login.  Secured mobile apps Droid and iOS.

Tips: PCI projects and Xamarin forms projects do not play nicely together.  Choose the project type wisely.  Also get some help from a Xamarin developer, it is saving me so much time when I hit an unknown.

Switch Master Page Minimum Permissions

Problem: Use the Client Side Object Model (CSOM C#) to add a new master pages to a site collection and switch the master page.

Initial Hypothesis: Writing to a site collection only required contribute rights or even "designer" rights at the web application permission level.

Resolution: The minimum permission set for changing master pages is "Full Permission" which a site owner and the site collection admin have.  So to switch master pages you need a high set of permissions.  UI allows master pages to be switched when the user only has "Design" permissions. This proof is flawed as the UI and CSOM permissions are different.  Can the UI have different permissions to the CSOM API???  Am I going mad.  
SPWeb object with Design user permissions cannot be updated and the API returns an "Access Denied Error" - Thanks to Sachin Khade for identifying this.

Updated 26/05/2017:  So the reply I got from the engineer who raised a Microsoft ticket is "SharePoint designer and  SharePoint GUI only need to have design permission to change the master page. This is because SharePoint designer is created as an extension of the SharePoint product. However, since CSOM calls are coded using Visual studio, the code flow involved in this is different and hence requires permissions that are higher than what SPD needs."

Summary: "Design" rights allow the user to change the master page using the UI however the same user cannot switch the master page using the CSOM C# approach.

Updated 26/05/2017: Thanks to Aswin Bhaskaran for working out a minimum permission set for using CSOM to switch the master pages on a site collection:

Note: "Design" rights can be applied at the Web Application Policy level allowing the accounts with "Design" rights the ability to add master pages.  The "Design" permission is only built into SP at the Site Collection level, I created the "Design" permission with the same permissions at the web application level to ensure my account in the Web app Design group has access to all site collections on my web app.

Note: Microsoft do not recommend customized master pages for O365 or future development.  Rather inject JavaScript to modify pages.

SharePoint Search to extract SharePoint list Data into SQL

Problem:  I have multiple lists that I need to get data changes updated into SQL on multiple site collections.  Think 1000 site collections with 5 lists in each so 5K lists are being updated (my actual requirement is much larger).

Initial Hypothesis: In SP 2010 or SP 2013 I would use a Full Trust Event Receiver and register each list using a feature but now we are in the new world of no full trust code.  The 1st thought is RER (Remote Event Receivers), I need to register 5,000 of these and they are notoriously unreliable for delivery.  Search has to pick up all changes to list items for indexing and I can break into the crawl pipeline during Content Enrichment.

Proposed Solution:

Points to Consider:

1. I need to create a Content Enrichment Web Service (CEWS) that I shall call from the crawl component.  I can only register a single web service on a crawl SSA so consider using the toolkit/Microsoft CEWS Framework on the Web Service endpoint this allows multiple Web Servies to be strung together.  Good idea to implement a toolkit so multiple pieces of logic can be applied.  So if BA Insights registered, then can't add another CEWS Web service. CEWS toolkit provides a pipeline of stages to add custom logic later.  So write our own stages and hook into the CEWS framework.
2. CEWS does not process item deletes so a possibility is using the crawl log to identify deletes
3. There is no CEWS on SharePoint Online/O365.

Note: Properties returned from CEWS has minimal data input (common custom fields such as Title) and need to augment with registered MP's to get the data in the CEWS web Service.  Do not get a dump of properties/need to register and ask for the properties.

27/04/2017 Note: An idea I had to deal with "Deletes", I could put the item into a status of "About to Delete" and the crawl would pick up the status and the CEWS could delete from he SQL database and then the Web Service could delete the list item from the SharePoint list.  It doesn't work, the CEWS pipeline does not pickup and item updated and then deleted.  So the crawl is smart but I can't enhance using this approach so more work to fix the delete ...

CSOM for Deployment of SharePoint sites and Assets

Problem:  Historically we built Site Templates and WSP to build up of SharePoint solutions.  With the current state of SharePoint it's not advisable to release Full Trust Code (FTC).

Initial Hypothesis: So I often tend to write CSOM for custom provisioning and asset deployment.  I utilise the Tenant Admin API for provisioning site collections and then CSOM SharePoint in C# to provision and deploy my assets.

Possible Resolution:
PnP has a great library to provision assets to base you solution on, customising and building XMLfiles allows you to quickly build repeatable SharePoint solutions that are easily configurable.
Also InstantQuick has a solution called IQApp that is worth a look at.

Displaying SharePoint site pages in an iFrame

Posts on the iFraming issue Series:
IE11 not saving field data iframe Issue
Term Store/MMS musing when using iFrames (WebTaggingDialog)
Displaying SharEPoinnt Site Pages in iFrames (this post)

Problem:  I need to display a SharePoint 2013 document library within an iFrame of another SharePoint 2016 site.  SharePoint by default does not allow iFrame itself.

Hypothesis: IE and the other browser respect different rules for allowing iframing.  The iFraming rules are dictated by http headers output from SharePoint.  SP 2010 and MOSS did not have iFraming restrictions.  Furthermore, chrome and safari do not follow the X-Frame-Options http output but use CSP to control ancestor framing.

Resolution:
1.> Remove the SharePoint generated X-Frame-Options header generated by SharePoint - step 2 below.
2.> Add the custom X-Frame-Options http header to allow a specific domain to iframe the site (keep the attack surface as small as possible) - step 1 below.

3.> Add the Content-Secuirty-Policy frame-ancestors entry to ensure Chrome and the other browsers will allow framing as shown below.

Posts on the iFraming issue Series:
IE11 not saving field data iframe Issue
Term Store/MMS musing when using iFrames (WebTaggingDialog)
Displaying SharEPoinnt Site Pages in iFrames (this post)

Create a Lookup column from a list in a separate web

Problem:  I have a site collection (SPSite) with multiple sites (SPWeb) and I have a reference list in the SPSite (root web) that I want to use from multiple lists in the sites (SPWeb).

Resolution:  I recorded this short video that shows how to create a lookup site column in the site collection and then at the list within a sub web, you can associate the lookup.