Thursday 19 March 2015

Identity Providers for SharePoint

Overview:  I have worked with and evaluated a couple of Services and Federation Server products.  Here is an old pot of setting up claims, at the bottom I have some thoughts on different services/server products.
Background: SAML and WS-Federation protocols are standard Single Sign-On protocols, the following version exist:
  • SAML 1.0, SAML 1.1, SAML 2.0
  • WS-Federation
Security Assertion Markup Language (SAML) is an XML-based protocol for exchanging authentication and authorization data between security domains.
SAML enables web-based authentication scenarios including cross-domain single sign-on (SSO).  SAML is a token representing a principal that normally represents a user but can represent an app.
  
Other terms to understand:
  • Identity provider (IdP) think ADFS/Azure ACS,
  • Service provider (SP) is the SAML consumer in our context this is SharePoint but this can be an MVC app.
  • Realm
OOTB SP2010 and SP2013 support SAML1.1 not SAML2.0, you can write custom code or use a Federation Server like ADFS to convert the SAML2.0 so it will work with SP.
Identity Provider (IdP) Products:
  1. Microsoft ADFS
  2. Ping Federate
  3. ThinkTexture Identity Server
  4. CA-SiteMinder
  5. IBM Tivoli (CAM)
  6. Oracle Access Manager
  7. ComponentSpace
  8. Shibboleth
  9. RSA Federated Identity Manager
  10. Entrust GetAccess
 IdP Services:
  1. Azure Active Directory
  2. LiveId
  3. Google
  4. Facebook
  5. LinkedIn
  6. Yahoo
This list is in no way exhaustive, pls post if you feel I am missing any providers.

Friday 13 March 2015

Capturing NFRs for SharePoint

Problem: Gathering Non Functional Requirements (NFRs) are always a tricky situation in IT projects.  This is because it is always difficult to estimate how the system will be used before you build it.  I often get business users stating extreme NFRs in the attempt to negotiate or show how world class they are (I generally think the opposite when hearing unreasonable NFR's). 

An example is a CIO at a fairly small NGO telling me the on-prem. SP 2010 infrastructure needs to be up all the time so an SLA of 99.99999.  This equates to 3.2 seconds downtime a year.  In reality, higher SLA's start to cost a lot of money.  SP2013 and SQL 2012 introduce Always On Availability Groups (AOAG) which helps improve SLA uptime but this costs in licensing infrastructure and management.  I need redundancy and the ability to deal with performance issues, so the smallest possible farm consists to 6 server, 2 for each layer in SP namely: WFE, App and SQL.

Here is an old post of SP2010 SLA's but still relevant today.

The key is gather you NFR's and ensure all your usage/applications on the production farm meet expected behaviours.  I have a checklist below.  Going thru the Microsoft's SP Boundaries, Limits and Thresholds document shall help highlight any issues.

The high level items I cover include the following topics:
  • Availability
  • Capacity
  • Compatibility (Browser, device, mobile)
  • Concurrency
  • Performance
  • Disaster Recovery (RTO, RPO)
  • Scalability
  • Search
  • Security
  • SLA

Capacity Example

Item
Day 1
Year 1
Year 3
Year 5
Site Collections
10
100
250
400
Database Size in GB
> than 1GB
490 GB
1220 GB
1960 GB
Search Index Size in GB
> than 1GB
120 GB
310 GB
490 GB
No of Content Databases
1
1
4
8
No of Search Items
10,000
10 Million
25 Million
40 Million
No of Index Partitions
1
1
3
4


Item
Day 1
Year 1
Year 2
Year 3
Number of Users
1,000
50,000
80,000
130,000

*Also calculate peak and average concurrency numbers

Average concurrency, for 20,000 users, the assumption is that 10% (2,000) users will be actively using the solution at the same time, and that 1% of the total user base (200) users will be actively making requests.  For for performance testing you are looking to handle 200 users without delays and a page response time of under 5 seconds.  Based on the simple guideline I've always used from Microsoft.

Peak concurrency depends on your situation for example the NFL playoffs game schedule in the when announced is not the simple 4 times the average concurrency tha would be suitable for most internal business applications.  Although this example may be considered a load spike rather than a peak concurrency.  

It also worth doing a usage distribution pattern for your users experience, so 80% may be light users, login, read 10 pages in your site and perform a single search with 1 minute gaps between interactions (wait times).  the remaining 20% perform a login, upload a 100kb document, view 10 pages and perform 2 searches.

RPO & RTO:

RPO - Max amount of lost data (in time)
RTO - Max time lost (rebuild farm and get the latest backups restored) to make the system operational again.   

SQL Server Sizing:
Option 1: work out the rows and bytes for storage and multiple by the number of rows and then add the tables together to get the size.
Option 2: Assume 100 bytes for each row, count the number of rows and get the storage requirements.

More Info:
https://technet.microsoft.com/en-us/library/ff758647.aspx

Monday 2 February 2015

Encrypting Content databases

Overview: TDE is Transparent Data Encryption, where you can encrypt your "data a rest", this encrypts the SQL mdf and ldf files.  Few enterprises require TDE for content database but if your customer has specific enterprise security requirements (Encryption at Rest for High Confidential data) or compliance requirements such as SOX, HIPAA, or Payment Card Industry Data Security Standard (PCI DSS) TDE may be an easy win.
Notes:
  • TDE is only available from SQL 2008, 2012 and 2014 Server Enterprise Edition.
  • SP Blobs are stored outside of mdf so they are not encrypted by TDE.
  • Only Content databases can be encrypted (not verified).
  • Search indexes are obviously not encrypted by TDE.
  • Encrypting the Connections to SQL or IPsec is needed to encrypt data between SP and SQL, not covered by TDE).  Nor are any call to web services or data in transit, use SSL.
  • TempDB is encrypted even if only 1 db is using TDE.
  • Applies to SP2013 On-prem. farms only.
  • I believe O365 uses BitLocker.
  • Vormetric and also offer encryption at Rest on SQL and other databases.

More Info:
Storage and SQL Server capacity planning and configuration (SharePoint Server 2013)
http://www.slideshare.net/michaeltnoel/transparent-data-encryption-for-sharepoint-content-databases
http://www.vormetric.com/search-results?query=SharePoint
http://web.townsendsecurity.com/bid/64783/4-Ways-to-Encrypt-Data-in-Microsoft-SQL-Server


Sunday 25 January 2015

Auditing in SharePoint 2013

Overview: SharePoint provides excellent logging capabilities, to retrieve the auditing logs Site Settings > Site Collection Administration > Audit log reports.
Notes:
  • By default auditing is enabled in SharePoint.  PB: I think this statement if false, all the farms I review are not logging information in the audit logs.
  • Auditing is done at a Site Collection level.
  • Audit logs are kept for 30 days by default and can be change via the UI in the site collection and the clean up is controlled by CA.
  • Audit logs are stored within the content database, so watch the size of auditing logs.  They can take up considerable space in the content database so don't just audit everything and keep the logs endlessly.
  • Permissions changes, check-in/check-out, search queries, edits, document views (not SPO), ... can be audited.
  • Various reports can be downloaded into excel for slice and dice such as the Security settings audit log report.
  • Each logged event roughly takes up 2k. Calculating content database storage reqs:

Audit logs can be shipped to a central storage area and removed from the Content Database, this is ensential for large CDB's that require full auditing and performance is suffering.  AvePoint and Metalogix offer tools as part of their products that perform the audit log storage & removal from the CDB.  Also see Varonis.

References:
https://support.office.microsoft.com/en-us/article/View-audit-log-reports-b37c5869-1b47-4a82-a30d-ea20070fe527?CorrelationId=9139de6c-b33b-45c1-9cc2-d3958a88eab3&ui=en-US&rs=en-001&ad=US
http://sureshpydi.blogspot.co.uk/2013/05/audit-log-reports-in-sharepoint-2013.html
http://sharepoint-works.blogspot.co.uk/2013/07/audit-logging-in-sharepoint-2013.html
Centralised Auditing Product:
LepideAuditor Suite – SharePoint
http://www.lepide.com/sharepoint-audit/
LogBinder SP
https://www.ultimatewindowssecurity.com/sharepoint/logbindersp/Default.aspx

Sunday 18 January 2015

What are SharePoints Competitor Products

Overview:  SharePoint does not stack up against any single COTS product easily due to the size and functionality offered.  This post is my opinion, and I am not an expert in any of the competing products.

Updated: 2018/06/13

A few years back I wrote a post that is still somewhat relevant form SP 2010 competitors.

This post aims to list competitor products in a broad sense and tries to highlight the areas that where the product and SharePoint perform a similar function.
  • Atlassian Confluence
  • Alfresco
  • Documentum
  • eRooms
  • Lotus Notes
  • EMC’s Center Stage
  • Google docs
  • Slack - Very similar to Teams from MS
  • ClickUp - project management but has chat, stores docs, calendars, time tracking, links, similar to MS Teams and Slack but from a project management focus. Update: 25/06/2022
Beehive from Oracle lines up to Lync, Exchange/Outlook and SharePoint
OpenText, lines up some of SP's functionality such as blogs, Wikis and document collaboration.
Search competitors:
  • Endeca (Oracle)
  • Autonomy (HP)
  • Google Search Appliance or Google mini
  • Coveo
  • Solr (check this out)
CMS Competitors
  • Sitecore
  • Umbraco
  • Drupal
OneDrive: Dropbox, Drop, Google docs
Skype: Zoom
Planner: Trello

More Info:
http://www.extended-content.com/wp-content/uploads/2013/05/Gartner-Magic-Quadrant-For-Enterprise-Search.pdf
If anyone has further information, please reply as this is not an exhaustive set of lists.  My experience with rival products is relatively limited.


Sunday 11 January 2015

Minification Tooling

Overview: Minification is the process of combining multiple css or js files, removing whitespaces and comments to improve web site performance.


Tools:
YUI Compressor(Yahoo)
Web Essentials(Microsoft)
Mavention(Microsoft)
Grunt
jscompress.com/
Google Code Compress (Google)


I'd always go for 1 of the Microsoft tools: Web Essentials or Mavention as the plug into Visual Studio, as a SharePoint guy this would be my preferred option.  Both the MS tools appear to use the same engine as the compression appears identical, work out to roughly 60% on both CSS and JS compression.

Thursday 1 January 2015

UI and Framework Notes

Assumed/Common Framework:
jQuery
Knockout vs Angular vs Backbone -
Modenrizr.js - detect browser capability
underscore - utility functions for arrays and collections.  Feels like LINQ queries
toastr - toast notifications

Data Access Modules:
1.> data.js - manage OData client access for JavaScript (supports caching & batch operations)
2.> Breeze.js - entity modelling and querying (ORM for JS.  Backend transformed to a data object i.e. 3 related customer tables becomes the customer object). Looks like LINQ queries.  Data.js is inside of breeze.

Presentation / App Pattern:
Knockout.js - (MVVM), declarative binding and dependency tracking.  Matches data to front end, then track changes and update.   Easy to use and understand.
Angular.js - MVC data binding,routing (url controls pages).
Compare: Knockout (good for SPAs) for more complex application whereas knockout is more light weight.

Responsive Web Design (RWD):
LESS - Minimises css, makes it simple to read not repeating attributes.
Bootstrap.js / Zurb Foundation 4 - Media queries to style for the device based on 12 column layouts.
Framework for cross cutting concerns (logging, caching,)....

Progressive Web Apps (PWA):
"Progressive Web Apps allow what you build to be installed by anyone, anywhere, on any device with a single codebase"

Flutter:  
Created by Google that uses DART .  Build once and HTML deploy for browser UI needs to be responsive.  Separate deployable code for Android and iOS, so PWA compliant.

PWA/RWD UI 3 break points:
  • Desktop: 1024+ 
  • Tablet: 600-1023
  • Mobile: 320-599

Useful JS modules:
JS Mapping module: