Notes on Compliance in Office365 & SharePoint

Also see: Data Protection Using SharePoint

Data Loss Prevention (DLP)
Historically used for email to identify, monitor and protect data.  This is the next step on from email policies/IT Policies where sent around and signed with the hope users would behave and only send appropriate information along.   DLP ensure that sensitive information such as patents, financial information, Payment Card Industry Data Security Standard (PCI DSS) , personally identifiable information (PII), or intellectual property (IP) are accidentally shared with external parties.  DLP can inform users before they send email or open access to OneDrive or SharePoint document libraries that the information being shared violates company policy (as configured in the DLP template).
https://blogs.office.com/2013/10/28/office-365-compliance-controls-data-loss-prevention/
https://blogs.office.com/2014/08/27/search-sensitive-content-sharepoint-onedrive-documents/

Information Rights Management (IRM)
IRM-Protect document libraries prevents sensitive information being copied, forwarded, printed

eDiscovery
Find related content with SharePoint, Exchange & files shares to assit with litigation and determining info on a topic ata point in time.

Records Management
Manage a documents life-cycle, stop key docs being amended or edited.

Information management policies
Enforce compliance such as expired content, usage auditing & retention policies

Office 365
All data is encrypted on the servers (encryption at rest) and TLS/SSL on all communications.

Other
DocAve can enhance IRM and archieving & they have a SharePoint Monitoring and Policy Enforcement module.

Reference:
https://support.office.com/en-us/article/Use-Office-365-to-help-comply-with-legal-regulatory-and-organizational-compliance-requirements-ce773cec-2151-4d06-9a4e-2818613bd7e0

Debug SharePoint ClientContext using IE Developer tools

Overview:  I have a redirect that looks at my client context and performs a redirect.  Fiddler is a good option for viewing JS and debugging but as IE developer tools is not my normal tool for JS debugging, I thought I'd try use it to debug my issue.  I need to debugging a JavaScript redirect (302) using IE Developer Tools and breakpoints.

Steps:
1. Fn + F12 to get the IE Developer Tools
2. Enter the redirection url https://demo.dev/sites/demo/SitePages/Redirection.aspx in IE
3. As the page is running hit the "Esc" key to stop the browser (I need it to get the js files in the redirection page).

4. The Folder Icon mark as point 1 in red on the diagram is used to find the js file to debug.
5. Open the js file step 2 and step 3 in the red annotation above.
6. Set break point as shown below

7. A watch windows is provided. Variables are easy to follow.
8. When you refresh if the breakpoints are set, you will get stopped at the break points you previously setup in the steps above for subsequent runs.

Azure Virtual Machine Basics

Azure Virtual Machine Basics

Windows Azure Virtual Machines

• Azure charges per minute pro-rated
• Disks explained:
• C: OS disk
• D: Temp Cache such as Page File (won't be persisted in a disaster)
• F: ... Persist disk storage (Add disks as shown below)

• Access the VM by default RDP sessions and remote PS are allowed.  

• Need to add endpoints to allow other means of access e.g. 443 for https traffic.

• Availability Sets are used to to load balance with more than 1 VM
• Add a new F drive to the VM

Smoke Testing SharePoint using Selenium IDE

Overview:  In SharePoint we need to retest code often as we make incremental changes.  Basic smoke testing is useful in that is allows you a certain degree of confidence that a bug has not crept into your latest deployment.

Most projects have varying degrees of control to ensure bugs do not cause unexpected behaviour and on the more advanced practices is unit testing and coded UI tests.  The unit tests are tricky with all the new SharePoint development methods.  Jasmine is a JavaScript testing framework and check out the @SPDoctor (Bill) for basic testing information.  Unit testing SP is difficult as the revs end up test SP and not the changes.  A lot of the code is UI driven which is hard to unit test.  I have previously written about code UI testing, MTM as part of the MS test and continuous testing.  And part of this is Selenium WebDriver.  I've used it once on a large project and it was awesome.  Now as you go into production you normally will do some manual smoke testing to check the deployment.

This post looks at automating smoke test.

There are various tools for recording smoke test or you can do the manual eyeball approach favoured my most SP project still.  I have used Powershell (both to gen http requests and to control IE).  MTM is good but it requires buy into the whole MTM process.  Personally I like Selenium IDE for Firefox.  It offers recording, and the capturing functionality is miles ahead of anything else.  These Selenium generate test can be export and used with Visual Studio or build into TFS or on a project I used TeamCity to run automated continuous build and integration.

Note:  Selenium IDE is the recording piece and Selenium WebDriver is the heavy duty real testing integration part.

Get Started:
1. Download Selenium IDE (make sure you get the Selenium IDE, I'm using version 2.9.0) 
2. Understand the UI and capabilities (YouTube basic Selenium IDE videos are great)
3. Install Selenium IDE on Firefox as shown below:

4. Launch Selenium IDE using FireFox

5. Record and run Selenium test (2 minute short video providing a Selenium IDE recorded test against a SharePoint Online Team Site).

Another option is to control IE using PowerShell
PS> $ie=New-Object -com internetexplorer.application  #open ie
PS> $ie.Navigate("http://www.radimaging.co.uk/Pages/default.aspx")

SharePoint 2016 Preview - Min Role

Overview:  SharePoint 2016 Min Roles before the public betas.  Things are going to change and this only obviously applies to SP2016 Preview on-prem.

"Min Role is basically a topology assistance service"

Installation:  The services are pretty similar to SP2013 and most of the PowerShell from AutoSPInstaller will work with SP2016.  Excel services has been removed.  The difference in the UI install is the ability to have server install min roles for a multi-server.  Min roles are collections of services installed on a machine.  I'd lean towards installing full/custom roles and then converting the specific servers to the specific min roles.

Using the min roles, SharePoint can verify the roles are in compliance and can be used to managed the farm.  In effect you'd need all 4 roles to have all the services on a SharePoint farm and you shall need 2 instances of each for high availability.  So excluding SQL you shall need 8 VM's for a High Availability (HA) farm.  If you install extra services on a min role server, SharePoint timer job shall stop the service on a daily basis (not proven).

There are 4 server roles:

1. Web Front end,
2. Distributed Cache (also has a witness/quorum),
3. Application,
4. Search

Servers can be changed from Custom to specific min roles or the other way around. 

Note:  FULL High Availaibility
Min 2 times each role (extra quorum for dist cache roles) plus 2 more for Search HA, so 11 for core SP.  Plus SQL AOAG 3 servers, plus WCA/OOS 2 + SP workflow + K2/AvePoint + SSIS

SharePoint 2016 Preview (Public Beta 1) on-prem Notes

Disclaimer:  This notes I made from workshops, what I have heard and the Unity Conference in Amsterdam 12-14 Oct 2015.  A lot of this information is from the workshop with Neil Hodgkinson and Spencer Harare), this is my takeaway summary.

Notes for Sp2016:

• Same HW req as 2013.  Farm servers min still Mem 12-16 CPU x64 1x4 Disk, Disk 80GB
• Pre-reqs: Win 2012 R2, Win Mgmt Framework 3.0 gives us DSC, .NET 4.5.2 ..., DSC can be used to pre-bake the VM image.
• Same DB rules as recommended by MS, loosing dbs from 2013;  No new DBs, need SQL 2012 or 2014 (except project Server DB which is not part of SP).
• Need Win 2012 standard or higher, not web edition, also dev can support windows 10
• Still no support for VMWare dynamic memory
• End-point encryption for SMTP

 - Upgrades and Patching

• No Foundation edition, SP2013 found to Sp2016 Server
• Path SP2013 > SP2016
• SC must be in 15 mode to upgrade
• Service Apps need to go SP2010 > SP2013 > SP2016
• SP2010 to SP2016 need to go to 2013 RTM baseline
• Changed patching, smaller packages and fewer restarts
• PSConfig not locking farm, can run multiple psconfigs and lower/zero down time patching (with HA farms)

 - Roles & Services

• Consider moving low impact services onto the traditional WFE role, keep the long running/batch processing (Crawl, search, MTS,  et al) on the app servers.
• WFE (Access services, SSS, Subscription Services, UPS)
• Distributed Cache has a quorum so need 3 not 2 for NA.
• Health Analyser rule for min role enforcement:  Puts min role in the correct state.
• Min Role does not manage the search topology
• Watch, switching min roles as index would be lost unless it is replicates (2 instances of each index)
• Services in Farm overrides the starting of services in the Min Roles,  so can never start "Request Management" in the "Services in Farm" but still use "Distributed Cache" min role.
• Can always switch min roles "Convert server role" or create custom roles (watch as needs multiple instances to keep running and index could disappear).

 - Key Thresholds for 2016:

• CDB sizing
• 100K SC per CDB
• Max file size 10GB
• Search index up to 50 million items

  - User Profile Sync:

• UPS Sync (FIM) is not Microsoft Identitiy Management (MIM)
• 2 modes: Active Directory Import (light weight, not useful for most large enterprise clients, e.g. Can't import pics or use BCS) or MIM 2016
• AD Import: faster than 2013, can only use AD, no profile picture. 
• MIM 2016 was FIM - Standalone product, only using the sync engine part for SharePoint (free if only use this service, does need Win 2012 and SQL Server licence)
• Using MIM management agent map AD properties to SP user Profile properties
• Syncing is driven by MIM not by SharePoint (UPS sync)

- What's New:

• Post to yammer from SP2016 doc library
• Improved integration
• Image and Video Preview (changed)
• Doc Lib accessibility (improved keyboard short cuts, VI user experience improved)
• SC creation faster on SP Site template using SPSite.Copy
• Project Server is part of the SP binaries/install, project server using it's own project db and adds 4/5 tables to the content database.  Project Server affects 3 DB (Project db, content db and config db)
• Save and share email attachments in SP2016

- Release Dates

• Preview = Beta 1 Aug 2015
• Beta 2 RC  = +- Nov/Dec 2015
• RTM Q1/Q2 2016

Hybrid Search 2016 Notes

Hybrid Search SP2016 (Also applies to SP2013):  Mixing on-prem and SPO results

• Search can add all crawls into a single index within SharePoint online (historically we have had to use Search Federation to try combine result sets).  So historically we use search federation whereby there are multiple indexes that are the shown on a single page this approach for search result federation is refereed to as "Search-time merging".

Federated Search provide by MOSS, SP 2010 and SP 2013

• The Index is held on SPO.  The new model is refereed to as "Index-time merging".

Single result set from multiple farms (joined the indexes into an SPO index). vNext hybrid search.

• Crawls done on SP2016, 2013 and maybe 2010 are pushed into an Azure queue which in turn is combined onto the SPO index (I believe the Index is encrypted as rest in SPO)
• Dir Sync is required between on prem AD and Azure AD 

UnityConnect Conference 2015 Amsterdam Search session - Architecture of Hybrid Search