Tuesday, 27 August 2013

VMware PowerCLI issue using templates to create VMs depending on AD forest level

Problem:  Within my DTAP environment I use VMware vShere v5.01.  Using PowerCLI and a Windows 2008 R2 syspreped template against a Windows 2008 R2 forest level DC, I was creating my VM's in CI and test.  When I got to UAT which needs to join the VM's to the existing enterprise domain, it fails.  The existing domain controller runs at Windows 2003 level.

Work Around: I can manually add login to the VM's and add them to the domain so I know the credentials are correct.  Or manually use PowerCli to join the VM's to the domain.

Resolution: Adjusted the sysprepeped template/OSCustomisation template - network domain user string and then could add the new machine/VM to the domain.

More Info:

Thursday, 22 August 2013

SSRS deployment for a developer machine

Problem:  I need to use SSRS 2012 in SharePoint mode on a single SharePoint 2013server.

Background:  On my developer VM I has SQL 2012 and SP2010.  SQL was installed with the "Shared Features":
1) Reporting Services - SharePoint &
2) Reporting Services Add-in for SharePoint products.
I installed the services and SSA and was successful in setting up SSRS.  The problem is when I went to UAT which consisted of WFE1, WFE 2 and SQL Server that will use SSRS.

Determine the Components you will need.  In my case this is:
SQL Server 2012 SP1SQL Server 2012 SP1SharePoint 2010.

Single server SharePoint 2010 and 2013 farms and SQL Server are easy to setup as all the components are installed as part of a Full SQL installation.

Once SharePoint and SQL 2012 SP1 is installed you will see the SQL reporting Services on the farm.  I needed to setup an SSRS SSA:
PS> Install-SPRSService
PS> Install-SPRSServiceProxy
PS> get-spserviceinstance -all |where {$_.TypeName -like "SQL Server Reporting*"} | Start-SPServiceInstance

Create a SQL Server Reporting Services Service Application within Central Admin

Other Info:
Install Reporting Services SharePoint Mode as a Single Server Farm

Summary:  AutoSPInstall proved too difficult for me to implement SSRS on SP2010.  There are a lot of permutations and writing a plug-in for autoinstaller does not warrant the effort.  Farm deployments get complex and it probably best to figure out you own scenario and automate from there.

See my blog post on Installing SSRS 2012 in SharePoint Mode on SP2010 as a starting point.

SharePoint BLOB storage for Dummies

Overview: BLOB Storage in SP2007 SP used EBS, SP2010 provided Remote Blob Storage (RBS) capability and SP2010 was backward compatible with EBS, SP2013 does not support EBS.  A common problem with using RBS on SharePoint is orphaned blobs (we need to keep blobs that are outdated by version no or deleted in case you need to do a restore).  This can lead to orphaned blobs – and you need to periodically go and clean-up the blobs.   RBS is enabled at a content database level whereas EBS is per farm. 3rd party RBS providers only need to be installed and configured on the WFE’s not the app servers.

FileStream (local) RBS:  Microsoft SQL server provides Filestream technology which effectively allows SQL to move blobs based on size to remote storage (so no longer stored in the SQL database).  You need to manage backup and recovery yourself and this tends to lead to unused blobs and orphaned blobs.  It also adds overhead to SQL's resources.  The diagram below is how I see Filestream working, the entire SP record is pushed to SQL, SQL adds the stub to the Content database and the blob is moved onto a network file share.

Why Remote RBS?
Microsoft does not have any Remote/externalised blob providers.  All data is stored in your SQL relational databases and not efficient at storing large BLOBs.

With large BLOBs in your content databases:
  • the size becomes harder to manage (think backup and recovery times);
  • database I/O is much more intensive when storing BLOBs; and
  • databases use expensive storage.
Enter the 3rd Party RBS providers : DocAve (AvePoint) has a nice BLOB provider, it can work in real time or retrospectively, backups are cleanly handles and the retention policy is great at cleaning up the old blobs.  This diagram shows how AvePoint provider stores the blob, instead of letting SQL handle the entire record storage, the provider at a SharePoint level, 1st will add the blob to external storage, the location/stub info is sent back to the SP server.  SP then sends the record to SQL including the stub location.  SQL does a lot less work.

Other vendors include: Metalogix, Stealth Software, StorageEdge from (Alachisoft), SimpleStor

Note: RBS does not change the Content Database size restrictions, the BLOBs are still added to the content DB size calc. The 200GB/4TB limit still use the external BLOB size as the records meta data reports the real/external file size, not the 1 kb stub size.

The thinking behind the 200GB/4TB content DB limit is chosen/used because of database density and not database size, i.e. the average number of objects (docs, items, lists etc…) at 200GB/4TB is approaching a point where indexes and query performance suffer. So externalising does not mean that the density is dropped as the stub is still an object that’s in the DB that’s indexed.  However, using blobs suggest there on average are fewer objects compared to another farm that would have smaller average sized objects within it's content db.  My head hurts just writing this.

The advantage of RBS on your farm is that fragmentation is reduced, the cost of storage is cheaper and performance/stability at larger concurrency is improved. Additionally, DR times are also greatly improved.

High Availability: Blobs are not HA, if the blob store/fileshare goes down you loose the blobs.  DFS can be used to replicate the blobs.  Note: Your blob storage pointers in the SQL Content databases will point to the DFS share.  The image below shows you how to setup a DFS share.  Tip: Use domain based DFS.

Windows Distribute File Share (DFS) provides a logical share for adding files/blobs.  The blobs/files are replicate to all physical disks.  If 1 goes down you don't notice and service continues.

More Info:

Sunday, 18 August 2013

Installing SSRS 2012 on SP2013 within a farm

Overview: SSRS in SharePoint mode had a big architecture change before this exact combination of software.  Since SP2010 and SQL 2012 the architecture has been improved to use SharePoint Service Applications. 

Below is a diagram to explain how SSRS works when using SQL Server 2012 SP1 and SharePoint 2013 on-prem.

The high level steps I followed are:
1.> Install SSRS - SharePoint mode using SQL Server 2012 SP1
2.> Install the SQL Server SharePoint Add-in for SSRS feature on 1 or more WFE/App SharePoint servers
3.> On the WFE server ensure the SQL Server Reporting Services Service status is "Online" as show below:
PS> Install-SPRSService
PS> Install-SPRSServiceProxy
PS> get-spserviceinstance -all |where {$_.TypeName -like "SQL Server Reporting*"} | Start-SPServiceInstance
4.> Create a SQL Server Reporting Services Service Application within Central Admin:
5.> Activate PowerView Site Collection Feature. Verify the site collection feature "Power View Integration Feature" is activated.

6.> Verify installation:

  • CA > Manage Servers in the farm > Verify the Service "SQL Server Reporting Services Service"is on the WFE/App server.
  • CA > Manage Services on Server > Select the Server where the SSRS-add-in has been installed and > Verify the "SQL Server Reporting Services Service" service is "Started".
  • Using the Discovery Reports discussed above verify that the SQL Server with SSRS (SP mode is the same SQL version as the web/app servers in the farm (all my servers are version 11.1.3000.0 - SQL 2012 SP1).

  • 7.> Open the Site Collection and create a new "Report Library" as shown below:

    8.> Navigate to the reports library e.g. SSRS Demo.  Click the "Library" tab and the "Library Settings" Ribbon button.  Click "Advanced Settings" and verify the setting "Allow management of content types?" is set to "Yes". 

    9.> Add 3 SSRS reporting content types to the reporting library.  Navigate to the reports library e.g. SSRS Demo. Click the "Library" tab and the "Library Settings" Ribbon button. Click "Add from existing site content types", then add the 3 Report Content Types shown below (if you don't see this check your install steps especially versions of SQL):

    Dodgy Note: Depending on the order that the Site Collection and adding the SSRS service is installed in you may need to activate the following features: on the Site Collection go > Site Settings > Site collection features > Ensure the "Report Server Integration Feature" is enabled.  If you don't see the SSRS reporting Content Types, check out this post.

    10.> Add reports Navigate back to the library and ensure you have the options to add Reports ect.  Upload or create reports and verify SSRS in SharePoint mode is working.

    Note: For more detailed steps look at my series on SharePoint 2010 SSRS using SQL 2012.  As mentioned the architecture is the same so and the steps work on SP2010 and SP2013.  The 1st post is here.

    Part 3 Update (This Post)  - Installing SSRS 2012 SP1 onto a SP2013 Farm

    Updated 01 May 2014: Here is a diagram to show the SSRS components in diagram form so you can understand where the pieces sit.

    More Info:


    Problem:  Installed SSRS on 2 App/WFE servers, PS working but the Servers in farm and services not seeing the SSRS Services are not displaying in CA.

    Resolution:  Installed the SSRS components on the CA machine.

    Problem: When using PowerGui to create the 'Report Library' that will host my SSRS reports, I can't create the library, while debugging I notice that when i look at the current web I can't retrieve the available 'ListTemplates' whichis returning "$null", I get the following error message on the property: "This operation failed because the QueryInterface call on the COM component for the interface with IID '{BDEADF28-C265-11D0-BCED-00A0C90AB50F}' failed due to the following error: Bad variable type"

    Resolution: My work around was to run the Powershell code out of the native Windows PowerShell cmd prompt.  I think PowerGui is the issue.

    Add SSIS to SQL install

    Problem:  I installed a dedicated database server for SSRS using SQL Server 2012 but I did not add the SQL Server Integration Services (SSIS). 

    Steps to Add SSIS:
    1.> Verify SSIS is missing from the Database Servers.    Tip: SSIS is 1 per machine, not instance based.

    2.> Run the installation binaries/SQL Server Installation Centre and follow the instructions below


    Tuesday, 13 August 2013

    AutoSPInstaller configuration for SharePoint 2013 Search

    Overview:  AutoSPInstaller is a great tool for building SharePoint 2010 ans 2013 farms.  One section of AutoSPInstaller relates to installing SharePoint 2013 search.  This post examines how to configure the xml that drives AutoSPInstaller and supplemental Ps to build the Search farm you require.
    Below is an example of a search xml config that will setup Search with full redunancy on 2 servers in the SP farm.  This design provides High Availaibility (HA).  MS recommend keeping each index partition under 10 million items.  So in this scenario if you follow MS recommendations you are limited to having 10 million items in your index (hardware, moving outher roles to more servers, usage of search to query all affect how quickly results will be returned so with tuning you can get your indexes to be larger and provide a good end user experience).

    Tip: Create a seperate SQL aliase fo your 4 Search databases, this will alow you to move the dbs to a new instance or at 40mill documents MS recommend you consider a dedicate search farm.  The SQL Alaise gives you room to adapt.

    Tip: Group your search roles onto servers such as:
    • Index & Query Processing
    • Analytics & Content Processing
    • Crawl, Content processing & Search Admin 
    Summary:  As of version 3.87 of AutoSPInstaller for an SP2013 installation, you only have 1 index partition.  To create additional partition indexes use PowerShell.  The example below is use the PS code given to me by Joseph Saad.


    $ssa = Get-SPEnterpriseSearchServiceApplication #Assuming you have a single Service application
    $active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
    $clone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone –SearchTopology $active

    New-SPEnterpriseSearchIndexComponent -SearchTopology $clone -SearchServiceInstance $WFE2 -IndexPartition 1
    New-SPEnterpriseSearchIndexComponent -SearchTopology $clone -SearchServiceInstance $WFE4 -IndexPartition 1

    Set-SPEnterpriseSearchTopology -Identity $clone

    More Info:


     4 Server Search farm created using AutoSPInstaller.  AutoSPInstaller version 3.87 only supports creating a farm using one partion index as shown below.
    To add "Partion Index 1" to the farm using WFE1 and WFE2 to hold the index components use the following PowerShell:
    $ssa = Get-SPEnterpriseSearchServiceApplication #Assuming you have a single Service application
    $active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
    $clone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone –SearchTopology $active
    New-SPEnterpriseSearchIndexComponent -SearchTopology $clone -SearchServiceInstance "WFE1" -IndexPartition 1
    New-SPEnterpriseSearchIndexComponent -SearchTopology $clone -SearchServiceInstance "WFE3" -IndexPartition 1
    Set-SPEnterpriseSearchTopology -Identity $clone
    Results in a farm that can handle up to 20 million documents due to the partioned index.

     Another Example:

    Sunday, 11 August 2013

    SQL Helper

    Problem: I need to to give a user permission to run all stored procs on a specific database.

    This is not relevant to SharePoint but I had to look up how to give windows users or groups access to run all stored procs in a database.  User will still need the underlying permissions to work on the database.  I create the "sp_executor" role for executing all stored procs on the db.

    use PBTransDB
    CREATE ROLE sp_executor
    GRANT EXECUTE TO sp_executor
    CREATE USER pbeck FOR LOGIN "demo\pbeck"
    EXEC sp_addrolemember 'sp_executor', 'pbeck'


    Thursday, 8 August 2013

    Stretched farms, Geo-replication and options

    Stretched farm: A stretch farm is where you put a single farm into 2 data centres.  This provides redunancy across locations improving HA and DR.   Generally the 2 data centres will be located relatively close to one another geographically as the ping time needs to be less than 1 ms from any WFE to the primaray SQL Server database and at least 1 gigabit per second bandwidth.  hospitals with multiple sites / data centres that have good network connectivity and are located close together is a perfect example.

    Geo-replication: SharePoint 2013 like previous versions does not have Geo-replication built into the product.  You can use PowerShell to provide this functionality.  A more achievable approach is to use a 3rd party tool such as DocAve to achieve and maintain Geo-replication.
    SharePoint 2013 Geo-Replication
    Another option, without the hassel to GEO-Replication is to use Riverbed devices distributed around your corprate network.  This decreates network requirments and improves the end user experience and does not involve syncronising data to multiple farms.  Riverbed's website.

    Finding Correlation Errors on a SP2013 farm

    Background: SP 2013 has rich and expansive logging/tracing capabilities.  Logging is done via the Unified Log Service (ULS).  This will add logs to the tracelogs (often refered to as the ULS logs or ULS trace logs or ULS, it doesn't matter except you need to understand the ULS service is not only the trace log) and the Windows Event Viewer.  Anything logged in the Event Viewer log will also be in the ULS trace logs.
    It is worth check how your logging is setup on your farm.  I change my default location for my ULS trace logs.  Change the logging so it matches your farms requirements.

    On a small farm, it's normally pretty easy to take a Correlation Id / the unique GUID generate for the SharePoint request, open the trace log using notepad and find the error.  The default is to create a trace log every 30 minutes, these log files have a lot of data in them on busy production farms, and as you may have a large farm you also have multiple logs to check. I use Microsofts'd unsupported ULSViewer to look at all my logs regardless of farm size.  You can trace the logs in a live format and then filter out what you need.  Another option is to open existing errors to get historical issues.  If you know the datetime and server where the error occured, you open the correct log file (it is labled with a datetimestamp) and then either filter for the correlationId or look around the time the error occured.

    Lastly, timer jobs ship entries from the ULS logs into the SharePoint Logging Database (SP_UsageandHealth).  You can directly query the SP_UsageandHealth database using T-SQL.

    Tracing Correlation Errors on a SharePoint 2013 Farm.
    User passes you a correlation Id and the date/time when the error occured, find the apprioate ULS trace log.  Open the log using ULSViewer and filter for the CorrelationId.  If you can reproduce the bug, you have the developer dashboard that can be turned on (performane penalty) selectively, their is a new SP2013 tab "ULS" this will show you the ULS trace snippet relating to this request.

    On a big farm you may want to1st find out which server in the farm had the error:
    Merge-SPLogFile -Path ".\error.log" -Correlation "5ca5555c-8555-4555-555b-f555af4d5555"
    Tip: Be aware this is a heavy process, so restrict which logs you will merge.
    Use ULSViewer to find the correlationId and review the logs.

    Use IT tools or Fiddler to examine the http response from SharePoint to get the correlation Id, this is the SPRequestGUID (assuming it is not show on the error message).


    More Info:

    Tobias Zimmergren has a great post on working with Correlation Id's.



    List of the ULS viewers


    SP2013 AutoSPInstaller Error Correction

    This post contains alist of issues I have had to correct on multiple farm builds. 

    Problem:  After a remote offline install the main CA box shows an upgrade is required.  CA > Manage servers in this farm :Upgrade Required
    Resolution: Run cmd > C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\BIN>PSConfig.exe -cmd upgrade -inplace b2b -force -cmd applicationcontent -install -cmd installfeatures

    Problem: PU or CU is not getting installed.
    Resolution: Ensure the .exe are extracted out e.g. ubersrv2013-kb2817414-fullfile-x64-glb.exe this is the June 2013 CU.  Ensure the March 2013 PU is extracted and in the same folder ubersrvsp2013-kb2767999-fullfile-x64-glb.exe.  Ensure they are placed in the correct location for me it is ..\2013\Updates\...  CU are cumulative so you only need the PU and then the latest CU.  Don't rename the CU - this caught me out.

    Problem: I kept receiving the error: 17303 Extracted file error for the June CU (ubersrv2013-kb2817414-fullfile-x64-glb.exe).
    Resolution:  Looks like the download was corrupt, re downloaded the CU and extracted the contents which fixed the issue.

    Problem: I built a 4 SP server 2013 farm.  I had 2 Search nodes setup via AutoSPInstaller and added another 2 search nodes (index partitions with it's corresponding replicated index).  After a reboot I got into the following state and could not get WFE2 to join the search farm.
    Using CA and looking at the Search topolgy on WFE2, I can see the error "Administrative status  The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel." and "Unable to retrieve topolgy component health states. This may be because the admin component is not up and running.".
    Resolution: Check the routing, my CA box is WFE, I also had a CA role on WFE2, while playing with the farm I added a host entry to work with CA on WFE2.  The issue is that this is the same host entry as where my Admin component sits.  Removing the host entry on WFE2 fixed the status of the index and query components.

    Monday, 5 August 2013

    Installing OWA 2013

    Series of Posts for Installing and Configuring Office Web Apps (OWA) 2013 to work with SharePoint 2013.

    Overview: Manually Install OWA 2013 on Windows 2008 R2 SP1.

    1. Download and install .Net 4.5 (dotNetFx45_Full_setup.exe)
    2. PowerShell 3 (Windows Management Framework 3.0)  Windows Server 2008 R2 SP1 (64-bit versions: Windows6.1-KB2506143-x64.msu)
    3. Install KB2670838 (Windows6.1-KB2670838-x64.msu)
    4. Run PS>Import-Module ServerManager
      PS > Add-WindowsFeature Web-Server,Web-WebServer,Web-Common-Http,Web-Static-Content,Web-App-Dev,Web-Asp-Net,Web-Net-Ext,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Includes,Web-Security,Web-Windows-Auth,Web-Filtering,Web-Stat-Compression,Web-Dyn-Compression,Web-Mgmt-Console,Ink-Handwriting,IH-Ink-Support
    5. Install OWA 2013 (wacserver.img)


    Load balancing in SharePoint

    Background: SharePoint 2010 required sticky sessions.  In effect when the 1st request from a user comes into SharePoint 2010 to the load balancer, the request will be directed to a particular SharPoint WFE server that will then serve the subsequent requests (the user now has session affinity/sticky session).

    SharePoint 2013 has distributed cache so you no longer need to use sticky sessions, the load balancer merely needs to direct the user to any server (preferably the quietiest).

    Possible options for Load Balancers in SharePoint:
    F5/BIG-IP (Level 2, 3, 4 &7)
    KEMP (Level 4 & 7)
    Barracuda (Level 4)
    Cisco - I don't think Cisco build dedicate load balancers and more however "Cisco IOS-based router product has load balancing capabilities".
    Not familiar with:
    http://www.a10networks.com/resources/files/DG_MS-SharePoint2013.pdf - NLB and SSL termination.

    Key features needed off a Load Balancer:
    Distributed Denial of Service (DDoS) attack protection
    Http compression
    SSL acceleration/ and termination
    Persistance/Sticky sessions
    HA via distribution of requests and verifying that SP is running.