Tuesday, 28 September 2010

SP2010 Connections Conference in The Hague - Day 1

Keynote by Steve Fox was good but felt too sales orientated.  It's always hard to deliver a keynote at a  technical conference with the variety of attendees.  So overall not the worst and it covered a lot from previous conferences so not much new information outside of prepare for Azure with SharePoint.  Also SP1 for SharePoint 2010 will be available sometime in 2011 Q1.

Andrew Connell - AC is always good value however today he had a problem with his laptop/VM and the demos didn't work, still a good presentation on building RIA Rich Internet Applications or as the slide read "Rich Business Applications".  Looked at 2 demos for building RIAs using Silverlight client object model vs Restful web services (*.svc), nothing ground breaking but useful if you are new to using the Client Object model.
  • SharePoint apparently in it's page headers returns how busy the SharePoint server is with a number between 1-10, 10 being very busy.  You can see this in fiddler.
  • Anonymous user sites should generally use RESTful WS rather than Client OM/Silverlight as you need authentication. 
Ted Pattison's Extending the Visual Studio 2010 SharePoint tools was a great session.  Forget T4 templates.  Extensibility projects (VSIX) uses .NET4.0 and is based on MEF.  It's pretty straight forward and I can't wait to build some templates for SP2010.

Michael Noel's session on on Disaster Recovery & High Availability cover load balancing (use sticky sessions, Hardware is better than Sw load balancing), search options and improvements of MOSS search, SQL clustering and mirroring, use SQL alias's for pointing your farm at the db, common farm architectures, SP2010 and virtualisation (not ideal for SQL server), 3types of mirroring, Split content db's but not too small, Backup options and i need to review MS's DPM product.

Went to Andrew Connell's LINQ to SharePoint 201 Best practices.  Key takeaway is SPMetal only works on native SP Foundation types, M+LINQ to SharEPoint can be extended as it is implemented using partial classes, the iCustomise interface helps extend SPMEtal for more complex types such as Html fields and Metadata.

See Razvan-Petru Radu conference notes

Monday, 27 September 2010

SharePoint 2010 BCS workshop with Scot Hillier

Overview:  attended a workshop on Business Connectivity Services (BCS) given by Scot Hillier in Den Haag (The Hague).  I have used BDC in MOSS and the BCS in SP2010  briefly.  I can honestly say I am now a BCS convert.  I know BCS was an improvement on BDC but looking at the tooling and going building advanced BCS solutions is really powerful and covers a lot more that I understood or had seen in a previous demo.

  • Office can use BCS - pretty obvious but I didn't realise the client application goes straight from the client application i.e. work/outlook to the SQL/WCF/.NET data type directly.
  • Double hop issue with NTLM.  Scot laid out a table to direct the options available for various scenarios that I will hopefully post here.
  • Secure Store Service is extensively used for authentication in BCS.
  • SPD has a good tool set for BCS.  Probably the place to start.
  • SQL 2000 doesn't work as an External Content Type (ECT) - this needs verification.
  • BCS doesn't support transactions and is not strongly typed however data access options can be intermingled to get transaction functionality if you need it. 
http://uscatu.wordpress.com/2010/09/30/business-connectivity-services-deep-dive-mdvwrk1-%e2%80%93-my-notes/  A far better summary was written by my fellow attendee Razvan-Petru.

BCS versions feature table

Thursday, 23 September 2010

Creating Site Columns / fields declaratively

Problem: Adding site columns decoratively has nuances.  This post adds information for getting fields/site columns correct when building them declarative using a feature to deploy to a Site Collection.

Resolution: You can use the following reverse engineering technique on the list.  This will show you the xml/CAML to create lists.  This works in MOSS & SP2010.  I used the list reverse engineering technique to figure out how to create a Lookup site columns with multiple values. 

Steps to Reverse engineering a SharePoint List:
1.> Create a list, content type and site column for the lookup (or field you want) using the SharePoint's UI
2.Then retrieve the list using a formatted url to see the native CAML used to create the site column.  http://url/_vti_bin/owssvr.dll?Cmd=ExportList&List=listguid
This approach saved me a lot of time today.

CAML attributes required for each field element are: ID, Name, DisplayName, Group, and Type.
MSDN field element attributes
Attributes explained
Type="Note" will create a site column in the UI showing "Multiple line of text".  Attribute to get full html imput capabilities are RichText="True" (default is false)  & RichTextMode="FullHtml".
Type="Choice" can have a default value selected as shown below:
Built-in Site Column Append-only Comment can be referenced in a content type to provide comments that are appended and basic html is available.
Lookup columns work in SP2010 without having to create an event receiver to setup the lookup relationship using the list guid as is the case in MOSS.  Multiple lookups are possible also simple set the field attributes Type="Lookup" & Mult="TRUE".

Tip: Updated 18 Oct 2010 - CKSDev has the functionality to import Site Columns, this is useful in that you may of created the SiteColumns via the UI on a prototype site and you can not easily get the xml to create the site columns/fields declaritively. http://blog.mastykarz.nl/cksdev-support-importing-site-columns/

Monday, 20 September 2010

CKSdev codeplex SPMetal SPI improvement

Matt Smith, David Mann, Todd Bleaker, Wes Hackett, Waldek Mastykarz & Wouter van Vugt have done a great job with CKSdev and they have upgraded the codeplex project recently that includes "Improved SPMetal SPI".  This is a visual studio template placed under the SharePoint 2010 grouping that allows you to add your SPMetal code to your project easily.  Previously you had to either create the LINQ to SharePoint class using the command prompt and provide switches such as the namespace.   Then I have to manually add/over right the class file generated into my source controlled code.  The other option was to add the SPMetal build to pre-build using the SPMetal cmds and each time and you need the SPMetal proxy file being overwritten to be out of source control i.e. this wouldn't work with your source control without a workaround.

Using the SPMetal Definitions included in the latest CKSDev project code is definitely a productivity improvement.

Data access options - SharePoint Lists or Database Tables

Problem:  I have built a composite application that uses SP 2010 lists for storing the application data.  There are 2 related lists within the application that are high transactional lists like a shopping cart (order with order items).  These 2 lists have had many fields added to them over the duration of the project and are no longer light lists. 
Hypothesis: SharePoint 2010 supports up to 50 million lists and I need to insert, edit and read often the information in these 2 related lists.  The lists are getting rather complex and transactional support is not available vie LINQ to SharePoint.  I need a full audit history which is pretty useful in SharePoint.  The volume and complexity of this transactional data is to heavy for SP especially as the lists grow.  I have to use SQL Server to store the data directly and link the appropriate SP lists using BCS and external content types.  My preferred methods of accessing data are the Entity Framework v4 or Linq to SQL.  As my data is pretty small, I don't need a multi tier system and we are using LINQ to SharePoint. 
Resolution: Create the SP lists in SQL, use LINQ to SQL to add and work with the SQL data.  I use tomb tables in SQL Server to keep a full audit history.  There are now 2 areas of storage: SP lists and directly to SQL tables.  Not ideal but worth that change.

Pros of using SharePoint List vs Database tables
SharePoint ListsDatabase Tables
SharePoint Security and PermissionsTransactional
WorkflowPerformance & higher transaction volumes
VisioningDistinct queries and aggregations
Office intregrationGreater Flexibility
Native / Easy setupEasier for existing .NET developers
SP generates UI
Update 18 Oct 2010 - Storing data for SharePoint 2010 comparison

Friday, 17 September 2010

Image dump - Ribbon actions are disabled

Problem: Adding a new domain account user to the "Farm administrators" group. The new account mydomain\mstest1 cannot use the "New", "Extend" or "Delete" buttons on the ribbon within "Web Application Management".
Rolling over the buttons on the ribbon display the following information: "This control is currently disabled." "You might not have the right permission level to use this, you might need ...".
Additionaly you will also notice the "Manage Service Applications" will not allow you to create or delete new Service Applications.
New user in the Farm administrators group
Hypothesis: The new farm administrator does not have the appropriate permissions on the configuration database.

Resolution: Give the new administrator db_owner rights and the ribbon action options are available.

Further Info:
By making a content database read-only as shown below all Site Collections become read-only, ribbon actions are disabled.  Not coentent can be added to the Site Collection.
Make the content db read-only

Resulting Site Collection (Can't use most ribon options)


Unified Logging Service (ULS)

Overview: ULS is made up of 3 components: Trace logs, Reporting database & the Windows Event viewer. 

Trace Logs:
Trace Logs by default are located in the 14 hive/SharePoint Root, by default they are created every 30 minutes and can be viewed in notepad. 
ULSviewer is available on codeplex to improve usability as the logs are easier to follow than if you use notepad.
Trace logs can grow extremely quickly.  Only log the appropriate messages for troubleshooting without making the logging to big.  Verbose logging cause huge amounts of logging but it is extremely detailed.  The amount of logging or what is logged can be set using the CA> Monitoring > Diagnotic Logging "Configure Diagnostic Logging".
SharePoint 2010 uses NTFS compression for log files to reduce storage on the WFE (logging) servers.
Consider moving the trace logs to a new drive so that the logs don't fill up the default drive (usually the c drive).  This change applies to all servers in the farm.
Logs by default are kept for 14 days, you can also hold them using sizes rather than duration.
Event Log Flood Protection in CA should always be turned on to stop repeated errors being recorded multiple time in you logs.
Correlation ID's as the name suggest helps us track errors from the UI through to the Trace Logs.  The end user when the receive an error are shown a unique correlation ID that can be used to track down the problem.  Correlation Id's as also shown on the developer dashboard and can be used to tract from the UI into the trace logs without errors.

Update 7 Oct 2010 - Useful Monitoring artlicle:
Update 14 Dec 2010 - ULS Viewer
Tool to view ULS log and filter data

Various SharePoint Logs Explained:

Tuesday, 14 September 2010

Recomended SP2010 Books

Professional SharePoint 2010 Administration - Todd Klindt, Shane Young, Steve Caravajal.  Wrox.   Good admin book, and useful for architecture.  Has a lot of best practice points and clearly explains core concepts such as service applications in SP2010 & claims based authentication.

Microsoft SharePoint 2010: Building solutions for Sharepoint 2010 - Sahil Malik.  APRESS.  Good all round book - especially useful for getting .net developers into SharePoint.   There are better reference developer books but it clearly works through major areas that SharePoint developer should know.

Thursday, 9 September 2010

SPAutoinstaller accounts

Change the setup.xml file to use 5 accounts:
  1. SP-Install - domain account with admin local rights on each WFE also need SQL dbcreator and dbsecurity roles (used to login and install binaries, use this account for add new servers to the farm);
  2. SP-Farm - domain account no permissions, will be the account to run timer job and other key roles;
  3. SP-Web-App-Pool - Content Web app account - Domain account only;
  4. SP-Services - Install all services to use the same domain account, this can be seperate for each services but for easy of setup and mainentance use 1 account. Exception is the User Profiles service, setup seperately using Spence Harbors post as the user domain account needs unique security, and
  5. SP-Crawl - Used to crawl SP content.
6th SP-SQL - Domain user account with no prividges used to start the SQL service (recommended) can use local/builin accounts but for Reporting services, and other advanced functionality need a domain account.
7th account should be used for User Profiles called SP-Service-UP.  This is a complex issue and needs full review don't try install this blindly hoping it will work.

Todd Klindt's Blog
Todd's netcast 49 about installation (specifically install accounts, user profile services and an SP2010 internet version enterprices edition licence bug).
Liam Cleary's list of SharePoint 2010 Service Accounts

Wednesday, 8 September 2010

SP 2010 Service Applications

SharePoint 2010 Serices Chart

DisableLoopbackCheck fix

I was listening to Todd Klindt's netcast 50 this morning and he mentioned that DisableLoopbackCheck is not the correct approach for allowing your SharePoint servers to see themselves (needed if crawling).  I have been doing this on my server installation and need to review my approach.

MS KB on disabling loopback using either approach.  DisableLoopback was introduced by MS to reduce prevention attacks.  As SharePoint needs to crawl itself and it's pretty nice to have a local browser for A records/renamed resources, I need to disable loopback however, the method of specify what to disable is definately more preferable.

Tuesday, 7 September 2010

SP2010 AutoSPInstaller NLB farm install issue - Trial period for this product has expired

Problem: I created a farm using the AutoSPInstaller project. All works perfectly. I then joined a 2nd SharePoint 2010 server using the script however the script was change to not create the services again. All works and I have a 3 server farm including the database server. I then added Windows 2008 R2 network load balancing on the 2 SharePoint 2010 servers. If try access the default Team Site created when I built the 1st server on the new server I get the error "The trial period for this product has expired." Event log doesn't show an error and the ULS shows the following error "Exception thrown while adding control 'Microsoft.SharePoint.Portal.WebControls.SearchBox' : Thread was being aborted ...".

Hypothesis: The install adds the IIS web site created on my initial farm build on the 2nd SharePoint server. Routing is working and the error occurs when the request is routed to the 2nd SP2010 server. The response is coming from the IIS web server and we know the issue is within the specific web app.  In MOSS there were 2 causes for the error message and this is clearly not the SP2 bug so it looks more like the app pool account bug.

Resolution: Create a new Web Application and create a new root Site Collection. The error goes away. OR
Using Powershell change the Managed account then using IIS Manager on each web server go to the approoriate app pool and change the account to the new account. 
Use Central Admin to change the service account.  CA > Security > Configure service account.  You will need to reset IIS on each web server.
Demo on changing app pool managed accounts

Friday, 3 September 2010

SharePoint 2010 PowerShell Useful Scripts

Get Content Types in a Site collection
$site = new-object Microsoft.SharePoint.SPSite(http://ww3.sp.demo.dev/)

$web = $site.rootweb.ContentTypes
ForEach ($id in $web)
'"' + $id.Name + $id.Id + '"' Out-File "d:\logfiles\contenttypes.txt" -append

Delete a List Instance
$site = new-object Microsoft.SharePoint.SPSite(“http://demo.dev/”)
$web = $site.rootweb
$list = $web.Lists["Orders"]
Run all time jobs 
$timers=Get-SPTimerJob|{$_.isDisabled -eq $false}
foreach ($timer in $timers) {$timer.RunNow()}
View ULS data for a CorrelationId
PS C:\Users\PaulB> get-splogevent ?{$_.Correlation -eq "fe45a7cd-e91b-4ad3-80dc-50cd2a238c6a"} select Area, Category, Level, EventID, Message Format-List 
Run the SharePoint Health Analyzer
$jobs = Get-SPTimerJob | Where {$_.DisplayName -match "Health Analysis Job"}
foreach($job in $jobs) {
  Write-Host "Running" . $job.DisplayName

Read an XML file:

Not all jobs are fired automatically so you can run all the SP Health Analyzer jobs using this PS cmd.  Ensure server can deal with extra load (run out of hours if possible).
Ari Bakker

Turn On-Off Publishing Workflow
John Livingston
PowerShell Basics
Zach Rosenfield's - Farm settings, retrieve solutions, basic SharePoint operations
Gary Lapointe

Brian Lalancette
AutoSPInstaller - Installation of SP2010
Use Windows PowerShell to Display Service Dependencies

Adam Preston
Disable Loopback in IE - New-ItemProperty HKLM:\System\CurrentControlSet\Control\Lsa -Name "DisableLoopbackCheck" -Value "1" -PropertyType dword

Todd Klindt

PowerGui SharePoint reference card
Download PowerGui

Add the SP cmdlets to PS window
Add-PSSnapin Microsoft.SharePoint.PowerShell

Get Help about PS Cmds
PS> Get-Help Get- SpWebapplication

Updated: 23 Oct 2012
Add a local user account

Add an account to AD and add the account to a group.

Apologies if I have ripped script directly.  I don't have the source for a lot of the work these help scripts are based.  On pls let me know if I need to credit a source.

Wednesday, 1 September 2010

SP2010 AutoSPInstaller Prerequisite Install Issues (Codeplex PS project)

Problem: Install SP2010 using the CodePlex PowerShell Installer (AutoSPInstaller) without internet access.

Add the following KB to the prerequisiteInstaller folder:
  • Windows6.0-KB968930-x64.msu (/POWERSHELL)
  • Windows6.0-KB976394-x64.msu (/KB976394)
  • Windows6.1-KB976462-v2-x64.msu (/KB976462)
  • Windows6.0-KB974405-x64.msu (/IDFXR2)
  • rsSharePoint.msi (/ReportingServices)
  • SpeechPlatformRuntime.msi (/Speech)
  • MSSpeech_SP_en-us_tele.msi (/SpeechLPK)

Error: Error.  An unknown error occurred installing prerequisites at c:\....\AutoSPInstaller.sp1:388 char:14 ...

Fix: Reboot server and retry.


Error: The tool was unable to install SQL 2008 R2 Reporting Services SharePoint 2010 add-in. Install process returned (0X643=1603).  Applies to the codeplex Powershell scripted install.
Fix: Only info I found suggested a re-run fixes the error. Simply re-running the prerequisite installer did not fix the issue for me. A similar error suggesting that i am using the incorrect version of rsSharepoint.msi relating to the Genva framework (claims) not Reporting Services.  I tried manually installing the Reporting services rsSharePoint.msi, it kept failing.  I also tried rsSharePoint_x64.msi, also failed.
Next I ran SharePoint install manually after the manual prerequisites was run as Administrator, failed.

Noticed I had 2 rsSharePoint.msi that are of a different size?  The file I used on 2 Sept 2010 on my Win 2008 R2 WFE was 36.9MB in size, my other was 39MB.  rsSharePoint.msi download

I use the version off the SQL media for getting Reporting Services wirking in the SharePoint mode.


Error: rsSharePoint.msi wont install
Fix: There are version versions of rsSharePoint.msi, ensure you use the correct 36.9MB file.
I downloaded the correct version from http://go.microsoft.com/fwlink/?LinkId=192588
Incorrect 39MB Version as of 2 Sept 2010 .
These file have the same name but they are different and using Windows 2008 R2 for my install only the 36.9MB version works.


Installation Notes for a 3 server NLB SharePoint 2010 farm