SPMetal Extender CodePlex project

I have release a project on codeplex, SPMetal Extender to be able to use additional field types.  Fields in lists can be extended using partial classes.  The project provides a Visual Studio plug-in vsix that generates Linq to sharePoint code for unsupport list fields such as publishing fields.  Additionally the vsix deploys a new template showing how to add 4 additionaly properties to SPMetal.  The Item template is guidance and needs to be adjusted by the developer after it is created.

SPMetal Extender VSIX extension installed

Add a new SPMetal Extended item to a SharePoint project

Generate code for addionnal list fields using Server Explorer using the VS IDE

SPMetal Extender is also published on Microsoft Visual Studio tools gallery 

Error adding webpart containing validation control

Problem: When adding a custom built webpart with asp validator controls to a page, the user will encounter the following error when attempting to save the changes "Error: This page contains content or formatting that is not valid.".

Initial Hypothesis:
This error only happens when the webpart control has validators and prevents the user being able to add the webpart to the page.

Resolution: Disable the validator controls when editing the page.
Add the following public method in the server control code ...
public void EnableValidators(bool enableVal)
{
this.validatorName.Enabled = enableVal;
}
In the Webpart's CreateChildControls() procedure only add the user control/validation when not in edit mode
if (SPContext.Current.FormContext.FormMode == SPControlMode.Edit || SPContext.Current.FormContext.FormMode == SPControlMode.New)
{
//validator.enabled = false
((UserControl1)_ctl).EnableValidators(false);
}
else
{
//validator.enabled = false
((UserControl1)_ctl).EnableValidators(true);
}
}
Add the EnableValidators method to the user control code behind:

 

The validators will then be disabled when adding the web part​ to the page / editing the page, but still work in presentation mode.

Thanks to Paul W

Change to session cookies for Claims Based Authentication

When you log into SharePoint using Claims Based Authentication, a cookie is written/persisted to ​disk (FedAuth) to persist your session, which prevents you having to be authenticated each time you open a new browser or re-boot.  So using a FedAuth cookie allows the browser to close and re-open windows as long as the FedAuth cookie has not expired.  For ADFS, FedAuth cookie expiry is by default set to expire 10 minute earlier than the SAML token.

You can change the cookie to be session based by running this PowerShell script:

$sts = Get-SPSecurityTokenServiceConfig
$sts.UseSessionCookies = $true
$sts.Update()
iisreset

You can revert back to a disk based cookie (default) by running this:

$sts.UseSessionCookies = $false
$sts.Update()
iisreset

Show cookies on a local machine for Internet Explorer

IE > Internet Options> General > Browser history "Settings" > Temporary Internet Files "View files".

Also see: http://blog.sharepointsite.co.uk/2015/08/fedauth-notes-for-problem-solving.html

Update 2016/03/10

Tip:  I wanted to examine a cookie to check the user being authenticate, and I opened the cookie using a base64 online decoder https://www.base64decode.org/

Update: 2016/11/23

IE Developer tool bar and Fiddler are great and easy to use and pretty feature rich but lately I have been using Chromes Developer toolbar press "Ctrl" + "Shift" + "i".



Linq to SharePoint - CRUD operations

Problem: Code to perform CRUD operations against a SharePoint list using Linq to SharePoint.
Resolution:
1) Read:
DataContext dc = new DataContext();
EntityList Customer = dc.GetList("Customers");
var CustomerItems = from Customers in Customer
select Customers;

Alternative approach

2) Insert:

DataContext dc = new DataContext();

CustomersItem Customer = new CustomersItem();

Customer.Title = "Title";

dc.Customers.InsertOnSubmit(Customer);

dc.SubmitChanges();

3) Update:

DataContext dc = new DataContext();

EntityList Customer = dc.GetList("Customers");

var CustomerItems = (from Customers in Customer

where Customers.Id == Convert.ToInt32(txtEditCustomerId.Text)

select Customers).Single();

CustomerItems.Title = CustomerItems.Title + txtCustomerTitle.Text;

dc.SubmitChanges();

4) Delete:

DataContext dc = new DataContext();

EntityList Customer = dc.GetList("Customers");

var CustomerItems = (from Customers in Customer

where Customers.Id == Convert.ToInt32(txtEditCustomerId.Text)
select Customers).Single();
dc.Customers.DeleteOnSubmit(CustomerItems);
dc.SubmitChanges();

Alternative approach 

Tip: If you are using read-only operation in your LINQ to SharePoint queries set ObjectTrackingEnabled to false to improve performance.

DataContext dc = new DataContext();
dc.ObjectTrackingEnabled = false;

Tip: Use the context on the generated proxy for performing LINQ to SharePoint queries.

More Info:
LINQ to SharePoint: CRUD operation on SharePoint 2010 list using SPLinq
LinqPad
LinqPad for SharePoint 2010

Extending SPMetal

Problem: Using SPMetal retrieve a list and display the basic columns and the following 3 columns types:

• Attachments;
• Html Publishing field; and
• Image Publishing field.

Resolution:
Extended partial class to retrieve attachments, a html publishing field and a publishing image.  The code link only does the read operation.
Extended partial class (advanced) to retrieve an attachment, a html publishing field, a publishing image and the hidden CreatedBy field.  You can use the parameters.xml when generating the SPMetal proxy to display additional hidden fields such as the CreatedBy field however, if the proxy has already been generated, it is easy to simple extend the partial class to get hidden fields.  This example also adds the code to insert and update the publishing html and the publishing image field columns.

More Info:
SPMetal only generates the proxy classes code against lists that site columns are available in SP2010 foundation.  So you also don't get Manage Metadata or any custom site columns you create.  You also don't get most of the built in columns with the exception of ID & Version using SPMetal.  However, by changing the parameters.xml file you can include them i.e. Modified, CreatedBy.  You can also use the extended partial class method to display the hidden built in site columns - I would use the parametes file and not the extended partial class coding for hidden fields(Field = Site Column).

Deploying a Sandbox solution

Problem: I created several wsp packages, unfortunately 1 of the solution projects was marked as a Sandbox solution when the project was created in Visual Studio 2010.  The lookup lists are not deploying correctly in the sandbox solution because I deployed the solution at farm level.

Initial Hypothesis: I notice my lists had strange behaviour.  I check my deployment and 1 of the solutions had not been deployed.  When I deployed the wsp using Powershell (PS> Install-SPSolution ...) it did not error so I had presumed it had installed correctly.  Using PowerShell (PS) I could see the solution holding the list wasn't installed. PS> Get-Solution

Resolution: Retract the solution and remove it from the farm.  Deploy the solution to the Site Collection's solutions and enable the feature/s.  Powershell Commands:

PS> Add-SPUserSolution -LiteralPath D:\packages\.Lists.wsp -Site http://demo.dev

PS> Install-SPUserSolution –Identity Lists.wsp -Site http://demo.dev
PS> Enable-SPFeature –Identity Feature_GeneralLists –url http://demo.dev/

Tip: Update 10 Dec 2010 - If you remove a solution (this also applies to sandbox solutions) (including deactivating the associated features), if the feature deploys content types or site columns, you will need to do an IISreset if you want the content types and site columns removed from the Site Collection (alernatively restart the Web Application so the whole farm is not reset).

Post on deploying fasm wsp's.

Custom Claims Membership Provider

Problem: We have written a custom claims membership provider, it works in development however if fails in the UAT environment.  The Membership provider goes to an LDAP directory (IBM TAM).

Initial Hypothesis: After checking the configuration was correct, I looked at the logs that were informing me that the provider was not returning results.  We can't use wireshark on the UAT environment to verify the call is being made to the LDAP directory.  And the error message is a generic failure message.

Resolution: Use Telnet to check that the WFE/WebService/STS/SharePoint token service can speak to the LDAP directory.
cmd>Telnet 192.168.1.101 389

The Claims web service cannot contact the LDAP directory, most likely cause is the firewall or the windows firewall.  Add a rule to the firewall to allow traffic on port 389.

Deploying packages into production using PowerShell

Problem: Developers build solutions and features for SharePoint 2010 using Visual Studio 2010 (VS), the farm wsp's need to be deployed in UAT, pre-production, production, CI potentially etc.  You don't have VS so you can use stsadm or Powershell.

Initial Hypothesis: This is a walk-thru on deploying your solutions and features in SharePoint 2010.  The main areas are:

1. creating solution files (wsp's);
2. moving them to the target environment;
3. adding the various solutions and features at the appropriate level in your farm (features have 4 steps to get into production namely:

• Add Solution,
• Install Solution,
• Install Feature and
• Activate Feature.

Resolution:

1.> Create wsp's
1.1> Using VS2010, put the project into "Release" mode

1.2.> Next create the package using VS

2.> Copy the wsp located in the bin\Release directory of your VS solution to the farm where CA is hosted.
3.> Add solutions and activate features.
PS> Add-SPSolution D:\Packages\Demo.wsp
PS> Install-SPSolution –Identity Demo.wsp -GACDeployment
PS> Install-SPFeature demo_feature1 -force
PS> Enable-SPFeature –Identity demo_feature1 –url http://demo.dev/sites/demo
Tip: The feature scope requires different parameters.  For example if the feature above ( ) was scoped at Site (Site collection) level, the cmd would be: PS> Install-SPSolution –Identity Demo.wsp -GACDeployment –WebApplication http://demo.dev/
Scoped as web application level
PS>Install-SPSolution –Identity Demo-WepAppScope.wsp -GACDeployment –WebApplication http://demo.dev/sites/demo
Remove a web or site scoped solution
PS> Disable-SPFeature –Identity demo_feature1 –url http://demo.dev/sites/demo
PS> UnInstall-SPFeature demo_feature1 -force
PS> UnInstall-SPSolution –Identity Demo.wsp -GACDeployment
PS> Remove-SPSolution Demo.wsp
Cmd to add the SharePoint snapin, allowing the administration to manage SharePoint
PS> Add-PSSnapin Microsoft.SharePoint.PowerShell

Post on deploying Sandbox solutions.

Update: 27 Feb 2013.  The Install-SPFeature is only used for manual install i.e. not with wsp deployments but when you manually copy the feature files to each WFE.  You can keep the step as it has no negetive impact but it is not required.

Update: 20 May 2013.  This just bit me again.  I have an SP Timer Job scoped to WebApplication, by default when the solution/wsp is deployed, the feature receiver is activate automatically on ALL web applications (including MySites).  Change the Feature xml to include the attribute ActivateOnDefault="FALSE".

Read More:
http://sppowershell.com/2010/06/25/provisioning-sharepoint-2010-with-powershell/
http://msdn.microsoft.com/en-us/library/ms442691.aspx
http://stsadm.blogspot.com/2010/06/deploying-sharepoint-2010-solution.html
http://patrickboom.wordpress.com/2010/05/31/using-powershell-to-deploy-sharepoint-solutions-wsp-2/
http://dotnet.sys-con.com/node/1208275

Powershell - Running as a farm administrator using the SharePoint plug-in on Powershell

Problem: I need to run SharePoint Powershell commands in Production/UAT, I get remote access to the servers however the account is not the SharePoint farm account as the Famr account runns windows services and should not have remote access to the servers. 

Initial Hypothesis: Remote desktop into the WFE server, use the command prompt to launch Powershell using the farm account.  Lastly, add the SharePoint snapin for SharePoint 2010.
Update 6 Dec 2010 - Alternatively, hold down the shift key and right click the program you wish to run, you are given the "run as .." menu option.

Resolution:
1.> Remote desktop into the machine using a remote desktop access account i.e. demo\
2.> Open the cmd prompt and run the following cmd:
cmd > runas /user:demo\farm_admin c:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.exe
3.> In the new Powershell windows add the PowerShell snapin
ps c:\windews\system32> Add-PSSnapin Microsoft.SharePoint.PowerShell

Read More:
http://social.technet.microsoft.com/Forums/en-US/sharepoint2010setup/thread/fcb77654-0f13-42e0-b181-6e52242fe9d6  Adding the SharePoint snapin to PowerShell.

DateTime control default behaviour

Problem: The SharePoint date time control's default behaviour is to returns the current date and time when no date has been selected.
Initial Hypothesis: Validate the control to pre insert to check if it is selected, if no date time is choosen don't update the record.
Resolution:  The Property IsDateEmpty returns "false" if the date has been input.
Code:
if (!dtMyBirthday.IsDateEmpty)  { // ... User has delected their birthday date so insert into SQL table or list }

Read More:
http://karinebosch.wordpress.com/sharepoint-controls/datetimecontrol-control/

Changing service account passwords - The Service is unavailable

Problem: Browsers return the following error "Service Unavailable  Http Error 503.  The Service is unavailable." on all SharePoint websites including central admin.

Initial Hypothesis: I changed my password yesterday causing the app polls to fail when logging in.  The domain account used on my development machine required a password change.  Starting the machine causes all the IIS web sites to display the error message "Service Unavailable".  I run various services and application polls using my domain account.  The services can no longer log on.  Application pool cannot be started after the reboot/iisreset.

Resolution:  Change the log on details for the application pools used by IIS that run using the domain account that's password was reset.  Also start the Windows services that run using the windows domain account.

Ensure the SharePoint services running

Installing SharePoint using a dedicated Installation account

Problem: Best practice is to install SharePoint servers using a dedicated install account.  This account is not the farm account.  Once the installation is complete, the installation account should be disabled.  You need to install in the role of the admin account to install correctly. 

Note:  If your install and farm account are the same as is often the case, this post does not apply to you.

Initial Hypothesis:  It is only worth following this post if you are using at least 5 or more accounts for your farm install.  The idea is that the installation account is disabled after installation.

Resolution:  The farm account needs the 2 SQL Server security roles namely: dbcreator and securityadmin.  The farm account still needs local admin permission rights on each Web Front End (WFE) server.  The Installation account does not need any SQL Server permissions.

Using the codeplex AutoSPinstaller.  Launch the installer using a cmd prompt using the runs cmd to run in the farm domain admin priviledge.
cmd> runas /user:demo\farm_admin %windir%\system32\cmd.exe
cmd> D:\SP2010\Script>Launch.bat


You can also do the runas shotcut to change the account installing SharePoint.
Update 6 Dec 2010 - Hold down the shift key and right click the cmd prompt program menu, you are given the "run as .." menu option.

Update: 22 June 2011 - the current version of AutoSPInstaller is 2.5, additionallly the codeplex project is called AutoSPIntaller not SPAutoInstaller as previously named.

AutoSPInstaller - Step-by-step guide: http://blog.lekman.com/2010/11/automated-sharepoint-2010-installations.html

Central Admin is not working correctly - You may be trying to access this site from a secured browser on the server

Problem: Using Central Administration (CA) on SharePoint 2010 you can't use the menu and you see the following information "You may be trying to access this site from a secured browser on the server. Please enable scripts and reload this page."

Initial Hypthesis: FireFix work so it is a I.E. 8 issue.

Resolution: As suggested on the MSDN formun turn of the IE

More Info:
http://social.msdn.microsoft.com/Forums/en/sharepoint2010general/thread/4cf3a740-dddf-4c08-bc36-03efc731eff8
http://social.technet.microsoft.com/Forums/en/winservergen/thread/e87585de-d365-48c8-b08f-1050d68724ed

Branding, Load testing, project success points

Branding - from designer to SharePoint
This is a 4 part series that is a great start for branding with a designer.
http://sharepointmagazine.net/technical/development/sharepoints-branding-limitations-part-1

SharePoint Project Checklist
http://sharepointmagazine.net/technical/administration/avoid-newbie-mistakes-10-steps-to-successful-sharepoint-deployments

Load Testing
http://sharepointmagazine.net/technical/load-testing-reveals-cause-of-sharepoint-server-performance-problem

BCS Limitations

A good post by Chakkaradeep Chandran on the limitations of BCS

Access SQL using a different domain account using SQL Windows authentication mode

Problem: Infrastructure has setup a new SQL Server 2008 R2 instance.  I can only remote desktop to the machine using the installation account.  AS SQL Server is setup to use Windows authentication I can't login using the local "Microsoft SQL Server Management Studio". 

Initial Hypothesis:  I can't connect remotely to the SQL instance, I can't login using remote desktop access with the appropriate account that has SQL access.  I need to remote desktop to the SQL Server using the remote access windows account i.e. demo\Installation and then runas the demo\sqladmin account to get access to the Management Studio.

Resolution: Run SQL Server Management Studio in the rights of the windows account that has SQL access.
Steps:

• Remote desktop into the SQLServer using the windows account that has remote desktop access i.e. demo\Installation
• Open you cmd prompt
• Execute the cmd> C:\Users\Installation>runas /user:demo\sqlaccount "C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE\Ssms.exe"
• Enter the SqlAccount password
• Management Studio opens up using the demo\SQLAdmin account

Update 6 Dec 2010 - Alternatively, hold down the shift key and right click the SQL Server Management menu, you are given the "run as .." menu option.

Could not load the Web.config configuration file when debugging in SharePoint

Problem: I can't debug my SharePoint projects using Visual Studio 2010 (VS).  I get the error "Could not load the Web.config configuration file.  Check the file for any malformed XML elements, and try again.  The following error occurred: Cannot connect to the SharePoint site.  If you moved this project to a new computer or if the Url of the SharePoint site has chnaged since you created the project, update the Site Url property of the project." in VS when I try debug.

Initial Hypothesis: I recenctly renamed all my VS projects in my VS solution.  The error tells me to check the "Site Url" on each of my projects.  I missed the project that was set as my start up project.

Resolution: Set the "Site Url" property in VS on the projects.

Dropdownbox population options for custom SharePoint applications

Problem: I need to populate combo boxes and drop down list boxes on a customised SharePoint page using a web part.  The web part has 5 of these lists that are populated choice fields.

Initial Hypothesis: I could use the Enum generated by Linq to SharePoint for the choice field options but to change the available options would require me to re-generate the SPMetal proxy.  I could use lookup lists and then use Linq to SharePoint but this is pretty heavy for simple choice columns.  So I chose to use the SP Server-side Object Model, this is not too heavy as it is only looking up the columns and looping thru the choices however I have 5 drop down lists doing this repeatedly for a lot of users.  So although it is working it could be better if the results were cached for a minute.  This would allow choice options to be change and the using a cache duration of 1 minute, the longest the wait would be is 60 seconds before users see the updated lists.

Resolution: I cached my list item collections, on the page I had 5 choice fields.  By adding the caching I am getting a 36% performance increase in my Page Request response time.  So not as good a return as I was expecting but still a considerable improvement. 

Test Information:
I used a sample of 10 items per request type.  The 10 pages requested each for the cached list item collections and the un-cached version.  I excluded the 1st page request for warm up. 
All the results were pretty consistent.
The testing was done on a local development machine.

Developer Dashboard showing improved response times using cacheing.

Code to cache list item collections:
HttpContext.Current.Cache.Insert(keyname, null, DateTime.Now.AddMinutes(1), Cache.NoSlidingExpiration)

Error occurred in deployment step 'Activate Feature': A duplicate field name "xx" was found.

Problem: A site column/ field occurs more than once.  I.e. there is a duplicate field name.
Initial Hypothesis:  If you are deploying site columns declaritively (CAML), you will have 2 site columns with the same name and type.  This commonly happens with "Email" so be more specific or reuse the exisitng site column. 
Resolution:  Ensure thre duplicate field name is unique or reuse an exisitng suitable site column.

If you have check that you don't have the issue above and you are deploying using Visual Studio 2010 use the following advice.  This occurs after you have renamed or deleted the site columns as explained above.

Problem: Deploying site columns using VS results in the error "Error occurred in the deployment step 'Activate Features': A duplicate field name "Email" was found. 

Initial Hypothesis: Ensure you don't have duplicate fields as show in the above problem solver.  Activate the feature using ststadm, it will now work.  The problem is that Visual Studio on it's 1st build and deployment caches SharePoint information, this was done by MS to speed up subsequent deployments of code.  However, the cached VS SP info is now wrong so restart VS 2010 and deploy again.

Resolution:

1. Retract the solution that holds the failing feature;
2. Close and reopen the Visual Studio Project; and
3. Deploy the feature using VS 2010. 

Reverse Engineer Lists, Content Types and Site Columns and retrieving the CAML

Problem: You need to create deployable lists, this should be done using a feature.  You can create a lists using the UI and then get the CAML that created the list.  The code will have some hicups but it a good place to start for building your Lists based on List definitions, content types and their corresponding site columns.

Resolution:
Get the CAML and create your feature based on the declaritive CAML
http://%7bwebapplication%7d/%7Bsite%7D/_vti_bin/owssvr.dll?Cmd=ExportList&List={ListGuid}
Thanks to Sahil Malik

Read More:
http://blah.winsmarts.com/2008-2-Dev_Tip__The_SharePoint_University_of_Reverse_Engineering.aspx
http://www.synergyonline.com/blog/blog-moss/Lists/Posts/Post.aspx?ID=24

Download: SharePoint 2010 sandbox project example that creates 2 lists with a lookup field. Updated 24/03/2011

SharePoint DateTime format is incorrect

Problem: The SharePoint DateTimeControl displays the date in US format i.e. 1033 or mm/dd/yyyy despite me changing the site collections regional settings.
Site Settings > Regional Settings > Locale > Select [English (United Kingdom)]

Inital Hypothesis: In MOSS changing the Locale did not change the format of the DateTimeControl. 
Resolution: This is still the case, you need to explicitly set the local on eash DateTimeControl.  You can set it to be the same as the site's local.
E.g. myDateTimeControl.LocaleId = SPContext.Current.Web.Locale.LCID 
Or set the "LocaleId"attribute of the SharePoint DateTime control
E.g. SharePoint:DateTimeControl runat="server" LocaleId="2057"

=========================
Tip: Changing the Regional settings will affect the curreny columns.

List of Locale's: http://msdn.microsoft.com/en-us/goglobal/bb964664.aspx

====================

Using CAML and SPQuery to query using dates

Problem: Using SPQuery I can't filter using a datatime parameter.
Hypothesis: I can see my custom web part code is formatting the DateTime selection (SharePoint:DateTimeControl) differently in my custom code CAML than the CAML generated by U2U.
Resolution:
Use the Microsoft.SharePoint.Utilities.SPUtility.CreateISO8601DateTimeFromSystemDateTime() method to get your DateTime control value into the correct format.
SPUtility.CreateISO8601DateTimeFromSystemDateTime(DateTime.UtcNow)

More Info:
http://blogs.msdn.com/b/saurabhkv/archive/2008/05/05/spquery-with-boolean-and-datetime.aspx
http://snahta.blogspot.com/2009/07/spquery-few-important-things.html
http://www.aidangarnish.net/post/Using-SPQuery-and-CAML-to-filter-and-order.aspx