Dynamic LINQ to SharePoint 2010

Problem: I have 2 connected Web Parts, the provider provide multiple values for lookup columns. I started with LINQ to SharePoint (SPMetal) but could not build up the query dynamically.
Hypothesis: My initial reaction was to use a dynamic CAML query as done in MOSS using U2U to work out my query. The issue is that the CAML is not safe (as we don't get validation until run-time) but at least I can dynamically build up my query.

Using Dynamic link with LINQ to SharePoint to can achieve the required result. The code is safe as least for run-time (my logic is still dodgy). Only issue is with the performance and the results if the list if large. Using a very broad LINQ-to-Sharepoint query that is converted into a CAML query anyway I get a large result set. I then filter using dynamic LINQ. Pretty heavy filtering and inefficient querying. Throttling concern: If I returned more than 5,000 items (default list throttling limit for SP2010 lists) I now loose results that should be queried (SharePoint would trim my SPMetal query and then Dynamic LINQ would work on the max of 5,000 items). Sure you can turn throttling off as a farm admin but not a great idea. You could programatically override the throttling (SPQueryThrottleOption.Override) using the server OM but this doesn't help for your SPMetal query.
In this case, my best option is to use a dynamic CAML query and get the exact set of data I am looking for.

Tip: Only get the fields you are using in CAML queries.
More Info:
Building CAML queries
Dynamic LINQ - Scott Guthrie

LINQ to SharePoint Posts

• Dynamic Linq
• Inefficient Linq Queries
• SPMetal issues

Installing Sharepoint 2010 options & Basic SP2010 manual installation tips

You have 4 options for installing SharePoint farms:

1. Manually sun the setup and follow the installation wizard (this is discussed below);
2. Deploy SharePoint 2010 via a slipstream install, this was my prefered method for MOSS.  I ran the install from a batch file that got it's configuration from an xml file;
3. PSConfig installation (sic); or
4. Use PowerShell to Install SharePoint. and technet scripted deployment

 Summary: For environments such as live the PowerShell/Slipstreamed options are best as they allow for recreation and input is always identical.  Manual install is fine for development servers however their is no advantage except for a lower learning curve for the IT admin.

Post below is a Manual Installation:
SP2010 install video
Install the pre-requisites

• Prerequisits will install roles and software you need internet access on the server to fetch the prerequisits software (this can be put on the server to stop the machine going to the Internet).
• Preferably have seperate instance of SQL 2008 R2 but for dev/demo machines. If 1 machine rather setup SQL devleoper or a instance (I dislike using SQL express).

Setup / SP 2010 install tips

• Install "Server farm" option not standalone
• "Complete" installs all component prefered option
• Connect to a new farm
• Database server name us name rather than IP (incase it changes)
• DB account (must already exist in AD)
• Passphrase used to connect new servers to this server farm (remeber/keep it)
• Kerbros - if your network supports it but use NTLM if you aren't sure.
• Wizard - follow screens, services can be heavy so add them when you need them, however for demo I select all services and create a new site collection - a good options is to use the Team Site Template.
• Need 3 accounts for min Best practices: 1) Managed Service account (domain user account) that SQL Server runs in, 2) Managed Service Account (domain user account) all services will be installed on this account (MS suggests using a seperate managed account for each service) on small farm s/dev I use 1 account,  and 3) Farm install account (domain account) this needs to be a local admin on each SP2010 server and have creator & dbsecurity accouts on SQL.
• 5 Accounts is a better option excluding the SQL services account namely:

1. SP-Install - domain account with admin local rights on each WFE also need SQL dbcreator and securityadmin roles (used to login and install binaries, use this account for add new servers to the farm),

2. SP-Farm - domain account no permissions, will be the account to run timer job and other key roles,
3. SP-Web-App-Pool - Content Web app account - Domain account only,
4. SP-Services - Install all services to use the same domain account, this can be seperate for each services but for easy of setup and mainentance use 1 account.  Exception is the User Profiles service, setup seperately using Spence Harbors post as the user domain account needs unique security, and
5. SP-Crawl - Used to crawl SP content.

Additional Info on accounts:

1. SQL Server needs to run as a windows service, you need an account, I would use a managed account in AD with no permissions called SP2010-SQLService.
2. Farm Installation account, you need to create a domain user account in AD, give the account local admin access to each SP2010 machine.  Call it SP2010-Admin.
3. SP2010 Service account/s, you need to create a managed service account with zero permissions in AD.  You can use 1 account or create a seperate account for each service (MS Best Practice).  I call my 1 account SP2010-Services. 

Use slipstreaming for SharePoint it's faster and consistant.
Use:

1. Windows 2008 R2 x64
2. SQL 2008 x64
3. On HyperV/VMWare except the db which should be a seperate physical machine/SAN

Update 08 November 2010:  Notes on deploying a 3 server farm consisting of 2 WFE's that are NLB using Windows NLB.  Installation done using AutoSPInstaller. 
Installation Notes for a 3 server NLB SharePoint 2010 farm

Update 10 November 2010: SharePoint install account - Todd Klindt.
Update 11 May 2011: SharePoint 2010 database management article

Logging custom error for SharePoint 2010 custom code

Problem: SP2010 has good logging in the ULS logs. Unlike MOSS you can write to the ULS logs in SharePoint 2010, allowing for a single consistant place to log your errors. The event viewer logs errors to a lesser extent than ULS where administrators tend to look first. However, on all projects the topic of how and where to log custom coding errors comes up. What level should the event be caught at and where should they be placed. I have seen multiple was of doing this and it really comes down to what the previous projects used and how do you want to monitor errors.
Tips:

1. Don't write code to function using error catching i.e. I have seen developers catch a specific error and from this position they know the code is to follow specific logic -it's very inefficient. Write code to deal with all situations.
2. Catch errors as specifically as possible, then decide if the error should be bubbled up or can it be dealt with via the logs. But catch the appropriate errors so they are logged.
3. You can write to Unified Logging Service(ULS) logs but these are often not checked or hard to find issues you have thrown up in your code, so consider using a logging block such as Microsoft's Enterprise Logging blocks or Log4net. Ted Pattison suggests writing to ULS and he know his stuff so if you don't have another specified logging policy write to the ULS. And if you do have another logging method consider writing to the ULS anyway.
4. There are a lot of logging applications for .NET, and most companies tend to have logging code ready for implementation on your SP 2010 project.
5. I have seen a MS gold partner use tracing on all projects. So when an error occurs they turn on tracing and try replicate the issue in production environments. Far better to catch errors so you can get your issues resolved quickly and don't need to change config setting or leave tracing enabled on the live production boxes.
6. A web part error can cause an entire page to throw an application error. However don't throw try catch blocks around all code, rather try catch the errors at more appropriate junctions such as at the service layer. Once again it really depends on the WP.

Summary: Avoid using errors for logic (pretty obvious). Log the errors to a distinct area (database, file system(xml files are pretty useful)) to identifying where and what the issue is. Catch specific errors -try not to catch general exceptions and if you do need to catch general exceptions make sure you have looked for more specific exceptions such nullrefobjectexceptions.

More Info:
MSDN SharePoint Logger - http://msdn.microsoft.com/en-us/library/ff798361.aspx
Writing to the ULS
Update: 5 Dec 2010 - Writing to ULS using SP2010 by Waldek Mastykarz
Update: 18 Jab 2011 - MSDN article on logging and debugging

The SharePoint Logger
using Microsoft.Practices.ServiceLocation;ILogger logger = SharePointServiceLocator.GetCurrent().GetInstance();
using Microsoft.Practices.SharePoint.Common.ServiceLocation;
using Microsoft.Practices.SharePoint.Common.Logging;
IServiceLocator serviceLocator = SharePointServiceLocator.GetCurrent();
ILogger logger = serviceLocator.GetInstance();
logger.TraceToDeveloper("Unexpected condition");

Update 14 Dec 2010 - ULS Viewer - Tool to view ULS log and filter data
Update 23 Dec 2010 - Tracing using CorrelationId

SharePoint 2010 using VS 2010 and Visual Source Safe 2005

Overview: Client is implementing SP 2010, they don't have TFS and the source control is Visual Source Safe (VSS) 2005.
Steps to integrate VS2010 into VSS 2005:

• Install the VSS client on your development server/environment;
• Open the VSS explorer and add the project (Subsequent developers need to locate the file and download a copy) and set the working directory if pulling the project;

• Open VS2010, select Tools > Options > Source Control > Plug-in Selection > Change the source control to "Microsoft Visual SourceSafe";

• I prefer to hold my solution file locally and pull my projects however a lot of architects prefer to keep the solution in Source Control. Open the solution in VS2010 and you should be ready to run.
• Tip: If you projects won't deploy click on the project and look at the "Site Url", this gets cleared down. Enter you local development server url i.e. http://demo1.app.dev/

Inefficient Queries - SPMetal join issue

Problem: LINQ to SharePoint 2010 (SPMetal) is not querying 2 SharePoint lists, when using SPMetal on a visual web part. I am trying to JOIN 2 lists inside a web part. The Application error message is "The query uses unsupported elements, such as references to more than one list, or the projection of a complete entity by using EntityRef/EntitySet.".



Hypothesis: At the SharePoint conference in Vegas Oct 2009 I remember in 1 of the sessions that inefficient LINQ queries are blocked. This was a little vague but a good start noting the stack trace message "InvalidOperationException: The query uses unsupported elements, such as references to more than one list, or the projection of a complete entity by using EntityRef/EntitySet.]
Microsoft.SharePoint.Linq.Rules.QueryEfficiencyProcessor.Process(Expression e)" .
Exception Details: System.InvalidOperationException: The query uses unsupported elements, such as references to more than one list, or the projection of a complete entity by using EntityRef/EntitySet.
Using LINQ I can query each of the lists.  Each list has a field of the same type that I am trying to join on. I was totally confused but realised it must be an issue with the efficiency/validity of the query. A simpler query worked so it looks like the CAML is not being correctly formed. I extracted the CAML using the DataContext Log method.
My choices are:

1. To get and fix the CAML query and then run a CAML query manually in my code;
2. Use LINQ to objects; or
3. Perform 2 queries and link the data manually.

Resolution:
Reading a post on MSDN on 2 stage queries gave me a quick fix. By adding the ToList() method, the query works.
var empQuery = from bug in Bugs.ToList()
join emp in Employees on bug.AssignedTo equals emp.Title
select new
{ emp.Title,
Project = bug.Project
};

Explanation: ToList() method forces immediate query evaluation and returns the generic that contains the query result.  As described in the MSDN article above LINQ can't convert the LINQ to SharePoint into a CAML query so by using the ToList() method, the query is broken into 2 stages.  This will apply to queries that use JOINS, UNIONS, and various other LINQ operators as described in the MSDN Unsupported LINQ queries article.

Tip: Turn off Object Change Tracking if you are only reading data.

LINQ to SharePoint Posts on this Blog

• Dynamic Linq
• Inefficient Linq Queries
• SPMetal issues

Links:
Linq to SharePoint 2010 examples
Tip: CAML query for optimised speed settings, only bring back columns you will use querySearch.ViewFieldsOnly = true;
querySearch.IncludeMandatoryColumns = false;
Display the CAML that LINQ to SharePoint is running

Connected Web Parts in SP2010

Good starting point for creating consumers & Provider web parts

Delete Web Parts from a page using the browser

Problem: A web part fails, is hidden and you can remove it using the edit mode on a SharePoint page.

Resolution: A custom crafted url will display a list of web parts and allow you to cleanup the page as required. Enter the domain plus http://url/_layouts/spcontnt.aspx?&url=page>

If the page with the problem had the url http://demo1.dev/myarea/printer.aspx

You would enter the url http://demo1.dev/_layouts/spcontnt.aspx?&url=myarea/printer.aspx

More Info

Alternatively Approach: Append ?content=1 to the url and you will be directed as above.

Example:

Page: http://www.demo.dev/sites/sponline/Pages/ViewProperty.aspx

Make it: http://www.demo.dev/sites/sponline/Pages/ViewProperty.aspx?contents=1

You get re-directed to: http://www.demo.dev/sites/sponline/_layouts/spcontnt.aspx?&url=%2fsites%2fsponline%2fPages%2fViewProperty.aspx