NullReferenceExcepetion when viewing/editing an existing list item after the list instance has been deployed

Problem: When viewing or editing and item in a list, I receive an application error (NullReferenceException).  This happens when I redeploy a list, my list is defined using site columns, content type and the list definition is based on the content type.

My ULS logs contained the following information:

Failed to cache field with id

Unable to open Lookup list 'Lists/xxxList'.[Error was 0x8007....]
System.NullReferenceException: Object reference not set to an instance of an object.   at Microsoft.SharePoint.Publishing.FieldCache.TryGetValue

Initial Hypothesis:  ULS logs show me that a lookup list is not being found.  By examining the columns on the list instance I can see they are not being created correctly.  My initial thought was my xml was wrong, once it was reviewed and I found it to be correct I realised it was a caching issue.

Resolution:  Clear the Visual Studio Cache

1. Right click on "Project" in Solution Explorer and click "Retract";
2. Close Visual Studio 2010; and
3. Open the "Project" in Visual Studio 2010 and "Deploy" the solution. 

SP2010 AutoSPInstaller NLB farm install issue - Trial period for this product has expired

Problem: I created a farm using the AutoSPInstaller project. All works perfectly. I then joined a 2nd SharePoint 2010 server using the script however the script was change to not create the services again. All works and I have a 3 server farm including the database server. I then added Windows 2008 R2 network load balancing on the 2 SharePoint 2010 servers. If try access the default Team Site created when I built the 1st server on the new server I get the error "The trial period for this product has expired." Event log doesn't show an error and the ULS shows the following error "Exception thrown while adding control 'Microsoft.SharePoint.Portal.WebControls.SearchBox' : Thread was being aborted ...".

Hypothesis: The install adds the IIS web site created on my initial farm build on the 2nd SharePoint server. Routing is working and the error occurs when the request is routed to the 2nd SP2010 server. The response is coming from the IIS web server and we know the issue is within the specific web app.  In MOSS there were 2 causes for the error message and this is clearly not the SP2 bug so it looks more like the app pool account bug.

Resolution: Create a new Web Application and create a new root Site Collection. The error goes away. OR
Using Powershell change the Managed account then using IIS Manager on each web server go to the approoriate app pool and change the account to the new account. 
OR
Use Central Admin to change the service account.  CA > Security > Configure service account.  You will need to reset IIS on each web server.
Demo on changing app pool managed accounts

Sandbox solution deployment error

Problem: Deploying a solution package to a sandboxed site using Visual Studio 2010 I receive the error "Error occurred in deployment step 'Activate Features': Timeout occurred while running the SharePoint Sandboxed Code service. This service is required to run sandboxed solutions in SharePoint.  Please ensure the service is configured correctly in SharePoint.". 
Resolution: I restarted my "SharePoint 2010 User Code Host" Windows Service.

More: Common error after initial install of SharePoint Error: Cannot start service SPUserCodeV4

http://www.sharepointdevwiki.com/display/spadmin2010/Configuring+Sandboxed+Solutions+-+Cannot+start+service+SPUserCodeV4+on+computer

SPMetal is throwing a ThreadAbortException

Problem: I am inserting new items into a list using Linq-to-sharepoint (SPMetal), my code throws a ThreadAbortException exception. 

Initial Hypothesis: When I run the code as a Site Collection Owner the new list item is inserted.  So this is a permissions issues.  I increase the permissions on the user that can't insert the list item.  It works so this is clearly a permissions issue.  After the user tries to perform the insert list item, they are directed to the "Error: Access Denied" message on the web page.  Clearly the error message is misleading however, SharePoint seems to understand the ThreadAbortException is a permissions issue.  The issue is the user does not have sufficient permissions to perform the action.  By elevating the user permissions the error goes away.  All well except the error logging is throwing an odd error, after some digging this issue has been blogged before however not for LINQ to SharePoint.

Resolution: Assign the correct permissions to the user performing an action.

SPMetal Invalid file name error

Problem: SPMetal fails with the following error:
Error: Invalid file name.
The file name you specified could not be used. It may be the name of an existing file or directory, or you may not have permission to access the file.

Initial Hypothesis: Permissions on the directory are wrong or the SPMetal proxy code file being created is locked.  Folder permissions haven't change and the file is removed from previous runs.  So my initial hypothesis is wrong.  Previously I was working on building lists created from custom content types and deployed via a feature.  There are 2 lists based on content types.  The list definition, custom content type & site columns have been removed.  The 2 lists still appear in my UI and when I click on the list I get an application NullReferenceException error.  Makes sense as SharePoint hasn't cleared up the list instances.
                            ============
Updated Problem: I have 2 lists on my SharePoint 2010 site that I can see in the web UI and through my Visual Studio Server Explorer however, I can't get into the UI as I am getting the NullReferenceException error.  I need to remove the lists to allow SPMetal to work.

Initial Hypothesis: Use another tool to delete the list, VS solution explorer won't let me update the list.  Use SharePoint Designer, on opening SPD it won't show me any lists or libraries.  So I'm left with using an stsadm cmd or Power Shell.  As I am loving PowerGui I went for the Power Shell Solution.
Resolution: Open the SharePoint Power Shell window, delete the lists using the following code:
$url = "http://demo1"
$site=new-object Microsoft.SharePoint.SPSite($url)
$web = $site.OpenWeb()
$list = $web.Lists["Customers"]
$list.Delete()
Now SPD works, SPMetal works - Problem resolved.
Summary: I created lists from content types, I removed the content types, list definitions however I left the list instance.  This cause SPD, SPMetal to err.  Cleaning the defunct lists using Power Shell corrects the issues.
More Info:
PowerGui sharePoint CheatSheet

SharePoint 2010 Error occurred in deployment of step ‘Activate Features’: the field with ID defined in feature was found in the current site collection or in a sub site.

Problem: I can't redeploy Site columns using Visual studio 2010 to an SP2010 site collection.  I get the error "Error occurred in deployment of step ‘Activate Features’: the field with ID defined in feature was found in the current site collection or in a sub site".
Resolution: Retract wsp, restart VS2010, open the project and deploy the solution.  VS2010 is caching data.
Microsoft kb

SharePoint 2010 console application System.IO.FileNotFound error

Problem: Built a local console app to access a SharePoint site, add System.Web & Microsoft.SharePoint references to the console project.  Run the code and get a System.IO.FileNotFound error.
Code SPSite site = new SPSite(http://demo1);
Hypothesis: Check url is correct (it was).  VS2010 console apps are set to be x86 by default.  SharEPoint only works on x64 so Microsoft.SharePoint is x64. 
Resolution:  Open VS2010, right click the console project.  "Properties" > "Build" tab, set "Platform target to "x64".
Tip: Uses the .NET 3.5 for console applications.
More info:  Top 2 search on Google yielded the solution.  Console FileNotFound error. and 2nd solution System.IO.FileNotFound Error

Custom error and call stack info is not showing on my SP dev machine

Problem: I have turned my web.config file customErrors mode to "Off" however the error is telling me to turn it off or RemoteOnly. Therefore my screen is not showing me the problem. You can debug but the feedback is not provided on the erring SP page.
Hypothesis: Changing the web.config in the IIS SharePoint 2010 web site does not take effect. Ensure the web.config is correct and ensure other related web.configs used by the site are updated.
Resolution: Update the following 3 web.configs:

• 14\CONFIG
• 14\TEMPALTE\LAYOUTS
• Current IIS web.config

web.configs should read:
customErrors mode="Off" />
SafeMode MaxControls="200" CallStack="true" ... AllowPage LevelTrace="true">
compilation batch="false" debug="true" >
More Info:

http://social.msdn.microsoft.com/Forums/en-US/sharepoint2010general/thread/2673ae4e-0b26-42ba-a158-1ceb63985721
http://blogs.msdn.com/b/amitsh/archive/2007/11/01/why-i-am-not-able-turn-off-customerrors.aspx

Logging custom error for SharePoint 2010 custom code

Problem: SP2010 has good logging in the ULS logs. Unlike MOSS you can write to the ULS logs in SharePoint 2010, allowing for a single consistant place to log your errors. The event viewer logs errors to a lesser extent than ULS where administrators tend to look first. However, on all projects the topic of how and where to log custom coding errors comes up. What level should the event be caught at and where should they be placed. I have seen multiple was of doing this and it really comes down to what the previous projects used and how do you want to monitor errors.
Tips:

1. Don't write code to function using error catching i.e. I have seen developers catch a specific error and from this position they know the code is to follow specific logic -it's very inefficient. Write code to deal with all situations.
2. Catch errors as specifically as possible, then decide if the error should be bubbled up or can it be dealt with via the logs. But catch the appropriate errors so they are logged.
3. You can write to Unified Logging Service(ULS) logs but these are often not checked or hard to find issues you have thrown up in your code, so consider using a logging block such as Microsoft's Enterprise Logging blocks or Log4net. Ted Pattison suggests writing to ULS and he know his stuff so if you don't have another specified logging policy write to the ULS. And if you do have another logging method consider writing to the ULS anyway.
4. There are a lot of logging applications for .NET, and most companies tend to have logging code ready for implementation on your SP 2010 project.
5. I have seen a MS gold partner use tracing on all projects. So when an error occurs they turn on tracing and try replicate the issue in production environments. Far better to catch errors so you can get your issues resolved quickly and don't need to change config setting or leave tracing enabled on the live production boxes.
6. A web part error can cause an entire page to throw an application error. However don't throw try catch blocks around all code, rather try catch the errors at more appropriate junctions such as at the service layer. Once again it really depends on the WP.

Summary: Avoid using errors for logic (pretty obvious). Log the errors to a distinct area (database, file system(xml files are pretty useful)) to identifying where and what the issue is. Catch specific errors -try not to catch general exceptions and if you do need to catch general exceptions make sure you have looked for more specific exceptions such nullrefobjectexceptions.

More Info:
MSDN SharePoint Logger - http://msdn.microsoft.com/en-us/library/ff798361.aspx
Writing to the ULS
Update: 5 Dec 2010 - Writing to ULS using SP2010 by Waldek Mastykarz
Update: 18 Jab 2011 - MSDN article on logging and debugging

The SharePoint Logger
using Microsoft.Practices.ServiceLocation;ILogger logger = SharePointServiceLocator.GetCurrent().GetInstance();
using Microsoft.Practices.SharePoint.Common.ServiceLocation;
using Microsoft.Practices.SharePoint.Common.Logging;
IServiceLocator serviceLocator = SharePointServiceLocator.GetCurrent();
ILogger logger = serviceLocator.GetInstance();
logger.TraceToDeveloper("Unexpected condition");

Update 14 Dec 2010 - ULS Viewer - Tool to view ULS log and filter data
Update 23 Dec 2010 - Tracing using CorrelationId

Using Session State in SharePoint 2010

Problem: I have setup session state but when I try add a custom object (class instance) to me ASP.NET session state (SQL) I get an error "unable to serialize the session state. in 'stateserver' and 'sqlserver' mode asp.net will serialize".
Initial Hypothesis:
I am adding a class to my session state
Client Instance = new Client("Paul", "Male");
HttpContext.Current.Session["ASPNETSession"] = Instance;
The Client class is not marked as Serializable
Resolution:
Mark the "Client" class as [Serializable]
http://dotnetguts.blogspot.com/2009/06/steps-for-session-inproc-mode-to.html