The Developer Dashboard in SharePoint 2013

Overview:  The Developer Dashboard is a great tool added to SharePoint in the 2010 version.  SharePoint 2013 has additional features and a new look.  This post explores the new developer dashboard.

SharePoint 2013 Dev dashboard, note the new tabs that have been added.

 

ULS log files are created but are empty

Problem: I have created a SharePoint 2013 farm with a custom location for my log files e.g. d:\logs.  On the 1st SP2013 VM that created the SP farm using AutoSPInstaller, the logs are present and logging correctly.  On the other/remaining SP VM's, the logs are created every 30 min however, the lof files are empty.

Initial Hypothesis: I had no idea and after looking at google the answer is permissions on the local file system.  Justin Kobel's post gives the fix.

Resolution:

Or run this Powershell on each VM:

PS> $computer= "$env:computername"
PS> $group="Performance Log Users"
PS> $domain="demo"
PS> $user="pbeck"
PS> $de = [ADSI]"WinNT://$computer/$Group,group"
PS> $de.psbase.Invoke("Add",([ADSI]"WinNT://$domain/$user").path)

PS> net stop SPTraceV4
PS> net start SPTraceV4

 

Tip: Using the Invoke-Command you can loop thru and connect to the SP VM's remotely and run the change on each VM in the SP far.  PS to loop thru and change local security groups is here and the xml is here.

 

 

More Info:
You will notice that the files within Windows explorer are shown with a blue colour.  This is to show the files/folders are compressed. 
Blue means compressed.
Green means encrypted.

Finding Correlation Errors on a SP2013 farm

Background: SP 2013 has rich and expansive logging/tracing capabilities.  Logging is done via the Unified Log Service (ULS).  This will add logs to the tracelogs (often refered to as the ULS logs or ULS trace logs or ULS, it doesn't matter except you need to understand the ULS service is not only the trace log) and the Windows Event Viewer.  Anything logged in the Event Viewer log will also be in the ULS trace logs.

It is worth check how your logging is setup on your farm.  I change my default location for my ULS trace logs.  Change the logging so it matches your farms requirements.

On a small farm, it's normally pretty easy to take a Correlation Id / the unique GUID generate for the SharePoint request, open the trace log using notepad and find the error.  The default is to create a trace log every 30 minutes, these log files have a lot of data in them on busy production farms, and as you may have a large farm you also have multiple logs to check. I use Microsofts'd unsupported ULSViewer to look at all my logs regardless of farm size.  You can trace the logs in a live format and then filter out what you need.  Another option is to open existing errors to get historical issues.  If you know the datetime and server where the error occured, you open the correct log file (it is labled with a datetimestamp) and then either filter for the correlationId or look around the time the error occured.

Lastly, timer jobs ship entries from the ULS logs into the SharePoint Logging Database (SP_UsageandHealth).  You can directly query the SP_UsageandHealth database using T-SQL.

Tracing Correlation Errors on a SharePoint 2013 Farm.
User passes you a correlation Id and the date/time when the error occured, find the apprioate ULS trace log.  Open the log using ULSViewer and filter for the CorrelationId.  If you can reproduce the bug, you have the developer dashboard that can be turned on (performane penalty) selectively, their is a new SP2013 tab "ULS" this will show you the ULS trace snippet relating to this request.

On a big farm you may want to1st find out which server in the farm had the error:

Merge-SPLogFile -Path ".\error.log" -Correlation "5ca5555c-8555-4555-555b-f555af4d5555"

Tip: Be aware this is a heavy process, so restrict which logs you will merge.
 

Use ULSViewer to find the correlationId and review the logs.

Use IT tools or Fiddler to examine the http response from SharePoint to get the correlation Id, this is the SPRequestGUID (assuming it is not show on the error message).

 

More Info:

Tobias Zimmergren has a great post on working with Correlation Id's.

http://www.sharepointblog.co.uk/2012/09/logging-capabilities-of-sharepoint/

http://habaneroconsulting.com/insights/An-Even-Better-Way-to-Get-the-Real-SharePoint-Error

List of the ULS viewers

http://www.sharepointblog.co.uk/2012/09/logging-capabilities-of-sharepoint/

SP2010 Logging Summary

Overview: ULS & Logging can be confusing for SharePoint.  It's really pretty simple and this post aims to clarify the basics of error logging on SP2010 farms.

Tip: Usage records analytics (referred to as Web Analytics).  Usage and ULS logs as well as other metrics and the event log are put into the Reporting database that is configured in "CA" under "monitoring" for "Configure Usage and health data collection".  This post only looks at ULS.

Summary:

1. Each event in SharePoint is given a unique correlationId, this is a unique identifier to help pull out the appropriate log entries from the ULS. 
2. ULS create .log files that can be opened individually using notepad.
3. ULSViewer is useful for examining multiple log files at once.  There are a few products that do the same thing but I use the Microsoft ULSViewer.
4. The amount of information logged by the ULS can be set in CA or using PS (see image 1 below).
5. If you enable timer jobs ("Diagnostic Data Provider: xxx" * 7), the data is shipped from the ULS logs on each server into the SP Logging DB.  Useful as it provides a central view and you can backup/achieve the data.  The Timer jobs pull in data from ULS, EventViewer, Timer Jobs and various other performance indicators.

Image 1.  Amending the default logging settings using PowerShell  (Can also use CA)

  

More Info:
Setting Logging levels: http://technet.microsoft.com/en-us/library/ff607887
Debugging & Logging: http://msdn.microsoft.com/en-us/library/gg512103.aspx
Logging explained by Waldek: http://blog.mastykarz.nl/logging-uls-sharepoint-2010/
ULS Explained: http://community.bamboosolutions.com/blogs/sharepoint-2010/archive/2011/07/07/sharepoint-2010-cookbook-the-usage-of-sharepoint-2010-uls-logging.aspx
ULS Viewer by Spencer Harbar: http://www.harbar.net/archive/2010/10/06/ulsviewer-ndash-the-tool-that-no-sharepoint-practitioner-should-be.aspx

Sandbox solution ULS logging

Overview: Step-by-step instructions to create a logging module for Sandbox solutions using a full trus proxy that logs to the ULS.

Steps:
1.> Ensure Sandbox solutions service is running:

1.1.> If the "Microsoft SharePoint Foundation Sandboxed Code Service" is stopped start it either thru CA (shown above or PS). 

PS> Start-Service -Name SPUserCodeV4

1.2.> Ensure the Windows service is running.

2.> Create project to test the full trust ULS logging proxy.

2.1> Create a Visual Studio project that deploys as a sandbox solution (test rig)

2.2.> Create a visual web part for sandboxed solutions (VSIX/Extension maneger: Visual Studion 2010 SharePoint Power Tools)

Note:  The submit button does not perform any action as the Full Trust Proxy must be created and deployed so we can use it to log code to the ULS.

3.> Create a ful trust proxy that code in the sandbox solution can call to add the error message to the ULS logs.

3.1.> Create an empty SharePoint project using Visual Studio 2010that deploys to the farm.
3.2.> CKSDev (VSIX extension has a SPI template to create a full trust proxy).  I used the SPI to create my ful trust proxy.
3.3.> Create the code to log to the ULS

3.4.> Edit the full trust proxy to use the ULS logging code created above.

4.> Add the proxy code call to the test rig/proxy caller i.e. sandboxed solution code.

Tip: Resart the Sandbox solution work process after each deployment

Tip: Use Powershell to check a correlation Id quickly:
PS>Get-SPLogEvent | ?{$_.Correlation -eq ""} | ft Category, Message -Autosize

References:
http://msdn.microsoft.com/en-us/library/gg622617.aspx
http://www.zimmergren.net/tag/sps
http://www.wictorwilen.se/Post/Working-with-SharePoint-2010-Correlation-ID-in-PowerShell-and-code.aspx

Creating a shared logging block

Overview:  A client host multiple applications on there SharePoint on premise farm and these need to be logged consistanly.

Hypothesis:  SP2010 provide easy logging ability to the ULS, Microsoft have the SharePoint Best Practices that includes the assemblies for logging, these can be extended to create a easy re-useable set of logging functions.  Key areas to cover are where are you logging to: ULS, Event Viewer.  Sandbox solution can be logged to ULS and the event viewer using a full trust proxy solution.  SharEPoint Online will need to log to a list on the hosted farm.

Microsofts Patterns and Practices code library for SharePoint 2010, is useful for providing the core logging functionality.  The Parrerns and Practices can be extended so the custom method for all application on the farm would call the logging code block that is hosted in the GAC.

More Info:
http://technet.microsoft.com/en-us/library/ee748656.aspx
http://blog.mastykarz.nl/logging-uls-sharepoint-2010/
http://www.alexangas.com/blog/2011/09/intro-to-sharepoint-2010-patterns-practices-logging/
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=23919

Updated 28 May 2020:
For C#, Power Platform, React, .... I advise to use: App Insights or https://serilog.net/
Log4Net is still very common

ULS Viewer

Problem:  The ULS log generated by SharePoint 2010 are difficult to read and review as the logging is store in flat text files.

Initial Hypothesis: ULS Viewer allows the administrator/developer to track, understand and solve issues on your SharePoint environment.  The ULS viewer is not a supported product my Microsoft however, it is alot easier to discover errors than using notepad and opening each ULS log.  Logs are located at C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\LOGS by default.

Resolution:
1.> Download the ULS Viewer
2.> Map a folder to the c drive on my UAT environment.

3.> Open the ULS Viewer.

4.> To view the logs as the are added, Click File > Open From > ULS, select the Logs location on your mapped drive.
5.> Add multiple WFE's log by using the File > Append From option.
6.> To view historic files, copy the log files from the server to your local machine and click File > Open From > File, select multiple ULS log files to see the server history.

How to use the ULS Viewer blogs:
Jie Li
Jeremy Taylor

Unified Logging Service (ULS)

Overview: ULS is made up of 3 components: Trace logs, Reporting database & the Windows Event viewer. 

Trace Logs:
Trace Logs by default are located in the 14 hive/SharePoint Root, by default they are created every 30 minutes and can be viewed in notepad. 
ULSviewer is available on codeplex to improve usability as the logs are easier to follow than if you use notepad.
Trace logs can grow extremely quickly.  Only log the appropriate messages for troubleshooting without making the logging to big.  Verbose logging cause huge amounts of logging but it is extremely detailed.  The amount of logging or what is logged can be set using the CA> Monitoring > Diagnotic Logging "Configure Diagnostic Logging".
SharePoint 2010 uses NTFS compression for log files to reduce storage on the WFE (logging) servers.
Consider moving the trace logs to a new drive so that the logs don't fill up the default drive (usually the c drive).  This change applies to all servers in the farm.
Logs by default are kept for 14 days, you can also hold them using sizes rather than duration.
Event Log Flood Protection in CA should always be turned on to stop repeated errors being recorded multiple time in you logs.
Correlation ID's as the name suggest helps us track errors from the UI through to the Trace Logs.  The end user when the receive an error are shown a unique correlation ID that can be used to track down the problem.  Correlation Id's as also shown on the developer dashboard and can be used to tract from the UI into the trace logs without errors.

Update 7 Oct 2010 - Useful Monitoring artlicle:
http://www.sharepointproconnections.com/content.aspx?topic=How-SharePoint-2010-s-Monitoring-Features-Can-Help-Admins&catpath=sharepoint
Update 14 Dec 2010 - ULS Viewer
Tool to view ULS log and filter data


Various SharePoint Logs Explained:
http://stackoverflow.com/questions/525197/where-is-the-default-log-location-for-sharepoint-moss

Logging custom error for SharePoint 2010 custom code

Problem: SP2010 has good logging in the ULS logs. Unlike MOSS you can write to the ULS logs in SharePoint 2010, allowing for a single consistant place to log your errors. The event viewer logs errors to a lesser extent than ULS where administrators tend to look first. However, on all projects the topic of how and where to log custom coding errors comes up. What level should the event be caught at and where should they be placed. I have seen multiple was of doing this and it really comes down to what the previous projects used and how do you want to monitor errors.
Tips:

1. Don't write code to function using error catching i.e. I have seen developers catch a specific error and from this position they know the code is to follow specific logic -it's very inefficient. Write code to deal with all situations.
2. Catch errors as specifically as possible, then decide if the error should be bubbled up or can it be dealt with via the logs. But catch the appropriate errors so they are logged.
3. You can write to Unified Logging Service(ULS) logs but these are often not checked or hard to find issues you have thrown up in your code, so consider using a logging block such as Microsoft's Enterprise Logging blocks or Log4net. Ted Pattison suggests writing to ULS and he know his stuff so if you don't have another specified logging policy write to the ULS. And if you do have another logging method consider writing to the ULS anyway.
4. There are a lot of logging applications for .NET, and most companies tend to have logging code ready for implementation on your SP 2010 project.
5. I have seen a MS gold partner use tracing on all projects. So when an error occurs they turn on tracing and try replicate the issue in production environments. Far better to catch errors so you can get your issues resolved quickly and don't need to change config setting or leave tracing enabled on the live production boxes.
6. A web part error can cause an entire page to throw an application error. However don't throw try catch blocks around all code, rather try catch the errors at more appropriate junctions such as at the service layer. Once again it really depends on the WP.

Summary: Avoid using errors for logic (pretty obvious). Log the errors to a distinct area (database, file system(xml files are pretty useful)) to identifying where and what the issue is. Catch specific errors -try not to catch general exceptions and if you do need to catch general exceptions make sure you have looked for more specific exceptions such nullrefobjectexceptions.

More Info:
MSDN SharePoint Logger - http://msdn.microsoft.com/en-us/library/ff798361.aspx
Writing to the ULS
Update: 5 Dec 2010 - Writing to ULS using SP2010 by Waldek Mastykarz
Update: 18 Jab 2011 - MSDN article on logging and debugging

The SharePoint Logger
using Microsoft.Practices.ServiceLocation;ILogger logger = SharePointServiceLocator.GetCurrent().GetInstance();
using Microsoft.Practices.SharePoint.Common.ServiceLocation;
using Microsoft.Practices.SharePoint.Common.Logging;
IServiceLocator serviceLocator = SharePointServiceLocator.GetCurrent();
ILogger logger = serviceLocator.GetInstance();
logger.TraceToDeveloper("Unexpected condition");

Update 14 Dec 2010 - ULS Viewer - Tool to view ULS log and filter data
Update 23 Dec 2010 - Tracing using CorrelationId