Tuesday 11 March 2014

Capturing data for SharePoint

I got an email from an old school friend that heard I may do some SharePoint stuff. 

"I need your advice on a Sharepoint question.  We have a client that need users to capture forms and the ablity to create new forms on the fly, does this sound possible?"

My dashed off reply is below  - comments are welcome

On the SharePoint thing, this is the deal with forms. InfoPath was the standard for creating web forms for SharePoint, saying that about 3 weeks ago, MS announce it is no longer the product of choice and it will not be support after 10 years. It really comes down to how hectic the requirement is where you want you data stored.

SharePoint out of the box allows for users to create lists, this are not too complex and the logic is generally pretty simple. It works really well if your requirement is simple web forms, lots of them and not relational data. All native, very little training but customizing the default look and functionality gets expensive real quick (inject custom JS), there is also a tool SharePoint designer that can be used to customers the forms. When you create a list the CRUD forms are all created for the list.

InfoPath - tool to draw forms custom logic, lots of issues when it get complicated but if you need a lot of forms fast and need some logic this is still a good option.

K2 and Ninetex have forms engines, I have used smartforms from K2, For forms for workflows and building complex forms this is a good option but more if you have a dedicated forms team/guy. If you told me you need 1000 forms with complex logic and more forms need to be added in time, there is workflow and you will have dedicate form requirements this is a good option but be careful it is not as easy as folks may make out.

Pdf share forms work with SP, so if your client has pdf forms - make sure you look at this.  I've never used this approach but it seems plausible.

Custom options, such as SharePoint Designer aspx, you can build and deploy aspx pages, slow but good for customization. Good option if you have a unique complex requirements (think a drawing tool) as basically you have full C# control. You can also create web apps and consume them in SharePoint.

With what I think your skill sets is at .., it is probably also worth looking at using MVC or creating the forms in .NET code (webforms), then display using iFrame or the new app model in SP2013. You secure the app using claims based auth/OpenId/OAuth.
Those are your basic options. Send me some more detail and I can try give you a clearer match.

Also see:
http://go.limeleap.com/community/bid/286331/Forms-in-SharePoint-Seven-Ways-to-Create-a-Form-in-SharePoint
http://blogs.office.com/2013/03/04/options-to-create-forms-in-sharepoint-2013/#comments

With SharePoint 2013, look at CSR.

Ultimate Forms from InfoWise offers a good option for form capture and output.
Stratus Forms looks like a great tool.

Update 09 March 2016:
·         SharePoint 2016 on-prem. shall support InfoPath Forms Services until 2026 (extended support only from 2021).
·         InfoPath Forms on Office 365  supported until further notice.
·         No InfoPath 2016 as part of Office 2016, use the InfoPath 2013 desktop application to build forms.

Friday 7 March 2014

EventLog Error Fix

Overview: After building my farms I trawl through the ULS and event logs to look for logs messages to identify any issues.  This post contains errors from my event logs that hopefully will help me in future.

Problem: My event log shows a Windows/IIS error whereby the IIS sites application pool uses a service account that does not have a user profile on the machine.  The error message reads "Event Id: 1511 Windows cannot find the local profile and is logging you on with a temporary profile. Changes you make to this profile will be lost when you log off."

Verify the issue:



Resolution: (IEDaddy's post gave me the resolution)
1.> Stop the processes that use the account (I stopped the web sites that used the application pool account "demo\OD_Srv")
2.> cmd prompt> net localgroup administrators demo\OD_Srv /add
3.> cmd prompt> runas /u:demo\OD_Srv /profile cmd
4.> in the new cmd prompt run > echo %userprofile%
5.> Check the user profiles and verify the profile store for the account (demo\OD_Srv) has a status of "Local"
6.> Remove the account from the local administrators group ie cmd> net localgroup administrators demo\OD_Srv /delete

More Info:
http://www.brainlitter.com/2010/06/08/how-to-resolve-event-id-1511windows-cannot-find-the-local-profile-on-windows-server-2008/
http://todd-carter.com/post/2010/05/03/give-your-application-pool-accounts-a-profile/

*************************

Problem: EVENT ID: 8321 - Task Category: Topology
A certificate validation operation took 120053.1569 milliseconds and has exceeded the execution time threshold. 

Resolution:  I performed various steps:
1.> Host entry add the host entry:
127.0.0.1  crl.microsoft.com
2.> Trust the SP root cert
http://support.microsoft.com/kb/2625048

Import the Trusted certificate

3.>  Reduce the time when the crl check is done (not a fix but it will fail quicker and carry on)

This post may also help - but it wasn't my issue: http://stevesps.blogspot.co.uk/2013/01/sharepoint-foundation-event-id-8321.html

 *************************

Problem: Event Id: 8313 - Task Category: Topology
A failure was reported when trying to invoke a service application: EndpointFailure
Process Name: w3wp
Process ID: 5640
AppDomain Name: /LM/W3SVC/1647355528/ROOT-1-13036555135555957
AppDomain ID: 2
Service Application Uri: urn:schemas-microsoft-com:sharepoint:service:649a3e7c090555059555c7a101555576#authority=urn:uuid:55b29cf855594c76555658fca66dac65&authority=https://sv-sp-web1:32844/Topology/topology.svc
Active Endpoints: 2
Failed Endpoints:1
Affected Endpoint: http://sv-sp-app2:32843/649a3e7c0904495552e4c7a555d64555/MetadataWebService.svc

Initial Hypothesis: It looks like the Web front ends cannot coomunicate with the MetadataWebService.svc, run mmc > file > add/remove snapin > snap-in "certificates" > Add > Computer Account > Local Computer > OK.
Expand "Certificates" > SharePoint > Certificates.  Open the certs and check if they are verified.  In my case my wfe's are good but my app servers do not have a valid certificate as shown below.

Resolution:
PS> $rootCert = (Get-SPCertificateAuthority).RootCertificate
PS> $rootCert.Export(“Cer”) | Set-Content C:\SharEPointRootAutority.cer –Encoding Byte




http://khalidstech.blogspot.co.uk/2012/11/certificate-validation-errors-in.html

Automation to add the SharePoint Root Certificate is done very nicely in this post: http://lennytech.wordpress.com/2013/06/18/powershell-install-sp-root-cert-to-trusted-root/

 **************

Disable CRL check (I believe this is from AutoSPInstaller)
Set-ItemProperty -path "HKCU:\Software\Microsoft\Windows\CurrentVersion\WinTrust\Trust Providers\Software Publishing" -name State -value 146944
set-ItemProperty -path "REGISTRY::\HKEY_USERS\.Default\Software\Microsoft\Windows\CurrentVersion\WinTrust\Trust Providers\Software Publishing" -name State -value 146944
get-ChildItem REGISTRY::HKEY_USERS | foreach-object {set-ItemProperty -ErrorAction silentlycontinue -path ($_.Name + "\Software\Microsoft\Windows\CurrentVersion\WinTrust\Trust Providers\Software Publishing") -name State -value 146944 

*************
 

Problem: Event Log is capturing EventId: 2159 Source: SharePoint Foundation Error message refers to Event 8306 within the ULS logs.

Resolution: Edit the web.config allowing the ULS to capture additional information relating to the error.  The resulting error show the common SharePoint COM class factory error.  In this scenario changing the "SecurityTokenService" app pool "Load User Profile" property to true correct the underlysing issue.

***************
 

Thursday 27 February 2014

Adding Additional Search Crawl database to a SP2013 Search farm

Problem: By adhereing to Microsoft recommendations for search farms, a threshold that is passed fairly quickly is that the Search Crawl database should not exceed 20 million items per database.  I have a 12 server search farm with four partitions (recommendation 10 million items per index partion).  I need to add a second Crawler DB.

Initial Hypothesis: Figure out your current crawl databases.


Add additional Search Crawl databases
 Review the existing crawl search databases

I proved this on a test system by resetting my index and recrawling, the results after adding 3 content db's are shown below.



Resolution:  Add more crawl database to the search service application.

$SSA = Get-SPEnterpriseSearchServiceApplication$searchCrawlDBName = "SP_Search_CrawlStore2"
$searchCrawlDBServer = "SP2013-SQL3" # SQL Aliase could be the Conn str
$crawlDatabase = New-SPEnterpriseSearchCrawlDatabase -SearchApplication $SSA -DatabaseName $searchCrawlDBName -DatabaseServer $searchCrawlDBServer
$crawlStoresManager = new-Object Microsoft.Office.Server.Search.Administration.CrawlStorePartitionManager($SSA)
$crawlStoresManager.BeginCrawlStoreRebalancing()

Powershell to see what is going on:

cls
$SSA = Get-SPEnterpriseSearchServiceApplication
$crawlStoresManager = new-Object Microsoft.Office.Server.Search.Administration.CrawlStorePartitionManager($SSA)
Write-Host "CrawlStoresAreUnbalanced:" $crawlStoresManager.CrawlStoresAreUnbalanced()

Write-Host "CrawlStoreImbalanceThreshold:" $ssa.GetProperty("CrawlStoreImbalanceThreshold")
Write-Host "CrawlStoresAreUnbalanced:" $crawlStoresManager.CrawlStoresAreUnbalanced()
Write-Host "CrawlPartitionSplitThreshold:" $ssa.GetProperty("CrawlPartitionSplitThreshold")
$crawlLog = New-Object Microsoft.Office.Server.Search.Administration.CrawlLog $SSA
$dbInfo= $crawlLog.GetCrawlDatabaseInfo()

Write-Host "Number of Crawl Databases:"
$dbInfo.Count$dbInfo.Values

More Info:
http://blogs.msdn.com/b/sharepoint_strategery/archive/2013/01/28/powershell-to-rebalance-crawl-store-dbs-in-sp2013.aspx

Wednesday 26 February 2014

Remote Debugging SharePoint 2013

Overview:  Object reference not set to an instance of an object is all the information I have in my customers ULS logs.  I need to debug an application in production that does not contain Visual Studio 2012.

My Steps:
  • Ran the Remote Debugguer (msvsmon.exe) on 1 of the production servers
  • Added a host file to point to my production server "SP-WFE1" (optional depending on where the web requests originate from)
  • Opened Visual Studio 2012 on a VM containing the projected (hooked up to TFS 2012) and opened the solution (.sln).
  • Select Tools > Attach to Process

..... Finish when time permits


More Info:
http://msdn.microsoft.com/en-us/library/ff649389.aspx
http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2008/03/05/how-to-remote-debugging-a-web-application.aspx
 

Sunday 23 February 2014

SharePoint Online Random Tips

1.> SPO and PowerShell:
To connect and use SharePoint online you cannot have a small business plan (P).  The Medium and enterprise Office 365 plans allow the SPO PowerShell scripts to run (very few so far).
Steps to use PS against SPO:
1.1.> Download and install the SharePoint Online Management Plugin
 1.2.> Add the snap-in/module to your PowerShell (or use the console with the snap-in already loaded)
 1.3.> Connect to your SharePoint site and validate it works.

=============================================================

2.> OOTB SPO Workflows:
P Plan only has three state workflow
E Plan has 5 basic workflows: signature, 3 state, approval, ..

Sandbox does not support Visual Studio based (code) workflows but you can create declaritive workflows via SPD.

SharePoint 365 Workflow Creation Process:
  • Prototype in Visio 2010 by using the SharePoint specific workflow, this can be demonstrated with the end user as they understand Visio workflow diagrams to explain there business process.
  • Export Visio 2010 diagram to SPD
  • Implement the workflow using SPD can integrate InfoPath forms
  • Custom actions are created using VS and can be deployed, the custom actions can be integrated into the SPD created workflow.
=============================================================

Saturday 8 February 2014

DevOps and SharePoint

Components of DevOps
  • IT Automation
  • Agile Development
  • Operation and dev teams working together
  • Service vitalisation
  • Continuous releases
  • Automated testing (Recorded Web UI & Unit testings)
  • Performance
Why DevOps
  • Collaboration btwn Dev & Ops teams; reduce unknowns early on, info easily available, better decision making, improved governance.
  • Increase apps faster; faster time to market
  • Improve App quality; environments available and not tedious to deploy
  • Reduce costs; fewer staff required as tasks are automated
DevOps Maturity Model
There are different levels of maturity and your final goal will depend on your company.  Operations departments are very different in a bank as opposed to Facebook.
The 4 main stages to DevOps Maturity are:
  1. Early - Manual traditional approach to software delivery.  Little or no automation.
  2. Scripted - Moving toward more automation.  Some scripting is used to assist the tracked recorded deployment process thru DTAP environments.
  3. Automation - Reusable common automation approach to releasing applications.  Workflow/orchestration using the automation capabilities used over all the DTAP environments.
  4. Continuous Delivery - end-to-end application delivery process from dev to production environments.  Probably includes Continuous Integration, nightly builds

Presentation at SharePoint Saturday 2013 on of Infrastructure as Code (IaC)
http://blog.sharepointsite.co.uk/2013/11/iac-presentation-for-sharepoint.html


Sunday 2 February 2014

Audit and documenting a SharePoint on prem farm

Overview:  I have had and done several SharePoint infrastructure audits over the past few years.  These can vary from running through checklists of best practices to see what the customer setup has and finding out why they may of chosen to diverge. 

There is a lot of good tooling and checklists to help you audit your farm such as the built in Health analyser, central admin, PowerShell scripts and a great tool to help document your farm SPDocKitSetup from Acceleratio.

Tooling:
  • PowerShell
  • Health analyser
  • Central Admin
  • ULS logs and event logs
  • IIS (IIS RealTime Log Viewer)
  • Fiddler
  • SQL Management Studio, SQL profiler
  • Tool to audit and document your SharePoint farm: SPDocKitSetup
  • Third party tools like: DocAve and Metalogix (Ideara had a decent tool previously) can help audit and document your system.
The basic steps are:
  1. Document the farms topology;
  2. Verify versions and components installed;
  3. Compare MS recommendations to the farm settings, provide findings and allow the customer time to explain why these setting differ; and
  4. Monitoring, troubleshooting and performance.
Below are a few items to review:

SharePoint:
SharePoint Logs - I put these into my d drive
Multiple install accounts
List all customisations especially wsp and custom code
External Access - DMZ, network,

IIS:
Change the IIS default log location to a separate disk
Reclaim old IIS logs (Backup and remove from WFE's)
IIS RealTime Log Viewer - good tool for reading IIS logs

SQL basic checks:
I/O, CPU, Memory on the base database machine/s
MaxDop = 1
Initital mdf sizing and ldf sizing
ldf on fast disk, mdf of content db's on slowest disk (or just put it all on fast disk)
Backups

Maybe Reminders:
Remove all certificate files that are not needed (*.cer & *.pfx)
Ensure local account user name/passwords are changed & secure (I have a local admin account on my Windows template used to create all my VM's ensure this account is disabled or secured).

More Info:
http://nikpatel.net/2012/03/11/checklist-for-designing-and-implementing-sharepoint-2010-extranets-things-to-consider/
https://habaneroconsulting.com/insights/Do-you-need-a-SharePoint-infrastructure-audit
http://blog.muhimbi.com/2009/05/managing-sharepoints-audit.html
http://www.portalint.com/thoughts-on-sharepoint-audit/