Monday, 24 December 2012

Digital Signatures and Install Software gotcha

Problem: In automating SQL Server and SharePoint images, the actual installation is taking a long time on my managed environment whereas my developer laptop is fast.  All installations are done without Internet access (offline).

I have a dev environment build on my laptop that runs SSD and I run 3 VM using VMware workstation 9 (all use Windows 2008 R2 SP1).  I create an 1) AD with 1GB or RAM and 1 CPU 2) SQL 20012 with 10GB RAM and 4 CPU's 3) SP2010 CU Aug 2012 10GB 4 CPU's.  All the installation is automated using slip streamed images.

So for simplicity on the CI environment I will explain a simplified comparable setup. I have 3 machines with the same roles however the SQL 2012 and SP2010 install take considerably longer.  The CI environment is on ESX (Cisco blades & chassis, and Violin (SSD) storage.  The CPU/compute is connected to the storage via SAS/Fibre channel made no difference either).  I have summarised the results below:

                                                                 SQL2012 (duration)      SP2010 (duration)
Laptop(VMworkstation Workstation)           15 min                              16 min
CI (ESX)                                                        22 min                              92 min 

Finding: My hardcore/good ESX infrastructure is taking 9 minutes longer to install SQL Server 2012 on beter hardware and an amazing 70 minutes longer to install SP2010.

Update 21 Feb 2013: Don't use PowerShell 3 with AutoSPInstaller (including using the version switch i.e. -version 2), it doesn't work and even changing AutoSPInstallers internal web call fail.  It can be made to work with the version 2 switch but it isn't worth the effort.

Initial Hypothesis:
After many many hours between service providers managing the infrastructure, it was not hardware or ESX configuration/setup.  However if the network card on the VM is disabled, the performance change improves to:

                                                          SQL2012 (duration)            SP2010 (duration)
CI (ESX)                                                     13 min                       5 min and 5 seconds

Pretty hefty improvement.  Using netstat is looks like there are requests to the Internet.  After adding Wireshark to monitor all traffic.  I can see requests being sent to (certificate revocation lists) and

Issue shown in Wireshark
Issue Shown in Fiddler
This is the 1st time I have seen this issue in a clients production environment.  If the WFEs/SP servers have internet access (less preferable) or the servers don't have access the install work in a timely fashion.  The symtoms of the issue are when the WFE's/SP Servers don't have internet access but think they do.  All the binaries are digitally signed and the install will try validate the signatures despite this being an offline install.

I confirmed the problem being how the networking is setup.  My issue shows up on the VM NIC adapter, Originally the IPv4 Connectivity has a status of "No Internet Access", once I ping google I get a reply and the status changes to "Internet".  I can ping google but not browse to it.

Resolution:  The problem is that executable code is digitally signed.  This is good, all code should be digitally signed so it can be authenticated.  However in this situation a lot of requests are being sent out from the VM as the install tries to verify all the SharePoint complied code.  The install on the local VM acts as if there is an Internet connection (which there is not).

It takes unique networking to get into this issue and SP/any digitally signed code will check the digital certs.

There are a few fixes such as:
1.> Allowing the servers to get out to the Internet, so open the firewall or set a proxy on the local VM.
2.> Add host entries to the cert fails immediately but will continue installing (This is not working for me).
3.> Make the following registry change:
set-ItemProperty -path "HKCU:\Software\Microsoft\Windows\CurrentVersion\WinTrust\Trust Providers\Software Publishing" -name State -value 146944
set-ItemProperty -path "REGISTRY::\HKEY_USERS\.Default\Software\Microsoft\Windows\CurrentVersion\WinTrust\Trust Providers\Software Publishing" -name State -value 146944
get-ChildItem REGISTRY::HKEY_USERS | foreach-object {set-ItemProperty -ErrorAction silentlycontinue -path ($_.Name + "\Software\Microsoft\Windows\CurrentVersion\WinTrust\Trust Providers\Software Publishing") -name State -value 146944}

More Information:

Certificates for installing sofware is cause slow install:

If you want to verify if a machine is having problems with a poarticlar process Process Explorer (Usefule if a machine has high memory, CPU or IO issues)

Sunday, 9 December 2012

SharePoint Saturday UK 2012

Great event again, it amazes me that such a great place to get info meet people learn about SharePoint attracts 250 people - that is free.  Sure you need to give up your time and it is a long way but more folks involved with SP in the UK should be at these events.

Mark Macrae, Anthony Pounder & Brett Lonsdale set up SharePoint Saturday, it's huge, the speakers are great and give their time.  People who attend are good to speak too.  So my thanks to the speakers, the 3 fellas mentioned above and Rik from BlackMarble (great conversation).

The presentations I saw were with my thoughts:
  • Wes Hackett (Bring SP into your Office with Apps for Office), I saw a fair amount of this at SPC but Wes brought some good ideas and presented the topic extremely well.  Apps are really powerful in Office and SP2013 but I'm not sold it's going to work as well as MS are preaching.
  • Bill Ayers (Lean-Agile Development with SharePoint), best session I went to.  Had a chat with Chris, Alan (Eardly) & Bill afterwards - really  good stuff, with a topic relevant to my current project.  Unit test and SP are not a great combination in my opinion sure you can using a Mocking framework but I'm not sold, the learning curve for the team is high (effective TDD takes time no matter how many converted devs tell me it's simple), you spend time mocking.  The list goes on and the right projects with the right management buy in 100% I'm behind it I just think it's probably less than 10% of SharePoint projects.   The main take away which I agree with is SCRUM for SP projects is great.  Implementing scrum has challenges but is worth the pain in most scenarios.  Testing is key, you can use Integration tests instead of Unit Tests in VS.  I advocate projects use Code UI testing as with most SP projects you get the most "bang for your buck" (not part of the talk).
  • Martin Hatch (Performance and Load Testing using Visual Studio), good overview with demo and walk thru.  It is amazing what VS 2012 ultimate.  You can definitely remove project risk by using these tools and it's no longer in the domain of load runner experts.  VS has firmly tooled dev and teams to monitor performance and determine bottlenecks.
  • Marjn Somers (Extending SP with Simple jQuery Solutions), this is a funny guy well at a SP conference the bar is pretty low in the comedy genre.  I felt it was too simple (and i'm a simpleton (not the pattern) - sic) but as a 101 into SP and jQuery excellent.
  • Adam Burcher (PowerShell - Let me script that for you!), was well presented.  It was an intro into PS and PS for SP. I didn't get much out of it but it was well presented and the demo of showing SP devs how easy it is to convert a C# event handler's code into PS was extremely effective.
Sessions I missed and would of like to see where Andrew Woodwards on Why you need a SharePoint Centre of Excellence (CoE), Paul Hunts session on Sift thru Search and deliver more, Chris O'Brien Getting to Grips with SP2013 Apps, Mark Macrae's BI talk would of been good to attend also.


Setting up VMs (VMware ESX5)

Overview:  VMware ESXi is a 1st class virtualisation platform and commonly used to host SharePoint VM's.  The post looks at the common tools for managing/setting up my VMware based infrastructure.  I'm certainly not an expert in virtualisation but this is a 101 in setting up VM's.
vSphere Client connects to the ESXi server infrastructure and provides a UI management tool.
PowerShell with PowerCLI to let you manage your ESX infrastructure using PowerShell. 
My preferred option is to use PowerGUI and user the add the PowerCli add in so I can interact with ESX but I have Intellisense and nice debugging capabilities.   The snippet below shows how to connect to the ESXi server.
Script expanded with more detail.  Really need to loop throu config to create multiple VM's and get their individual settings.
Before running the script ensure you have the OSCustomizationSpec and OS template as you need them to build.
Check the VM is created.
Summary:  Useful scripts for building a unique set of VM's on VMware.  For Continuous Integration it is better to start with pre-build environments.  My next step would be to fire off the SQL Server 2012 builds on 1 or more of these VM's as shown in the SQL Server 2012 slipstreamed install.

Thanks to Wayne Senior for info in this post.