Showing posts with label Smoke testing. Show all posts
Showing posts with label Smoke testing. Show all posts

Wednesday 31 January 2018

Looking for a cheap quick UI testing and monitoring Tool - end test and Ghost Inspector Review

Problem:  My client is looking for a simple tool to monitor a website is up and running and can run a small set of UI tests and asserts to verify it is working as expected.

Initial Hypothesis:  There are a lot of monitoring sites like uptime that meet this requirement, but I reviewed Ghost Inspector and endtest.  I am not looking to do full CI as I would look at Selenium WebDriver for an enterprise solution for UI testing.

Resolution:  Trial endtest and Ghost inspector on my O365 subscription to validate it monitors and alerts, can perform advanced logins and it can validate custom pages after JavaScript injection.  Price and feature wise both tools are pretty similar.

Ghost Inspector Initial Thoughts
Easy to use and there is a recording function for Chrome.  This review has put me off Ghost Inspector to some degree but definitely an excellent product to evaluate.
Bad review for Ghost Inspector but it does assume enterprise level UI testing more suited to tooling like Selenium.

endtest Initial Thoughts
Easy to use, setup testing in a matter of minutes, recorded actions and assertions.  The trial is limited as I could not check the scheduling mechanism, but end test looks like the ideal tool for my requirement.  Would need to go for the pro licence at $79 per month.  A simpler smaller option would be more attractive but let's see what the client thinks.

Other Tools for UI Testing:
Selenium
Selenium IDE is an excellent tool for UI testing and UI automation testing.  Here is a post on Selenium I did a few years back.

qTest Explorer
This is a Manual recording and documentation tool that records browser and desktop interaction.  It's straightforward to use and great for Manual UI testing.  It is not for automation or re-running test but great if the project requires manual testing and proof.


Sunday 6 December 2015

Smoke Testing SharePoint using Selenium IDE

Overview:  In SharePoint we need to retest code often as we make incremental changes.  Basic smoke testing is useful in that is allows you a certain degree of confidence that a bug has not crept into your latest deployment.

Most projects have varying degrees of control to ensure bugs do not cause unexpected behaviour and on the more advanced practices is unit testing and coded UI tests.  The unit tests are tricky with all the new SharePoint development methods.  Jasmine is a JavaScript testing framework and check out the @SPDoctor (Bill) for basic testing information.  Unit testing SP is difficult as the revs end up test SP and not the changes.  A lot of the code is UI driven which is hard to unit test.  I have previously written about code UI testing, MTM as part of the MS test and continuous testing.  And part of this is Selenium WebDriver.  I've used it once on a large project and it was awesome.  Now as you go into production you normally will do some manual smoke testing to check the deployment.

This post looks at automating smoke test.

There are various tools for recording smoke test or you can do the manual eyeball approach favoured my most SP project still.  I have used Powershell (both to gen http requests and to control IE).  MTM is good but it requires buy into the whole MTM process.  Personally I like Selenium IDE for Firefox.  It offers recording, and the capturing functionality is miles ahead of anything else.  These Selenium generate test can be export and used with Visual Studio or build into TFS or on a project I used TeamCity to run automated continuous build and integration.

Note:  Selenium IDE is the recording piece and Selenium WebDriver is the heavy duty real testing integration part.

Get Started:
1. Download Selenium IDE (make sure you get the Selenium IDE, I'm using version 2.9.0) 
2. Understand the UI and capabilities (YouTube basic Selenium IDE videos are great)
3. Install Selenium IDE on Firefox as shown below:




























4. Launch Selenium IDE using FireFox








5. Record and run Selenium test (2 minute short video providing a Selenium IDE recorded test against a SharePoint Online Team Site).

Another option is to control IE using PowerShell
PS> $ie=New-Object -com internetexplorer.application  #open ie
PS> $ie.Navigate("http://www.radimaging.co.uk/Pages/default.aspx")


Thursday 20 August 2015

Non Functional Testing for SharePoint

Overview:  Functional Requirements are the business requirements that the business define for the application being built.  Non-functional testing is concerned with performance, reliability, scalability, recovery, load,  security and usability testing.  For SharePoint it is a good idea to test this at a platform level and then verify the individual application non functional testing is appropriate.

SharePoint Non Functional Testing:
All of these test should be performed against your various SharePoint platforms and will dictate the SLA's offered to the business using SharePoint as a service.  Baseline testing is a good idea as the differences can be used to determine the efficiency of the individual application being created.

Proxies:
Fiddler is my favourite (other tools for capturing web traffic Charles, BurpSuite, WireShark and you can use the developer tools shipped with the browsers). tcpdump, goPacket are awesome for network monitoring.

Use Fiddler to:
  • Observe traffic (http/https requests, headers,..)
  • Replay sessions, 
  • Evaluate performance,
  • Set break point
A common misconception is the function of performance vs load testing.  

Performance is primarily concerns with looking at typical user usage scenarios and see how long each page takes to load.  So a recorded script with wait time between recorded calls is useful  It's also worth looking at the same page with minimal data and also with large amounts of data.  

Load Test involves recording the standard users interaction (ensure some users query heavy and some light amounts of data), there are no wait times.  The norm is to multiple this concurrent number of users by 100 to know the amount of users the farm business application can support.  e.g. a concurrent user load of 100 users where the performance is acceptable means the farm should handle 10,000 users (100 users times 100).  You run the 100 users in a steady state for a few hours.

Stress testing is similar to load testing but you keep stepping up the number of concurrent users until the SharePoint farm starts running out of resources and throttling requests.  Once the system degrades and performance is not acceptable is pretty much how many users the application on the SharePoint farm are supported.  So if the performance throttles at 170 users, the farm can handle 17, 000 normal usage users.

References:
http://www.guru99.com/non-functional-testing.html