Showing posts with label Automation. Show all posts
Showing posts with label Automation. Show all posts

Friday, October 28, 2011

Selenium WebDriver tips 2

1. "Xpath expression cannot be evaluated or does notresult in a WebElement" error

During our recent automation project we've been encountering really irritating error using WebDriver
"The xpath expression '//given xpath' cannot be evaluated or does notresult in a WebElement"

This error was appearing randomly, for example script was working for 10 times but failed on the 11th. Eventually after several experiments we were able to narrow the problem to following factors:

- this problem appears on the IE only(IE8 in our case)
- this problems appears when WebElement is identified using XPath selector
- this problems appears often shortly after webpage is loaded

Possible Solutions:

Get rid of the XPath selectors
This is great idea and Alister Scott presents sound arguments for this here. This approach however is not always applicable in our context. We know that sophisticated XPath selectors can be brittle but having to automate application based on Microsott SharePoint there is not much choice

Add pause before first FindElement (with XPath selector) method
This solution worked out however you need to remember about adding this pause every time new page is opened. Because in general we don't like pauses in our automation scripts for many reasons we were looking for something else.

Update current FindElement method
We found this solution indirectly thanks to Alister Scott and his SpecDriver example on github. Because this problem appears usually at the first attempt to find element using XPath it's easy to update FindElement method and create your own where you can catch the exception for given number of times. An example in C#:

for (int i = 1; i <= 10; i++)
{
try
{
return driver.FindElement(selector);
}
catch (OpenQA.Selenium.WebDriverException e)
{
Console.WriteLine("Error " + i + " raised: " + e.Message);
}

It really works and often above loop doesn't need more than two iteration to return correct WebElement.

2. WebDriver loses focus on IE8

Another issue which appears randomly. Fortunately we were able to identify one of the reason of this issue - Microsoft Office Communicator. In order to run tests using WebDriver on IE (Windows 7, IE8 in our case) Microsoft's IM needs to be switched off if not then WebDriver randomly switches from IE to IM window during script execution.

Alek

Monday, October 10, 2011

My story with software testing (checking) automation (part 1)

It's started as usual. There is a web based software product with many manual test cases already written. There is a need in project to shorten the regression time for each iteration. There is a few days free where testers wait for the new build. There is a company orientation toward software automation (no matter what, let's automate !). So eventually there must appear someone powerful in project who doesn't know testing at all and suggests to automate manual testing. Keeping in mind that it is very easy to automate some activities on web page (Selenium IDE, QTP recording, etc.) it was a great recipe for automation testing disaster. However this time based on successes and failures of other teams we wanted to start differently and finsish successfully.

Why to automate at all?
We started with this question. What is the reason for automating anything? Can we really automate manual testing? According to great blog post from Micheal Bolton, we should switch labels and use automation "checking" word instead of "testing". But is "checking" word as catchy as "testing" for people who drive the project? It is not and thanks to this new label it was easier for us to explain that what we automate can never be replaced with human "testing". As testers our goal is to provide information about the product. What information we can deliver using automation?

1. We can verify if product being checked automatically is in same shape as it was at the time of automation scripts being written (but only to the extent that the check scripts verify this product).
2. We can reuse the automation scripts on every regression phase
3. We can free testers from repetitive tasks and given the time for real testing.
4. We can use automation to support our manual testing. To check things that are too complicated for human activity or simply too boring for human (read more on Alan Page blog post) It means that we can challenge our previous assumptions about the product and learn new things about the product being tested.

So we decided that we would try to cover with automation some of the regression part. We wanted to have more time for testing and for creating new testing ideas. We also wanted to use automation framework for creating tests that have never been executed before on our software like testing with more types of inputs, with more combinations, with repetitive activities.

What to automate?
Before we started with automation we had already manual regression test cases created. We decided to use them as background for regression automation. Should we really use manul testing for automation? According to one of the lesson from Lesson Learned In Software Testing it can be trap and we should avoid it. But as I mentioned before we wanted to use them as a background only. Based on the manual regression test cases we created a list of things that needed to be obligatory verified during regression. However this list of things to check wasn't our end goal. We needed also to think about the ways of verification some test cases, about the oracles. How we will know that something works or not or better how the machine will know that this works or not. And this discussion leaded us to the tools and question with "how" - how we will do this?.