My story with software testing (checking) automation (part 1)

It's started as usual. There is a web based software product with many manual test cases already written. There is a need in project to shorten the regression time for each iteration. There is a few days free where testers wait for the new build. There is a company orientation toward software automation (no matter what, let's automate !). So eventually there must appear someone powerful in project who doesn't know testing at all and suggests to automate manual testing. Keeping in mind that it is very easy to automate some activities on web page (Selenium IDE, QTP recording, etc.) it was a great recipe for automation testing disaster. However this time based on successes and failures of other teams we wanted to start differently and finsish successfully.

Why to automate at all?
We started with this question. What is the reason for automating anything? Can we really automate manual testing? According to great blog post from Micheal Bolton, we should switch labels and use automation "checking" word instead of "testing". But is "checking" word as catchy as "testing" for people who drive the project? It is not and thanks to this new label it was easier for us to explain that what we automate can never be replaced with human "testing". As testers our goal is to provide information about the product. What information we can deliver using automation?

1. We can verify if product being checked automatically is in same shape as it was at the time of automation scripts being written (but only to the extent that the check scripts verify this product).
2. We can reuse the automation scripts on every regression phase
3. We can free testers from repetitive tasks and given the time for real testing.
4. We can use automation to support our manual testing. To check things that are too complicated for human activity or simply too boring for human (read more on Alan Page blog post) It means that we can challenge our previous assumptions about the product and learn new things about the product being tested.

So we decided that we would try to cover with automation some of the regression part. We wanted to have more time for testing and for creating new testing ideas. We also wanted to use automation framework for creating tests that have never been executed before on our software like testing with more types of inputs, with more combinations, with repetitive activities.

What to automate?
Before we started with automation we had already manual regression test cases created. We decided to use them as background for regression automation. Should we really use manul testing for automation? According to one of the lesson from Lesson Learned In Software Testing it can be trap and we should avoid it. But as I mentioned before we wanted to use them as a background only. Based on the manual regression test cases we created a list of things that needed to be obligatory verified during regression. However this list of things to check wasn't our end goal. We needed also to think about the ways of verification some test cases, about the oracles. How we will know that something works or not or better how the machine will know that this works or not. And this discussion leaded us to the tools and question with "how" - how we will do this?.