Monday, 23 February 2009

Don't blame the bread-making machine - thoughts on test automation

If you have a bread-making machine, it may or may not make good bread.

If it doesn't, the reason might be the ingredients you put into the machine. If you put in too much salt (or no salt at all), the bread won't be edible. Is that the fault of the bread-making machine?

Some people blame the test automation for things which are actually the responsibility of the testing. For example, if you mistakenly think that finding lots of bugs is the main aim of your regression test automation, you are setting yourself up for disappointment, if not failure of your automation effort. Finding bugs is what testing (and testers) do - it is not the job of automation - the job of automation is to run tests.

The tests are the ingredients that determine whether the testing is effective or not. The effectiveness of the tests is the same whether those tests are run manually or using an automation tool. 

The tool is no more responsible for the quality of the testing than the bread-making machine is responsible for the taste of the bread.