Since the millennium bug there has been an explosion in test automation. The expansion of automated tools from vendors such as Mercury Interactive, IBM Rational, Segue and others has really delivered automated testing into the heart of just about every major organisation in the western world. And this can be seen to continue with new development methodologies such as eXtreme programming which has test automation at the heart of its design.



So why Automate? Well, the major benefits are:

Speed

Automated tests can run much faster than the manual equivalent, so more testing can be achieved in a smaller time frame. It is not unusual to see regression suites that take two weeks manually reduced to 24 hours with test tools!

Consistency/repeatability

Automated tests can be repeated over and over and will always perform the test exactly as recorded. Manual testers can easily make mistakes or perform the same test slightly differently when repeated.

Unattended – reduce cost

Once developed, the automated testing can be executed unattended overnight, so repeating the testing requires very little resource or cost.

Audit trail

The tests are stored as automated scripts and the test results are stored for every execution of the tests providing a full audit trail of all testing performed.

Improved Test Coverage

As the test pack develops, more & more tests can be added and as the execution time is much faster than manual testing and less resource intensive, more testing can be completed, ensuring greater coverage of the application under test.

Free up testers to perform other tasks

If the structured testing is automated, testers will hopefully have more time to perform other types of testing (accessibility, usability, ad-hoc/random testing) which commonly get pushed aside.

Some testing may only be possible with automation

For example how do you manually simulate 5000 concurrent customers performing transactions on your on-line web site?

What types of tools are available?

Functional/regression automated tools

These tools capture user interaction with the application under test and can then be replayed. However they go beyond this and allow you to data-drive tests, to capture the state of your application whilst performing business processes and provide fully-featured development environments allowing you to manipulate applications in almost any kind of way. Tools such as Mercury’s WinRunner or QuickTest Professional demonstrate the pinnacle of what these kinds of tools are capable of.

Load/Performance tools

These are used to emulate multiple (concurrent) users performing tasks against your application, for example if your business has an on-line banking system – how many users can it support? What kind of performance will it deliver under load? Where are the bottlenecks? All of these kinds of questions can be answered using automated load testing tools such as Mercury’s LoadRunner.

Test Management

Tools such as Mercury’s TestDirector or Quality Center offer Test Management facilities; a central repository to store test requirements, test scripts, execution results and defect tracking, integrating seamlessly with the automated tools such as WinRunner, LoadRunner and QuickTest Pro.

So what are the pitfalls to test automation?

It is easy for these tools to become ‘shelfware’ as purchasers do not consider the complexity and power of some of these tools.

They all share a steep learning curve, so some time and money need to be invested into getting your team up to scratch with the tools.

Some solutions to good automation implementation:

Technical Testers. Ensure your testers have a reasonable grasp of technology, some programming experience would be ideal.

Training, Training, Training. Good training programs are offered for these tools and are well worth the investment. The trainers usually have good project background experience and can demonstrate how the tools should be used and explain the pitfalls.

Hand-holding for the basic infrastructure. Following the training it is well worth getting a consultant in for a few days or weeks to build the basic automated infrastructure, provide on-site training against your application and to develop templates for your team to expand the test automation out from. They can achieve in weeks what could take you months to learn.

Tools such as LoadRunner, WinRunner, QuickTest Pro, TestDirector, Quality Center can provide a great return on investment in the longer term, but to make them effective do invest in training and assistance at the beginning.