Test automation can add a lot of complexity and cost to a test team’s effort. In addition problems like including unrealistic expectations, poor testing practices, a false sense of security, maintenance costs, and other technical and organizational problems might arise. But it can also provide some valuable assistance if its done by the right people, in the right environment and done where it makes sense to do so.
Automated testing is an expensive process. Studies show that it can take between 3 to 10 times longer to develop automated Test Suite than to create and execute manual test cases. Costs of test automation include personnel to support test automation for the long term, dedicated test environment as well as the costs for the purchase, development, and maintenance of tools.
A common type of automated tool is the ‘record/playback’ type. For example, a tester could click through all combinations of menu choices, dialog box choices, buttons, etc. in an application GUI and have them ‘recorded’ and the results logged by a tool. The ‘recording’ is typically in the form of text based on a scripting language that is interpretable by the testing tool. If new buttons are added, or some underlying code in the application is changed, etc. the application can then be retested by just ‘playing back’ the ‘recorded’ actions, and comparing the logging results to check effects of the changes. The problem with such tools is that if there are continual changes to the system being tested, the ‘recordings’ may have to be changed so much that it becomes very time-consuming to continuously update the scripts. Additionally, interpretation of results (screens, data, logs, etc.) can be a difficult task. Note that there are record/playback tools for text-based interfaces also, and for all types of platforms.