5

I understand the concept of creating scripts to perform tasks, but I've read re-creating my manual tests as scripts isn't the direction I should be headed. What is the difference and what types of tasks should I be looking at to automate?

Note this was extracted from How can Manual QA's without a programming background learn Automated Testing? which has 4 questions and thus is too broad. I'm posting this from that as it seems like a good question.

9

Main difference is that in manual testing, a sapient, thinking processor (human) is looking at the results and can see "something is weird here".

Automated testing does not exist, it is wrong terminology, misnomer, even if widely accepted. In automated checking, totally dumb processor (your test script) checks for condition which programmer thought about, and blatantly ignores anything else, even if obviously wrong for a human manual tester.

What makes sense is automated regression testing (also a misnomer, it is checking again) when your automated scripts checks for known/suspected conditions (and ignores everything else, as all scripts do).

Human can do exploratory testing. Automated scripts, not so much :-)

But: to perform action in automated script, you need to know how to do it manually (and usually do it multiple times). You cannot automate something you don't know how to do manually.

Your automated test can check for small subset of what human tester would notice (so your manual tests are a good start). Some tasks (like pattern recognition) is beyond abilities of most automated tests, so you need to pick which conditions to check in script, and what will be ignored.

There are many cases where my automated tests perform action but do not check for errors, because I never seen any error in that action. So such test can silently fail to detect error, because programmer failed to anticipate it. To deal with such situation, you (programmer) have to use "defensive programming" techniques and i.e. check if page title is what you think it should (if not: something broke, but none of your checks detected it). This is not how human would detect such error, but that is all you have in the script.

And scripts are good in something humans are bad: Running exactly same checks day after day, very efficiently.

  • Or as I like to put it: Automation is a great way to find out wether human testers should even bother yet. – Guran Jan 31 '17 at 14:02
  • 1
    Not sure I understand what you mean. Care to expand to answer? I think just the opposite: Automated checks is just just a small subset of most commonly used checks what humans do during manual testing. – Peter M. - stands for Monica Jan 31 '17 at 14:07
  • I mean that with some automated sanity checks, I can avoid wasting human testers time. Just a simple script that simulates a user clicking through the menu in a web application menu and checks that every page loads means that human QA doesn't have to bother with an installation that has access issues for example. – Guran Feb 1 '17 at 9:06
5

Different sorts of tests include

  • web / api / database tests
  • functional / performance

Most likely you will want to focus on the most the very most repeated tests using the ide/ language your dev's use. e.g. java/eclipse/selenium/junit so the dev's can help you sort out issues.

Your best bet will be to take a class right up front. Stability is a key element in these tests, if you have lots of tests with a false positive rate of say 5% after a year you will have a load of automation that are all unstable and give you lots of false positives. Its hard to write very stable selenium tests with no code background.

You will want to focus on the most repeated tests. looking at your regressions and smoketests this will come into focus: e.g. if you run test #200 - as part of a bi-weekly regression - the smoketest you run as you move between qa->stage and stage-> prod Then it is a good candidate for automation. Keep in mind it will take you (without a code background) 10x the time to automate a given test then it will take you to run manually. So if you don't re-run the test often, there seems little benefit to automation

Your question is still very broad. I think I can really give you a better idea of what you want to do if you put more information on your specific needs.

  • Is your company agile?
  • Are you talking about a web page?
  • What kind of budget are you working with?
  • Team size
  • What are your most repeated tests, how often are they repeated
  • Are the tests well defined (e.g. with test steps that include what to do, and what to verify) or do you do mostly ad-hoc
  • Good response with helpful general principles, especially on the point of the most-repeated tests. The only proviso (in the spirit of the other response) would be the question of whether the results can be readily evaluated by an automation script or whether they require human judgment. – JDM Jan 27 '17 at 15:53
2

As already stated in the comments above, automated tests are:

  • prone to errors unless you spend enough time to ensure they are stable;
  • unable to perform creative tasks;
  • but at the same time faster and more durable when it comes to a lot of simple repeated actions (test steps).

So the best way is to automate those areas of testing which include routine work while more complex use cases (e.g. security related ones) should be left for manual verification.

As for scripts, they would be different for different tools. The main difference with manual ones is that you have to specify all preconditions and provide more detailed steps. Besides there are some scenarios a software tool cannot perform correctly (e.g. detect misaligned content, or export data into .csv file and check the consistency of this file).

2

While I agree with facets of the other elements that people are highlighting in the other answers, there is another type of automated testing that appears to be missing: automation designed to reduce subjectivity.

Humans vary in their ability to detect certain issues. If you ask people if a sound plays the same time as a visual cue occurs, there is ~200ms of variability where some people would not be able to detect a problem. Things like performance tests (as mentioned by Dan) are designed to concretely measure elements that were previously solely up to manual interpretation.

If there is a likelihood of disagreement about whether or not something is an issue, having concrete data to back your case can often make the difference between it being fixed and it being ignored.

Another thing to keep in mind about automated tests is that if the amount of time that it takes to develop and maintain the test is less than the amount of time that it takes a user to run the test for the remainder of that project's lifetime, you should automate it if possible to save resources.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.