Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I come out of quite a few years as a software engineering, but now I'm completely focused on testing. Our team has been trying to expand our automated tests and, to me, this means a lot of coding. I wanted to get an idea of what other people in this field thought though, in case this is simply a result of my development background. Do any of you find that there are other ways to achieve extensive test automation without a big focus on coding those tests?

Thanks for any input.

share|improve this question
    
What field are you doing this in? web testing? network? application? –  Mark Mayo Jun 3 '13 at 3:18

4 Answers 4

up vote 9 down vote accepted

Welcome to SQA, Daniel. Sometimes there are dedicated tools/products you can purchase for specific kinds of testing. For example, there are companies that provide software packages specifically for testing network devices.

For most of us though, test automation means writing code. Sometimes there are ways to cut down on the amount of code you write. If you aren't using a test framework (e.g. JUnit for Java), you should consider doing so. A test framework can cut down on some of the low-level coding work and can handle reporting for you, too.

Sometimes there seems to be a lot of code to write because you face a combinatorial test problem, i.e. a problem in which you have many independent variables to test, and each test has many possible values. See http://sqa.stackexchange.com/search?q=combinatorial%20testing for more details.

Coming from a development background, your inclination may be to write unit tests for developers who choose not to write their own. That can be a lot of work, and once you're done, you may be stuck maintaining them yourself; if a developer doesn't want to write unit tests, they may not want to maintain them either. I recommend focusing on integration- and system-level tests.

You don't necessarily need to write tests for everything. If a part of the product isn't used much or is unlikely to break (the classic example of the latter is setters and getters on a Java bean), automated tests may not be worth the effort. James Whittaker and some other Google engineers wrote a pretty good book that covers those trade-offs; I'm sure there are many other books that cover similar material.

share|improve this answer

One more way to limit development effort is to build your own test framework.

It will hide behind the scene boilerplate code that your testers write in each and every test case again and again. Those are: establishing database or Web service connections, generating test data of the same type, assertions, etc. For sure, there is no need to re-invent a wheel in all cases. There are test frameworks that address technical problems common for different business domains, like TestNG for test driving and reporting, WebDrivers for testing browsers, XML libraries for comparing XML documents and so on. However, there are also patterns specific only for your business domain, your system and your project that none of existing frameworks has covered. Bank domain will be interested in simplifying creation of bank statements and their verification, in airline industry creation of bookings and verification of booking processing can be simplified.

By identifying both technical and business patterns in your test cases, you will be able build your test framework, often on the top of existing technical frameworks like TestNG, Selenium, etc. You will wrap those patterns with Domain Specific Language (DSL), so your tests will stop looking like:

BankStatement statement = new BankStatement().
Transaction t = new Transaction();
t.setAmount(...);
statemet.addTransaction(t);
try {
   DBConnection conn = DB.getConnection();
   conn.execute("INSERT INTO Statement (...) VALUE (...)");
   conn.close();
} catch(SQLException e) {
   ...
}

and starting looking like

createBankStatement()
   .withTransaction()
      .ofAmount(...);

This way your testers will be able to focus on investing good test cases.

Obviously, some development will still be necessary, as no test framework is complete once and forever. Your testers will be inventing test cases that cannot be supported by your test framework, so they will need to extend it with new features. Those features can be written by them and incorporated into the framework by you and then shared with others, or testers may submit feature requests that will drive your work on developing the framework. Either way, it will grow and become useful to everyone in your project.

share|improve this answer

Good question I must say. It seems from your question that you are thinking in a proper direction.

Actually it is to be taken into consideration that Full automation for any system is not possible as there are some checklists that are to be done manually like load, stress testing etc and some more.(Obviously it will depend on the system under consideration)

The amount of coding required for doing automation depends upon how much part of the system you want to automate. The more you want to lessen the efforts for testing and cover the test cases the more code you may have to write (In most of the cases).

Now a days Selenium IDE like tools provide you with the facility to write less code and automate your web applications just by recording via IDE. In those cases you have to enhance the script for maximum test coverage.

share|improve this answer
    
Why do you think load and stress testing are to be done manually? –  dzieciou Jun 2 '13 at 18:08
    
I know that loadrunner like tools can be used to avoid load and stress testing to be done manually. In the above case a system built from flash + HTML can be considered as the flash portion can't be automated using Selenium alone while HTML portion may be fully automated. Also the http requests may or may not be automated using Selenium. –  talktokets Jun 3 '13 at 3:08
    
What's flash-capability got to do with load testing? –  Mark Mayo Jun 3 '13 at 3:18
    
@MarkMayo, I had just given example there about partial/full automation of any system. Hence I used that term here. It is nothing to do with load testing :) –  talktokets Jun 3 '13 at 4:37

At my company we utilize Robot framework and have over 1000 robot keywords that act as a front end to selenium. We have 1 team dedicated to maintaining and creating new keywords, overseeing the nightly runs and other automation support issues. We then have a team of about 30 QA who only write test cases in robot.

By using robot framework to create high level keywords, a tester doesn't need to focus much on coding - only on passing the right page element id's and inputs to the robot keywords.

That said, depending on the script, there can be a great deal of coding still - especially in scenarios where our team does not already have keywords created to support the test.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.