Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I'm working on a project that involves 4 developers and three QAers.

I've tried to encourage the other devs to write tests for every new feature they have added, and we have built up a suite of around 50 Selenium UI tests. I don't doubt that this has improved the quality of the product but the developers have become slaves to getting the tests working and I don't think we know enough about Selenium yet to really use it efficiently.

I'm thinking that for the next project I will pass automation of tests and building an automated regression suite over to the QA people. One of them has had good success using the Selenium add-on for Firefox before so the skills and the will do exist. Is this the right decision to make?

share|improve this question
    
From the answer you accepted, it seems that your QA are not competent enough to develop test by themselves. Other organizations (like ours) dedicate competent coders to develop automated test, so developers can focus on the application don't have to become experts in Selenium. And our developers like it that way. Solution is obvious: hire competent programmers to write automated tests. –  Peter Masiar Jul 9 at 17:17

5 Answers 5

up vote 8 down vote accepted

Extending and maintaining automated tests costs time, but if your team are slaves to updating tests you are doing it wrong. :)

One of the biggest reasons to create automated test coverage is to create a fast feedback cycle loop. Letting the test-team fix the tests afterwards slows down this process and will always lag behind. There is a high risk the tests will always be an after thought and are untrusted by the developers.

I suggest using a BDD framework. My previous team combined Cucumber and Selenium into a testing framework. When a developers checks in new code the current tests should not break. Also you want new coverage for new features. Schedule the tests in a continuous integration cycle to make sure the tests do not break.

  • Testers write test-cases
  • Developers automate them

Of-course testers can also automate them and developers can extend tests with more corners cases, but keep in mind that you need good developers to create a maintainable testing framework.

Use best of both worlds and pair program (1 dev + 1 qa) on implementing the more complex test-cases.

share|improve this answer
    
I'd agree, except that it doesn't take Developers to automate the test cases, it takes someone trained or who has experience generating automation frameworks. Developers can help, and should code review the tests when they can, but it's not just up to them to automate, QA should be able to find capable people to do this as well –  MichaelF Jul 3 at 12:13
1  
Agreed, but in my experience most "Test engineers" are the lesser coding gods. Therefor as an general guideline I am convinced that developer should assist with setting up automated testing frameworks. Also because this makes adoption by the developers easier, both technical as mental. –  Niels van Reijmersdal Jul 3 at 12:29
    
Niels, your approach will require developers to become/keep updating expert skills in Selenium. In my experience, developers have hard time to keep with tools they need and unit tests, for which they are responsible. It is just the question of mental bandwidth IMHO. System integration etc test are different animal, have different goals and assumptions. Our developers are happy that we have enough competent test automation programmers so they don't have to write integration tests. –  Peter Masiar Jul 9 at 16:55
    
Correct. As a true believer of agile methodologies I think a team should be cross-functional, preferable in small teams (up-to 6 members) the team-members have a shared skill set. Having a team-member who's primary skill is test engineering is a big plus. But each iteration all the PBI's should meet the "Definition of Done", which in my book includes automated test coverage. In smaller teams you cannot depend on one or two testers having the ownership of all the automated tests. The world is not black and white it is not a silver bullet. Great that the separation is working for your teams :) –  Niels van Reijmersdal Jul 9 at 18:08

Why either/or? Work as a team with testers providing test ideas and expertise that the devs can code up. I would also shy away from an emphasis on GUI automation and get the devs working at a unit/system level unless they are doing this already

share|improve this answer

Here's how I would approach this kind of situation:

  • Unit tests - Evangelize the living daylights out of unit testing. Any kind of business logic automation should be handled by unit tests, as should any other automated tests whose goal is to check the functionality of a single unit of code. This is developer-level code, but tester guidance is essential if you want your unit tests to be useful.
  • API/Integration tests - These work more or less the same way as unit tests from the developer perspective, but they test slightly bigger chunks of code. Again, coded by your developers with tester input on what makes a good test.
  • "Headless" end-to-end tests - These are lighter-weight than full-GUI tests, but will not find problems with components and validation code. They're also not possible in all circumstances. They're somewhere between API-level and full GUI-level automation.
  • End-to-end tests - These are usually your GUI-level automation tests. They can be coded by developers with tester input, or coded by testers with developer input.

As a general rule you want most of your automation to be at the unit test level because it's the fastest, lightest-weight level to work with and it's relatively trivial to set up most build environments to run unit tests on every compile/build. Thousands of unit tests can be run in the time it takes to complete a single GUI-level test.

The next biggest number of tests should be the API/Integration level tests. They're slower than unit tests, but may still be able to run with every build. Depending on how they're designed, they may or may not require the executable to run.

End to end tests in any form should be the smallest grouping for several reasons:

  • they're orders of magnitude slower than unit-level and API-level tests.
  • they're more likely to report false positives than lower-level tests (as a rule, the more pieces you have to link together in automation, the more likely you are to get a failure that isn't caused by a bug)
  • they're more difficult to maintain than unit-level and API level tests (because they're dealing with more complexity).

Your developers and testers are working together on the software - why wouldn't they work together on the automated testing? Who does what is a matter of who is best suited to which task, but the goal is still the same.

share|improve this answer
    
Our QA started with manual testers. Then manager realized that when requiring time from developers to add something to simplify testing, developing has higher priorities. So he created his own small team of dedicated "test automation engineers" who work with developers, but report to him, to solve problems according to QA priorities. Situation which @Niels van Reijmersdal describes is theory, and can work in small teams up to maybe 5-8 developers and QA. In my in experience (15+ devs), manager of the dev team solves his own priorities first, so separate QA team is needed. –  Peter Masiar Jul 9 at 17:00

It depends on what sort of resources you can afford. Test Automation, done well, is not a part time effort. If you can't afford to have a Toolsmith on your team to work on the Automation Framework you will need help, or expect it to take a long time, or expect it to take effort to keep it going as it gets reprioritized. While most Testers do not spend a lot of time coding and learning to code, many do and they can help provide the assistance, and boost, your team may need. I'm no Developer but I do know HOW to make a Test Framework and get it working, with input from Developers as needed. After all many Developers did not know how to code at some point, they needed to learn how to do it as well. It's also a skill that many Testers need to work on, as the thought process is different but you can usually make a very effective Framework utilizing Coding Skills and Practices with Testing Skills and Practices.

If you want a Framework that does what YOU want, QA needs to own it. QA needs to run it, and QA needs to direct it. You should get buy in from your Developers as well, some Code Review (this is code after all) will be good now and again not just on the Test but on the Framework as well. Make it robust, and most of all make it useful. Other teams should be able to run it, even Developers should have a way to do some localized Smoke Testing in their environments to get their buy-in, and show the usefulness of the Framework for them - get them catching issues early on. The last Framework I used (a BDD Web Driver that used SpecFlow) was able to be used by QA, DEV and the BA's - I just had to show them how to run it. I am not a Developer but I do a lot of automation.

It's good you have the Developers working on Unit Tests, and in the Framework, but remember their goal is to Develop a solution to a problem and your role is to show that their solution solves the problem.

While the browser add-ons are useful, look at something like WebDriver for Selenium and another tool like Cucumber or SpecFlow. Build a framework that will allow you to run a suite of tests, or a select few, so you can cover Regression, Functional and other tests as needed. You also want reports coming from your tests, you should be able to see the historical Pass/Fails for Tests so you know if they are valid, failing only at specific times or show that your pass rate is getting better - management will like that! Any tool takes time to learn, you just need to dedicate that time to your team and give them the resources to improve.

share|improve this answer

I think it is the right decision to let the QA Engineers develop the test automation. Developers aren't testers and do not have the know-how to create good GUI test cases.

But you can involve the developers in the development of test automation, if needed. They have useful knowledge about programming and can help you to setup the infrastructure of your test automation framework. If you have troubles or do not know how to develop something, ask them.

share|improve this answer
2  
I didn't vote down, but just as developers are not testers, the other way around testers are not developers. Testers should not code automated tests without super vision of developers. I have seen testers with crappy coding skills build monster (un-maintainable) automated, but well defined testcases. You need both :) –  Niels van Reijmersdal Jul 3 at 11:53
2  
@NielsvanReijmersdal there are people who do automation frameworks and can follow Development Methods and Practices and work solely as Testers. I've seen Developers with bad Unit Test skills and Testers with good Test Frameworks, there is a lot of cross over in these to realms so its not all set in stone. –  MichaelF Jul 3 at 12:11
    
@NielsvanReijmersdal The term 'testers' in my answer includes all possible roles testers can have, also Test Automation Engineers. And the term "developers" means the people who work on the software to test. The focus of the developers should be on the development and unit tests, no GUI tests. Maybe this was unclear written. Hope it is clear now. –  Twaldigas Jul 3 at 12:26
    
+1. Seems like in opinion of many "developers first" people around here, testers are low-or no-skilled developers. Well, of course if they have no skills, you cannot expect them to develop properly structured test. Test are just programs - they just use your application to simulate user interaction. Competent test engineers will write well-structured tests. If incompetent tester will write test, result is predictably poor. But it is caused by their low skills, not by the fact they are subhuman test engineers. –  Peter Masiar Jul 9 at 17:13

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.