Should developers design unit based test cases prior to writing the tests

Question

Background

We are currently moving API testing from the Technical testing team to the development team. The technical testing team has a formal process where they write the test objectives and test cases prior to developing the script in SOAPUI. Therefore, analyzing the specification and supporting requirements document are important for deriving the tests.

In the new world the developers will code/point and click their unit tests and there will be no other QA other than the UAT of the consumption of API, which can happen between a month to three months after the API has been created. The API is not an open API, only for third party suppliers and internal web and app resources.

There is an argument that additional QA work is not needed if the developers are writing the unit tests, so developers do not need to have an analysis phase or write test cases prior to setting them up in SOAPUI or coding the test.

We create enterprise level financial websites and applications. The teams is 30+ strong.

Question

Should developers still perform such analysis and write test cases?

Answer 1

I have seen Unit Testing performed by the developers writing the application code. I have seen it help drive the design and implementation of the code. This works well and is a great improvement from 20 years ago when testing, regardless of who was doing it, was largely manual.

In your case I would still look to have the Unit Tests to drive the development code. This is something a developer should be able to do well.

You have a great practice if you currently write tests ahead of the code. Preserve that practice.

Just adapt it to the new environment with whatever simple tests can support the code as it is being written Doesn't have to be ahead as taught in TDDD/BDD, but imho as long as tests are written within a few hours/days and committed together, then is fine. I think this also represents the real world more than folks like to admit. It's not the best way but often the most practical way given developers experience and expertise.

I would usually point out that this is too limited as Unit Testing is just one of the 4 Agile Testing Quadrants, however for an API and in your case with facts such as "The API is not an open API, only for third party suppliers and internal web and app resources. " this doesn't apply as much.

I would also make sure that some of the other areas such as security, performance, load, smoke tests, integrations, etc. will still be covered by this model

Answer 2

One statement that caught my eye in the original post is: "The API is not an open API, only for third party suppliers and internal web and app resources. ". The implication is that since this is for internal use it doesn't have to be as user-friendly or robust as external-facing projects. I would argue that internal-facing projects should be more robust and user-friendly than external-facing projects because doing so will make the work force more productive. There is a tangible dollar amount that can be connected to a tool that is easy to understand and use vs one that is not. There is also the security question: If a malicious user were to access this, what harm can they cause and can we live with that?

While developers are thinking about how to make something work, testers are thinking about how it might not work. Unit tests are a good idea because they test the logic at a layer that allows for high-speed execution. The flaw in only relying on unit tests is not the tests themselves, but the folks writing them. Because of a developer's natural bias toward the Happy Path, unit tests written by developers tend to follow those paths.

Testers are generally not programmers and as such are not good at writing unit tests, even though they may have some good ideas on negative, edge, and corner cases to exercise the code.

A solution that uses the strengths of both groups in the team would be a Test Review. Have developers explain to testers what the unit tests are doing. Testers can then ask questions about negative, edge and corner cases that the developers might not have considered. Developers would then write unit tests against those scenarios as well. The bonus is that after a few sessions developers will begin to anticipate the questions the tester is going to ask, consider some previous unhappy paths that were discussed, and proactively write unit tests and code against those scenarios. Developers get smarter, Test Review meetings get shorter, and the code gets more robust. The downside is that developers and testers still need to collaborate, albeit on a much smaller scale.

Answer 3

Yes, this is a fundamental feature of what Test Driven Development entails.

Since these are new APIs, you have a tremendous opportunity here to pursue contract first web service development. Consider SOAPUI as a development tool not just a test tool.

SOAPUI rather quietly supports a contract-first approach. The API can be designed via its UI or an imported WADL/WSDL. It can then generate service mocks for the client developers and unit tests for the service developers. It can generate the boilerplate code for both. The inbuilt Groovy Scripting is very powerful and will go down well with developers.

Answer 4

In a TDD/BDD approach we should write the unit test case first and then write the development code which will pass the test. That's becoming a standard in the world of devops.

Now the important part here is the code coverage and the functionality developed. Now as a developer you should ensure that the unit tests give 100% code coverage. Use tools like Sonar or EclEmma to get the code coverage. That would definitely reduce your defects if the unit tests are written correctly.

The problem that I've seen here is developers do not write the whole spectrum of tests possible. If there are 10 possible unit tests for a certain requirement and the developer has written only 5. Then it's a miss from his side. For example, if the requirement is 'Given I'm an admin, I should be able to login to the application'. The tests could be:

• If correct username password the user logs in and displays a success message.
  • Incorrect password the user could not login and displays an error message.

Now if the developer writes the test for only successful login. Then the unit tests would pass. Code coverage would be 100%. But there's a missing functionality. What about the incorrect password scenario?

So a QA is definitely required along with unit tests, code coverage tools.

Answer 5

The unit tests should be code reviewed by the team. If you point out numerous problems with the tests, that may lend weight to the argument that more thought should be put into test design.

Answer 6

Test design is just as important for unit testing as for other, higher-level forms of testing. Good test design practices such as equivalence partitioning will often allow you to write tests that are more complete and more efficient to write and run than "off the cuff" unit testing because it helps you maximize the number of conditions covered while minimizing the number of tests actually performed.

Another consideration is that if you just sit down and bang out unit tests based on the code, without the detailed study of the specification and requirements that you mentioned, you risk testing what the code does, rather than what it's supposed to do.

I personally have found the book The Art of Software Testing to be very helpful in learning good unit test design. It was my first exposure to test design and I found the discussion on equivalence partitioning in particular to be very effective in guiding my unit testing efforts.

Answer 7

"where they write the test objectives and test cases prior to developing the script in SOAPUI. "

So, ^ that's what testers used to do. Certainly a good practice to write the cases down ahead of writing the actual tests, based on the business requirements.

If developers now write the tests, they should certainly apply the same care - analyze the application objectives - before writing test cases, ideally before then writing application code in a TDD fashion.