Skip to main content
Software Quality Assurance & Testing

Return to Answer

added 176 characters in body
Source Link
Michael Durrant
Michael Durrant
  • 25.2k
  • 3
  • 41
  • 113

For small test suites only

I've changed my opinion on the "rule" of only 1 assertion per test over time.

A few years ago, when entering the QA field, I considered it a great approach and almost too obvious to explain and justify. As other poster have written, it's the way to be specific about what is being tested, for failures to be more specific, etc, etc.

However I've seen that in real world situations, most significantly as test suites grow, treating this as a fixed rulesrule becomes a problemproblematic due to the following reasons:

Note: An example of "multiple assertions" is that for a given UI operation one might expect 1) to be on a certain screen 2) to see certain data in a certain place 3) To see correct error messages 4) To see correct action buttons to proceed 5) To see the database updated correctly. So this is 5 assertions for the same action.

  • test suites run length. If all tests have to perform setup and teardown (they should) and only test 1 assertion this can lead to very lengthy test suites times. This is a huge problem as once test suites get slow a whole host of bad things happen (not run, slow development, increasing use of shared seed data, etc).

  • test suite size. If 3 tests are written that have a total of 19 assertions, changedchanges to those tests are relatively easy. However if the 19 assertions are used to break down the tests into 19 tests with 1 assertion per test, maintaining and updating the code becomes error-prone, time-consuming and boring for many programmers.

  • Not DRY. When a lot of examples repeat the same setup and teardown, you start to get a lot of cut-and-paste programming and code in multiple places that needs to be updated instead of code in one place. I have frequently seen "some" tests have their setup changed without the changes being applied to other tests leading to a mish-mash of styles

  • programmers have different opinions and experience. I work with a lot of programmers who don't follow the "1 assertion per test" rule. Making every follow this rule would be a challenge (worth it in some organizations but not in others).

Note: An example of "multiple assertions" is that for a given UI operation one might expect

  1. to be on a certain screen
  2. to see certain data in a certain place
  3. To see correct error messages
  4. To see correct action buttons to proceed
  5. To see the database updated correctly.

So this is 5 assertions for the same action.

One thing I do agree with and find hellps is @ethal evans mention of
"only permit them at one point in the execution of the test."
I also expect to put them at the end of the test (excepting any required teardown).

For small test suites only

I've changed my opinion on the "rule" of only 1 assertion per test over time.

A few years ago, when entering the QA field, I considered it a great approach and almost too obvious to explain and justify. As other poster have written, it's the way to be specific about what is being tested, for failures to be more specific, etc, etc.

However I've seen that in real world situations, most significantly as test suites grow, treating this as a fixed rules becomes a problem due to the following reasons:

Note: An example of "multiple assertions" is that for a given UI operation one might expect 1) to be on a certain screen 2) to see certain data in a certain place 3) To see correct error messages 4) To see correct action buttons to proceed 5) To see the database updated correctly. So this is 5 assertions for the same action.

  • test suites length. If all tests have to perform setup and teardown and only test 1 assertion this can lead to very lengthy test suites times. This is a huge problem as once test suites get slow a whole host of bad things happen (not run, slow development, etc).

  • test suite size. If 3 tests are written that have a total of 19 assertions, changed to those tests are relatively easy. However if the 19 assertions are used to break down the tests into 19 tests with 1 assertion per test

  • Not DRY. When a lot of examples repeat the same setup and teardown, you start to get a lot of cut-and-paste programming and code in multiple places that needs to be updated instead of code in one place. I have frequently seen "some" tests have their setup changed without the changes being applied to other tests leading to a mish-mash of styles

  • programmers have different opinions and experience. I work with a lot of programmers who don't follow the "1 assertion per test" rule. Making every follow this rule would be a challenge (worth it in some organizations but not in others).

For small test suites only

I've changed my opinion on the "rule" of only 1 assertion per test over time.

A few years ago, when entering the QA field, I considered it a great approach and almost too obvious to explain and justify. As other poster have written, it's the way to be specific about what is being tested, for failures to be more specific, etc, etc.

However I've seen that in real world situations, most significantly as test suites grow, treating this as a fixed rule becomes problematic due to the following reasons:

  • test suites run length. If all tests have to perform setup and teardown (they should) and only test 1 assertion this can lead to very lengthy test suites times. This is a huge problem as once test suites get slow a whole host of bad things happen (not run, slow development, increasing use of shared seed data, etc).

  • test suite size. If 3 tests are written that have a total of 19 assertions, changes to those tests are relatively easy. However if the 19 assertions are used to break down the tests into 19 tests with 1 assertion per test, maintaining and updating the code becomes error-prone, time-consuming and boring for many programmers.

  • Not DRY. When a lot of examples repeat the same setup and teardown, you start to get a lot of cut-and-paste programming and code in multiple places that needs to be updated instead of code in one place. I have frequently seen "some" tests have their setup changed without the changes being applied to other tests leading to a mish-mash of styles

  • programmers have different opinions and experience. I work with a lot of programmers who don't follow the "1 assertion per test" rule. Making every follow this rule would be a challenge (worth it in some organizations but not in others).

Note: An example of "multiple assertions" is that for a given UI operation one might expect

  1. to be on a certain screen
  2. to see certain data in a certain place
  3. To see correct error messages
  4. To see correct action buttons to proceed
  5. To see the database updated correctly.

So this is 5 assertions for the same action.

One thing I do agree with and find hellps is @ethal evans mention of
"only permit them at one point in the execution of the test."
I also expect to put them at the end of the test (excepting any required teardown).

Source Link
Michael Durrant
Michael Durrant
  • 25.2k
  • 3
  • 41
  • 113

For small test suites only

I've changed my opinion on the "rule" of only 1 assertion per test over time.

A few years ago, when entering the QA field, I considered it a great approach and almost too obvious to explain and justify. As other poster have written, it's the way to be specific about what is being tested, for failures to be more specific, etc, etc.

However I've seen that in real world situations, most significantly as test suites grow, treating this as a fixed rules becomes a problem due to the following reasons:

Note: An example of "multiple assertions" is that for a given UI operation one might expect 1) to be on a certain screen 2) to see certain data in a certain place 3) To see correct error messages 4) To see correct action buttons to proceed 5) To see the database updated correctly. So this is 5 assertions for the same action.

  • test suites length. If all tests have to perform setup and teardown and only test 1 assertion this can lead to very lengthy test suites times. This is a huge problem as once test suites get slow a whole host of bad things happen (not run, slow development, etc).

  • test suite size. If 3 tests are written that have a total of 19 assertions, changed to those tests are relatively easy. However if the 19 assertions are used to break down the tests into 19 tests with 1 assertion per test

  • Not DRY. When a lot of examples repeat the same setup and teardown, you start to get a lot of cut-and-paste programming and code in multiple places that needs to be updated instead of code in one place. I have frequently seen "some" tests have their setup changed without the changes being applied to other tests leading to a mish-mash of styles

  • programmers have different opinions and experience. I work with a lot of programmers who don't follow the "1 assertion per test" rule. Making every follow this rule would be a challenge (worth it in some organizations but not in others).