My web system has roles which allows the users to access different resources depending on the role.

My questions are: should permissions be tested by automated integration tests? Are there best practices about it?


There are a lot of ways to answer the first question.

Should this specific feature be tested?

That depends on how important the feature is and whether it is possible that the feature was broken by a change since the last time the feature was tested.

Should the association between permissions and users be tested?

See above.

Should this specific role be tested?

See above.

Should permissions be tested by automated tests?

That depends on whether the permissions will already be tested by manual testing, and whether you think you will save time or reduce risk by automated those tests compared to testing manually. Remember: it takes time to write automated tests, and you have to maintain them afterwards.

Should permissions be tested by automated integration tests?

Assuming the answer to the previous question is Yes, you should decide whether you can test permissions adequately with automated unit tests, which may be easier to write and faster to run. If you can't then you will want to write integration tests.

I don't know how to answer the best practices question.


As user246 says, whether you automate your permissions tests depends on the application and whether it can be properly tested that way. The question of what you test and how always depends on the application, its user base, and how critical each feature is to that user base.

There are no best practices. This is something that a lot of people find difficult to believe or accept. The reason there are no best practices is very simple: every application is different. The best way to test a given application may be completely different than the best way to test another application.

Within the context of permissions testing, here are a few examples:

  • If the software gives users with admin permissions the ability to modify other people's financial information, there are legal and regulatory requirements for access and logging of their use of the information. That makes it essential that this type of software be thoroughly tested and have the testing documented. I would definitely automate permissions testing in this case and include checking that everything which should be logged is logged.
  • If the software allows logged in users to store non-identifying data like site preferences, testing can be limited to verifying that the storage occurs and that each user gets the correct preferences applied. This may or may not be automated depending on how often you need to test it (probably not very, since this kind of thing tends to be a set-and-forget operation so only needs to be tested when changes are made to that module).
  • If the software requires logged in users to run and maintains multiple levels of permissions, automation may or may not be the best choice depending on how often changes are made to which level of access can access what items. With software like this, I'll typically manually test new permissions which are unlikely to get much use, and automate permissions that are critical (such as in retail point of sale software making sure that a user with sales clerk level permissions can make sales).

For just those three examples, the "best practice" for user permissions is completely different for each one, and could be different for different applications within each grouping. It's always a judgment call - I will make that judgment based on factors like this:

  • if I have to test something often enough that it gets boring, I will consider automating it. Whether I actually automate it or not depends on how often I have to test it and how much work I would have to do to automate it.
  • if I have to test something for every release, I will automate it unless it's something where automation would be either impossible or too fragile to be worthwhile
    • device interaction is rarely if ever worth automating. Yes, it's possible to simulate device input, but it's usually too fragile and/or doesn't provide the test you need (an example of this is PIN pad entry)
    • interaction with third party software is rarely worth automating. There are too many external factors involved.
    • anything involving printouts, look and feel, and appearance is better done manually. Automation is too time-consuming and fragile.
    • if it's something that's typically done once and forgotten, why am I testing it every release? If I automate this, it's automated for my convenience, such as a lengthy setup sequence that I use to build testing data and run when I need it rather than all the time.
  • if it's something that's critical to the application user base, and the user base is critical to my employer's finances, I will automate as much as I can, following the 80/20 rule of aiming for the 20% of the application getting 80% of the use.

Broad heuristics like this make a better guide to what you should be doing than best practices, because they're intended to work within a context, where a best practice typically is not.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.