Should the tester check usability, security, reliability, anything else?
Personally, I think yes, but no.
Yes in the sense that you need to make sure it's "good" in a common sense way. But once you've identified where it's not "good" you need to break down why into a functional requirement. Consider the categories:
Usability is not something that can be tested. How can you put into a specification "Must be easy to use" or something like that?
Security is something that can be tested, but how do you know something is secure if you don't have a requirement that says what the definition of secure means? You could lock down every port on a server and call it secure, but it won't do anything.
Reliability is another thing that can't really be defined outside of the requirements. If it doesn't meet the requirements all the time, it doesn't meet the requirements. Period.
Something else that might fall in this category is efficiency. Someone might say "the app runs too slow." Well, how fast does it need to run? "Well I don't know, but this is too slow." Please. Come back when you have a requirement. Sure, I can take a look at the routines, and maybe come up with something, but I need to know what to shoot for.
|show 3 more comments|
In my opinion, testers should absolutely be testing non-functional attributes (performance, usability, privacy, security, reliability, compatability, supportability, etc.). In general, I think testing should have less emphasis on functional requirements and more emphasis on non-functional requirements, but I don't want to side track your question.
If you're looking for a list, start with the ilities. If you're looking for a recommended list, it sort of depends on your product and customers. Overall, I find the list in my first paragraph a good place to start, but there's no one size fits all answer here.
Alan's answer is spot on with the "-ilities". I also recommend that you buy a copy of Release It! by Michael Nygard - it's the key book for software developers and testers to help them understand what kind of things make software fail to operate well in Production. It will give you great insights into the kinds of "non-functional" or operational testing which you need to undertake on any non-trivial software system.
I identified several key chapters/pages in Release It! in terms of making software work well in production in a blog post here: http://blog.softwareoperability.com/2013/01/26/which-sections-of-release-it-help-to-make-software-operable/ These pages would be a good starting point for the kinds of operational testing you should be doing.