How to test different tools to be able to compare them?

Question

I've been looking into software testing since we want to test and compare several tools (tools = software programs) made for the same goal, eg. all html editors. Although the tools we have now are open source, we are not interested in the code.

So that made me look at black-box/functional testing, but that's described as using the programs specification to create your test cases. What we wanted to do though is to use a more general test case for all the tools. That test case would fit the general specification of the domain these tools are build for, but not explicitly for any one of them.

Would that still be called black-box/functional testing or is there a different approach used for this?

And are there other recommendations for this type of 'test and compare' situation?

EDIT: after further reading, I think what I am looking for are System or Acceptance Testing techniques. But still, any other recommendation or correction is welcome.

Answer 1

As with most other tests, let your Requirements drive your testing, without being tool-specific.

Do you have Requirements for what your tool-under-test must do? If so use that. Don't let the specifics of how the tool fulfills the Requirement become part of your test cases.

For example, if your tool must be able to open an existing HTML file, then don't include the keystrokes used within your test case. Just refer to the fact that you Open the file.