I'm quite interested in the Scipy/Numpy project which is basically a Python scientific computation/numerical methods library. I'm also interested in testing. What are some good approaches for testing such a library, or any software library? There already are extensive unit tests, which run pretty well. But what would an acceptance test look like? Thoughts?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
An acceptance test is supposed to actually simulate the actual functionality a user would use. For example the acceptance test of using an email application would involve both logging in and opening the inbox. You may have a unit test on logging in, and a unit test on opening the inbox, but the acceptance test says that the two actually work together to give the user some functionality.
Key point: acceptance tests are written by the customer (or at least from the customer's point of view.)
So how is the library being used? When I think of most libraries I use, each piece of the public API of that library is independent of the rest of the public pieces of the API (i.e. I'm not forced usually forced to use one part just so I can use another part. Not usually.)
For most libraries (as opposed to applications) the unit tests will give you the same information as acceptance tests will. It would seem strange for extensive unit tests to pass, and yet the library not be in a user-acceptable state. But, like any product, I would find out how your users are actually using the library and try to write a test around it. You might take some open source applications that use your app and compile against them, and run functional tests on those. If your changes break the app, it's a good indicator that the users would actually be displeased with that build of your library.
Define your view of the term acceptance test first, although it usually agreed to be something like "testing from the customer perspective" each company have variations on it, for example by choosing different customers- a customer can be the end user of a calculator or a developer using the library.