I'm quite interested in the Scipy/Numpy project which is basically a Python scientific computation/numerical methods library. I'm also interested in testing. What are some good approaches for testing such a library, or any software library? There already are extensive unit tests, which run pretty well. But what would an acceptance test look like? Thoughts?
asked May 11, 2011 at 1:42
Add a comment
|
2 Answers
An acceptance test is supposed to actually simulate the actual functionality a user would use. For example the acceptance test of using an email application would involve both logging in and opening the inbox. You may have a unit test on logging in, and a unit test on opening the inbox, but the acceptance test says that the two actually work together to give the user some functionality.
Key point: acceptance tests are written by the customer (or at least from the customer's point of view.)
So how is the library being used? When I think of most libraries I use, each piece of the public API of that library is independent of the rest of the public pieces of the API (i.e. I'm not forced usually forced to use one part just so I can use another part. Not usually.)
For most libraries (as opposed to applications) the unit tests will give you the same information as acceptance tests will. It would seem strange for extensive unit tests to pass, and yet the library not be in a user-acceptable state. But, like any product, I would find out how your users are actually using the library and try to write a test around it. You might take some open source applications that use your app and compile against them, and run functional tests on those. If your changes break the app, it's a good indicator that the users would actually be displeased with that build of your library.
-
Some great points. I would like to add that, yes, if the unit tests are passing, that's a good sign. In fact, one kind of form of user acceptance test would be running all unit tests for a particular module, and seeing if there are any problems. Commented May 12, 2011 at 2:31
-
It will be good to know whether the unit tests pass, but whether they do so is orthogonal to whether the acceptance test will pass. The unit test should verify that the library does what the developer intended. The acceptance test verifies that the library does what the customer intended. If a unit test fails but the customer doesn't care about that part of the library, the acceptance test may still pass.– user246Commented May 12, 2011 at 13:19
-
If the Unit Tests are failing the build should never have made it into Test in the first place, the Acceptance Test will tend to be more generic and often times be "Happy Path" since its only going to demonstrate that functionality has been achieved. glowcoder definitely links the two rightly though, as they are both Tests and may cover the same things but not in the same way.– MichaelFCommented May 24, 2011 at 13:05
Define your view of the term acceptance test first, although it usually agreed to be something like "testing from the customer perspective" each company have variations on it, for example by choosing different customers- a customer can be the end user of a calculator or a developer using the library.
