What is the best way to perform informal testing?

Question

Sometimes I got the projects which don't have any formal requirement, Client provides us AUT directly and ask to perform testing on it. In this case, what is the best way to start testing?

Answer 1

As mentioned before, you need some sort of requirements to tell if the result of an action is correct. This is also known as the oracle problem, which says that testers rely on partial oracles in order to decide whether a piece of software behaves correctly. Of course, there might be situations in which you can tell that the observed behavior is wrong, although you don't exactly know what the SUT is supposed to do:

A tester might recognize a result of a calculation as impossibly large even though she doesn’t know what the exact result should be. (You might not know offhand what 1.465732 x 2.74312 is, but if a program said 7,000,000 you could reject that as obviously wrong without doing any calculations.)

Therefore, to be effective and efficient, talk to your client. You don't need super-formal, machine-readable requirements, but at least an overview as well as some core processes and features. This will make it at lot easier to define, execute, and interpret test cases.

However, there're techniques you can use without any knowledge about the SUT. For instance, you can set up a monkey testing environment to find crashes and hangs. This scales easily since you can run an arbitrary number of instances simultaneously in the cloud. Some examples for monkey testing tools are:

• Web: gremlin.js
• Android: UI/Application Exerciser Monkey
• Java Swing: ReTest (disclaimer: I'm a software engineer at ReTest)

Moreover, you can then use these "generated" test cases to perform regression testing when you get a new version of the SUT. Regression testing sidesteps the oracle problem by using the software itself as the oracle. That is, the results of a previous (typically stable) version serve as the oracle for the tests (also known as consistency oracle as it compares the consistency between two versions).

Answer 2

I think all the time we are facing such issue , a product without any documentations, So the approach should be like this:

1. Define you testing technique Exploratory Testing
2. Try to know the main purpose or target business function that the AUT is handling (Happy Path) by discussing with product stockholders.
3. if you are working in a group, try to dived work between team, some peoples to focus on GUI (Look and Feel issues), Some for Security , some for performance if applicable.

Answer 3

You need a spec of some sort. Otherwise, you really have no idea what the program is supposed to do, or what requirements it has. For example, say it doesn't start at all when you execute it. Is this because it's just fundamentally broken, or because you're missing a license file somewhere, or because it's only supposed to work on Windows 8 and above, and you're running it on Windows 7?

However, there are probably things that you can use as unofficial specs, with the caveat that they're almost certainly incomplete. For example, there's probably some sort of documentation available, that lists system requirements, goes through the setup process, and talks about the program's functionality. You could test to make sure that what's in the documentation is correct. If the program has certain certifications, like "Works with Windows x," you can validate that those certification requirements are met. For a web application, you could potentially do load/stress testing.

When you were done, I'd write some sort of test report with your assumptions, and what you tested for, and what problems you found. For example, even if the documentation doesn't say so specifically, I'd work under the assumption that the program shouldn't crash/freeze while doing things it's documented to be able to do, or corrupt data.