Strategies and Techniques for Exploratory Testing and finding new bugs?

Question

From what I understand, "Exploratory Testing" is about free-style testing that largely depends on creativity of the testers and how well they know the product or feature under test. It is something that, usually in practice, "delivers" a higher defect detection ratio than scripted/pre-written scenarios based testing.

What are the general recommended strategies and techniques for Exploratory Testing?

---

To put some context here: I've heard about one method which, I think, was called "pair testing" - it involves one person thinking/inventing a use case and the other actually going through it; or just simply inter-exchanging ideas during the pair testing. What worked for you?

Answer 1

What are the general recommended strategies and techniques for Exploratory Testing?

Recommended Strategies with ET:

1. Use Exploratory Charters to guide your work. Even if you don't timebox your exploration, charters will give you just enough specifics to be useful.

   • It will take time to develop your ability to write good charters. Start with crappy ones and then work on improving them over time. As you get more comfortable, the easier this becomes.

2. Take basic notes that will help you summarize and learn from your charters. Whether these are mindmaps, or simple notes, etc.

3. Time box your charters into sessions. This is the basis for Session Based Test Management (SBTM).

4. Continue to practice, try new things and incorporate feedback.

Techniques with ET:

You can use any test techniques you'd like in an exploratory way:

• Functional, Domain (boundary + equivalence analysis), Scenario, Usability, Performance, etc.
• You can even use automation in an exploratory way.

Other ET / blended approaches:

You mentioned pair testing which you can definitely do in an exploratory way. I'd recommend using charters and then switch off between the pair for who does the driving (hands on work).

Answer 2

Gather an audience from a wide background and expertise

• Different people with different background offer different ways of thinking. It is surprising how a person sitting next to you offers a new way of thinking.

Brain-storming session with this gathered crowd

• When people are working together, they stimulate each other into getting better ideas faster.

Brainstormed ideas can then be used as exploratory testing methods.

• It is also beneficial to develop a culture of brain-storming exploratory testing ideas.
• During brain storming sessions, I prefer having everyone writing down their testing ideas first on a piece of paper without showing to anyone. This way, people are less likely to be influenced by others' logic, and hence a wider spread of ideas. Of course, testing ideas are shared among everyone after everyone has exhausted their creativity for a given brainstorming session.

Some tools can be helpful

• Mind map can be very helpful to visualize relationship among test ideas. You may visually discover new test ideas and etc.

No test plan is needed

• In general, exploratory testing does not require a pre-written test plan; some people will argue a test plan is always helpful for bookkeeping purpose. In my personal experience, I prefer no written test plan before exploratory testing is completed.

Pair testing or even group testing

• When working in groups, people motivate each other and can manage time more efficiently in general.

Answer 3

"free-style testing that largely depends on the creativity of the testers and how well they know the product or feature under test."

I would switch 'creativity' (which I also see at Wikipedia) with 'experience and training'. How well they know the product is definitely helpful.

I break down exploratory testing like this:

The different paths:

• Happy - user enters everything correctly and clicks correct buttons and links

• Sad - user enters invalid data or fails to enter data - error reporting to the end user should be done correctly - correct markup, not relying on the color red, etc.

• Optional - many sites have 'optional' functionality. For example additional orders, 2nd shipping address, different payment methods, etc. Optional workflows will also have happy and sad flows within them

So the above is for workflows. Now to examine individual elements (fields, buttons, text, etc): You want to test for the following:

• required data
• invalid data type (string for date, integer used for date, etc.)
• invalid data formats being used
• data validations, e.g. 'added date' must be in the past, 'dob' must be > 1900, etc.
• boundary testing (1 under limit, at limit, 1 over limit, etc.)
• valid relationships between data

I feel that the main part of 'free-style' for exploratory testing is spotting new or changed functionality not previously - or recently - tested.
Part of an approach can include 'I only check x every 5 runs, I only check y once a week, I only check z once a month' and I myself have used this approach. It allows a tester to cover a lot more with their testing and helps with the human balance between monotony, boredom and rote and the need to be comprehensive in testing.

One element that might relate to creativity is tester experience and attitude. For example, I know one exploratory tester whose goal tends to be to give a thumbs-up for release. Another tester I know often sees their mission as 'there is always something I can break, I just have to to try hard enough' and they nearly always manage to find some sort of bug - though often minor and not necessarily related to recent change. They have even said that release testing is often a good excuse to check the overall health and functionality of the product. A big part of exploratory testing is also that you learn more each time you do a session and that knowledge gets used by your brain in future sessions without updating any scripts.

In the more broader goal of 'find more bugs' I would also recommend learning more about:

• The industry you are in
• Characteristics of the users of the product
• The product and how it is used as part of other processes
• The business aims and goals of your company
• 'new user' testing services such as usertesting and usabilitysciences

Answer 4

Because developers usually just test "happy path", any detour from it may uncover lurking problems. Omit some input fields. Enter fields in different order. Go back and forth between pages. Change page size. Enter invalid input.

Happy path should be automated (including check for standard errors), but humans are very inventive in a way how to break the system if they stray from happy path.

With experience, you will find ways which broke some pages - chances are, other pages cannot handle such breakage too.

Looking at OP profile, and answers in SO like https://stackoverflow.com/questions/30600738/difference-running-protractor-with-without-selenium : OP is quite experienced tester, so basic answer like this one is likely not be of much value for OP - but maybe it will benefit others.

Answer 5

Pre-written test cases are usually going to be written against a spec and cover the anticipated scenarios. Usually these things are going to work because the developer has checked for basic functionality.

The reason exploratory testing finds more bugs is because subtle interactions between features or false assumptions that the code makes in unusual situations are hard to account for completely. For instance if cancel out of a user flow, maybe the application assumed some data that did not end up getting entered.

You can also have user scenarios that were not anticipated at all, like if the application allows me to enter an unlimited amount of input that ends of causing a problem.

In practice exploratory testing feels kind of random, but I think I am actually trying to find something the application allows, but that was unanticipated.