Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Historically at our company the chosen past by our prodecessors has been to take screen shots of every step of a test in a test script, even if no bug has been found.

We are no moving from a waterfall method to interative development cycles.

Over the last 6 months we have been trialing, using rapid test design using mindmaps and not taking screen shots, instead stating that the test cases in the mindmaps and the mangement spreadsheets are evidence that the test has been thought of and then executed.

It's our belief that taking screenshots does not prove anything and actually uses valuable testing time. However there is some resistance with people believing the mind map or similar processes do not give them sufficent coverage if questioned at a later date about that test.

I would be interesting in hearing what everyones thoughts and opinions are, specifically relating to the following questions. Either way it has been ruled out that we will purchase recording tools.

1) Do you take screen shots to prove a script has been executed or just when a bug is found?
2) Have you found not taking screen shots to evidence a test, has caused any confrontation or issues? 3) Have your found any disadvantages using rapid test case design and mindmpas for test analysis?

Please can you state whether you work for a Transnational, national, regional or startup company. We are Small National company and would like to hear from everyone, but need take a balance view as to what we can achieve and not achieve.

thankyou for taking the time to read this.

Tim

share|improve this question
    
In your experience have these screenshots ever been useful in a dispute? How much time is spent taking them? –  Phil Kirkham Jul 8 at 16:42
    
What industry do you work in - for safety critical and some other domains there could be a need for recorded evidence –  Phil Kirkham Jul 8 at 16:47

7 Answers 7

up vote 3 down vote accepted

Anecdote: I worked on one project for a national company. Testing was being done and overseen by one of the Big Name consultancies. Their testers were told to take screenshots for every test step. They abandoned this practice as (1) it was taking too much time and test progress was too slow and (2) I was fnding 90%of the defects on the project by doing exploratory testing.

share|improve this answer

We record a video of our test after design of what a "good" test case pass looks like and put it in shared storage. After that videos are only kept on fail. This way we can always go back and look at what the original good run looks like.

share|improve this answer

To document testing scenarios, we use a wiki with text-based steps. Our wiki (FOSWiki) has version control, so you can even see how steps changed over time (and who made changes). Way better than docs on shared drive - hyperlinks make navigation a breeze. And FOSWiki allows to attach images to pages. Linking to other wiki pages with related/similar scenarios and/or bug to bugzilla is trivial. We also use (highly customized) bugzilla (which also allows attaching images) but we use it only if it helps to clear something - in 95% cases, text is enough.

We are international company with agile process and high-availability environment (100% Python) and have web-based app.

Id we started from scratch, we would go with TRAC, which integrates webSVN, wiki, bug database and has markup in code to create hyperlinks to wiki and bug database when code changes are viewed in browser via webSVN. It is pretty slick, free/open source, but alas we are too deep in our bugzilla and wiki. But we lustily look at TRAC integration and considering converting anyway.

http://trac.edgewall.org/

share|improve this answer

In my experience, the requirement to collect evidence (like screenshots) of tests being executed is driven by regulations in your industry. I had to collect evidence while testing military avionics (MIL-STD-2167), telecommunications equipment (TL9000), and financial reporting for publicly traded company (Sarbanes-Oxley).

When working with software that is not regulated, the evidence collection was not required, and generally not done. (I can't remember anytime when we did it)

You may want to check if your are required to collect that evidence before dropping it. Otherwise, if you are not required - then its a business decision on your part. Is it worth the time to collect detailed results of tests that pass? You would know by how many times you actually used the data that you collect. In my experience, its better to focus on collecting data/screenshots for failed tests to help diagnose the failures quicker.

Good luck.

share|improve this answer

I am working for a top large enterprise software company. I've worked for internet company and consumer product companies. Assume you mean taking screenshot for manual test execution. I never heard that screenshot is taken even no bug is found. If the intention is to simply prove the test is executed, you team got a trust and management issue, address it first.

share|improve this answer

Everything we do as testers requires some trade off between time (planning and executing) and achieving our testing mission. Taking screen shots to reference later or to have proof of coverage is fine as long as you balance that with time to achieve your mission (including adequately testing the system). Realistically this means if we want to record our steps we must have some way to automatically take the screen shots through a tool or just use video.

If you are using Windows (7 and later I believe) you can open up Problem Steps Recorder, hit record and it will do all of this for you. I'm sure there are other tools out there that will accomplish the same things.

I'll use tools like PSR while doing exploratory testing so I can review what I've done and even perhaps find a few interesting things that I missed while my attention was somewhere else. This often leads to follow up testing.

Note: Recording every step for review later is not the same thing as having to upload or place each photo into a tracking device. Doing this would probably defeat the efficiency gains in automatically recording the steps.

Note 2: Outside of the original tester these recordings don't mean much anyways. It's the testers job to report on what they found, to summarize and provide useful information back to the product team.

share|improve this answer

Other comments have covered the screenshot taking, so I address the third questions: Encountered disadvantages in using mindmaps for test analysis.

So: Sometimes projects are long and complex, and on those the state of testing is hard or next to impossible to interpret from mindmaps. Still, capturing screenshots is overkill in most of the situations. The best alternative in many situations is somewhere in between. A bit boringly it is nothing new - just documenting what is relevant to test in reusable format, tracking the outcome of testing and maintaining this information so that it is easy to find out what is best to test next. And that can be done with well ordered test cases. For test cases do consider what is relevant to write down. Is it fully scripted instructions, or is it possible just write down instructions what to test, not how to test. That is often faster and thus tields better bang for testing time.

Test cases are not always not a bliss. It is possible to go wrong with them in many ways, but in complex, long projects the "old method" provides good tools to work efficiently and maintain understanding what is going on in the project.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.