Tag Info

Hot answers tagged

21

If test automation does the job it is intended to, when a defect is introduced, it should be found and fixed very quickly, hopefully within a single daily cycle. If this is how the automation suite is working, then automation should become a "barrier" that prevents new bugs from being created in code that is already working and tested. When you think of it ...


13

Do you think you would find the remaining 5% of bugs that are currently found through a suite of manual tests if you spent that time doing exploratory testing instead? Would managers have the confidence in test coverage that they need to make decisions without those functional tests? I think the biggest risk of not having a manual functional test suite ...


12

I don't like the statement because it assumes that the only thing you can automate are regression tests. If you rewrote the statment as "Regression tests don't find new bugs", then I think it's accurate. I often write automation (e.g. performance, data driven, or model-based tests) that find new bugs - and I think the automation effort is incomplete without ...


6

As well as SnagIt (good shout Lyndon) I am starting to use Rapid Reporter which is proving to be a v. useful exploratory testing tool. It doesn't do video, but I use SnagIt for the video (once a defect has been found) to support the other documentation provided / generated by Rapid Reporter


6

"Is it worth while spending the time to write a full suite of manual functional tests if hardly any defects are found from them?" That depends - is the purpose of writing them to find defects? If so, then it sounds like you would be better off putting your energy into spending more time testing. If you're considering dropping them, then it doesn't sound ...


6

You may find yourself tempted to write out a bunch of charters early on to show management types so that it doesn't look like you're sitting on your hands. Resist this urge if you find it upon you. There's no real need to overcomplicate charters. If start a session and you realize the scope is overambitious, modify the scope and note down whatever new ...


5

Not saying that it's cheap, but, tools like OneNote tend to fill this void very well. We use a notebook for each application. For projects, we use Tabs/Sections for Modules/areas, and for operational type changes, we use a new section for each Release. Each session takes up 1 page. These get stored on either a sharepoint or in a shared folder that we all ...


5

The theory behind it is that an automated test will only catch exactly what it tests for. Since someone had to code it manually, someone is aware of how it will fail. By this definition it's not a new bug - someone already knows about it when they write the test. One could argue that it can catch new bugs, but not new symptoms. A lot of it depends on ...


5

I use two tools for these kinds of problems. One which I found because it is recommended by Michael Bolton (just to credit the source) is BB TestAssistant - http://www.bbsoftware.co.uk/BBTestAssistant.aspx which has a free express license available. The other is a tool called Jing which is free http://www.techsmith.com/jing/. BB TestAssistant I use for ...


5

I think there's one more piece to the answer that I see being hinted at in others' responses, but not directly called out. Often, automated tests are created in such a way that even non-essential details are locked in and executed the same way. This leads to what has been referred to as the Pesticide Paradox or the minefield analogy. Bugs are found when ...


5

I will predicate my answer on the assumption that you believe you are leveraging exploratory testing as well as you can; in other words, that you do not believe you can discover more than 95% of your defects via exploratory testing. I think the gist of your question is, "Is it worth the additional work of having a full suite of manual tests in order to ...


5

I recently discovered PSR.exe, which is part of Windows 7 (just type in psr in the search field). It allows the user to record everything he or she does (like Snagit), but also makes it easy to add comments and to highlight certain regions of the screen. Once you finish recording, everything will be saved in a zip file as a so called ".mht" file. When you ...


4

Hey. Other possibility is Test Explorer. It has very nice features: record test as 'steps' which reflect what tester did Capture screenshot for every step record movie add notes to test during recording save test as baseline record local files usage record registry changes record system resources used replay test (which is actually displaying steps from ...


4

Use whatever you are comfortable with. I'd suggest starting with something basic for recording notes like Notepad++. The important thing is to be able to keep detailed notes about your session, to setup a session charter, etc. Once you've got some practice with it, you can see what works and what doesn't. I've used Rapid Reporter and Session Tester as ...


4

1) As a free solution: You can use FFmpeg for full screen recording. For windows it requires ScreenCapturer (also free and opensource). For linux it can record the screen natively. http://svnpenn.blogspot.com/2011/10/ffmpeg-screencast-windows.html 2) Camstudio is also a good solution. The output files may be huge, but you always can encode it with FFmpeg ...


4

Caveat: I've discussed and thought about testing playbooks a little - I'm not convinced that I necessarily have the same interpretation as others, partly because there hasn't been enough discussion of the idea (that I'm aware of, would love references if people have them) for a consensus to develop. This is my best attempt at this point in time, it will ...


3

I like to keep things simple. I work with a small team of testers and programmers. We like to keep things lightweight. For managing charters and information flow, I use a spreadsheet and a whiteboard. I categorize charters by product area. I haven't found getting more detailed than that to be useful to me (or others). I name charter files by their product ...


3

In my experience, I tend to find more bugs while I'm creating the automated tests than I find by executing them. Still, it's not true that automated tests don't find bugs - some automated tests find some bugs, even if it's not many. And done well, the cost of executing automated tests is low enough that it's worth doing. I don't view the automation choice ...


2

Recently I learned about Screenpresso http://www.screenpresso.com/ It's not as good as SnagIt but much better than Jing. In free version you can record videos only up to 20 secs. This limitation is gone in paid version.


2

Have you tried qTrace? They seem to be coming out with a free version of their screen capture and defect submitting tool. It integrates directly with a bunch of bug trackers, I used it with Bugzilla nicely. Found it here: http://qasymphony.com/qtrace-features.html


2

I use a combination of three tools for screen capture while I am testing, be it exploratory or scripted. For screen recording BB Flashback Express is the free version of BB Test Assistant (http://www.bbsoftware.co.uk/BBFlashBack_FreePlayer.aspx). I set it to record constantly and retain 10 minutes of footage. This keeps the file size down, and resource ...


2

Really depnds on the person testing. I've yet to find a solution that works for everyone. Since I'm normally testing on windows machines, I tend to use a combination of psr.exe with OneNote and occaisionally SnagIt for when I need a screen shot to remind me exactly what to look at later if it's not in the scope of what I'm working on. Normally I'm not a ...


2

What I currently (well, over one year already) "misuse" as a kind of session reporter is the "BB Testassistant Expert". I heard about it in the RST class of Michael Bolton and got me the budget to buy it. Its under 200 EURO, plus one update of 60 EURO last winter. I just start it in the beginning and let it run. On my test machine it is not intrusive and ...


2

What do you do with your scripted manual tests? If they are just executed by testers once or twice then tossed you probably can change your process & reduce/eliminate the extra paperwork. But if any one else is using them, then consider the impact outside of QA before elimating them. Do you have requirements so that the legal department is expecting ...


2

We find 95% of the defects during exploratory testing so isit worth while spending the time to write a full suit of manual functional tests if hardly any defects are found from them??? The fact that you find 95% of the bugs you currently find using one approach is interesting. But the real questions are: are you being effective? Are you finding all ...


2

In using "exploratory" testing we've found that it's VERY dependant on the testers experience and understanding of the system. If the tester has a broad knowledge of how the software should work and what should be displayed then they'll be able to catch all those little bugs. The manual process may provide better coverage but might miss an element that ...


2

Exploratory tests can help to plan and write an effective functional test suite. When we are required to test a new feature it is always better to explore it before planning or automating it. Exploratory testing can generate new testing ideas/path and can reveal uncovered areas for testing as we explore the software. As Ethel Evans said with automation we ...


1

One more documentation tool for session-based test management for your consideration: qTrace qTrace includes the option to capture and display timestamp associated with every screen and action along with user's action. Besides, the duration of the record is also captured. Also, the timestamp feature is available in free edition.


1

I've just started using session based testing in place of scripted test cases. My charters have been fairly simple so far: The area of the feature to be tested, and how I plan to test. (Functional, performance..) Any changes to this area since last session. (Has a bug been corrected? Could it impact other areas?) Limitations, ie. I know of a piece that is ...


1

I assume for the purposes of your question you are not concerned about testing anything upstream from the JMS queue. You may want to check whether the values in the XML are consistent with any constraints in the databases, e.g. uniqueness or maximum length. Depending on your circumstances, you may also want to test what happens when the values are not ...



Only top voted, non community-wiki answers of a minimum length are eligible