Questions tagged [test-design]
Use for questions about designing tests, specifically selecting test techniques to use in specific circumstances.
532 questions
63
votes
17
answers
29k
views
How should a Software Tester deal with missed Defects/bugs in Production?
I work as a QA/test engineer in a small software company. It is an e-commerce based product. I was assigned a few projects by my manager. I mostly do functional/blackbox testing. Before the production ...
- 611
49
votes
16
answers
16k
views
Can a Manual Tester survive in software industry without learning Automation?
I mean to say is that if I dislike coding and don't understand complex code, then will I be able to survive as a QA. Or do I need to learn test automation to grow position and pay in my career, as I ...
- 491
44
votes
12
answers
10k
views
What should Testers do if they are not able to find good defects in the product?
I work as manual/black-box tester. My responsibility primarily includes functional testing of a web application and to some extent database testing using sql server, based on the functional ...
user15122
29
votes
10
answers
9k
views
How can developers be assured that their software is fault free?
For example when creating software for critical real time applications, like those used in aviation, how certain are you that it's going to run without problems? Are you 100% certain or are you >90% ...
- 445
27
votes
8
answers
5k
views
What should a Software Tester do to become more valuable and helpful for the Development Team?
When I write detailed defect descriptions or bug reports, then my team's Development manager feels satisfied. What are few things a blackbox tester can do to help the Developers become more productive ...
user15122
23
votes
9
answers
5k
views
Should application and automated test languages match?
If an application is written in a certain language, e.g. Python, Ruby, Clojure, etc. should Quality Engineers use the same language to write the tests?
Should this be a decision that we can apply ...
- 331
22
votes
7
answers
4k
views
How to report a bug that has no technical solution?
When we were testing a project, which would require an Excel sheet as an input. The only validating mechanism of this whole file importing process was offered by Excel macro (validate against inputs' ...
- 1,495
22
votes
6
answers
13k
views
Is an inverted test pyramid really an anti-pattern?
I know that the inverted test pyramid, i.e., having more end-to-end tests than unit tests is an anti-pattern.
However, I've started to think, what could be advantage of having less unit tests then ...
- 10.5k
21
votes
2
answers
1k
views
Systematic approaches to selection of test data
I've used Phadke's design of experiments approach to select parameter values for testing (software) systems.
In a nutshell, the approach uses minimum, typical and maximum values of test (method) ...
- 586
18
votes
9
answers
741
views
How to obtain "fresh mind" results?
I noticed that when you start on a project with a beginner mind (for that particular product) you have a fresh approach and start noticing a lot of bugs.
After a while and some experience with that ...
18
votes
5
answers
2k
views
What Are Some Examples of Why Explicitly Sleep Statements Are Bad?
I've read various places that explicit sleep statements are bad practice in automated tests, even if I've used them occasionally. They can be annoying (creating unnecessary waiting in tests) but I can'...
- 2,205
18
votes
4
answers
3k
views
Is refreshing multiple times a test case for web applications?
I have been faced with a QA test case where refreshing a page multiple times almost instantly (6x, 7x, 8x) creates a bug. This is totally random and every time the number of refreshes require to ...
- 183
17
votes
8
answers
59k
views
Difference between requirement, use case, and scenario?
I've taken a class on software testing best practices and test design. In the content, they have a model that basically shows that the three items are derived in this way:
Requirements > Use case &...
- 301
17
votes
7
answers
13k
views
How can I structure Selenium tests in a way that minimizes the maintenance work?
What is the best way to build/structure the tests in a way that will minimize the work needed to keep the tests up to date as the product changes and improves?
- 171
16
votes
4
answers
322
views
What techniques are available to determine which browsers to test?
When websites being deployed to the internet are about to go live, the age old question always comes up. "Which browsers should I test?"
I am not asking that question because it will soon become ...
- 9,750
15
votes
7
answers
2k
views
The test team as an enemy of development? And how can this be avoided?
Details
Forming a Scrum team should include all the skills necessary to develop a user story in order to deliver a potentially deliverable product increment with each sprint.
In traditional ...
- 1,117
15
votes
4
answers
3k
views
Test strategy to bring automated tests from UI to lower(API/Unit) layers?
Background/Problem Statement: Recently I came across a project team, which is having thousands of UI automated tests covering all kind of manual tests including positive, negative etc., and obviously ...
- 5,573
15
votes
4
answers
807
views
Good strategy for automated regression where initial state can't be guaranteed or predicted
The situation
I need to start building automated regression for a very large web application with thousands of users. It is impossible to start from a known state, or even to be sure that data ...
- 31.5k
14
votes
4
answers
831
views
How do you know if you are writing high quality tests?
Inspired by Exercises on software testing
We can learn to write automated tests and we can learn to do tdd and bdd but what tools / resources / guides are available to tell us if the actual tests we ...
- 25.2k
14
votes
1
answer
4k
views
Is Selenium's LoadableComponent a good solution as test design pattern?
I need to refactor my Selenium tests, and come across their LoadableComponent wiki page. I began trying to use it, though I couldn't really understand what is it good for. I took a look at their ...
- 583
13
votes
9
answers
5k
views
How do quality assurance engineers test calculations?
How do quality assurance engineers test calculations done inside the code and calculations in stored procedures?
Also if the calculation is complex what should be the approach to test something like ...
- 1,197
13
votes
5
answers
4k
views
What to do when a bug is fixed, unfixed and fixed
A blocker bug I raised last week had been fixed and ready for re-testing.
This morning, I re-tested it, it passed and subsequently closed.
In the afternoon, by accident, I noticed this feature was ...
- 1,495
13
votes
11
answers
2k
views
What is a Test Automation Framework?
Some automation engineers use an underlying automation framework as a jar dependency in their selenium web driver project in eclipse. Then, they use the framework to write functional scripting of the ...
- 139
13
votes
4
answers
2k
views
How long should a test wait to assume that the result remains fixed
We had a defect where in a situation the state of the application changed into something undesired. It has been fixed, we added a test:
Set up state
Action that triggered the wrong state change
Check ...
- 32.6k
13
votes
5
answers
2k
views
Test for unknown issues
We have been developing wifi/2g boards that communicate to a server and store data.
This project has been running for several years already, and we are constantly updating both boards' firmware and ...
- 231
13
votes
5
answers
385
views
Heuristics for identifying "one-time" tests?
I understand the importance of having good "regression tests" to ensure fixed bugs don't get reintroduced. But I think it's possible to go too far and end up with a massive "regression test suite" for ...
- 2,699
13
votes
5
answers
3k
views
Are Mind Maps useful as a testing tool?
I don’t know much about Mind maps but I’ve seen expert testers (James Bach and Michael Bolton) use them to plan out their test strategy – usually in a demonstration. It seems like a plausible way to ...
- 3,741
13
votes
3
answers
6k
views
Random test data: How to do it right?
Several months ago I started an experiment in test automation. Each test, before it starts, picks up random test data. For instance, when my test requires a user with administration privileges, I draw ...
- 10.5k
12
votes
6
answers
37k
views
Test Suite vs. Test Plan
I am having a difficult time pinning down an official description of what a test suite is compared to a test plan.
In Test Computer Software they don't even mention suites. (I assume because the book ...
- 175
12
votes
9
answers
4k
views
Tools for Server application testing [closed]
I'm currently working (almost finishing) a Java project that consists on a server application and some client applications on the same network, that will communicate with the server only to request ...
12
votes
6
answers
5k
views
Whose responsibility is it to design test cases?
The question: Is it the developer's responsibility to design test cases, or the QA tester's? Or is it someone else's?
I am a software engineer at a small company in the United States. We have an ...
- 219
12
votes
4
answers
7k
views
Is there a guide for systematic way of writing test scenarios?
Are there best practices in coming up with effective test scenarios? Are there documented test scenarios that can be reused across different systems?
- 761
12
votes
6
answers
575
views
What online games (ideally with a bunch of glitches) would make for a good software testing training exercise?
I'll be giving a 3-day course on software test design later this month with lots of hands-on exercises. I'd like to include one or more exercises that would involve testing an online game for ...
- 2,357
11
votes
6
answers
4k
views
How many test steps are allowed in a single manual test case?
I have been writing Test Cases for half a year as of now. My question is: How many test steps are allowed only in one test case? For example more than 6 steps is not allowed or depends on ...
- 137
11
votes
7
answers
30k
views
How to test a toaster? (interview question)
I want to know how we should approach for testing real world objects. The problem that I faced before is to get the list of use cases from the interviewer. Whenever I asked what the real world object ...
- 271
11
votes
5
answers
819
views
Signal to noise ratio during early testing
The Context
For the past several years our QA team has been very active in manual and automated testing of our products which led to a very small amount of real-user complaints about the functioning ...
- 11.4k
11
votes
4
answers
341
views
How do you design your test method/function
This question has been revolving on my mind from quite some time.
Do you develop one method for one test scenario or club similar scenarios in one method and develop automated tests for it. Both ...
- 3,447
10
votes
10
answers
763
views
Should a test method test only one condition?
While writing automated test I come across situations when I include multiple verification points in one method but then I have also read/heard that one tests should make just one verification. Which ...
- 3,447
10
votes
10
answers
2k
views
What type of tests you should do first?
I had this question in a job interview:
Imagine that you have got a new application. You have never seen it
before, it is absolutely new. What types of testing will you do first?
As far as I ...
- 111
10
votes
5
answers
1k
views
What proportion of different types of software testing does your team use?
Note: I've asked this over on SO and p.SE and it got closed. I'm not trying to game the system, just looking for a home :)
I'm seeking a little "wisdom of crowds" estimation or a pointer to an ...
- 328
10
votes
3
answers
403
views
What criteria do I apply to a highly complex development project to warrant getting "real world" data?
Both the product I used to work on and the one I'm working on now have a fairly large and complex matrix of possible configurations, work-flows, and data scenarios. The one product had, no joke, ...
- 2,637
10
votes
5
answers
7k
views
How common is testing based on log files analysis?
Here is a document that outlines challenges and solutions of testing of Mars Science Laboratory.
I found the approach of log files analysis unexpected but quite beneficial on many levels. It seems to ...
- 203
10
votes
1
answer
5k
views
What goes in a Testing playbook?
The concept of the Testing Playbook has been recently added to the list of possible tools in your Context-Driven toolset. The Playbook has been described as being similar to the list of plays that a ...
- 221
9
votes
5
answers
3k
views
Are state transition diagrams a good software testing technique
I have been looking at different testing techniques and wondered if anyone had some "real world" examples of using state transition diagrams rather than the bog standard clock and light switch ones ...
- 113
9
votes
6
answers
981
views
How to prevent cheating by manual testers?
I'm the guy who is motivated to bring some quality assurance into our team. The problem is, that our developers very often hate testing and even if they have a test protocol, not all of them are ...
- 119
9
votes
7
answers
952
views
Testing without direction, how not to miss things? What about layout?
So this might sound like a "complain" post but it really isn't meant to be. But often-times I will be handed a project, lets say a Website. And be told to give it a "Full run through".
So I take a ...
- 1,720
9
votes
3
answers
160
views
Designing an open source test framework without compromising security
Context
The robot framework website mentions robotframework-autoitlibrary in
its external library listing and it points to
http://code.google.com/p/robotframework-autoitlibrary/.And I found a
...
- 11.1k
9
votes
4
answers
2k
views
How to implement qa test process for large scale application?
There is a large scale application with thousands of functional points and which never had any QA team or person. So when a QA person or team starts working with this type of product what should be ...
- 181
9
votes
4
answers
2k
views
Test case design standards- friend or foe?
Currently we use an agile team based approach to testing, where each team is responsible for their specific area of a product. While we have found this to be effective and easily manageable, over ...
- 1,326
9
votes
4
answers
1k
views
How to make sure your test data is correct?
We are doing integration tests where we simulate manually data coming from external systems to our application. Our integration tests have become the bottleneck in our process, because preparing test ...
- 10.5k