Questions tagged [test-design]

Use for questions about designing tests, specifically selecting test techniques to use in specific circumstances.

Filter by
Sorted by
Tagged with
61 votes
17 answers
27k views

How should a Software Tester deal with missed Defects/bugs in Production?

I work as a QA/test engineer in a small software company. It is an e-commerce based product. I was assigned a few projects by my manager. I mostly do functional/blackbox testing. Before the production ...
user avatar
  • 591
48 votes
15 answers
15k views

Can a Manual Tester survive in software industry without learning Automation?

I mean to say is that if I dislike coding and don't understand complex code, then will I be able to survive as a QA. Or do I need to learn test automation to grow position and pay in my career, as I ...
user avatar
  • 481
44 votes
12 answers
9k views

What should Testers do if they are not able to find good defects in the product?

I work as manual/black-box tester. My responsibility primarily includes functional testing of a web application and to some extent database testing using sql server, based on the functional ...
user avatar
29 votes
10 answers
8k views

How can developers be assured that their software is fault free?

For example when creating software for critical real time applications, like those used in aviation, how certain are you that it's going to run without problems? Are you 100% certain or are you >90% ...
user avatar
  • 445
28 votes
8 answers
4k views

What should a Software Tester do to become more valuable and helpful for the Development Team?

When I write detailed defect descriptions or bug reports, then my team's Development manager feels satisfied. What are few things a blackbox tester can do to help the Developers become more productive ...
user avatar
23 votes
9 answers
4k views

Should application and automated test languages match?

If an application is written in a certain language, e.g. Python, Ruby, Clojure, etc. should Quality Engineers use the same language to write the tests? Should this be a decision that we can apply ...
user avatar
22 votes
7 answers
4k views

How to report a bug that has no technical solution?

When we were testing a project, which would require an Excel sheet as an input. The only validating mechanism of this whole file importing process was offered by Excel macro (validate against inputs' ...
user avatar
  • 1,515
21 votes
2 answers
1k views

Systematic approaches to selection of test data

I've used Phadke's design of experiments approach to select parameter values for testing (software) systems. In a nutshell, the approach uses minimum, typical and maximum values of test (method) ...
user avatar
  • 586
20 votes
6 answers
9k views

Is an inverted test pyramid really an anti-pattern?

I know that the inverted test pyramid, i.e., having more end-to-end tests than unit tests is an anti-pattern. However, I've started to think, what could be advantage of having less unit tests then ...
user avatar
  • 10.4k
19 votes
5 answers
2k views

What Are Some Examples of Why Explicitly Sleep Statements Are Bad?

I've read various places that explicit sleep statements are bad practice in automated tests, even if I've used them occasionally. They can be annoying (creating unnecessary waiting in tests) but I can'...
user avatar
18 votes
9 answers
730 views

How to obtain "fresh mind" results?

I noticed that when you start on a project with a beginner mind (for that particular product) you have a fresh approach and start noticing a lot of bugs. After a while and some experience with that ...
user avatar
18 votes
4 answers
2k views

Is refreshing multiple times a test case for web applications?

I have been faced with a QA test case where refreshing a page multiple times almost instantly (6x, 7x, 8x) creates a bug. This is totally random and every time the number of refreshes require to ...
user avatar
17 votes
8 answers
53k views

Difference between requirement, use case, and scenario?

I've taken a class on software testing best practices and test design. In the content, they have a model that basically shows that the three items are derived in this way: Requirements > Use case &...
user avatar
17 votes
7 answers
12k views

How can I structure Selenium tests in a way that minimizes the maintenance work?

What is the best way to build/structure the tests in a way that will minimize the work needed to keep the tests up to date as the product changes and improves?
user avatar
  • 171
15 votes
7 answers
2k views

The test team as an enemy of development? And how can this be avoided?

Details Forming a Scrum team should include all the skills necessary to develop a user story in order to deliver a potentially deliverable product increment with each sprint. In traditional ...
user avatar
  • 1,173
15 votes
4 answers
3k views

Test strategy to bring automated tests from UI to lower(API/Unit) layers?

Background/Problem Statement: Recently I came across a project team, which is having thousands of UI automated tests covering all kind of manual tests including positive, negative etc., and obviously ...
user avatar
15 votes
4 answers
306 views

What techniques are available to determine which browsers to test?

When websites being deployed to the internet are about to go live, the age old question always comes up. "Which browsers should I test?" I am not asking that question because it will soon become ...
user avatar
  • 9,670
15 votes
4 answers
790 views

Good strategy for automated regression where initial state can't be guaranteed or predicted

The situation I need to start building automated regression for a very large web application with thousands of users. It is impossible to start from a known state, or even to be sure that data ...
user avatar
  • 30.8k
14 votes
5 answers
4k views

What to do when a bug is fixed, unfixed and fixed

A blocker bug I raised last week had been fixed and ready for re-testing. This morning, I re-tested it, it passed and subsequently closed. In the afternoon, by accident, I noticed this feature was ...
user avatar
  • 1,515
14 votes
4 answers
2k views

How long should a test wait to assume that the result remains fixed

We had a defect where in a situation the state of the application changed into something undesired. It has been fixed, we added a test: Set up state Action that triggered the wrong state change Check ...
user avatar
14 votes
4 answers
756 views

How do you know if you are writing high quality tests?

Inspired by Exercises on software testing We can learn to write automated tests and we can learn to do tdd and bdd but what tools / resources / guides are available to tell us if the actual tests we ...
user avatar
13 votes
9 answers
4k views

How do quality assurance engineers test calculations?

How do quality assurance engineers test calculations done inside the code and calculations in stored procedures? Also if the calculation is complex what should be the approach to test something like ...
user avatar
  • 1,133
13 votes
5 answers
2k views

Test for unknown issues

We have been developing wifi/2g boards that communicate to a server and store data. This project has been running for several years already, and we are constantly updating both boards' firmware and ...
user avatar
  • 231
13 votes
5 answers
3k views

Are Mind Maps useful as a testing tool?

I don’t know much about Mind maps but I’ve seen expert testers (James Bach and Michael Bolton) use them to plan out their test strategy – usually in a demonstration. It seems like a plausible way to ...
user avatar
  • 3,725
13 votes
3 answers
6k views

Random test data: How to do it right?

Several months ago I started an experiment in test automation. Each test, before it starts, picks up random test data. For instance, when my test requires a user with administration privileges, I draw ...
user avatar
  • 10.4k
13 votes
1 answer
4k views

Is Selenium's LoadableComponent a good solution as test design pattern?

I need to refactor my Selenium tests, and come across their LoadableComponent wiki page. I began trying to use it, though I couldn't really understand what is it good for. I took a look at their ...
user avatar
  • 575
12 votes
6 answers
35k views

Test Suite vs. Test Plan

I am having a difficult time pinning down an official description of what a test suite is compared to a test plan. In Test Computer Software they don't even mention suites. (I assume because the book ...
user avatar
  • 175
12 votes
9 answers
4k views

Tools for Server application testing [closed]

I'm currently working (almost finishing) a Java project that consists on a server application and some client applications on the same network, that will communicate with the server only to request ...
12 votes
11 answers
2k views

What is Test Automation Framework?

I am a complete beginner in Test Automation and could not find a good answer on the web. Some automation engineers use an underlying automation framework as a jar dependency in their selenium web ...
user avatar
  • 129
12 votes
6 answers
4k views

Whose responsibility is it to design test cases?

The question: Is it the developer's responsibility to design test cases, or the QA tester's? Or is it someone else's? I am a software engineer at a small company in the United States. We have an ...
user avatar
  • 219
12 votes
4 answers
7k views

Is there a guide for systematic way of writing test scenarios?

Are there best practices in coming up with effective test scenarios? Are there documented test scenarios that can be reused across different systems?
user avatar
12 votes
5 answers
359 views

Heuristics for identifying "one-time" tests?

I understand the importance of having good "regression tests" to ensure fixed bugs don't get reintroduced. But I think it's possible to go too far and end up with a massive "regression test suite" for ...
user avatar
  • 2,666
12 votes
6 answers
480 views

What online games (ideally with a bunch of glitches) would make for a good software testing training exercise?

I'll be giving a 3-day course on software test design later this month with lots of hands-on exercises. I'd like to include one or more exercises that would involve testing an online game for ...
user avatar
  • 2,347
11 votes
6 answers
3k views

How many test steps are allowed in a single manual test case?

I have been writing Test Cases for half a year as of now. My question is: How many test steps are allowed only in one test case? For example more than 6 steps is not allowed or depends on ...
user avatar
  • 137
11 votes
7 answers
27k views

How to test a toaster? (interview question)

I want to know how we should approach for testing real world objects. The problem that I faced before is to get the list of use cases from the interviewer. Whenever I asked what the real world object ...
user avatar
11 votes
5 answers
773 views

Signal to noise ratio during early testing

The Context For the past several years our QA team has been very active in manual and automated testing of our products which led to a very small amount of real-user complaints about the functioning ...
user avatar
  • 11.4k
11 votes
4 answers
334 views

How do you design your test method/function

This question has been revolving on my mind from quite some time. Do you develop one method for one test scenario or club similar scenarios in one method and develop automated tests for it. Both ...
user avatar
  • 3,427
10 votes
10 answers
695 views

Should a test method test only one condition?

While writing automated test I come across situations when I include multiple verification points in one method but then I have also read/heard that one tests should make just one verification. Which ...
user avatar
  • 3,427
10 votes
10 answers
2k views

What type of tests you should do first?

I had this question in a job interview: Imagine that you have got a new application. You have never seen it before, it is absolutely new. What types of testing will you do first? As far as I ...
user avatar
  • 111
10 votes
5 answers
6k views

How common is testing based on log files analysis?

Here is a document that outlines challenges and solutions of testing of Mars Science Laboratory. I found the approach of log files analysis unexpected but quite beneficial on many levels. It seems to ...
user avatar
10 votes
1 answer
5k views

What goes in a Testing playbook?

The concept of the Testing Playbook has been recently added to the list of possible tools in your Context-Driven toolset. The Playbook has been described as being similar to the list of plays that a ...
user avatar
9 votes
5 answers
3k views

Are state transition diagrams a good software testing technique

I have been looking at different testing techniques and wondered if anyone had some "real world" examples of using state transition diagrams rather than the bog standard clock and light switch ones ...
user avatar
  • 113
9 votes
6 answers
958 views

How to prevent cheating by manual testers?

I'm the guy who is motivated to bring some quality assurance into our team. The problem is, that our developers very often hate testing and even if they have a test protocol, not all of them are ...
user avatar
  • 119
9 votes
5 answers
1k views

What proportion of different types of software testing does your team use?

Note: I've asked this over on SO and p.SE and it got closed. I'm not trying to game the system, just looking for a home :) I'm seeking a little "wisdom of crowds" estimation or a pointer to an ...
user avatar
  • 308
9 votes
7 answers
941 views

Testing without direction, how not to miss things? What about layout?

So this might sound like a "complain" post but it really isn't meant to be. But often-times I will be handed a project, lets say a Website. And be told to give it a "Full run through". So I take a ...
user avatar
  • 1,636
9 votes
3 answers
148 views

Designing an open source test framework without compromising security

Context The robot framework website mentions robotframework-autoitlibrary in its external library listing and it points to http://code.google.com/p/robotframework-autoitlibrary/.And I found a ...
user avatar
  • 10.8k
9 votes
4 answers
2k views

How to implement qa test process for large scale application?

There is a large scale application with thousands of functional points and which never had any QA team or person. So when a QA person or team starts working with this type of product what should be ...
user avatar
9 votes
4 answers
2k views

Test case design standards- friend or foe?

Currently we use an agile team based approach to testing, where each team is responsible for their specific area of a product. While we have found this to be effective and easily manageable, over ...
user avatar
9 votes
4 answers
1k views

How to make sure your test data is correct?

We are doing integration tests where we simulate manually data coming from external systems to our application. Our integration tests have become the bottleneck in our process, because preparing test ...
user avatar
  • 10.4k
9 votes
3 answers
383 views

What criteria do I apply to a highly complex development project to warrant getting "real world" data?

Both the product I used to work on and the one I'm working on now have a fairly large and complex matrix of possible configurations, work-flows, and data scenarios. The one product had, no joke, ...
user avatar
  • 2,589

1
2 3 4 5
11