Hot answers tagged test-design
16
A hard coded sleep statement is generally supposed to represent some sort of mocked delay in the application that doesn't exist during testing.
The harmful aspect of this is that a hard coded value can't represent the complexity that actually exists in what its mocking.
Take for example a network delay. Your production system usually takes 4 seconds to go ...
14
Peter,
Good question. And an extremely important one for software testers.
I have studied the question of how can software testing inputs be combined most efficiently and effectively pretty steadily for the last five years.
By coincidence, slightly over five years ago, I started with almost precisely the same question you're posing now. At that point, ...
13
How I think about state models for testing. A state is the system's readiness to respond in a planned way to each of a set of defined events. You know that a system is in a new state (compared to a moment ago) if:
The set of events to which it now responds differs from the set a moment ago.
The system now responds to given events differently than it ...
12
Katrina,
It is possible to include some important approaches and goals for software testing that tend to be beneficial in creating effective test case scenarios in a wide variety of testing situations. They include:
Don't repeat yourself (e.g., keep scenarios "DRY"); don't repeat combinations of test inputs more than you need to because you would find ...
10
You should talk to your marketing department and understand why they have no specification for that particular area and explain to them why it is important for specification to exist and that there cannot be any short-cutting in this area. You may find out that they lacked resources or were unsure about this area. Regardless of the reason you can help them ...
10
Combinatorial testing is an approach to testing an operation that has many inputs. For that website, I would start by identifying the operations. Sometimes there will be a specification for the system, but often there is not, and in any case the specification is unlikely to match the system in every detail. I recommend reading the specification and then ...
9
You could setup personas which are designed around real world of users. We have found this quite useful and it really helps to provide a fresh perspective e.g.
Today I'm going to be Andy, the super user of the system. Andy is very sharp with numbers and is the user that is responsible for the administration of the system. He enjoys watching sports on ...
9
Being QA Manager with about 3 years of experience, I just give my team mates testing tasks, which are NOT related to software, e.g.:
Compose test cases for blender / vacuum cleaner / etc. - any kind of familiar device / equipment. This results in brain refreshing, and for the cost of 2-4 hours I get team "reloaded"))
The same is applicable for testing such ...
8
The problem here isn't white-box testing. The problem is that there is no specification. This is where domain knowledge and product familiarity really matters. A tester who just tests that the code does what it was intended to do (from the dev's point of view) by reading the API could very well miss a variety of issues, such as a business requirement that ...
8
How much testing is sufficient? Meaning, what's the minimum effort required to test reliability?
When looking at the question of "how much testing?" you have to consider "how lucky do you feel?" You could do no testing at all if you feel really lucky, or if the consequences to being wrong are extremely low. You could test everything for a really long ...
7
Actually, the underlying heuristic for combinatorial testing of multiple input parameters is that they are interdependent and the specific values assigned affect on a common output condition or state.
Based on the concept of testing various combinations of input variables that affect a common output condition or state, combinatorial testing may not be ...
7
You're asking a unicorn question.
Why?
Because you've skipped over the point of all that work. It doesn't matter what proportion of different types of testing you do, it doesn't matter what your organisation is, and it doesn't matter what technologies you use, if you don't know what information your stakeholders want to discover from your testing.
That's ...
7
I think the simple answer is, do something else for a while. Our jobs require a lot of repetition, and we automatically develop habits in response to repetition. That behavior is a deeply ingrained survival technique; habits allow us to do things quickly without thinking them through. Sometimes those habits allow us to discover new things, but other times ...
7
This is a terminology issue: what Beatty is saying is that conventional testing methods are unable to detect those conditions.
Essentially, they don't manifest in typical testing activities (and detection requires detailed analysis of the code base by someone with access to and knowledge of the code - which many testers lack). Certainly in my career I've ...
6
RN.
You've got several options here. One is, as you mentioned, to reset the database before each run. That's probably a good idea anyway to ensure that you've got the same starting condition and the beginning of the run.
For the individual tests within the run, you can operate a simple counter through the run and use that to generate your business key - ...
6
The name for this type of testing is combinatorial testing or sometimes referred to as pairwise testing. There is a very effective free tool from Microsoft that we also use for combinatorial testing called PICT.
More info is also available at http://www.pairwise.org and my blog post on combinatorial testing with negative values.
Essentially the input file ...
6
The main harm occurs when the time value you use in your Sleep statement isn't appropriate for the current instance of the test.
Your script might Sleep for 4 seconds, because that's how long it took for some important object to appear when you created your script. Then the script moves on with actions using that new object.
But when you re-run the script, ...
6
Generating test data is a difficult problem because if you don't understand the symantics of the data then you are likely to generate test data that will throw false positives in your tests (test failure due to faulty data, not a bug in the product).
The approach I have used with great success is parameterized test data generation from equivalent ...
6
Some ideas for the GPS part, based on my experience testing GPS's:
Do field tests, and choose you locations wisely- from totally open skies to crowded tall buildings with limited to no GPS reception, from standing still to driving slow and fast, change heights during the tests (GPS is less accurate in reporting heights), choose different times of day, ...
6
You may also want to check
Locale appropriateness of colors, icons, etc
Paper size issues
Sort order issues
Tax issues
Holiday issues
Time Zone issues
etc, etc.
These may help:
All Things Quality: Pseudo-Translation of Strings as an Aid to Internationalization Testing
All Things Quality: Internationalization and Localization
6
allquixotic,
How important is it that you get these string correct in all situations?
For example, if this is a life-critical application that depends intrinsically on the correctness of your strings (the difference between "Send police to the Town Hall" and "Send fire equipment to the Town Hall" for example), then you must do a deeper dive. Consider ...
6
You could try the 'tours' concept and try out different tours of the software.
I'd also disagree somewhat with your premise - the more you use a program the more you notice any slight changes. You also understand more how all the parts interact which in turn gives more ideas. I think at the start you may notice more but they are shallower than ones you find ...
5
Ahmet,
I agree your thought of a wiki for testing techniques could be a valuable idea for software testers.
The best testing technique repositories that I am aware of are of the "dead tree" variety.
Lee Copeland's "A Practitioner's Guide to Software Test Design" is the best book on the topic in my view. Available only in book format.
Also well worth ...
5
My preference is to have one test per element. This aids in diagnosability (test failure points directly to missing element) and maintainability (e.g. if elements are added or removed). I also like to use namespaces and classes to group the tests - e.g.
namespace TestPrerequisites
{
public class GuiElements
{
public bool VerifyElementOne() ...
5
Short answer: - I use them to evaluate my understanding of a product, and to communicate my understanding of the product. Secondary (test) use is to walk through the CLD and ask typical tester "what if?" questions.
Longer answer: You can use CLDs pretty much as designed - as a method for understanding the system you are testing, including assumptions as ...
5
Answering your second question first: yes, you absolutely need test case design standards. They don't have to be monsters. As with all things agile, they need to be enough.
For your first question there's no "right" answer, but I can give a few guidelines. Your standards should be based around a kind of test case triage: first priority is being able to set ...
5
Yes. I call them Contract Tests. One easy way to understand them:
Start with tests for a specific implementation of the interface. For example, consider ArrayList implements List. You write tests for ArrayList. One test could be this:
testEmptyIffZeroSize:
list = new ArrayList()
assert list.isEmpty()
assert_equals 0, list.size()
Notice that ...
5
David,
I've thought about this exact question quite a lot and I'm optimistic that I'd be able to help share some tips that you'd find useful. Unfortunately, explanations in a few paragraphs may not be sufficient to answer your question in detail. I'd be happy to talk by Skype today if you want to talk; I'll send you my Skype details via LinkedIn.
What ...
4
We never start with clearly written, complete, unambiguous, mutually-consistent specifications, and sometimes we start with nothing but an untested product. You can't test without some kind of model of what the product should be. You should ask yourself how important it is for your model to match the model in the heads of the rest of the company, and your ...
4
Excellent question. I think this comes under the heading of the "Application is its own specification". In the absence of requirements or specs, the best way of knowing what an application is supposed to do is to start using the application. If you actually have access to the code of the application, that extends this idea where the code itself becomes ...
Only top voted, non community-wiki answers of a minimum length are eligible
