Hot answers tagged test-design
16
A hard coded sleep statement is generally supposed to represent some sort of mocked delay in the application that doesn't exist during testing.
The harmful aspect of this is that a hard coded value can't represent the complexity that actually exists in what its mocking.
Take for example a network delay. Your production system usually takes 4 seconds to go ...
12
Katrina,
It is possible to include some important approaches and goals for software testing that tend to be beneficial in creating effective test case scenarios in a wide variety of testing situations. They include:
Don't repeat yourself (e.g., keep scenarios "DRY"); don't repeat combinations of test inputs more than you need to because you would find ...
10
Combinatorial testing is an approach to testing an operation that has many inputs. For that website, I would start by identifying the operations. Sometimes there will be a specification for the system, but often there is not, and in any case the specification is unlikely to match the system in every detail. I recommend reading the specification and then ...
9
You could setup personas which are designed around real world of users. We have found this quite useful and it really helps to provide a fresh perspective e.g.
Today I'm going to be Andy, the super user of the system. Andy is very sharp with numbers and is the user that is responsible for the administration of the system. He enjoys watching sports on ...
9
Being QA Manager with about 3 years of experience, I just give my team mates testing tasks, which are NOT related to software, e.g.:
Compose test cases for blender / vacuum cleaner / etc. - any kind of familiar device / equipment. This results in brain refreshing, and for the cost of 2-4 hours I get team "reloaded"))
The same is applicable for testing such ...
8
How much testing is sufficient? Meaning, what's the minimum effort required to test reliability?
When looking at the question of "how much testing?" you have to consider "how lucky do you feel?" You could do no testing at all if you feel really lucky, or if the consequences to being wrong are extremely low. You could test everything for a really long ...
7
I think the simple answer is, do something else for a while. Our jobs require a lot of repetition, and we automatically develop habits in response to repetition. That behavior is a deeply ingrained survival technique; habits allow us to do things quickly without thinking them through. Sometimes those habits allow us to discover new things, but other times ...
7
This is a terminology issue: what Beatty is saying is that conventional testing methods are unable to detect those conditions.
Essentially, they don't manifest in typical testing activities (and detection requires detailed analysis of the code base by someone with access to and knowledge of the code - which many testers lack). Certainly in my career I've ...
6
The main harm occurs when the time value you use in your Sleep statement isn't appropriate for the current instance of the test.
Your script might Sleep for 4 seconds, because that's how long it took for some important object to appear when you created your script. Then the script moves on with actions using that new object.
But when you re-run the script, ...
6
The name for this type of testing is combinatorial testing or sometimes referred to as pairwise testing. There is a very effective free tool from Microsoft that we also use for combinatorial testing called PICT.
More info is also available at http://www.pairwise.org and my blog post on combinatorial testing with negative values.
Essentially the input file ...
6
Generating test data is a difficult problem because if you don't understand the symantics of the data then you are likely to generate test data that will throw false positives in your tests (test failure due to faulty data, not a bug in the product).
The approach I have used with great success is parameterized test data generation from equivalent ...
6
Some ideas for the GPS part, based on my experience testing GPS's:
Do field tests, and choose you locations wisely- from totally open skies to crowded tall buildings with limited to no GPS reception, from standing still to driving slow and fast, change heights during the tests (GPS is less accurate in reporting heights), choose different times of day, ...
6
You may also want to check
Locale appropriateness of colors, icons, etc
Paper size issues
Sort order issues
Tax issues
Holiday issues
Time Zone issues
etc, etc.
These may help:
All Things Quality: Pseudo-Translation of Strings as an Aid to Internationalization Testing
All Things Quality: Internationalization and Localization
6
allquixotic,
How important is it that you get these string correct in all situations?
For example, if this is a life-critical application that depends intrinsically on the correctness of your strings (the difference between "Send police to the Town Hall" and "Send fire equipment to the Town Hall" for example), then you must do a deeper dive. Consider ...
6
You could try the 'tours' concept and try out different tours of the software.
I'd also disagree somewhat with your premise - the more you use a program the more you notice any slight changes. You also understand more how all the parts interact which in turn gives more ideas. I think at the start you may notice more but they are shallower than ones you find ...
5
Yes. I call them Contract Tests. One easy way to understand them:
Start with tests for a specific implementation of the interface. For example, consider ArrayList implements List. You write tests for ArrayList. One test could be this:
testEmptyIffZeroSize:
list = new ArrayList()
assert list.isEmpty()
assert_equals 0, list.size()
Notice that ...
5
David,
I've thought about this exact question quite a lot and I'm optimistic that I'd be able to help share some tips that you'd find useful. Unfortunately, explanations in a few paragraphs may not be sufficient to answer your question in detail. I'd be happy to talk by Skype today if you want to talk; I'll send you my Skype details via LinkedIn.
What ...
4
Read MindMapping 101 from Darren McMillan - http://www.bettertesting.co.uk/content/?p=956
and this discussion ( prompted by myself ) on the STC - http://www.softwaretestingclub.com/forum/topics/im-the-map-im-the-map-im-the
They are nothing new, use of them seems to come in waves - as your question demonstrates :)
3
Yes, we use log monitoring as part of our testing process. Each manual tester is asked to monitor the logs during test (tail -f), to look for hidden errors/exceptions during testing.
Also, our UI will pop-up a dialog if a client side exception is thrown. These usually cause automated tests to fail, and starts the investigation.
Finally, we have a script ...
3
In software, there are a couple of ways to wait for an event to occur. One way is to cause the event to notify you. Sometimes that is possible, sometimes not. Another way is to poll: you wait a while, check whether the event has occurred, and if it hasn't, you do it again.
A variation of polling is to wait a while and then assume that the event has ...
3
First of all, it is understandable that a former developer might approach testing in terms of individually testable layers or sub-components. However, as a tester, if you do nothing else, you must verify that the finished product behaves correctly when exercised using whatever interfaces the end-user will use. Everything else is secondary to that -- ...
3
Welcome to SQA, David. I suspect many of us have been in the same position -- I certainly have.
You did not ask for advice on what to automate, so I assume you are mostly interested in how to document and execute a test of so many options. One way to reduce the effort is to investigate how the 46 options are implemented. The more code they share, the ...
3
One approach is to actively recruit team members and other stakeholders for testing of the product. I have found that it is useful to try different approaches and use whatever works best with each person, which can include:
Paired testing with a code developer (emphasize the thrill of having a function work on the first build :) )
Asking a content ...
3
Use a variable attenuator between your device and AP, there are manual and controllable models and it's relatively cheap.
You'll have to connect it to the Wi Fi antennas using an RF cable , or put your device into an RF insulated box, which in turn is connected to the attenuator.
Your application probably doesn't see the signal strength reports directly, ...
2
I wrote this recently - http://rosiesherry.com/2012/05/mindmaps-testing-pinterest/
My theory is that we can learn from other people's mindmaps and it would be useful to have a bunch in one place - I've used Pinterest for this because it is easy and visual, am sure there are other ways.
2
Yes, my team has been using Mind Maps to write and run our test cases for over a year now with much success. My team like to use them as you can visually see what has been tested, and not been tested. I've blogged about it some as well. http://testerstrek.blogspot.ca/
2
It depends on the goal of your testing, the system and how complex it is, how easy it is to monitor, how much information is logged and I'm sure some other factors that I am missing.
I can give you specific examples of where I have used MFAT in "performance" testing and it made a lot of sense. I put performance in quotes because it is generally used as a ...
2
The question is what your test objectives are: What exactly do you want to test?
I see two options here. Let's take the monitoring of free disk space as an example. By your tests, do you want to ensure that you can actually correctly measure the free disk space, or do you want to ensure that the graphs and statistics based on the free disk space data are ...
2
I think unit tests are inevitable. And unless your application is a standalone and no other applications are using it, integration test is also inevitable. The question you should really ask is how deep? Do you think it is necessary to provide 90+% unit test coverage? Do you need to provide close to complete coverage for endpoints? That will always be a ...
2
There are different approaches possible:
if you have system requirements document, you may write one or more test cases to each requirement
another approach is based on system / user manual, you read this document and then write test scenarios trying to cover described functionalities
in Agile / Scrum teams you may use acceptance criteria definition for ...
Only top voted, non community-wiki answers of a minimum length are eligible
