Test Design is the selection of test aims, methodologies and implementation methods to satisfy the goals of a testing scenario.
14
votes
7answers
346 views
How to obtain “fresh mind” results?
I noticed that when you start on a project with a beginner mind (for that particular product) you have a fresh approach and start noticing a lot of bugs.
After a while and some experience with that ...
14
votes
2answers
392 views
Systematic approaches to selection of test data
I've used Phadke's design of experiments approach to select parameter values for testing (software) systems.
In a nutshell, the approach uses minimum, typical and maximum values of test (method) ...
10
votes
6answers
298 views
What online games (ideally with a bunch of glitches) would make for a good software testing training exercise?
I'll be giving a 3-day course on software test design later this month with lots of hands-on exercises. I'd like to include one or more exercises that would involve testing an online game for ...
8
votes
1answer
651 views
What goes in a Testing playbook?
The concept of the Testing Playbook has been recently added to the list of possible tools in your Context-Driven toolset. The Playbook has been described as being similar to the list of plays that a ...
7
votes
5answers
260 views
What Are Some Examples of Why Explicit Sleep Statements Are Bad
I've read various places that explicit sleep statements are a bad practice in automated tests, even if I've used them occasionally. They can be annoying (creating unnecessary waiting in tests) but I ...
7
votes
5answers
216 views
Workflow shortcut to meet the release deadline
When you design a test scenario or even when you run your tests and you stumble to lack of specifications from your marketing Dept. should you bypass them and take the initiative of making your own ...
6
votes
5answers
2k views
Are state transition diagrams a good software testing technique
I have been looking at different testing techniques and wondered if anyone had some "real world" examples of using state transition diagrams rather than the bog standard clock and light switch ones ...
6
votes
4answers
227 views
How do you design your test method/function
This question has been revolving on my mind from quite some time.
Do you develop one method for one test scenario or club similar scenarios in one method and develop automated tests for it. Both ...
6
votes
4answers
346 views
Testing techniques repository?
I was wondering if there are any repositories or wikis of formal testing techniques - test design ideas, like pairwise or boundary. I couldn't find one so far.
Thanks!
Ahmet
6
votes
4answers
454 views
Testing/Verifying Google Analytic code
I need help figuring out a test methods for Google Analytic. Our client pushes out a lot of marketing campaigns and they love using GA to track what is working, what is not working and where to focus ...
6
votes
1answer
1k views
Is Selenium's LoadableComponent a good solution as test design pattern?
I need to refactor my Selenium tests, and come across their LoadableComponent wiki page. I began trying to use it, though I couldn't really understand what is it good for. I took a look at their ...
5
votes
5answers
628 views
Is there a guide or systematic way of writing test scenarios?
Are there best practices in coming up with effective test scenarios? Are there documented test scenarios that can be reused across different systems?
5
votes
3answers
135 views
Internationalization testing
By internationalization testing I intend testing application with one or more additional languages and one or more additional regional settings (locale).
We expect a number of risks with ...
5
votes
3answers
330 views
Are Mind Maps useful as a testing tool?
I don’t know much about Mind maps but I’ve seen expert testers (James Bach and Michael Bolton) use them to plan out their test strategy – usually in a demonstration. It seems like a plausible way to ...
5
votes
4answers
408 views
Test case design standards- friend or foe?
Currently we use an agile team based approach to testing, where each team is responsible for their specific area of a product. While we have found this to be effective and easily manageable, over ...
5
votes
3answers
150 views
What criteria do I apply to a highly complex development project to warrant getting “real world” data?
Both the product I used to work on and the one I'm working on now have a fairly large and complex matrix of possible configurations, work-flows, and data scenarios. The one product had, no joke, ...
5
votes
4answers
185 views
How common is testing based on log files analysis?
Here is a document that outlines challenges and solutions of testing of Mars Science Laboratory:
http://compass.informatik.rwth-aachen.de/ws-slides/havelund.pdf
I found the approach of log files ...
5
votes
1answer
206 views
How can I use causal loop diagrams to help me test?
Causal loop diagrams seem like a really powerful tool for modelling a system - but I have no experience with applying them to my testing.
Do you use causal loop diagrams? How do you use them, and ...
5
votes
1answer
107 views
Where to start with introducing a testing framework
This I hope won't come across too broad of a question.
If you've been tasked with introducing testing process' for a web application developed in an 'agile' environment, released in 2 weekly cycles. ...
5
votes
1answer
218 views
What are the tiers of testing that should be done on a large scale distributed system?
Let's say you have a large distributed system in the cloud that's got dozens of developers working on it and actively adding code to. Your job is to make sure all of the quality gates are used and the ...
5
votes
1answer
4k views
How to use Selenium's PageFactory annotations with dynamic loaded elements?
I'm rewriting some Selenium tests according to the page object design pattern taking advantage of Seleniums' annotations. My problem is that I have some html select elements whose option elements are ...
5
votes
1answer
1k views
How does the DataSource attribute on a test run the test multiple times?
This is going to be a bit long/wordy/verbose, but I've been looking at it for a while and I think that I'm bad at explaining things because people seem to get confused about what I'm asking.
You know ...
4
votes
3answers
266 views
What types of testing are LEAST suited to using pairwise testing (and similar test design approaches)?
For those of you who have used pairwise test design methods (or similar combinatorial software testing methods) in an attempt to zero in on a manageable number of unusually powerful tests from among a ...
4
votes
4answers
228 views
Where to focus testing? On integration level or unit level?
Consider a modern web application that serves JSON through a REST API to a rich Javascript-based application with one or more single-page apps within it. Our own application is written using Angular ...
4
votes
5answers
299 views
What proportion of different types of software testing does your team use?
Note: I've asked this over on SO and p.SE and it got closed. I'm not trying to game the system, just looking for a home :)
I'm seeking a little "wisdom of crowds" estimation or a pointer to an ...
4
votes
2answers
375 views
When is it inappropriate to use white box testing?
Sometimes, tasked with testing a piece of software for which there are no requirements, a tester will resort to white box testing. Are there times when that is inappropriate?
Here is an example. At ...
4
votes
2answers
219 views
Test isolation and automation
We are using Selenium webservices for our automated functional testing .
I have a test which tests creation of some objects. If I run the same test again, I need to make sure that each test has a ...
4
votes
2answers
134 views
Should A Formal Test Plan For Dynamic String Testing Include All Combinations?
Situation:
Large enterprise web application
Formal test plan in HP Application Lifecycle Management (née Quality Center)
Many dynamic text strings in the web application with several variables
An ...
4
votes
2answers
176 views
How to test a location based application
Our application is location based. We have tested its main functionality, (using mock locations) and now it is time to test how it reacts on real cases using GPS.
So, what are the main limitations ...
4
votes
2answers
94 views
Is there a name for this type of vector operation?
I often find myself wanting to generate a lot of negative test cases by mutating a known good set of inputs.
Let's say I have a function that will accept (a, b, c) as an expected valid input for a ...
4
votes
2answers
208 views
How to make sure your test data are correct?
We are doing integration tests where we simulate manually data coming from external systems to our application. Integration tests task has become bottleneck in our team, because preparing test data is ...
4
votes
2answers
156 views
Turning software testing into an internal service
I've been tasked with changing the focus of my team from a "can do everything" team into one that is more service orientated, i.e. we can provide the framework for your testing rather than being there ...
4
votes
1answer
75 views
In performance testing, when have MFAT approaches been successfully applied (in addition to an OFAT approach)?
Background:
MFAT = "Multiple Factors at a Time"
OFAT = "One Factor at a Time"
In the field of Design of Experiments, practitioners in a large variety of fields (including software testing) have ...
3
votes
7answers
118 views
Is it necessary to test third party tools?
I have a question about best practices for writing test cases. At my job, I am using a closed source, third party tool that parses files into a database. Since this is a third party tool, is it ...
3
votes
1answer
186 views
What are the first steps a tester should perform when using combinational testing?
Hypothetically, lets assume that I am testing the "Search and compare flights" functionality of this website*, and I want to apply a combinational testing technique.
Where should a tester start this ...
3
votes
3answers
171 views
Why testing, by definition, cannot find deadlocks and stack overflows?
Sean M. Beatty says in "Where testing fails" article that "deadlocks, stack overflows, race conditions and timing problems cannot be detected by testing (whether it is code inspection, whide-box ...
3
votes
1answer
99 views
Are there common techniques for testing the conformance of an implementation to a general contract?
Say you have defined some abstract interface and you specify a general contract for that interface to which all implementations must adhere. Are there common techniques that can facilitate testing the ...
3
votes
2answers
162 views
How to test the graphical contents of a bitmap or a PDF file?
The module I'm testing creates some graphical reports which include things like scatter plots and multipage line graphs. The bitmap will never be exactly the same between test runs, since some ...
3
votes
1answer
82 views
Multiple Assembly QA Systems?
I'm having difficulty deciding whether or not using multiple assemblies for a QA system is the right approach to take. Here are some important details:
The underlying system is a single assembly ...
3
votes
1answer
106 views
xUnit tests patterns for testing threaded applications
Are there either any xUnit (or otherwise) design patterns or anti-patterns for testing threaded code?
3
votes
2answers
219 views
XMLs and JMS queue trigger to database - what to test?
I'm testing a system that involves XMLs from a JMS queue getting inserted into a database using a JMS trigger (webmethods).
Firstly, I turn off the JMS trigger to let messages accumulate on the ...
3
votes
1answer
392 views
PageObjects: Aggregation/Composition vs Inheritance for nested components
I need to refactor my Selenium tests, and was wondering what the best design solution would be for a portal with a hierarchical page organization. We are using tiles, and we have defined a baselayout ...
2
votes
2answers
115 views
IPv4/IPv6 dependency testing
Research from the Software Improvement Group has shown that 1 in every 12 applications behaves incorrectly when confronted with a dual-stack IPv4+IPv6 environment. How does Quality Assurance usually ...
2
votes
1answer
48 views
What particular testing requirements do data driven applications have?
Different sorts of software need different approaches to testing. An interactive GUI-driven application will be tested in a very different way from a piece of middleware, for example.
Specifically ...
2
votes
3answers
112 views
Repeat/common Test steps in multiple test cases - to do or not to?
I am trying to seek some input on what might be an efficient best practice. I have been told that end-to-end test cases might have certain steps that are being repeated on other test cases. Let's take ...
2
votes
2answers
410 views
Web application GUI automation best practices
Our team develops automation solutions for various web based products.
I'd like to compile a list of patterns and best practices from the GUI web automation space that have been used and proven as ...
2
votes
2answers
277 views
What new things need to be covered in Embedded Systems testing compared to normal manual testing?
Can you provide me with some common checklist for Embedded Systems testing please? What are the new things that are covered in this?
And can you also advise me some interesting references for reading ...
2
votes
3answers
134 views
Do I Need To Create Mock Classes For Every Class?
Let's say I have a system composed of three classes: A,B,C that share certain dependencies.
To test A independently, I would need to create mock objects for B and C
To test B independently, I would ...
2
votes
2answers
92 views
Any good test plan/cycles or experiences without using crowdsourcing?
I'm trying to rethink our test cycle that we use at my company and am wondering if anyone could give any leads, examples, experiences, etc. to how you test without the use of crowdsource data. The ...
2
votes
1answer
114 views
Encouraging project teams to define Edge cases
Can you please give me concrete examples of Edge cases preferably for Financial systems (General Ledger) and STP Settlements to help us kick start the right kind of thinking /approach by both the ...
