Tag Info

Hot answers tagged

18

A hard coded sleep statement is generally supposed to represent some sort of mocked delay in the application that doesn't exist during testing. The harmful aspect of this is that a hard coded value can't represent the complexity that actually exists in what its mocking. Take for example a network delay. Your production system usually takes 4 seconds to go ...


15

How I think about state models for testing. A state is the system's readiness to respond in a planned way to each of a set of defined events. You know that a system is in a new state (compared to a moment ago) if: The set of events to which it now responds differs from the set a moment ago. The system now responds to given events differently than it ...


14

Peter, Good question. And an extremely important one for software testers. I have studied the question of how can software testing inputs be combined most efficiently and effectively pretty steadily for the last five years. By coincidence, slightly over five years ago, I started with almost precisely the same question you're posing now. At that point, ...


12

Katrina, It is possible to include some important approaches and goals for software testing that tend to be beneficial in creating effective test case scenarios in a wide variety of testing situations. They include: Don't repeat yourself (e.g., keep scenarios "DRY"); don't repeat combinations of test inputs more than you need to because you would find ...


10

You should talk to your marketing department and understand why they have no specification for that particular area and explain to them why it is important for specification to exist and that there cannot be any short-cutting in this area. You may find out that they lacked resources or were unsure about this area. Regardless of the reason you can help them ...


10

Combinatorial testing is an approach to testing an operation that has many inputs. For that website, I would start by identifying the operations. Sometimes there will be a specification for the system, but often there is not, and in any case the specification is unlikely to match the system in every detail. I recommend reading the specification and then ...


10

Being QA Manager with about 3 years of experience, I just give my team mates testing tasks, which are NOT related to software, e.g.: Compose test cases for blender / vacuum cleaner / etc. - any kind of familiar device / equipment. This results in brain refreshing, and for the cost of 2-4 hours I get team "reloaded")) The same is applicable for testing such ...


9

The problem here isn't white-box testing. The problem is that there is no specification. This is where domain knowledge and product familiarity really matters. A tester who just tests that the code does what it was intended to do (from the dev's point of view) by reading the API could very well miss a variety of issues, such as a business requirement that ...


9

How much testing is sufficient? Meaning, what's the minimum effort required to test reliability? When looking at the question of "how much testing?" you have to consider "how lucky do you feel?" You could do no testing at all if you feel really lucky, or if the consequences to being wrong are extremely low. You could test everything for a really long ...


9

You could setup personas which are designed around real world of users. We have found this quite useful and it really helps to provide a fresh perspective e.g. Today I'm going to be Andy, the super user of the system. Andy is very sharp with numbers and is the user that is responsible for the administration of the system. He enjoys watching sports on ...


9

I've read that if a requirement can't be verified by black-box testing or inspection, then it's not a requirement, but a design specification. I don't think that's true. There are plenty of requirements which are difficult-to-impossible to test with black-box techniques; for example, a client-server application might have a requirement that all ...


9

As user246 says, tricks to force developers to test can always be gamed: you're much better off finding out why they don't like testing and what the actual problem is then building a culture of testing and quality from that. You're working in PHP - there are unit test frameworks available for PHP that your devs can use. If they have no idea how much trouble ...


8

A requirement is typically a general statement, whereas a use case is typically a specific statement implied or derived from the requirement. A requirement may map to multiple use cases. A scenario might be a set of background assumptions that put a use case in context, or it might be grouping of use cases. Here is a contrived example. The requirement is ...


8

Welcome to SQA, Ana. I see at least two questions: Does it make sense to distinguish between focused tests and workflow tests? Yes, it does. A focused test concentrates on a particular feature, including things like edge conditions. It attempts to answer the question, "Does this feature work in all circumstances?" A workflow test verifies that the ...


8

It's not exactly clear what question you are asking, but let me take a stab. I would deal with it by creating a bug report. In it, I would mention what you are seeing in Commit A, and Commit B. I'd mention that the combination of the two pushes performance past the prescribed limits. From a QA point of view, it's not important at all in which Commit the ...


7

Actually, the underlying heuristic for combinatorial testing of multiple input parameters is that they are interdependent and the specific values assigned affect on a common output condition or state. Based on the concept of testing various combinations of input variables that affect a common output condition or state, combinatorial testing may not be ...


7

You're asking a unicorn question. Why? Because you've skipped over the point of all that work. It doesn't matter what proportion of different types of testing you do, it doesn't matter what your organisation is, and it doesn't matter what technologies you use, if you don't know what information your stakeholders want to discover from your testing. That's ...


7

The main harm occurs when the time value you use in your Sleep statement isn't appropriate for the current instance of the test. Your script might Sleep for 4 seconds, because that's how long it took for some important object to appear when you created your script. Then the script moves on with actions using that new object. But when you re-run the script, ...


7

Generating test data is a difficult problem because if you don't understand the symantics of the data then you are likely to generate test data that will throw false positives in your tests (test failure due to faulty data, not a bug in the product). The approach I have used with great success is parameterized test data generation from equivalent ...


7

Some ideas for the GPS part, based on my experience testing GPS's: Do field tests, and choose you locations wisely- from totally open skies to crowded tall buildings with limited to no GPS reception, from standing still to driving slow and fast, change heights during the tests (GPS is less accurate in reporting heights), choose different times of day, ...


7

I think the simple answer is, do something else for a while. Our jobs require a lot of repetition, and we automatically develop habits in response to repetition. That behavior is a deeply ingrained survival technique; habits allow us to do things quickly without thinking them through. Sometimes those habits allow us to discover new things, but other times ...


7

This is a terminology issue: what Beatty is saying is that conventional testing methods are unable to detect those conditions. Essentially, they don't manifest in typical testing activities (and detection requires detailed analysis of the code base by someone with access to and knowledge of the code - which many testers lack). Certainly in my career I've ...


6

RN. You've got several options here. One is, as you mentioned, to reset the database before each run. That's probably a good idea anyway to ensure that you've got the same starting condition and the beginning of the run. For the individual tests within the run, you can operate a simple counter through the run and use that to generate your business key - ...


6

Read MindMapping 101 from Darren McMillan - http://www.bettertesting.co.uk/content/?p=956 and this discussion ( prompted by myself ) on the STC - http://www.softwaretestingclub.com/forum/topics/im-the-map-im-the-map-im-the They are nothing new, use of them seems to come in waves - as your question demonstrates :)


6

The name for this type of testing is combinatorial testing or sometimes referred to as pairwise testing. There is a very effective free tool from Microsoft that we also use for combinatorial testing called PICT. More info is also available at http://www.pairwise.org and my blog post on combinatorial testing with negative values. Essentially the input file ...


6

You may also want to check Locale appropriateness of colors, icons, etc Paper size issues Sort order issues Tax issues Holiday issues Time Zone issues etc, etc. These may help: All Things Quality: Pseudo-Translation of Strings as an Aid to Internationalization Testing All Things Quality: Internationalization and Localization


6

allquixotic, How important is it that you get these string correct in all situations? For example, if this is a life-critical application that depends intrinsically on the correctness of your strings (the difference between "Send police to the Town Hall" and "Send fire equipment to the Town Hall" for example), then you must do a deeper dive. Consider ...


6

You could try the 'tours' concept and try out different tours of the software. I'd also disagree somewhat with your premise - the more you use a program the more you notice any slight changes. You also understand more how all the parts interact which in turn gives more ideas. I think at the start you may notice more but they are shallower than ones you find ...


6

At some point you have to trust the third-party software you work with. That's not to say the software won't have bugs; of course it will. But you can never test everything. For example, I suspect you do not test the operating system, compilers, text editors, router firmware, printer drivers, and web browsers that you depend upon. (At a previous job, our ...


6

While the entire system isn't under your control (which is clearly the superior option), it might still be possible for some slice of the system to be under your control. You say it's a payroll system. Can you create a "Test Company" that everyone else would agree to leave alone? Can you create "Test Employees" that everyone else would agree to leave ...



Only top voted, non community-wiki answers of a minimum length are eligible