Tag Info

Hot answers tagged

19

A hard coded sleep statement is generally supposed to represent some sort of mocked delay in the application that doesn't exist during testing. The harmful aspect of this is that a hard coded value can't represent the complexity that actually exists in what its mocking. Take for example a network delay. Your production system usually takes 4 seconds to go ...


15

How I think about state models for testing. A state is the system's readiness to respond in a planned way to each of a set of defined events. You know that a system is in a new state (compared to a moment ago) if: The set of events to which it now responds differs from the set a moment ago. The system now responds to given events differently than it ...


14

Peter, Good question. And an extremely important one for software testers. I have studied the question of how can software testing inputs be combined most efficiently and effectively pretty steadily for the last five years. By coincidence, slightly over five years ago, I started with almost precisely the same question you're posing now. At that point, ...


12

Katrina, It is possible to include some important approaches and goals for software testing that tend to be beneficial in creating effective test case scenarios in a wide variety of testing situations. They include: Don't repeat yourself (e.g., keep scenarios "DRY"); don't repeat combinations of test inputs more than you need to because you would find ...


10

You should talk to your marketing department and understand why they have no specification for that particular area and explain to them why it is important for specification to exist and that there cannot be any short-cutting in this area. You may find out that they lacked resources or were unsure about this area. Regardless of the reason you can help them ...


10

Combinatorial testing is an approach to testing an operation that has many inputs. For that website, I would start by identifying the operations. Sometimes there will be a specification for the system, but often there is not, and in any case the specification is unlikely to match the system in every detail. I recommend reading the specification and then ...


10

You could setup personas which are designed around real world of users. We have found this quite useful and it really helps to provide a fresh perspective e.g. Today I'm going to be Andy, the super user of the system. Andy is very sharp with numbers and is the user that is responsible for the administration of the system. He enjoys watching sports on ...


10

Being QA Manager with about 3 years of experience, I just give my team mates testing tasks, which are NOT related to software, e.g.: Compose test cases for blender / vacuum cleaner / etc. - any kind of familiar device / equipment. This results in brain refreshing, and for the cost of 2-4 hours I get team "reloaded")) The same is applicable for testing such ...


9

How much testing is sufficient? Meaning, what's the minimum effort required to test reliability? When looking at the question of "how much testing?" you have to consider "how lucky do you feel?" You could do no testing at all if you feel really lucky, or if the consequences to being wrong are extremely low. You could test everything for a really long ...


9

The problem here isn't white-box testing. The problem is that there is no specification. This is where domain knowledge and product familiarity really matters. A tester who just tests that the code does what it was intended to do (from the dev's point of view) by reading the API could very well miss a variety of issues, such as a business requirement that ...


9

You're asking a unicorn question. Why? Because you've skipped over the point of all that work. It doesn't matter what proportion of different types of testing you do, it doesn't matter what your organisation is, and it doesn't matter what technologies you use, if you don't know what information your stakeholders want to discover from your testing. That's ...


9

A requirement is typically a general statement, whereas a use case is typically a specific statement implied or derived from the requirement. A requirement may map to multiple use cases. A scenario might be a set of background assumptions that put a use case in context, or it might be grouping of use cases. Here is a contrived example. The requirement is ...


9

Welcome to SQA, Ana. I see at least two questions: Does it make sense to distinguish between focused tests and workflow tests? Yes, it does. A focused test concentrates on a particular feature, including things like edge conditions. It attempts to answer the question, "Does this feature work in all circumstances?" A workflow test verifies that the ...


9

I've read that if a requirement can't be verified by black-box testing or inspection, then it's not a requirement, but a design specification. I don't think that's true. There are plenty of requirements which are difficult-to-impossible to test with black-box techniques; for example, a client-server application might have a requirement that all ...


9

As user246 says, tricks to force developers to test can always be gamed: you're much better off finding out why they don't like testing and what the actual problem is then building a culture of testing and quality from that. You're working in PHP - there are unit test frameworks available for PHP that your devs can use. If they have no idea how much trouble ...


9

This is - sadly - rather more common than anyone here would like. It's where I was when I started at my current position: two major applications, both stable, but the company has never had dedicated test specialists before. The first thing I did was make sure that everyone knew there weren't going to be any quick changes. No matter how skilled a person is, ...


8

The main harm occurs when the time value you use in your Sleep statement isn't appropriate for the current instance of the test. Your script might Sleep for 4 seconds, because that's how long it took for some important object to appear when you created your script. Then the script moves on with actions using that new object. But when you re-run the script, ...


8

The traditional definitions would be something like this: A test suite is a collection of test cases related to the same test work. You might have a suite for regression, one for build verification tests, a suite that is specific for a component, and so on. A test plan is generally a document which describes testing approach and methodologies being used for ...


8

It's not exactly clear what question you are asking, but let me take a stab. I would deal with it by creating a bug report. In it, I would mention what you are seeing in Commit A, and Commit B. I'd mention that the combination of the two pushes performance past the prescribed limits. From a QA point of view, it's not important at all in which Commit the ...


7

Generating test data is a difficult problem because if you don't understand the symantics of the data then you are likely to generate test data that will throw false positives in your tests (test failure due to faulty data, not a bug in the product). The approach I have used with great success is parameterized test data generation from equivalent ...


7

In my experience with documenting system tests, I've found a multi-layered approach works. I really like Microsoft Test Manager for this because of two things: the ability to define input parameters for manual tests and the concept of shared test steps which can be used by any test case. You don't mention if you're using a test case tool, word, excel or ...


7

You're already doing it! The core principle here, which you're already following, is: Prepare some data Enter it into the input field Check whether it is displayed correctly in the output. To make your testing more sophisticated and thorough, consider the possible variations at each stage: 1. Data: Basic numbers and letters Basic punctuation ...


7

Actually, the underlying heuristic for combinatorial testing of multiple input parameters is that they are interdependent and the specific values assigned affect on a common output condition or state. Based on the concept of testing various combinations of input variables that affect a common output condition or state, combinatorial testing may not be ...


7

Some ideas for the GPS part, based on my experience testing GPS's: Do field tests, and choose you locations wisely- from totally open skies to crowded tall buildings with limited to no GPS reception, from standing still to driving slow and fast, change heights during the tests (GPS is less accurate in reporting heights), choose different times of day, ...


7

I think the simple answer is, do something else for a while. Our jobs require a lot of repetition, and we automatically develop habits in response to repetition. That behavior is a deeply ingrained survival technique; habits allow us to do things quickly without thinking them through. Sometimes those habits allow us to discover new things, but other times ...


7

This is a terminology issue: what Beatty is saying is that conventional testing methods are unable to detect those conditions. Essentially, they don't manifest in typical testing activities (and detection requires detailed analysis of the code base by someone with access to and knowledge of the code - which many testers lack). Certainly in my career I've ...


7

While the entire system isn't under your control (which is clearly the superior option), it might still be possible for some slice of the system to be under your control. You say it's a payroll system. Can you create a "Test Company" that everyone else would agree to leave alone? Can you create "Test Employees" that everyone else would agree to leave ...


7

It depends, and there are no industry standards. Seriously. Any metric can be gamed (and will be, if you use it for assessment). I'm not aware of any standard approaches, not least because the teams are - or should be - evaluating themselves regularly and looking for ways to improve their own processes (if they aren't then they're probably using ...


6

Read MindMapping 101 from Darren McMillan - http://www.bettertesting.co.uk/content/?p=956 and this discussion ( prompted by myself ) on the STC - http://www.softwaretestingclub.com/forum/topics/im-the-map-im-the-map-im-the They are nothing new, use of them seems to come in waves - as your question demonstrates :)


6

RN. You've got several options here. One is, as you mentioned, to reset the database before each run. That's probably a good idea anyway to ensure that you've got the same starting condition and the beginning of the run. For the individual tests within the run, you can operate a simple counter through the run and use that to generate your business key - ...



Only top voted, non community-wiki answers of a minimum length are eligible