Hot answers tagged

30

They cannot be 100% sure of a bug free system. They can increase their confidence that the system is bug-free by use of static analysis and testing. Some people advocate bebugging as a way of estimating how many bugs remain in a system. There are at least two forms of static analysis. If the system goes wrong then it goes wrong. There are assorted methods ...


24

NASA's Software Assurance Technology Center at Goddard once did a test to see how few defects they could get in some code for the shuttle. After a truly rigorous and vastly, vastly expensive process, with multiple levels of review, using very small functions to minimize the risk in each, they managed to get it down to 1 defect per 10,000 lines of code ...


22

A hard coded sleep statement is generally supposed to represent some sort of mocked delay in the application that doesn't exist during testing. The harmful aspect of this is that a hard coded value can't represent the complexity that actually exists in what its mocking. Take for example a network delay. Your production system usually takes 4 seconds to go ...


19

All non-trivial software has bugs. Risk is part of life. It is foolhardy to attempt to remove all possible risk, because in the process, you introduce the risk that you will accomplish nothing. Even if your algorithm and your implementation are perfect, your software does not live in a vaccuum: it depends on compilers, interpreters, libraries, operating ...


16

How I think about state models for testing. A state is the system's readiness to respond in a planned way to each of a set of defined events. You know that a system is in a new state (compared to a moment ago) if: The set of events to which it now responds differs from the set a moment ago. The system now responds to given events differently than it ...


15

Peter, Good question. And an extremely important one for software testers. I have studied the question of how can software testing inputs be combined most efficiently and effectively pretty steadily for the last five years. By coincidence, slightly over five years ago, I started with almost precisely the same question you're posing now. At that point, ...


13

Katrina, It is possible to include some important approaches and goals for software testing that tend to be beneficial in creating effective test case scenarios in a wide variety of testing situations. They include: Don't repeat yourself (e.g., keep scenarios "DRY"); don't repeat combinations of test inputs more than you need to because you would find ...


12

A requirement is typically a general statement, whereas a use case is typically a specific statement implied or derived from the requirement. A requirement may map to multiple use cases. A scenario might be a set of background assumptions that put a use case in context, or it might be grouping of use cases. Here is a contrived example. The requirement is ...


12

The traditional definitions would be something like this: A test suite is a collection of test cases related to the same test work. You might have a suite for regression, one for build verification tests, a suite that is specific for a component, and so on. A test plan is generally a document which describes testing approach and methodologies being used for ...


12

If I was being interviewed and was asked this question, first of all i would start off by trying to quench my curiosity. How much time do i have? What type of a toaster is it? How much power limit does it operate on as per the vendor? Does the vendor provide any user manual or claims documentation? How does it work? Is it timer based or manual toaster? ...


11

Combinatorial testing is an approach to testing an operation that has many inputs. For that website, I would start by identifying the operations. Sometimes there will be a specification for the system, but often there is not, and in any case the specification is unlikely to match the system in every detail. I recommend reading the specification and then ...


11

You could setup personas which are designed around real world of users. We have found this quite useful and it really helps to provide a fresh perspective e.g. Today I'm going to be Andy, the super user of the system. Andy is very sharp with numbers and is the user that is responsible for the administration of the system. He enjoys watching sports on ...


10

You should talk to your marketing department and understand why they have no specification for that particular area and explain to them why it is important for specification to exist and that there cannot be any short-cutting in this area. You may find out that they lacked resources or were unsure about this area. Regardless of the reason you can help them ...


10

The main harm occurs when the time value you use in your Sleep statement isn't appropriate for the current instance of the test. Your script might Sleep for 4 seconds, because that's how long it took for some important object to appear when you created your script. Then the script moves on with actions using that new object. But when you re-run the script, ...


10

How much testing is sufficient? Meaning, what's the minimum effort required to test reliability? When looking at the question of "how much testing?" you have to consider "how lucky do you feel?" You could do no testing at all if you feel really lucky, or if the consequences to being wrong are extremely low. You could test everything for a really long ...


10

Being QA Manager with about 3 years of experience, I just give my team mates testing tasks, which are NOT related to software, e.g.: Compose test cases for blender / vacuum cleaner / etc. - any kind of familiar device / equipment. This results in brain refreshing, and for the cost of 2-4 hours I get team "reloaded")) The same is applicable for testing such ...


10

As user246 says, tricks to force developers to test can always be gamed: you're much better off finding out why they don't like testing and what the actual problem is then building a culture of testing and quality from that. You're working in PHP - there are unit test frameworks available for PHP that your devs can use. If they have no idea how much trouble ...


10

This is - sadly - rather more common than anyone here would like. It's where I was when I started at my current position: two major applications, both stable, but the company has never had dedicated test specialists before. The first thing I did was make sure that everyone knew there weren't going to be any quick changes. No matter how skilled a person is, ...


9

The problem here isn't white-box testing. The problem is that there is no specification. This is where domain knowledge and product familiarity really matters. A tester who just tests that the code does what it was intended to do (from the dev's point of view) by reading the API could very well miss a variety of issues, such as a business requirement that ...


9

You're asking a unicorn question. Why? Because you've skipped over the point of all that work. It doesn't matter what proportion of different types of testing you do, it doesn't matter what your organisation is, and it doesn't matter what technologies you use, if you don't know what information your stakeholders want to discover from your testing. That's ...


9

Read MindMapping 101 from Darren McMillan - http://www.bettertesting.co.uk/content/?p=956 and this discussion ( prompted by myself ) on the STC - http://www.softwaretestingclub.com/forum/topics/im-the-map-im-the-map-im-the They are nothing new, use of them seems to come in waves - as your question demonstrates :)


9

Generating test data is a difficult problem because if you don't understand the symantics of the data then you are likely to generate test data that will throw false positives in your tests (test failure due to faulty data, not a bug in the product). The approach I have used with great success is parameterized test data generation from equivalent ...


9

Welcome to SQA, Ana. I see at least two questions: Does it make sense to distinguish between focused tests and workflow tests? Yes, it does. A focused test concentrates on a particular feature, including things like edge conditions. It attempts to answer the question, "Does this feature work in all circumstances?" A workflow test verifies that the ...


9

I've read that if a requirement can't be verified by black-box testing or inspection, then it's not a requirement, but a design specification. I don't think that's true. There are plenty of requirements which are difficult-to-impossible to test with black-box techniques; for example, a client-server application might have a requirement that all ...


8

Some ideas for the GPS part, based on my experience testing GPS's: Do field tests, and choose you locations wisely- from totally open skies to crowded tall buildings with limited to no GPS reception, from standing still to driving slow and fast, change heights during the tests (GPS is less accurate in reporting heights), choose different times of day, ...


8

It's not exactly clear what question you are asking, but let me take a stab. I would deal with it by creating a bug report. In it, I would mention what you are seeing in Commit A, and Commit B. I'd mention that the combination of the two pushes performance past the prescribed limits. From a QA point of view, it's not important at all in which Commit the ...


8

In my experience with documenting system tests, I've found a multi-layered approach works. I really like Microsoft Test Manager for this because of two things: the ability to define input parameters for manual tests and the concept of shared test steps which can be used by any test case. You don't mention if you're using a test case tool, word, excel or ...


8

It depends, and there are no industry standards. Seriously. Any metric can be gamed (and will be, if you use it for assessment). I'm not aware of any standard approaches, not least because the teams are - or should be - evaluating themselves regularly and looking for ways to improve their own processes (if they aren't then they're probably using ...


8

Things are expected to slip through if you are just given a once through correct? Im not talking about a lot of major stuff. But little things (especially if it's a large website). How do I not beat myself up over this? Yes. There are innumerable levels of quality and bugs will slip through at each of them. Consider these levels: Is it possible to ...


7

My preference is to have one test per element. This aids in diagnosability (test failure points directly to missing element) and maintainability (e.g. if elements are added or removed). I also like to use namespaces and classes to group the tests - e.g. namespace TestPrerequisites { public class GuiElements { public bool VerifyElementOne() ...



Only top voted, non community-wiki answers of a minimum length are eligible