Tag Info

Hot answers tagged

9

This is - sadly - rather more common than anyone here would like. It's where I was when I started at my current position: two major applications, both stable, but the company has never had dedicated test specialists before. The first thing I did was make sure that everyone knew there weren't going to be any quick changes. No matter how skilled a person is, ...


8

If I was being interviewed and was asked this question, first of all i would start off by trying to quench my curiosity. How much time do i have? What type of a toaster is it? How much power limit does it operate on as per the vendor? Does the vendor provide any user manual or claims documentation? How does it work? Is it timer based or manual toaster? ...


8

The traditional definitions would be something like this: A test suite is a collection of test cases related to the same test work. You might have a suite for regression, one for build verification tests, a suite that is specific for a component, and so on. A test plan is generally a document which describes testing approach and methodologies being used for ...


8

It depends, and there are no industry standards. Seriously. Any metric can be gamed (and will be, if you use it for assessment). I'm not aware of any standard approaches, not least because the teams are - or should be - evaluating themselves regularly and looking for ways to improve their own processes (if they aren't then they're probably using ...


6

Selenium is a tool that remote-controls a browser. You can use it to simulate a user interacting with a web site. JUnit is a framework for writing Java unit tests. It takes some of the grunt work out of organizing tests and generating reports. You can express each test as a method on a class; typically, you have multiple tests per class. JUnit will run ...


5

There's a few extra factors here that can impact the way you handle this problem: Do you get results for each test as it completes or do you have to wait until all tests complete? Do you have multiple machines on which to run the tests (and is it possible to do this) or are you tied to a single system running your tests in sequence? Can you break your ...


5

As already stated in a comment, it depends. You have to consider a lot of things in order to decide your test strategy. Usually you have limited resources for testing (time, budget, testers, infrastructure etc.), so you cannot test everything equally thorough. A risk-based approach helps you decide where to concentrate your testing efforts and how thorough ...


5

Instead of an "either/or", it is better to think of test case or scenario testing as a tree structure: Trunk: This is common test set that are always run (e.g. launch the application) Branch: These are major scenarios that have few common tasks (e.g. "admin" vs "user" scenarios) Branch: ... (if needed) Branch: ... (if needed) Twig: Test cases that are ...


5

I prefer to test in isolation, test should not be depended of each other in order to be able to run them on their own. This makes testing a smaller part of the application easier for developers. Also if each test has its own environment to run in you can run tests in parallel without tests conflicting each others runs. There will be some extra overhead in ...


4

Great question. I see two issues, as follows. Using random data may lead to unrepeatable results. You can mitigate this by logging (or otherwise recording) every random choice you make, and then playing those choices back. That could be as easy as recording the initial seed to your random number generator, assuming your data does not change over time. ...


3

As always, it depends. At my previous employer, we had some 400-odd customers (business-to-business software). The one customer who was the only user of certain features was one of the largest customers so it was absolutely critical to make sure those features worked well. In anything business-to-business that's pretty common: the biggest customers will ...


3

If you have specifications that say a confirmation must be displayed, the test fails. If you have user requirements or user expectations that there will be a confirmation dialog, the test fails. In the situation you describe, I would create a bug from the test, but it would have a lower severity rating because it doesn't interfere with functionality. The ...


3

There are tools that will generate a set of tests that cover every combination at least once without hitting all possible combinations (these work on the principle that most bugs will be surfaced by exercising pairs of settings). Since the feature hasn't gone live, your business analyst is probably your best reference for the combinations that are most ...


3

Here are pros and cons of running tests in a sequence. I understood you ask in a context of automation, not manual execution. Pros: Execution time can take shorter, because previous test sets prerequisites for the next one. Cons: Harder to run in parallel. If you have a sequence of tests that depends on each other, you cannot run them in parallel. You ...


3

This is a pretty common situation - and one I've dealt with myself. Based on your comments and the question itself, I can offer a few suggestions. Document your test dependencies - if you're going to be running tests that depend on the result of other tests, make sure you document them. Build in per-test checks - it's a lot more complex to check the ...


3

Based on your response to my comment, you're not actually looking to test database operations but an application that employs the operations. That makes things a little simpler. If you have access to the data store - the simplest method you can use is to perform an operation with your application and treat the data store as an oracle for verification ...


2

I think it is called a "Combinatorial test matrix" . With more combinations the matrix could become infinitely large, have a look at Pairwise Testing to simplify this, but still have a high coverage with combinations. Pairwise (a.k.a. all-pairs) testing is an effective test case generation technique that is based on the observation that most faults are ...


2

It could be called anything as long as the stakeholders of the project understand what it is and how to use it. You can call it combinatorial input matrix, all pairs matrix, exhaustive input matrix and so on!


2

You avoid this question completely by integrating testing into software development life-cycle. I assume you mean that the bulk of testing is done after development, rather than "how do we re-factor in the 11th hour of final testing"? General advice for re-factoring is to have good test coverage so changes can be made with confidence and have a high degree ...


2

Whenever possible, instead of managing dependencies, I work very hard to eliminate them, or at least reduce them. Another high priority goal for me is to eliminate any technology that is not directly involved in the feature I'm testing. Every additional element of technology offers possibilities for the tests to fail independent of whether the system ...


2

If we refer to ISTQB glossary of terms then test suite: A set of several test cases for a component or system under test, where the post condition of one test is often used as the precondition for the next one. test plan: A document describing the scope, approach, resources and schedule of intended test activities. It identifies amongst others test items, ...


2

• For Android Lockito application should work for you. It allows to emulate movement from one location to another one. You can set the speed, the route is based on Google Maps. • For iOS There are at least 2 approaches: use Xcode with GPX files Here's good article on this. Also there is a script to generate GPX files with routes. or add ...


2

Does the presented style of cascading tests violate the "one assertion per test" principle? Yes, it does. While the style is quite clean, a disadvantage to your solution is that it becomes harder to run test_login and test_write_post with a different user, for example with one that is already registered. I personally find a better solution is to have ...


2

There are many such tools. Basically if you do a search with term "test management tool" you will find a lot of alternatives. There are also many sites listing the tools, but they seem to be biased towards some commercial alternatives. One site with a quite exhaustive listing for Test tools is https://www.testtoolreview.de/en/ The company I'm working in ...


2

Answer is "when its not feasible/possible to perm a test" It is used for "multi-environment" test-cases and/or if it is some type of generic test-cases script My real example, it would been need to test feature of upload several files same time. But old versions of IE doesn't support possibility to select several files in dialog - so it is "No apply" here, ...


2

No one here can definitively answer an ambiguous question like this. Go ask the person who wrote the test cases, or the test manager, or someone else on the team who has already figured out the answer.


2

To expand on bish's answer a bit: Your fixtures aren't supposed to be testing end-to-end functionality of all the subsystems - that's something for manual testing and/or automated functional testing and should be done as little as possible (because it's slow and prone to error - you do that as the top-level sanity testing). Each fixture is - or should be ...


2

You can go either way. You might include these steps in each test, because they're actions that an actual user will take. Meaning that they can potentially fail in specific circumstances, and thus you'll want them linked to that specific test case. However, signIn() and signOut() look like actions every tester will perform manually as well for every ...


2

However you obtain test data, you would like to know that it is appropriate to the intention of the tests. When the test data is defined outside of the tests themselves, there is always the chance that the data will drift away from the intentions of the tests. The chance increases if the data is maintained by someone other than the test automators. If you ...


1

There are two ways of proceeding with this. Go to the actual class file testSuite.TestOrder and change the @Test tag on the method to @Test(enabled="false") But you will have to recompile your class files every time you have to make a change. @Test(group={"ignore"}) TestNG has a group option which help you categorize the tests. Mark all test that you want ...



Only top voted, non community-wiki answers of a minimum length are eligible