Tag Info

Hot answers tagged

49

Where can we start? If you have a bug tracking system, get some sort of report of bugs for the past few years, broken down by module (if that's possible). Then make a histogram/pareto chart to see which are the buggiest areas (each bug counts as +1, each re-opened bug counts as +1, each "that didn't fix it you developer you" counts as +3). Those areas ...


30

Good question. As others have answered, the amount of detail will depend upon your particular context. When I'm asked that question by clients, I've started drawing a simple 2 X 2 matrix that looks like this:


25

I seem to recall there is a book that specifically addresses this exact scenario. Just a sec while I search Amazon... ...here we go: Working Effectively with Legacy Code I admit I haven't read this myself, but I've heard good things about it, and it has 4.5 stars.


22

You are about to embark on a journey without end, so it is the means, not the ends, that are important. Deciding where to start Business first: As Laura said, make sure that your work directly improves areas important to the business Metrics: It is critical to get good metrics both before you start and as you progress. Keep your metrics current, ...


14

Any ideas for reducing test time for GUI tests? One tip that many seem to overlook is to ensure that all your tests are actually useful. Many times, tests decay over time - their usefulness fades to the point that the tests themselves are a waste of time, no matter how quick they are. I once joined an organization that ran automated overnight tests ...


13

Yes! Me too! Unstructured you say? You have my deepest sympathies; even so, it sounds like you have more weight to enforce rules and compliance than yours truly in my small but passionate team. Broadly speaking, this is what I did: Picked a module and began writing test cases for it - shooting for the moon, but still accepting what I managed to document ...


13

I find the Page Object pattern very useful, and use a modified PageFactory (parameterized for custom timeouts). I also use WidgetObjects (basically, PageObjects with a parent reference via constructor) to represent common themes across pages. I don't think URL association is necessary on all pages, so I keep it out of my base class. URL params? I keep ...


9

I've been in this situation before. We were running real-time tests for similar reasons. I wouldn't optimize prematurely, but if you see a looming problem I also wouldn't wait until it's already an issue. Some things we did: Parallelized testing across many machines. Make it easy to select a subset of your tests, so you can break one test run into ...


8

That's a good question. I wish there were a hard and fast answer, but you need to calibrate the level of detail with the capabilities of your tester. I think you should always describe the intent of the test case, i.e. the requirement you are testing. For someone with more experience, you can leave the rest to the tester's imagination. For someone with ...


7

In addition to the given answers, it would also depend on the current members of the team, and the kind of members you intend to hire in future. The current testers will be well aware of the basic interaction with the system, and the new hire should be given appropriate training of the system and hence, every single step need not be documented in a TC. It ...


7

My first idea might be to break the system up into smaller components (not literally just from a planning perspective). From here you can prioritise them based on which are more important and work with those more important sections first. With such a big project this modular approach would help prevent a sense of your team getting no-where and perhaps keep ...


7

Yes, however this is one of those circumstances when it depends, an the final answer on "how efficient" will be different for everyone. Why it matters If you use test automation for a smoke or BTV test, then you want to be able to execute broad coverage quickly If you need to debug or re-run a test to see if a failure occurrs you don't want to be sitting ...


6

What you really need to interested in is testing each component in isolation. Integration level and above becomes more challenging to test. If you are using automated tests then these cannot rely on putting data into a database and expected it to be there straight away. There is a replication time that you would need to wait which could significantly slow ...


5

For better or worse, I was taught years ago to write a test case as if a tester were walking in off the street to execute it. Thus my test cases tend to have a lot of detail. However, I have executed other's test cases so entrenched in detail that I had to take pressure time to sort out the meaningful bits & stick to those. That is very frustrating, ...


5

My first thought, reading your description was to go after where the largest chunck of sales run through the system. Keeping 80% of the dollars flowing correctly is more important than a workflow that maybe accounts for 3% of them. Since you said the testers also write the specs I'm assuming they have a good idea what business flow/transactions represent ...


5

There's no problem being BA+QA, but it looks that you're also an Architect in your project. I see a direct evil in combining Architect and QA roles. You have noticed for yourself every concern: the natural role of Architect is to stand up for idea that "the program is working". The natural role of QA is a direct opposite: to prove that "the program is not ...


5

Efficiency is relative to your needs. Rather than ask, "Should automated tests be efficient?", it might be more interesting to ask, "When does automated efficiency matter?" For example, if you expect a developer to run your automated test before checking in a change, it matters a great deal how long it takes that test to run. You could probably poll your ...


5

Yes. I call them Contract Tests. One easy way to understand them: Start with tests for a specific implementation of the interface. For example, consider ArrayList implements List. You write tests for ArrayList. One test could be this: testEmptyIffZeroSize: list = new ArrayList() assert list.isEmpty() assert_equals 0, list.size() Notice that ...


4

If you need to be sure every option/path is tested (click the button, use the hot key, tab to it & enter) then you want to detail every step. And unfortunately sometimes that tedious details is necessary. On the flip side, for a test focused on something else I find putting in specifics can narrow the test & prevent problems from being found. I'm a ...


4

As well interrogating your bug cases, it might be worth speaking to the Business about which features provide them the most value / money. What feature would cost them the most money / bad reputation should it stop working.


4

Once you have the table, you should be able to generate the paths through those options. It's the path I find most important as then it can be related to the real world or the deliverable. The 'User Journey' is a thing you can prioritise. Eg. It is more important that a customer can place an order, than the customer viewing their order history. Once the ...


4

Not to negate one anybody else has already said because I think it is all good advice, but the first step I would take is to run one very basic test. That moves you from having a system with zero tests to having a system with an incomplete set of tests that can be improved upon. My experience is that it's easier to complete a job that's already under way ...


4

I tried both ways over the years and still having doubts each time. At the end it comes down to who is going to use the document- if you are sure it is going to be someone with in-depth knowledge of the system and with good testing skills you can easily give only the headlines. An exception is when details are needed by a standard or certification, Medical ...


4

Step 0, Get the build automated when they check into the source control system. TeamCity pro could probably be made to checkout and run a command - it is and it's free and easy to set up. Step 1, Write one test. Don't get hung up on unit versus intergrtion - just get one test. Step 2, Think of it as an oppotunity: What is it your business is about? Try ...


4

Scaling is always important to keep in mind. If you doubled the number of tests (which will happen eventually) to so now it goes up to 20 hours, would this be a problem? Maybe. You would start it at 5 when you leave, and get no feedback until after lunch the next day. Another thing I'd consider is how fast could they be? If you were to take them from 4-10 ...


4

I think you should work towards fast feedback. Using parallel test execution your testrun can theoretically take as long as your slowest test. A testrun which completes in minutes instead of hours helps you work in smaller faster cycles, catch errors faster, improve your testsuite faster etc. Developers want to know a change they've made caused a bug in ...


4

Automated tests should be just efficient enough, and no more. "This creates tests that run much longer than they could, but is this a problem? Most of the time the tests I run are run overnight unattended, so it's not too relevant if they take 4 or 10 hours to run. On the other hand, even small efficiencies could make a difference." Not trying to use ...


3

You need enough detail to ensure that all the requirements of what your testing can actually be tested. And that's where it should stop. To quote Einstein, "as simple as possible, but no simpler." The more specific you are, the more likely you are to become too focused on one part of the requirements that you miss out on the others. There is an exception to ...


3

I would try and answer this question,with whatever context you have mentioned in the question... to me the term "testing behavior" reflects what the requirements or behavioural expectations are ,from the registration form ( that you have included as an example) in simple words -- "what does this form do ?" answers could be -- it accepts inputs,it accepts ...


3

As most other posts have pointed out, the amount of detail included in a manual test plan depends on the testing experience and abilities of the tester. Prior to recent organizational changes, all of the tests that were written by QAs within my department were also executed by fellow QAs within the same department. What this meant was that we could ...



Only top voted, non community-wiki answers of a minimum length are eligible