70
Where can we start?
If you have a bug tracking system, get some sort of report of bugs for the past few years, broken down by module (if that's possible). Then make a histogram/pareto chart to see which are the buggiest areas (each bug counts as +1, each re-opened bug counts as +1, each "that didn't fix it you developer you" counts as +3). Those areas are ...
63
You are about to embark on a journey without end, so it is the means, not the ends, that are important.
Deciding where to start
Business first: As Laura said, make sure that your work directly improves areas important to the business
Metrics: It is critical to get good metrics both before you start and as you progress. Keep your metrics current, preferably ...
35
I seem to recall there is a book that specifically addresses this exact scenario. Just a sec while I search Amazon...
...here we go: Working Effectively with Legacy Code
I admit I haven't read this myself, but I've heard good things about it, and it has 4.5 stars.
34
Good question. As others have answered, the amount of detail will depend upon your particular context. When I'm asked that question by clients, I've started drawing a simple 2 X 2 matrix that looks like this:
17
Yes! Me too! Unstructured you say? You have my deepest sympathies; even so, it sounds like you have more weight to enforce rules and compliance than yours truly in my small but passionate team.
Broadly speaking, this is what I did:
Picked a module and began writing test cases for it - shooting for the
moon, but still accepting what I
managed to document (...
15
Any ideas for reducing test time for GUI tests?
One tip that many seem to overlook is to ensure that all your tests are actually useful. Many times, tests decay over time - their usefulness fades to the point that the tests themselves are a waste of time, no matter how quick they are.
I once joined an organization that ran automated overnight tests which ...
14
I find the Page Object pattern very useful, and use a modified PageFactory (parameterized for custom timeouts).
I also use WidgetObjects (basically, PageObjects with a parent reference via constructor) to represent common themes across pages.
I don't think URL association is necessary on all pages, so I keep it out of my base class. URL params? I keep my ...
11
I've been in this situation before. We were running real-time tests for similar reasons. I wouldn't optimize prematurely, but if you see a looming problem I also wouldn't wait until it's already an issue. Some things we did:
Parallelized testing across many machines. Make it easy to select a subset of your tests, so you can break one test run into ...
10
My first idea might be to break the system up into smaller components (not literally just from a planning perspective). From here you can prioritise them based on which are more important and work with those more important sections first. With such a big project this modular approach would help prevent a sense of your team getting no-where and perhaps keep ...
9
What you really need to interested in is testing each component in isolation.
Integration level and above becomes more challenging to test. If you are using automated tests then these cannot rely on putting data into a database and expected it to be there straight away. There is a replication time that you would need to wait which could significantly slow ...
9
That's a good question. I wish there were a hard and fast answer, but you need to calibrate the level of detail with the capabilities of your tester. I think you should always describe the intent of the test case, i.e. the requirement you are testing. For someone with more experience, you can leave the rest to the tester's imagination. For someone with ...
8
In addition to the given answers, it would also depend on the current members of the team, and the kind of members you intend to hire in future. The current testers will be well aware of the basic interaction with the system, and the new hire should be given appropriate training of the system and hence, every single step need not be documented in a TC.
It ...
8
My first thought, reading your description was to go after where the largest chunck of sales run through the system. Keeping 80% of the dollars flowing correctly is more important than a workflow that maybe accounts for 3% of them.
Since you said the testers also write the specs I'm assuming they have a good idea what business flow/transactions represent ...
7
Yes, however this is one of those circumstances when it depends, an the final answer on "how efficient" will be different for everyone.
Why it matters
If you use test automation for a smoke or BTV test, then you want to be able to execute broad coverage quickly
If you need to debug or re-run a test to see if a failure occurrs you don't want to be sitting ...
7
There's no problem being BA+QA, but it looks that you're also an Architect in your project.
I see a direct evil in combining Architect and QA roles.
You have noticed for yourself every concern: the natural role of Architect is to stand up for idea that "the program is working". The natural role of QA is a direct opposite: to prove that "the program is not ...
6
Step 0, Get the build automated when they check into the source control system. TeamCity pro could probably be made to checkout and run a command - it is and it's free and easy to set up.
Step 1, Write one test. Don't get hung up on unit versus intergrtion - just get one test.
Step 2, Think of it as an oppotunity:
What is it your business is about?
Try ...
6
As well interrogating your bug cases, it might be worth speaking to the Business about which features provide them the most value / money.
What feature would cost them the most money / bad reputation should it stop working.
6
Not to negate one anybody else has already said because I think it is all good advice, but the first step I would take is to run one very basic test. That moves you from having a system with zero tests to having a system with an incomplete set of tests that can be improved upon. My experience is that it's easier to complete a job that's already under way ...
6
For better or worse, I was taught years ago to write a test case as if a tester were walking in off the street to execute it. Thus my test cases tend to have a lot of detail. However, I have executed other's test cases so entrenched in detail that I had to take pressure time to sort out the meaningful bits & stick to those. That is very frustrating, ...
6
Yes. I call them Contract Tests. One easy way to understand them:
Start with tests for a specific implementation of the interface. For example, consider ArrayList implements List. You write tests for ArrayList. One test could be this:
testEmptyIffZeroSize:
list = new ArrayList()
assert list.isEmpty()
assert_equals 0, list.size()
Notice that ...
5
I tried both ways over the years and still having doubts each time.
At the end it comes down to who is going to use the document- if you are sure it is going to be someone with in-depth knowledge of the system and with good testing skills you can easily give only the headlines.
An exception is when details are needed by a standard or certification, Medical ...
5
If you need to be sure every option/path is tested (click the button, use the hot key, tab to it & enter) then you want to detail every step. And unfortunately sometimes that tedious details is necessary.
On the flip side, for a test focused on something else I find putting in specifics can narrow the test & prevent problems from being found.
I'm a ...
5
I do it the same way you do - list broad areas like functionality, performance, security, then list pieces and approaches for each, then break those down into test cases if necessary.
In addition, I like to add a "goals" and "non-goals" section sometimes, especially where there are legacy tools or APIs that I am not going to explicitly test. Sometimes I'...
5
Efficiency is relative to your needs. Rather than ask, "Should automated tests be efficient?", it might be more interesting to ask, "When does automated efficiency matter?" For example, if you expect a developer to run your automated test before checking in a change, it matters a great deal how long it takes that test to run. You could probably poll your ...
4
Just begun to play with one of implementations of no-sql, so only that I can advice to you is my guess + actually I'm not QA engineer, so advice is not formal.
However, hope it will help.
So, what to take into account:
eventual consistency(is time required for getting data consistent on write acceptable);
fail-safety(will data stay alive if one of, let'...
4
My answers below are based on the framework that we've built, which uses a state machine to traverse the application we're testing.
The URL (or, rather, a compiled regex representing the URL) is indeed a property of the individual page objects in my test suite. They all descend from a base page object very similar to terryp's (though we've converted to ...
4
Scaling is always important to keep in mind. If you doubled the number of tests (which will happen eventually) to so now it goes up to 20 hours, would this be a problem? Maybe. You would start it at 5 when you leave, and get no feedback until after lunch the next day.
Another thing I'd consider is how fast could they be? If you were to take them from 4-10 ...
4
I think you should work towards fast feedback. Using parallel test execution your testrun can theoretically take as long as your slowest test. A testrun which completes in minutes instead of hours helps you work in smaller faster cycles, catch errors faster, improve your testsuite faster etc.
Developers want to know a change they've made caused a bug in ...
4
Automated tests should be just efficient enough, and no more.
"This creates tests that run much longer than they could, but is this a problem? Most of the time the tests I run are run overnight unattended, so it's not too relevant if they take 4 or 10 hours to run. On the other hand, even small efficiencies could make a difference."
Not trying to use ...
4
Once you have the table, you should be able to generate the paths through those options. It's the path I find most important as then it can be related to the real world or the deliverable.
The 'User Journey' is a thing you can prioritise. Eg. It is more important that a customer can place an order, than the customer viewing their order history.
Once the ...
Only top voted, non community-wiki answers of a minimum length are eligible
