Tag Info

Hot answers tagged

14

Any ideas for reducing test time for GUI tests? One tip that many seem to overlook is to ensure that all your tests are actually useful. Many times, tests decay over time - their usefulness fades to the point that the tests themselves are a waste of time, no matter how quick they are. I once joined an organization that ran automated overnight tests ...


10

To answer the question "are they worthwhile?", you need to explore: How expensive (in terms of time and money) are they to create, maintain, and execute? What value are these tests providing? Are they finding bugs? Providing confidence? Are there other, less expensive, ways to provide similar value? I once worked at a software company which built desktop ...


9

I've been in this situation before. We were running real-time tests for similar reasons. I wouldn't optimize prematurely, but if you see a looming problem I also wouldn't wait until it's already an issue. Some things we did: Parallelized testing across many machines. Make it easy to select a subset of your tests, so you can break one test run into ...


7

Yes, however this is one of those circumstances when it depends, an the final answer on "how efficient" will be different for everyone. Why it matters If you use test automation for a smoke or BTV test, then you want to be able to execute broad coverage quickly If you need to debug or re-run a test to see if a failure occurrs you don't want to be sitting ...


5

Efficiency is relative to your needs. Rather than ask, "Should automated tests be efficient?", it might be more interesting to ask, "When does automated efficiency matter?" For example, if you expect a developer to run your automated test before checking in a change, it matters a great deal how long it takes that test to run. You could probably poll your ...


4

Automated tests should be just efficient enough, and no more. "This creates tests that run much longer than they could, but is this a problem? Most of the time the tests I run are run overnight unattended, so it's not too relevant if they take 4 or 10 hours to run. On the other hand, even small efficiencies could make a difference." Not trying to use ...


4

Personally I prefer the approach that you are currently implementing, i.e. scale out with a lot of test machines. The reasons I prefer it is: Machine level isolation of tests from each other. Even with the best intention, it is possible to get resource contention and tests interfering with each other, especially when web testing is involved (session ...


4

I think you should work towards fast feedback. Using parallel test execution your testrun can theoretically take as long as your slowest test. A testrun which completes in minutes instead of hours helps you work in smaller faster cycles, catch errors faster, improve your testsuite faster etc. Developers want to know a change they've made caused a bug in ...


4

Scaling is always important to keep in mind. If you doubled the number of tests (which will happen eventually) to so now it goes up to 20 hours, would this be a problem? Maybe. You would start it at 5 when you leave, and get no feedback until after lunch the next day. Another thing I'd consider is how fast could they be? If you were to take them from 4-10 ...


2

I'm in the same boat, where 12-16 hours for tests to run is acceptable. However, if for no other reason than editing and debugging, efficient is the way to go. Although it may take you a while to write the clever search, it will save you a lot of time in the long run. Additionally, if anything ever happens where you have less of a window, there's no need ...


2

Answering your question requires knowing why you were asked to write those tests. Unlike solving applied math problems, we usually do not write automated UI tests for the sheer pleasure or the intellectual challenge. I assume you were asked to write them, in that style, for a reason. If there is a reason to believe the UI is particularly buggy and that it ...


1

I agree with Aniket, this kind of exhaustive UI test is not worth it. Another thing to consider: Awhile back, I inherited a manual test case plan from someone who left the company. He had been doing things like checking the list sorting behavior in every area of the app. He wasn't a programmer and didn't understand that the code behind the dropdown box for ...


1

IMHO, no. Here's why: Depending upon your UI, cost of implementing will probably be high. Don't forget to sum the cost of maintaining it when your UI changes in the next release and so on. This itself should warrant second thoughts. UI tests can be flaky. In my experience accuracy of UI test cases is pretty low. I'd imagine this to get worse with brute ...


1

Answer @corsiKa - Any ideas for reducing test time for GUI tests? One point that I have experienced is for the GUI test, many times a tester add delay / think / wait time between every action, so that objects become available before the click / action. Usually automation tools provide the function / api as waitUntilVisible(object). If the function is used ...


1

Fast Feedback Loops Are Good The faster the feedback loop the quicker you can act upon errors. Assuming this is a nightly run 24 hours is a long feedback loop, especially if you are working in an agile environment. I would aim to get your entire test run running in 15-30 mins on a CI server that automatically runs the tests when code is checked in. This ...


1

When you are assuring the quality of the product, the quality of your test script is as important. Hence, the automation should be very efficient. (This is more of a good attitude than a requirement.) In the real world, the automation should be as efficient as practically possible. Some ways: **Modularize the tests** - If the tests can run in ...



Only top voted, non community-wiki answers of a minimum length are eligible