Tag Info

Hot answers tagged

7

There is a concept of "steel thread" that we used back at one of my previous jobs where the user story itself is a specific requirement that has a very particular expectation. Anything outside of that particular expectation should be its own user story. The acceptance tests can then be focused and targeted to verifying that the user story's "steel thread" ...


6

It sounds like this is the only regression testing of existing functionality you do, is that correct? It sounds like you're finding valuable bugs with your testing, and that your manager appreciates its value. It also sounds like the other work you have conflicts with performing this testing. "We have to test 1 platform on 3 versions, and, as we usually ...


4

It sounds as if you are referring to a smoke test, so that's how I'll refer to it. The length of a smoke test depends on your product, the risk aversion of your organization, and the resources available for smoke-testing. In my first QA job, the smoke test could take up one individual's entire morning, but detecting a catastrophic defect during the smoke ...


3

I'd call it Production Monitoring as well. In my company we have "Production Engineers" performing this sort of task, not QA folks.


3

Cel, As always, it depends. If there's a requirement that the link contain a label then obviously the test needs to check that a label exists (whether automated or not). Here are some of the possible things I'd consider with this scenario: Is the code dynamically generated or static? For a static "once and done" page, the mark 1 human eyeball might be ...


3

I would agree with Laurent, and we have faced a very similar situation. Our team tests the APIs for data transport and parsing. The developer unit/component level tests were all against mock objects and data stores. Unit tests are not always comprehensive so we built a 'fake server' to emulate various web services that we used for running additional ...


3

Yes, we use log monitoring as part of our testing process. Each manual tester is asked to monitor the logs during test (tail -f), to look for hidden errors/exceptions during testing. Also, our UI will pop-up a dialog if a client side exception is thrown. These usually cause automated tests to fail, and starts the investigation. Finally, we have a script ...


3

Writing and maintaining automated tests is a big investment and it is ok to start slowly It is expensive to write and maintain automated tests. If the tests are written in an ineffective way, the investment may not pay off. Similarly, if the tests require a great deal of maintenance, either because of how the tests are written or because the interfaces ...


3

Spiff, As you suspect, there are no magic bullet answers to your question, but I can offer some suggestions to consider. Wherever possible, look to code any particular action exactly once and reuse it multiple times. Whereever possible, drive your tests with a combination of object-oriented and data-driven test code. Where you can string together a ...


2

Any amount of time is OK as long as it's saving you more time than it costs. So if the tests need to run an hour to prevent 30 testers testing a bad build for five hours while a new deployment is created, so be it. I would seriosly try to invest in automation to get the time down as much as possible. Curating automation can be almost as much hassle as just ...


2

Time taken to validate a build varies based on various factors. One way to identify the optimal time that should be taken in your scenario is to put in numbers for the following: No. of testers No. of features Avg. time taken per tester per feature (And in your case) No. of versions to be tested Once you have these numbers, come up with the time taken ...


2

When I was at Netflix, I called the team Operational Quality. The focus was on insuring that the code that was built & validated during the dev & test process actually worked the way it was supposed to in the production environment. Some of the tools included: continuous measurement of the performance of the different applications (down to the ...


2

I think "acceptance tests" and "mocking services layers" don't go along. My feeling is that when you do acceptance/functional testing, then you run your whole system in a manner close to the way it will be used although you try to do it in a QA/preprod environment to be confident of the state of your system. If, for the reasons you mention, you can not do ...


2

This is the kind of scenario where I'd use a mix of automated and manual testing - manual testing using the iPad, because one thing GUI automation always does is simulate the human inputs. I've found that simulation often has crucial differences from actual input. For automation, the crucial question here is where the data sent from the ios app is stored. ...


2

To make this change for "As an admin" you will need to change the code that is generated in specflow. This would be the best place for this change IMHO. As Lyndon suggests, the code that I have written for WatiN should work, but it really depends on how the application is developed. My other answer copied below. What I found to solve the "run test as user ...


2

Full regressive acceptance testing can only, really, be a static value if the application never gets new features. The moment you add a new feature, and that includes support for new web browsers, new operating systems, etc., you need to add tests to cover that new feature. To say that you need to limit it to 30 minutes is unrealistic. If all you are ...


1

In addition to what Kate said, here's one thing I always consider when writing up scenarios: I am writing them up as an user of the application, so I need to provide all the details that the user would be aware of, and not much more than that. The rest can be handled by the code behind it.


1

keep it short enough so it doesn't disturb other testing activities (much), that's what I usually try to get to.


1

Although I haven't had a chance to try it in this particular circumstance, you could try the answer that I received to a similar question. If it is just as simple as logging in as the admin, I would first ensure that you can log in as an admin, add that code to your setUp, and then do a verification in the Given


1

We rely extensively on log analysis, but not for Web applications but for testing embedded code (from low level drivers to applications). As the presentation says one of the first decisions you'll have to take is online vs. offline , this changes your strategy completely. Our Perl framework has a simplified API for analyzing offline logs, and/or handling of ...


1

Web applications are a little bit different in that the web server deals with multiple, unrelated clients. If you are looking for a pattern, it is not enough to know that page Y followed page X -- you also need to know whether those events occurred in the same session. Sometimes this information is available in the logs; sometimes, not. Of course in a ...


1

I wouldn't worry about redundant tests at first. It's much better to test a piece of code twice than not at all. The only way to accurately determine whether your tests are redundant is by using a code coverage tool (or extensive logging). However, as Kate Paulk states, there are no magic/silver bullets with testing, or software development in general (of ...



Only top voted, non community-wiki answers of a minimum length are eligible