Tag Info

Hot answers tagged

10

If they can be run unattended, run them over the weekend every week. That'll give you enough run-time, and enough break between runs to fix bugs and perhaps improve the automation. As for analyzing the results, that's a different issue. I have found it useful to group the results into features or categories. That way, I can quickly see from the result set ...


10

Usability testing, because while QA may assist in the process, the actual test participants are probably not QA people.


7

Being a developer and tasked with testing at times for me things that can slip your mind is Usability and those annoying grammar/spelling mistakes.


7

Basic exploratory testing for the most simple of functions.. Historical usage examples tend to push people into a direction of "that's always broken" or "that's always worked" .. the worst thing is not checking either, to find out whether they work now.


6

I'll start by saying that "automated" and "user acceptance" do not fit in the same sentence very well :) The point of user acceptance testing is to have end users test your product in its final state as to validate it is doing what it has to do & it is doing it correctly. Human interaction just can't be automated (yet). The value that our human ...


6

There are two supporting features that are great for testers. Technical ability Domain knowledge Technical ability is more along the lines of "I'm testing a game, and I know graphics libraries. I'm more likely to be able to spot rendering flaws." Domain knowledge "I'm an accountant, I will recognize a problem with accounting software faster than most ...


4

Negative testing is one that, while not forgotten, gets left behind when the time crunch comes in. For example, say a message response has a limited set of valid numeric values (1, 2, 3, 4). What if a response is sent that is outside of that range of numerics (6, 10, 0, -1)? What if the data type sent back doesn't match (1.5, 0.2, "blue", BLOB)? What if ...


4

It depends. For me, every time there is a deployment to test environment we would run the automation. What might change though is the number of test cases that we would run and the number of machines that we would use to run them. After we go-live with 1.0, we normally hand select a number of key test cases, that give us a high level test pass. This test ...


4

In general you should use tests as a spec for your product. Changing the tests should be done as little as possible, unless you are making a change to your spec. And you should never change tests and code at the same time, otherwise you won't necessarily know if they are both correct. It is not uncommon for 'big requirement changes' to change the spec. I ...


4

When a customer asks you for something using words that you don't understand, you must ask for clarification. I'm guessing that the customer wants to understand how the whole UAT process will work - who does what, how, and when. But that's just a guess. The term "UAT Protocol" could mean anything to this particular customer. Guessing and hoping for the ...


4

Does the application you are testing have on-line help? or a manual? Those might be a good place to start. Also, what about access to designers, support or coders? If you can find someone who has worked with the software for a long time they may be able to give you an overview to get you started.


4

As you say - 'it depends' but a few comments from my experiences: If you supply exact steps and data then what is the point of UAT? You might as well get your testers to run the scripts. I'd rather give the users some training on the system and give them scenarios to follow - which will have been developed with their input. Should anyone be able to run the ...


3

One of the key areas I have seen improvements to applications is when end users are actively involved testing and development, ideally from start to finish as part of the team. The more removed that users are from the development and implementation and testing the less likely it is that the application will meet their business need. I have actually seen, ...


3

Where I work, load testing is often forgotten because it's not easy to coordinate and it consumes a lot of scarce resource. On a more generic note, functions or features that don't get a lot of use in the general run of things are very easy to forget. A lot of "one time" "set and forget" features fall into this category, especially if you work with ...


3

We tend to skimp on performance testing. We all know it needs done, but it always seems to be left for last/dropped completely when we get to the end of a project window. My guess is you are going to get as many different answers as there are testing methods out there. Depends on the person & the shop they are currently working in.


3

To be honest, this is not something I've ever done from the professional side. However, from a user side here's a suggestion that I get from World of Warcraft by Blizzard. When I log into my WoW app on my machine, there's a "splash" screen that comes up before I get to my main account login that gives different news feeds, articles, ads, etc., related to ...


3

I'd agree with Bruce, it depends. If your requirements need these to run more often fit them in where you can. Otherwise what you may need to do, if you can afford it, is to set up an environment to constantly run them. I did this in one company where we needed to continually run these tests and did it in a Longevity environment where the tests always ...


3

Anything that is automated should run the same way every time. When you're running it the same way every, single, time you are more likely to get results that compare to eachother. If you're relying on non-automated tests to do these things, you greatly increase the chance you're not running the exact same test. It might be done with different steps or in a ...


3

Disclaimer This is solely based on my experience. I'm sure there are others who will have a radically different experience with trying to get good UAT feedback. Note I consider UAT to be more informal than formal. I'd love formal UAT, but there simply isn't budget for that where I am. :-( Unfortunately, in a situation where SOX compliance is involved, ...


3

Approvaltests.com calls them unit tests, so I'm not going to argue :) Although I'm not sure why you are looking for a hard definition. It's taking the value from unit tests, adding readable output and screenshoting them. Kinda a neat process that could be used for UAT and forces dev's to use TDD. Tools that I would consider similar are Cucumber, jBehave, ...


3

If you want to build your own there is a screencast on how to do beta invites in Ruby on Rails here railscasts.com/episodes/124-beta-invitations. You might be able to adapt that solution to fit your needs.


2

Verifying what version is loaded on the machine I'm testing... I tend to miss the very basic things and jumps to the more interesting things, checklists helps to avoid misses like that.


2

From the User perspective you should also understand who the Customers are, they may be in the same boat as you and just told to use the software without training. Some may seem familiar to them with Domain Knowledge, but that is picked up and often testing in that realm can discover very different bugs because you are exploring the software and doing ...


2

Unfortunately, we do not have formal UAT. A handful of support personnel walking through new features is the closest thing to UAT we do have.


2

There is a concept called the 'Golden Master' testing which is what approval tests uses, and tries to automate. The reason that the concept is orthogonal to Unit Tests, functional tests, and user acceptance is that all of these are still doing verification by asserting against primitive values. All tests have 2 parts: 1) do 2) verify The real difference ...


2

I found this site which does pretty much most of what we needed and seems to fit the bill nicely, at least for distribution of the app. http://www.airdrop.me/


2

I am looking for a specific tool which will help me document what I see when I perform web site usability evaluations. At the end of each evaluation, I need to create an report for my client. As your demand above, I would think a tool called qTrace could help you out. It's a complete screen capture tool that helps a tester easily submits clear and ...


2

Here are a few considerations: There are multiple reasons why a unit test might fail. Unit tests have to be maintained just like the system under test does. When a unit test breaks, it might point to a bug in the system under test, or it might point to a bug in the unit test (e.g. a timing issue), or it might even point to a unit test that no longer makes ...


2

On some projects testers certainly can benefit from knowing the status of unit tests. 1) To initiate discussions. If unit tests have not passed before the testing / release, I would like to know why. 2) On some cases testers being able to see the status of unit tests and ask these questions has positive impact on maintaining unit tests. Sometimes testers ...


2

Have you considered using post actions to either pull the UATs from the tasks or push the UATs to the task? This is relatively easy to accomplish if you have some experience of configuring your own workflows. It allows you to generate the tasks required based on the Acceptance Criteria (i.e. each would create a new task) and copy the details of that into ...



Only top voted, non community-wiki answers of a minimum length are eligible