Tag Info

Hot answers tagged

10

If they can be run unattended, run them over the weekend every week. That'll give you enough run-time, and enough break between runs to fix bugs and perhaps improve the automation. As for analyzing the results, that's a different issue. I have found it useful to group the results into features or categories. That way, I can quickly see from the result set ...


6

I'll start by saying that "automated" and "user acceptance" do not fit in the same sentence very well :) The point of user acceptance testing is to have end users test your product in its final state as to validate it is doing what it has to do & it is doing it correctly. Human interaction just can't be automated (yet). The value that our human ...


6

There are two supporting features that are great for testers. Technical ability Domain knowledge Technical ability is more along the lines of "I'm testing a game, and I know graphics libraries. I'm more likely to be able to spot rendering flaws." Domain knowledge "I'm an accountant, I will recognize a problem with accounting software faster than most ...


4

Does the application you are testing have on-line help? or a manual? Those might be a good place to start. Also, what about access to designers, support or coders? If you can find someone who has worked with the software for a long time they may be able to give you an overview to get you started.


4

It depends. For me, every time there is a deployment to test environment we would run the automation. What might change though is the number of test cases that we would run and the number of machines that we would use to run them. After we go-live with 1.0, we normally hand select a number of key test cases, that give us a high level test pass. This test ...


4

In general you should use tests as a spec for your product. Changing the tests should be done as little as possible, unless you are making a change to your spec. And you should never change tests and code at the same time, otherwise you won't necessarily know if they are both correct. It is not uncommon for 'big requirement changes' to change the spec. I ...


4

Negative testing is one that, while not forgotten, gets left behind when the time crunch comes in. For example, say a message response has a limited set of valid numeric values (1, 2, 3, 4). What if a response is sent that is outside of that range of numerics (6, 10, 0, -1)? What if the data type sent back doesn't match (1.5, 0.2, "blue", BLOB)? What if ...


4

When a customer asks you for something using words that you don't understand, you must ask for clarification. I'm guessing that the customer wants to understand how the whole UAT process will work - who does what, how, and when. But that's just a guess. The term "UAT Protocol" could mean anything to this particular customer. Guessing and hoping for the ...


3

As you say - 'it depends' but a few comments from my experiences: If you supply exact steps and data then what is the point of UAT? You might as well get your testers to run the scripts. I'd rather give the users some training on the system and give them scenarios to follow - which will have been developed with their input. Should anyone be able to run the ...


3

Approvaltests.com calls them unit tests, so I'm not going to argue :) Although I'm not sure why you are looking for a hard definition. It's taking the value from unit tests, adding readable output and screenshoting them. Kinda a neat process that could be used for UAT and forces dev's to use TDD. Tools that I would consider similar are Cucumber, jBehave, ...


3

I'd agree with Bruce, it depends. If your requirements need these to run more often fit them in where you can. Otherwise what you may need to do, if you can afford it, is to set up an environment to constantly run them. I did this in one company where we needed to continually run these tests and did it in a Longevity environment where the tests always ...


3

One of the key areas I have seen improvements to applications is when end users are actively involved testing and development, ideally from start to finish as part of the team. The more removed that users are from the development and implementation and testing the less likely it is that the application will meet their business need. I have actually seen, ...


3

Disclaimer This is solely based on my experience. I'm sure there are others who will have a radically different experience with trying to get good UAT feedback. Note I consider UAT to be more informal than formal. I'd love formal UAT, but there simply isn't budget for that where I am. :-( Unfortunately, in a situation where SOX compliance is involved, ...


3

Anything that is automated should run the same way every time. When you're running it the same way every, single, time you are more likely to get results that compare to eachother. If you're relying on non-automated tests to do these things, you greatly increase the chance you're not running the exact same test. It might be done with different steps or in a ...


3

If you want to build your own there is a screencast on how to do beta invites in Ruby on Rails here railscasts.com/episodes/124-beta-invitations. You might be able to adapt that solution to fit your needs.


3

To be honest, this is not something I've ever done from the professional side. However, from a user side here's a suggestion that I get from World of Warcraft by Blizzard. When I log into my WoW app on my machine, there's a "splash" screen that comes up before I get to my main account login that gives different news feeds, articles, ads, etc., related to ...


3

Where I work, load testing is often forgotten because it's not easy to coordinate and it consumes a lot of scarce resource. On a more generic note, functions or features that don't get a lot of use in the general run of things are very easy to forget. A lot of "one time" "set and forget" features fall into this category, especially if you work with ...


2

We tend to skimp on performance testing. We all know it needs done, but it always seems to be left for last/dropped completely when we get to the end of a project window. My guess is you are going to get as many different answers as there are testing methods out there. Depends on the person & the shop they are currently working in.


2

From the User perspective you should also understand who the Customers are, they may be in the same boat as you and just told to use the software without training. Some may seem familiar to them with Domain Knowledge, but that is picked up and often testing in that realm can discover very different bugs because you are exploring the software and doing ...


2

There is a concept called the 'Golden Master' testing which is what approval tests uses, and tries to automate. The reason that the concept is orthogonal to Unit Tests, functional tests, and user acceptance is that all of these are still doing verification by asserting against primitive values. All tests have 2 parts: 1) do 2) verify The real difference ...


2

I am looking for a specific tool which will help me document what I see when I perform web site usability evaluations. At the end of each evaluation, I need to create an report for my client. As your demand above, I would think a tool called qTrace could help you out. It's a complete screen capture tool that helps a tester easily submits clear and ...


2

Have you considered using post actions to either pull the UATs from the tasks or push the UATs to the task? This is relatively easy to accomplish if you have some experience of configuring your own workflows. It allows you to generate the tasks required based on the Acceptance Criteria (i.e. each would create a new task) and copy the details of that into ...


1

Zigster, as regards site loading speed and search speed, I would add non functional requirements at the user story level. They can be expressed in the form of acceptance criteria. For example if one user story is: As a customer I want to run a search based on product name So that the products with matching name will be displayed Among the acceptance ...


1

Updating to reflect exit criteria for testing. You can look at MSDN article - Adventures with Testing BI/DW Application:On a crusade to find the Holy Grail - http://msdn.microsoft.com/en-us/library/gg248101.aspx Critical areas to focus are Validating the ETL Scenarios for full, delta pull Testing with Production Data for multiple regions, multiple data ...


1

It is not possible to do exhaustive testing in the context of data warehousing. I had been into ETL testing. There was a corner case bug in which if there is a lock created on one of the tables in the source database, the migration of data to the destination DB failed. Is it possible for any tester to imagine a scenario where table gets locked automatically ...


1

Although I don't have the first clue about provisioning Android apps, it sounds as though all that you really need is a couple of hidden pages on your existing site, or a new one just for this and a database table to store the info. We've done similar things in my company. We'd have our developers make a couple of hidden pages and a new database table that ...


1

What has been mentioned in the previous post is very true. At my current project we use automated user acceptance test, we also use the same suit as build acceptance test. Since automated tests make sure each and every part of the AUT is as it was expected to be w.r.t the SRS, we can always be sure that both the functionality and GUI lay out is as expected. ...


1

Automated tests save time and increase accuracy during regular execution of test cases over a long period of time. There is of course a trade off, since it takes a long time to create automated tests. Related discussion is here and here.



Only top voted, non community-wiki answers of a minimum length are eligible