Questions tagged [automated-testing]

Use for questions involving problems with automated tests. Relevant for designing test automation, debugging test automation, automation tooling questions, and questions about when it is appropriate to automate. Questions regarding specific tools should tag those tools as well.

Filter by
Sorted by
Tagged with
17
votes
5answers
9k views

What is the difference between “automated testing” and “automated regression testing”?

"Automated Testing" as a term seems to have a number of different uses. For some, it means built in unit tests that, when a new build is run, execute to make sure code refactors have not broken ...
17
votes
9answers
8k views

How do you calculate your return on investment on automated tests?

I've been looking at calculating the ROI on automated test on a project that I've been working on. However, I'm a little stuck when it comes to determining a good method for this. Simply counting the ...
17
votes
4answers
5k views

Screen Play vs Page Object pattern

I've recently stumbled upon Senenity/JS framework which advocates the Screen Play pattern, which was proposed as a better alternative to Page Objects. What are the key fundamental differences between ...
16
votes
6answers
2k views

Good practices to audit test automation code

One test automator quit his job a few days ago. We noticed today all most all of his test programs produced fake results. What he did was either producing random test results or assert true == true. ...
16
votes
4answers
144k views

Selenium: How to identify the button WebElement

In my web page I'm having following two buttons: <button onclick="addToSelected('newApplicationForm');">Add Strategy</button> <button onclick="submitAddNewApplication('...
16
votes
4answers
17k views

Moving from Selenium to Protractor

I have been using Selenium/Java/Maven for functional automation testing in my project. But since angular is getting a lot of popularity, the development team has started using it for the development ...
16
votes
5answers
59k views

How to handle dynamic changing ID's In XPath?

How to Handle Dynamic Changing IDs In XPath? for example : //div[@id='post-body-3647323225296998740']/div[1]/form[1]/input[1] In this xpath id "3647323225296998740" is changing every time when ...
16
votes
6answers
5k views

How to deal with flaky tests that have intermittent failures?

Having flaky tests that sometimes fail and sometimes pass is very frustrating: there is no single reason each of the flaky tests fail the failures don't seem to happen to real users using the ...
16
votes
5answers
3k views

Can knowing too much about the tested code be a disadvantage?

Shift left testing is an approach in software testing to involve testers into software lifecycle as early as possible. I always understood this as helping in finding conflicting requirements, but also ...
15
votes
12answers
3k views

In house built testing tools

I've been having to deal with a lot of testing tools lately for testing tasks that should/must be automated. These include things like performance/load testing, webservice testing, vulnerability ...
15
votes
6answers
4k views

Why are automation testers interviewed the same way as developers?

A junior automation developer I work with who has around two years of experience interviewed for a QA position. He was asked to write some basic programs for whiteboard exercises like checking if a ...
15
votes
7answers
1k views

Should QA be able to code tests?

Having a development background, I am convinced that record and replay testing tools have their drawbacks, especially by making tests brittle. See Automated Testing != Record-Playback Tool for a good ...
15
votes
4answers
323 views

If I didn't use TDD but want to transition to agile, should I go back and create those unit tests?

In an transitioning agile environment, if there was no Test Driven Development (no unit tests created), should you create unit tests to catch up on technical debt? and to what extent? Should ...
15
votes
2answers
9k views

Good resources/tutorials/tips for beginner doing automation?

I've just started automating all the tests our team has (~5000). I'm using Telerik Test Studio for the Silverlight automation. I have some tests written, but I'm quickly realizing that with each test ...
15
votes
4answers
1k views

Testing interactions with external systems

I'm trying to find a way to test the interactions of our platform with external servers, like Twitter, for example. On our online photo sharing application, we have, for example features which let ...
15
votes
7answers
10k views

How can I structure Selenium tests in a way that minimizes the maintenance work?

What is the best way to build/structure the tests in a way that will minimize the work needed to keep the tests up to date as the product changes and improves?
15
votes
8answers
363 views

Testing of automated scripts

How much testing of your automated scripts do people generally do? Is it best to treat the automated test suite as a separate project or part of the Agile development cycle of the product your are ...
15
votes
5answers
354 views

What is best practice for test failures that have a low priority fix?

If I have a test that fails due to a known bug, but that bug is deemed lower priority to fix than other work, what should be done with the test? I can think of 2 options: Leave the test in the test ...
15
votes
4answers
287 views

What techniques are available to determine which browsers to test?

When websites being deployed to the internet are about to go live, the age old question always comes up. "Which browsers should I test?" I am not asking that question because it will soon become ...
15
votes
3answers
9k views

What are some tools for fuzz testing?

Fuzz Testing as defined by Wikipedia is: a software testing technique, often automated or semi-automated, that involves providing invalid, unexpected, or random data to the inputs of a computer ...
15
votes
5answers
7k views

What are the trade-offs of using page objects instead of a collection of functions in UI automation?

I've been using Selenium off and on for the past two years. Recently I've been seeing more references to page objects. (I'm referring to page objects as an means of organizing UI automation code -- ...
14
votes
4answers
4k views

Why do medium to large companies not adopt test automation?

Is test automation simply not a right fit for some medium/large organisations, regardless of whether they "care" or not and assuming it can be applied there? What are the most common reasons for this? ...
14
votes
7answers
3k views

What are anti-patterns in test automation?

anti-pattern: There must be at least two key elements present to formally distinguish an actual anti-pattern from a simple bad habit, bad practice, or bad idea: A commonly used process, ...
14
votes
10answers
73k views

Best free (or very cheap) automation tool for Windows GUI testing? [closed]

I used to use vTask Studio, but it seems they've abandoned it. No updates in almost a year. Is there are any other automation tool for Windows GUI testing?
14
votes
7answers
2k views

The test team as an enemy of development? And how can this be avoided?

Details Forming a Scrum team should include all the skills necessary to develop a user story in order to deliver a potentially deliverable product increment with each sprint. In traditional ...
14
votes
4answers
7k views

What is the Real Business Value of Software Testing in an IT Project?

I work as a manual tester on a big project. My project follows Agile methodology. I test a Web-based application. I sometimes ask myself, am I giving any value to the stakeholders? What is the Real ...
14
votes
5answers
20k views

What are features of MS Visual Studio Coded UI Test framework that are not present in other tools?

I've recently started working at an organization that has no automated testing in place. They are migrating to Team Foundation Server for project management and source control, and use Visual Studio ...
14
votes
3answers
11k views

Is Selenium the right tool for browser compatibility testing?

I have been seeing a lot of customers pushing for browser compatibility testing using Selenium. Was wondering whether this is really necessary. The push is mostly due to Selenium grid supporting multi ...
14
votes
2answers
24k views

Selenium: How to delete single characters from a textbox?

Using Selenium how to delete single characters from a textbox using Selenium? I have to delete the last 2 characters from text box. Ex: ABCD to AB
14
votes
3answers
791 views

What are the challenges around the integration of Manual and Automated Tests?

This is more a request for your professional opinion than a question... We are seeing how test automation is spreading to more and more organizations. This in itself is a good thing, but at the same ...
13
votes
9answers
804 views

Bugs keep reappearing in our GUI software during development. How should this be addressed?

We have a software product has a lot of usability features, and we're constantly changing it. We have issues with things like persisting scroll position which seem to get fixed, but then are broken ...
13
votes
4answers
2k views

How long should a test wait to assume that the result remains fixed

We had a defect where in a situation the state of the application changed into something undesired. It has been fixed, we added a test: Set up state Action that triggered the wrong state change Check ...
13
votes
3answers
2k views

Test strategy to bring automated tests from UI to lower(API/Unit) layers?

Background/Problem Statement: Recently I came across a project team, which is having thousands of UI automated tests covering all kind of manual tests including positive, negative etc., and obviously ...
13
votes
9answers
1k views

Performance testing - a way in?

We have an a browser based application that is built on the Microsoft Stack, specifically .NET 3.5. It also uses Ajax, iframes, jQuery etc. At the moment we are just guessing at the concurrency ...
13
votes
5answers
943 views

How often do you run long automated user acceptance tests?

Let's say you have product or project that is coming near to a next release and that has previous versions released a while ago and that has a big suite of automated user acceptance tests that just ...
13
votes
4answers
2k views

How can I make a smooth transition from being a Black-Box Tester to an expert Automation Engineer?

I have worked as a functional, black-box tester for over 8 years in the Software Industry. I am a skilled exploratory tester and have found a lot of critical problems in many products over the years. ...
13
votes
4answers
749 views

Good strategy for automated regression where initial state can't be guaranteed or predicted

The situation I need to start building automated regression for a very large web application with thousands of users. It is impossible to start from a known state, or even to be sure that data ...
13
votes
2answers
3k views

How to avoid redundant tests

We are building an application using a completely test-driven approach. As developers, we are very familiar with unit tests but haven't been exposed to integrated / functional / acceptance tests. ...
13
votes
2answers
666 views

Maintaining automation framework for different browsers like IE and FF

How to maintain automation framework for different browsers like IE and FF? In my current set up, I maintain two different configuration properties files, one with xpath locators that FF supports ...
13
votes
3answers
3k views

Test Automation for SAP

I'm evaluating a number of test automation framworks to implement on a SAP project. Right now I'm looking at Fitnesse, which we believe we can write some fixtures for, to call SAP Green Hat, which ...
13
votes
1answer
4k views

Is Selenium's LoadableComponent a good solution as test design pattern?

I need to refactor my Selenium tests, and come across their LoadableComponent wiki page. I began trying to use it, though I couldn't really understand what is it good for. I took a look at their ...
12
votes
13answers
912 views

Should automated tests be efficient?

I've written and edited a bunch of automated GUI tests. In some cases, the tests are inefficient (eg to select an item from a list, instead of a clever search, a dialog is opened and starting fro the ...
12
votes
9answers
14k views

Whether to choose Selenium over Watir or Lightweight Test Automation when testing web applications created using C Sharp?

I am currently researching testing frameworks to test C Sharp developed web applications. I have been leaning more towards the use of Selenium over Watir or Lightweight Test Automation but hesitant ...
12
votes
9answers
3k views

How to test a product without domain knowledge?

We have one product to test but we don't have the resource who have the domain knowledge of the specific field my company don't want to lose the client as it a breakthrough for the company & we ...
12
votes
6answers
35k views

How should you interview for QA *Automation* positions?

A variant on the How should you interview for QA positions? I am interviewing candidates for an Automation position, we don't use commercial automation tools, but rather use a framework using mainly ...
12
votes
4answers
1k views

How can I report coverage on an n-dimensional test matrix?

I have a test matrix for my product's installation, which is n-dimensional, where n is about 11 or 12 at this point. For instance, the OS is one dimension, and the DB is another, and the Reporting ...
12
votes
6answers
693 views

How to know if an existing set of automated tests are effective?

I came to a new project and it was already written 100 autotests. The product is new. How to determine whether the tests are effective? What is the best way?
12
votes
3answers
13k views

Automated tests pass every time when run individually, but when they are clumped into a test suite it's a toss up if the tests will pass

I have around 200 test classes for my web application at the moment. I use Selenium2/Webdriver and the tests are written in java, with TestNG used as my framework. When I run each test by itself, ...
12
votes
4answers
35k views

Automation and bypassing Google No CAPTCHA reCAPTCHA

My client want to use Google's No CAPTCHA reCAPTCHA (NCRC) to prevent robots completing an application. QA have automated this process, but if the business introduce NCRC into the form, then this ...
12
votes
3answers
5k views

Random test data: How to do it right?

Several months ago I started an experiment in test automation. Each test, before it starts, picks up random test data. For instance, when my test requires a user with administration privileges, I draw ...