6

Take a very specified example of web application testing with selenium: It seems obvious that (almost) everyone is suggesting using element ID as the best locator strategy, and that your GUI-automation tasks shouldn't be depend on the HTML structure of the web app under test etc.

I'm getting the feeling that we are trying so much to automate things efficiently, and hence forgot about the testing aspect: Using element ID, how do I make sure that, for example, my "Login" button is exactly at the top of the page and not just "somewhere" on the page? And isn't it the page structure that I would like to test to make sure that the GUI-elements are following a specified, business-related order?

Nevertheless, the recommended best practice works great when I'm only concerned with functional testing of my web app. Of course!

Improve this question
3
  • 1
    On top of beatngu13's excellent answer:: people comment that you should use the css selector of an element instead of the xpath, because there may be a structural change without a visible change to the philosophy of the page, yet you ll still be needing to re-write all your xpaths, if for example your designer changed a div to span on the top of the page etc. Also,what you re missing is that you CAN use css selectors to essentially describe the page structure and that is in the desired business-related order. That being said,automation is intended as a functional testing tool.
    – Leon
    Commented Sep 12, 2017 at 6:31
  • @Leon Please kindly share me some resources for using CSS to describe the page structure in desired business order. I've never thought about that :D
    – nhle
    Commented Sep 13, 2017 at 5:40
  • 1
    From a quick search, those are some to get you started and get some ideas saucelabs.com/resources/articles/selenium-tips-css-selectors w3schools.com/cssref/css_selectors.asp Basically the premise is using siblings(direct or not)/descendants(direct or not)/ascendants(direct or not) as well as first/last/n-th child an element "lower" in the page through matching what should be "above" it in the DOM. Hope this gives you an idea how to approach this. p.s. ofc this will lead to complex & big selectors so consider if you really need to check the business order while testing.
    – Leon
    Commented Sep 13, 2017 at 7:31

5 Answers 5

Reset to default
13

There is definitely a focus on test automation, not automated testing—simply because the latter is hard to do. I think this is related to the testing vs. checking debate (started by James Bach and Michael Bolton):

Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc.

Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product.

We can easily let machines do automated checks, but automated testing is still an open research question. This is also why good manual testers are priceless; to quote a tweet by Adam Goucher:

The worst thing you can do productivity wise is take a talented manual tester and turn them into a crappy coder.

IMHO a solid QA needs both. That is, (smart) test automation and (smarter) manual testing.

Nonetheless, there are tools you can leverage to address some parts of the things you have mentioned. Whereas Selenium is perfectly suited for functional testing, you can use the Galen Framework to check the layout of your web app. Moreover, Depicted is a neat tool to reveal visual regressions (have a look at this introduction).

Improve this answer
2
  • 1
    +1 for Line:- "This is also why good manual testers are priceless || The worst thing you can do productivity wise is take a talented manual tester and turn them into a crappy coder."
    – Nitin Rastogi
    Commented Jan 3, 2018 at 5:57
  • +1 for Galen. Coders and non-coders alike can use it for UI testing. It also works well for functional and end-to-end testing.
    – Alan Larimer
    Commented Feb 24, 2018 at 20:58
2

As usual, for different tasks you use different tools.

For (automated) testing the functionality, you use Selenium.

For (automated) testing of the "look and feel", you use "compare the images" tools like Applitools.

Or you can wait for General Artificial Intelligence. Problem with that approach is that after it's arrival, if it is self-improving, some researchers are concerned we (humans) have just few weeks to be replaced as a species on the top of the food chain, so after GAI we will not have problems with any kinds of testing anymore :-)

Improve this answer
1

To me what suffers most in testing when we focus more on automation is-

What do most of us do after learning a automation tool or strategy!

  • Trying to apply it at every available opportunity we get.
  • Searching for possible areas where we can use that strategy.
  • Changing the problem statement to suit our solution which is easy to forget the problem statement.

We should get ourselves out from these and then focusing on automation would not be questionable any more!

Improve this answer
1

I love automation and what it has done for QA side of things. It ironically outsourced all the offshore testing shops that was industry-craze ten years ago back to onshore. I love push-button sanity checks on releases.

That said, I spend about a third of my time debugging my own code anymore as I do on the code I'm allegedly QA for. Who is QA for original QA guy when original QA guy is basically a developer of a side-app bundled into the solution anymore? Industry hasn't figured out org-chart on that yet.

Improve this answer
1
  • I do fully agree with you!
    – nhle
    Commented Feb 2, 2018 at 14:10
1

We need to evaluate what we are trying to automate in the first place. Using the id of the element is one of the best ways to locate an element in Selenium. If we use the test to validate whether a button is present, clickable or takes us to the correct page on clicking; it could be a valuable test. Especially, if it has to be repeated a number of times during our testing. In terms of using it for checking the position of a button on a page, would be a BAD IDEA as it is going to result in flaky tests due to constant changes in element position.

This is where we testers need to come in and do some exploratory testing to ensure the GUI elements are in the right location and whether it makes business sense to have it there. Meanwhile our automated tests can be running on the side doing its own thing.

I view automation as complementary to the overall testing effort. It is a tool/aid to help out in testing. In my experience, I have used automation when there are number of mundane tasks/scenario that have to be repeatedly tested. So I write automated scripts to test these repeatable tasks while I concentrate my efforts on exploring other critical aspects of the application manually. I am a big proponent of Session Based Exploratory Testing.

Also, another reason why we may get a feeling that we are focusing more on automation is because in the recent years, CI/CD Integration, DevOps and other approaches/practices have become popular which needs automation to begin right from the start of the SDLC. They are trying to align with the "Shift Left paradigm". Sometimes during these efforts people forget why they are doing automation and what they need to test in the first place.

In summary, I think automation is a tool to help in the overall testing effort. We need to weigh the cost vs value of doing automation and also understand what we are trying to automate. Automation definitely has a place in testing when used wisely and when constantly reminding ourselves why we are automating something compared to just manually testing it.

-Raj

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.