So I'm trying to make an automated test for a type of survey. Just imagine a bunch of questions each page with some options like "agree/strongly agree/etc..."

Now the issue is, while the questions themselves on each page are always the same, but their order of appearance is always random. So I've been trying to think of a way to possibly put this is an automated test. (it was a task given to me).

Since the order of the questions changes, the labels themselves change...and TBH I'm not super familiar with Selenium (I'm using IDE) so I'm not sure how I would go about creating a test for this, since every time the test is run the order of the questions changes on each page. Also it's worth noting that one page of questions can be entirely different next time...so even on the same PAGE the questions that appear isn't always the same.

Any ideas?

  • What do you want to test about this random page?
    – user246
    Sep 28, 2015 at 15:33
  • IDE will not help you much here. But using webdriver + programming language of your choice you can interrogate page about the questions, and write tests reflecting current questions as rendered on the page. I did similar tests for our system. For me, key was being able to generate complex data structures on the fly, which Python provided. Sep 28, 2015 at 15:40
  • Im wondering if the time invested is going to be worth it for this though, when it was assigned to me I don't think they realized the questions themselves come up in a random order unfortunately.
    – Mercfh
    Sep 28, 2015 at 15:41

1 Answer 1


The first question is - do the questions needs to be in a random order? Part of being a QA is to give feedback on product decisions.. if it doesn't make sense, why is it part of the product design?

Given that there's a good reason why they're random..

If all of them share a similar div/html element ID, then you can use a CSS selector to grab a list of the similar elements, e.g:

By questionLocator = By.cssSelector("div[id*='question-']");
List<WebElement> questions = getDriver.findElements(questionLocator);

You can then iterate through that list to answer your questions. If it's multiple pages, loop this per page, like so:

By questionLocator = By.cssSelector("div[id*='question-']");
List<WebElement> questions;

while (morePages) {
        questions = getDriver.findElements(questionLocator);
        for (WebElement question : questions) {
            // do something with question
        morePages = areThereMorePages();
        if (morePages) clickNext();

You'll have to code some extra logic to work out whether you're on the last page or not.

This approach only works if each question is indistinguishable from one another. It'll work pretty bluntly, and you can randomly fill in each question, but won't allow for any logic - nor will you be able to do any recall of your answers.

If you want to do something more complex, you will need to capture some identifying information from the web element. I've built tests for dynamic questionnaires, and the approach I took was to parse a unique question ID from the HTML element, and then query the application API/Database with that question ID, returning a Question object with which I could work out how to answer that specific question. This allowed a high degree of complexity, including multiple answers, optional answers, different types of question, quiz completion percentages..

  • Sadly unfortunately due to design decisions it's basically staying that way. And while the order changes so does the label (ie like question_81 for the first page may be question_5 labels on the next pass). And theirs like...200 possible questions it can grab from.
    – Mercfh
    Sep 28, 2015 at 16:05
  • I would ask the developer if they could attach unique IDs to each question element, as it would make testing far easier. Not every organisation takes testing seriously though, so you may have to be creative. It may be possible to do a pattern match against the text of the question to uniquely identify the question, and then try to query the API/database with that - or at least store the text of the question and the answer you inputted for recall later. If they don't allow you any way to identify the question, your logic will have to be blind for each question. There's no way around it. Sep 28, 2015 at 16:14
  • I talked to a Co-worker and he said there may be a way to just "predict" how many questions there are (as the test) but the thing is.....I still need to be able to "select" them to continue on with the survey. So im really not sure if theirs a good way around this.
    – Mercfh
    Sep 29, 2015 at 18:23
  • As said, if all of your questions are the same (strongly disagree -> strongly agree) you could just randomly fill in each question. Is it necessary to fill in specific answers for specific questions? If you need to get results back afterwards (e.g. 50% agree 50% disagree) you could still store each of your questions/answers in a list/hash/etc as you answer them, and then do queries on that. Sep 30, 2015 at 9:29
  • yeah I think that would work, im using the IDE however so im not used to Selenium completely and writing rspec so Im not even sure how you would do that. Is there some way to use the IDE part of the way then fill in code?
    – Mercfh
    Sep 30, 2015 at 13:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.