Differentiating between two items with the same class name using selenium

Question

As a learning project, I am trying to use selenium webdriver with python 3 to collect my shopping for the week from sainsburys website.

At the moment, the script will loop through a bunch of URLs for different foods, find the "add" item button by its class name and click it. If I have the item already, it will instead search for the "plus" button to add one more. I have to find each of these elements by class name rather than xpath, as if the item is on offer or something else, its positioning changes and I can't generalize it for each URL.

Below is my function for collecting the item.

def collect_item(url, old_url):
if url != old_url: #If you are collecting multiple items, dont reload the page each time.
    driver.get(url)
    time.sleep(2)
else:
    pass
try:
    try: #if getting item for the first time
        one_more_class_name = "pt-button__inc"
        click_add = driver.find_element_by_class_name(one_more_class_name)
        if click_add.is_displayed() == True:
            click_add.click()
            time.sleep(2)
        else:
            pass
    except: #if getting item for multiple times
        one_more_class_name = "pt-button__inc"
        add_class_name = "pt-button"
        click_add = driver.find_element_by_class_name(add_class_name)
        if click_add.is_displayed() == True:
            click_add.click()
            time.sleep(2)
        else:
            pass

    old_url = url
except:
    failed_urls.append(url)
    old_url = url
    pass

I run into an issue for two reasons:

1. If an item is unavailable, the item will not have an "Add" button. The script will continue to search the page for an add button, finding one eventually on the "suggested items" section as they have the same class name, and it will add a random item to my basket.
2. If an item I want to buy for the first time has a "suggested item" that is already in my basket, the function will click to add one more of the suggested item, rather than one of the items I wanted.

The first issue I can't solve yet. The second is due to the function looking for "plus" buttons first, but it's like this as, for some reason, the script will prefer to click the "add" button of a suggested item rather than the "add" button of the pages main item.

So how can I get selenium to search for a class name in only a specific region of the html, or have it only search for elements visible to a user?

Answer 1

I am not familiar with Python, but I can make some suggestions.

First, visit the Selenium documentation site. You can find examples and detailed documentation using Selenium with Python here.

Next, to answer your specific question, instead of searching by class name, look for something that is unique to the item/s you are searching for. You may need to find a parent element that contains the information you need, or you might need to search for an element containing specific text.

If you can find a distinct parent element, your next step is easy: you do something like this (pseudo code because I do not know Python)

target = parentElement.FindElement(By...)

You can call FindElement or FindElements on any web element, so this allows you to easily drill down to the item you want. Note that this will return any/all element(s) meeting your search criteria no matter how many nodes are between them.

If you have to use something like item text and you need to navigate up the node structure you would need to use XPath and a call something like (again, pseudo code - use the selenium documentation to find the correct syntax for the call in Python):

target = childElement.FindElement(By.XPath("parent::*"))

This will return the immediate parent of childElement, so you may have to make several jumps. An alternative is to wrap the call into a convenience method so you can do something like:

target = childElement.getParent().getParent().getParent()

I'd suggest starting with small steps: identify the closest element with a unique identifier to the element you want to use, then work out how to use that identifier. The Selenium python API documentation will help.

Note: I have no affiliation with Selenium, but I use it with C# in my test automation