Clicking on a tab using selenium not working

Question

I'm trying to go through this site "https://www.dream11.com/cricket/create-team/1278/17702" and once this site is loaded, I want to click the skip button (which I can do) and then go on each of the individual tabs "WK", "BAT", "AR" and "Bowl" and click them and then scrape data from them once they are clicked.

But I can't seem to figure out any way of clicking the individual tabs using find_element_by_xpath, when I've tried various different class settings and still can't figure out.

I know another way would be to actively move the mouse to find the coordinates or to do a search by image but was wondering if there was a simpler way or if I was missing anything basic.

Here's the code I have currently.

from selenium import webdriver 
import time
from selenium.webdriver.common.by import By 
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC 
from selenium.common.exceptions import TimeoutException
import urllib

option=webdriver.ChromeOptions()
option.add_argument(" - incognito")
browser = webdriver.Chrome(executable_path=r'C:/Users/mohammadh/Downloads/chromedriver', options=option)

browser.get("https://www.dream11.com/cricket/create-team/1278/17702")

timeout=20

try:
    items = WebDriverWait(browser, timeout).until(EC.visibility_of_element_located((By.XPATH, "//button[@class='btn btn--flat btn--bordered btn--bordered--white text-color--white']")))
except TimeoutException:
    print("Timed out waiting for page to load")
    browser.quit()

element = browser.find_element_by_xpath("//div[contains(@class,'spotlight_skip_button spotlightSkipButton_4b62a')]")

element.click()
WK = browser.find_element_by_xpath("//div[contains(@class,'createTeamTabTitle_99a5a')]")
WK.click()
BAT = browser.find_element_by_xpath("//div[contains(@class,'createTeamTabTitle_99a5a')]")
BAT.click()
AR = browser.find_element_by_xpath("//div[contains(@class,'createTeamTabTitle_99a5a')]")
AR.click()
BOWL = browser.find_element_by_xpath("//div[contains(@class,'createTeamTabTitle_99a5a')]")
BOWL.click()

Any help would be greatly appreciated.

Answer 1

Use this:

WK = browser.find_element_by_xpath("//div[contains(@class,'createTeamTabTitle_') and text() = 'WK']")
WK.click()
BAT = browser.find_element_by_xpath("//div[contains(@class,'createTeamTabTitle_')and text() = 'BAT']")
BAT.click()
AR = browser.find_element_by_xpath("//div[contains(@class,'createTeamTabTitle_')and text() = 'AR']")
AR.click()
BOWL = browser.find_element_by_xpath("//div[contains(@class,'createTeamTabTitle_')and text() = 'BOWL']")
BOWL.click()

Answer 2

The problem is that this xPath: //div[contains(@class,'createTeamTabTitle_99a5a')] finds more elements than just one you want to click on:

The solution would be to find a different xPath, something like: //div[contains(@class, 'createTeamTabsContainer')][1] for the first tab, //div[contains(@class, 'createTeamTabsContainer')][2] for the second and so on.

For locating elements in a browser and checking xPath and what elements are returned by a particular xPath, I use Try XPath add-on in Firefox. You just type in your xPath and if it matches some elements on the site, they will be highlighed just like you see in the screen couple of lines above. It's priceless for such kind of problems.