1

I'm trying to scrape a javascript website. The website contains a list of products and I need to find out a specific product through this list. All product names are between strong brackets from an HTML perspective.

example: <strong>product name</strong>

I'm able to find only the first product in the list using this code below, how can I find other products ?

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time


my_url = 'https://order.koomi.com/comptoir-sainte-cecile-VwRWLYwwkd/fr/2018030601/takeout/menu/vins-rouges-36?fbclid=IwAR0jMjQtHoZyNLSlafvvPKHn9OkNMsCN4BdAjgfaigdVvyed3aZRS0Ij4tw'
    
    chrome_options = Options()
    chrome_options.add_argument("--no-sandbox")
    chrome_options.add_argument("--headless")
    
    driver = webdriver.Chrome("/home/pi/chromedriver")
    driver.get(my_url)
    time.sleep(5)
    #htmlSource = driver.page_source
    
    entries = driver.find_element_by_xpath('//strong')
    print(entries.text)
2
  • 2
    Welcome to the community. find_element_by_xpath() only finds 1 instance. Have you tried find_elements_by_xpath()? This will put all found entries into an array.
    – Lee Jensen
    Commented Dec 1, 2021 at 20:51
  • it does but instead of having product name it returns the code below for each product.<selenium.webdriver.remote.webelement.WebElement (session="64c8ef24cac37169cfd92572fc1937c4", element="83a11039-9c74-4e89-905b-d254b9796456")>
    – DBL
    Commented Dec 2, 2021 at 13:56

1 Answer 1

1

To get all the content within every <strong></strong> tag in a web page, here is a sample code in Java:

//Get list of web-elements with tagName  - strong
 List<WebElement> allProducts = driver.findElements(By.tagName("strong"));
 
 //Traversing through the list and printing its text
 for(WebElement product:allProducts){
    System.out.println(product.getText());
 }

You can use something on the similar lines in Python.

Edit:

I haven't tried this personally, but based on online search I've got this for Python,

list = driver.find_elements_by_tag_name('strong')

for li in list:
    print(li.text)

You can try and let me know if that works.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.