0

I want to scrape this marketplace data https://www.tokopedia.com/search?st=shop&q=dapur

but to get the data, that I want, I must go through pass some layers in the page.

In this case this web have 3 layers, 1st layer is the shop catalogue or the Starting URL page:

(in this page users must click the Lihat Toko button to go to the next layer):

enter image description here

2nd layer, is the Shop profile (In this page users must click the Info Toko button to go to the final layer/pop up content):

enter image description here

Final layer, is the pop up content. I want to get the data from this pop up box:

enter image description here

This is my code:

from selenium import webdriver

chrome_path = r"E:\Chromedriver\chromedriver.exe"

driver = webdriver.Chrome(chrome_path)

driver.get("https://www.tokopedia.com/search?st=shop&q=dapur")

driver.find_element_by_xpath('//div[@class="css-gmuwbf"]//button[1]').click()

driver.find_element_by_xpath('//div[@class="css-ais6tt"]//button[3]').click()

I've run the code, but always have an error messages.

3
  • 2
    Agung, please show the exact error messages you are receiving. Most likely you have problem of waits, because the page has AJAX behavior between clicks. Additionally, Selenium may not be the necessary tool for this; have you consider simply making calls to the Tokopedia API? developer.tokopedia.com/openapi/guide/#/shop Jul 22, 2020 at 11:22
  • the error message: Jul 23, 2020 at 2:20
  • NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[@class="css-ais6tt"]//button[3]"} (Session info: chrome=84.0.4147.89) Jul 23, 2020 at 2:34

1 Answer 1

2

I think you have to switch the window first then only you will be navigate to the second-page (layer):

please use below piece of code to switch your window :

public boolean switchToWindow(String windowname) 
    {
        
        Set<String> windowsname=driver.getWindowHandles();
        try {
        windowsname.forEach(window->{
            if(window.equalsIgnoreCase(windowname))
            {
                 driver.switchTo().window(window);
                 
            }
        });
        return true;
        }
        catch(NoSuchWindowException e)
        {
            return false;
        }
        
    }

now please use the below piece of code to switch the alert and get the UIData :

public String getAlertData()
    {
        try {
        WebDriverWait wait=new WebDriverWait(driver,30);
        wait.until(ExpectedConditions.alertIsPresent());
        Alert alert =driver.switchTo().alert();
        return alert.getText(); 
        }
        catch(NoAlertPresentException e)
        {
            return null;
        }
    }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.