Selenium, Python: Write, click next page loop

Question

I'm trying to learn selenium with python and have been trying to automate:

1. Search of the next button on the page.
2. Click on this button.
3. Save each HTML page from the next page in a file.
4. Repeat until the next button can't be clicked anymore.
5. Quit.

Unfortunately the button persists even when it cannot be clicked. Here is the difference between the two:

When clickable:

<input type="image" id="OptionsBar1_imgNext" title="Next" src="/images/toolbaricons/nextsmall.gif" onclick="changeSelect(1);return false;" value="">

When not clickable:

<input type="image" id="OptionsBar1_imgNext" title="Next" src="/images/toolbaricons/disabled/nextsmall.gif" onclick="changeSelect(1);return false;" value="" disabled="" readonly="">

The number of pages is not consistent so I can't define a range of pages. I have had success running through and capturing HTML until the end, but can't get the driver to quit. I imagine this is because I'm not understanding the loop function correctly.

Here is what I have come up with:

with open('{}.html'.format(file_name), 'w', encoding = 'utf-8') as file:
    file.write(driver.page_source)
while True:
    driver.find_element(By.ID, "OptionsBar1_imgNext").get_attribute("src") is "/images/toolbaricons/nextsmall.gif"
    driver.find_element(By.ID, "OptionsBar2_imgNext").click()
    with open('{}.html'.format(file_name), 'a', encoding = 'utf-8') as file:
        file.write(driver.page_source)
while False:
    driver.find_element(By.ID, "OptionsBar1_imgNext").get_attribute("src") is "/images/toolbaricons/nextsmall.gif"
    driver.quit()

I'm sure this is an elementary question and any help is appreciated!

Answer 1

Here's one way to achieve the desired result:

while True:
    next_button = driver.find_element(By.ID, "OptionsBar1_imgNext")
    src = next_button.get_attribute("src")
    if src == "/images/toolbaricons/nextsmall.gif":
        next_button.click()
        file_name = 'page_{}.html'.format(counter)
        with open(file_name, 'w', encoding = 'utf-8') as file:
            file.write(driver.page_source)
        counter += 1
    else:
        driver.quit()
        break

In this code, we use a while loop to repeat the process until the next button can no longer be clicked. Inside the loop, we find the next button using its ID, get its src attribute, and check if it is equal to "/images/toolbaricons/nextsmall.gif". If it is, we click the button and save the current HTML page to a file. If it's not, we quit the driver and break out of the loop. To keep track of the number of pages, we use a counter variable that is incremented each time a new page is saved.