How to get src link using Selenium?

Question

The task is simple: to extract the link of an audio of pronunciation for a word from a Yahoo Dictionary Webpage: e.g. Yahoo's Dictionary@ "real"

Using "Chropath", I can locate the Xpath of the element that contains the ".mp3" src link. The Xpath is

//div[@class='compText ml-10 d-ib']//span[contains(@class,'d-ib dict-sound va-mid audio-0')]

However, when I try to use the below coding, it seems that the find_element_by_Xpath method returns nothing. (Remarks: note the "SoundURL " part)

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
import os

# instantiate a chrome options object so you can set the size and headless preference
options = Options()
options.add_argument("--headless")
word = "real"

print("start driver...", end='')
driver = webdriver.Chrome(options=options, executable_path="F:\Python_Module\chromedriver.exe")
driver.get('https://hk.dictionary.search.yahoo.com/search?p='+ word)


Pronunciation = driver.find_element_by_class_name(" fz-14").text
Meaning = driver.find_element_by_xpath("//div[@class='compList mb-25 ml-25 p-rel']//ul").get_attribute('innerHTML')

SoundURL = driver.find_element_by_xpath("//div[@class='compText ml-10 d-ib']//span[contains(@class,'d-ib dict-sound va-mid audio-0')]").get_attribute('innerHTML')


print("Print Function started")

print("begin pronunciation")
print(Pronunciation)
print("begin pronunciation")

print("begin Meaning")
print(Meaning)
print("end Meaning")

print("begin sound")
print(SoundURL)
print("end sound")

As shown in the screencap, I would like to extract the following element:

<audio src="https://s.yimg.com/bg/dict/ox/mp3/v1/real@_us_2.mp3" xpath="1"></audio>

Answer 1

The problem is that for 1-2 seconds, the span element is present in the DOM, but the audio child element hasn't been injected yet.

You can verify this by adding a time.sleep(3) before grabbing your soundURL var.

How you want to solve this problem in your script depends on your requirements. There's basically 3 sets of options:

1. time.sleep() - simple but inefficient
2. selenium implicit wait
3. selenium explicit wait - more complicated to setup but efficient

If you want to learn about Selenium waits, refer here: link

With a wait strategy, you'll probably want to find the audio element itself rather than getting it thru the containing span element. Here's an example along those lines (using implicit wait):

driver.implicitly_wait(3)
sound_url = driver.find_element_by_tag_name('audio').get_attribute('src')
# sound_url now contains 'https://s.yimg.com/bg/dict/ox/mp3/v1/real@_us_2.mp3'