0

I'm working on a project with www.NJMLS.com. I'm try to extract email of agent name "Edward An":

import dryscrape as d
d.start_xvfb()
br = d.Session()
br.visit('http://www.njmls.com/members/index.cfm?action=dsp.results&city=&county=&nametype=firstname&name=Edward&x=70&y=33')
#open url and try to extract the email of Edward An

Now how to get email of "Edward An". If I try:

for x in br.xpath("//*[@class = 'realtor-info']/*")
# selecting all elements in <div class="realtor-info">....</div>
 if "Edward An" in x:
  print x['herf']
#Not responding

Whats wrong with my script?

if "Edward An" in x.text()
 print x

Is also not working.. Still No Error and No response

  • I think the issue is if "Edward An" in x. Looks like x is not a text but a web element. Try to change to if "Edward An" in x.text:. Let me know if it works so that I add this answer since currently I am not sure if it is correct. – Alexey R. Jan 28 at 11:30
0

Have a look at the example here - https://dryscrape.readthedocs.io/en/latest/usage.html

Looks like you need to start the session object by giving it a base url

br = d.Session('http://www.njmls.com')

and then when you call visit give in the URI

br.visit('/index.cfm?action=dsp.results&city=&county=&nametype=firstname&name=Edward&x=70&y=33')

your if statement seems unlikely to work as its implying that X is is a string when its likely to be an element object but it seems there might be more issues.

  1. Does the page require a login ?
  2. Does the server let you visit that page ?
  3. Does the server accept city and county being empty ?

You should be able to add some error handling or use a debugger (pycharm has awesome python debugger integration) or logging statements to see what else is happening.

My guess is that you are not getting the page you expect or it is not loading.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.