2

I need to write a crawler that crawls throughout exercises available on my website (1200 exercises) and save a copy of DOM in my local as I have to use that DOM to create an exercise DVD. Can anyone please suggest a way to do it?

Please see the screenshot to get more clear understanding. enter image description here

  • 2
    What does this have to do with testing? – SlightlyKosumi Nov 16 '16 at 8:19
  • I'm using selenium webdriver to write a crawler that's it.. and trying to figure out if there is any way to do it using java/C# or any inbuilt method in selenium. – Pankaj Dubey Nov 16 '16 at 11:57
  • I would look into tools like Jsoup it's a highly effective HTML parser – DEnumber50 Nov 17 '16 at 16:14
1

You can try to dump page source and parse it, or dump only element source.

Page source to pageSource variable (Java):

String pageSource = driver.getPageSource();

Element source to elementSource variable (Java):

WebElement element = driver.findElement(By.id("id"));
String elementSource = element.getAttribute("innerHTML");

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.