I need to write a crawler that crawls throughout exercises available on my website (1200 exercises) and save a copy of DOM in my local as I have to use that DOM to create an exercise DVD. Can anyone please suggest a way to do it?

Please see the screenshot to get more clear understanding. enter image description here

  • 2
    What does this have to do with testing? Nov 16, 2016 at 8:19
  • I'm using selenium webdriver to write a crawler that's it.. and trying to figure out if there is any way to do it using java/C# or any inbuilt method in selenium. Nov 16, 2016 at 11:57
  • I would look into tools like Jsoup it's a highly effective HTML parser
    – DEnumber50
    Nov 17, 2016 at 16:14

1 Answer 1


You can try to dump page source and parse it, or dump only element source.

Page source to pageSource variable (Java):

String pageSource = driver.getPageSource();

Element source to elementSource variable (Java):

WebElement element = driver.findElement(By.id("id"));
String elementSource = element.getAttribute("innerHTML");

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.