I need to write a crawler that crawls throughout exercises available on my website (1200 exercises) and save a copy of DOM in my local as I have to use that DOM to create an exercise DVD. Can anyone please suggest a way to do it?
2What does this have to do with testing?– SlightlyKosumiNov 16, 2016 at 8:19
I'm using selenium webdriver to write a crawler that's it.. and trying to figure out if there is any way to do it using java/C# or any inbuilt method in selenium.– Pankaj DubeyNov 16, 2016 at 11:57
I would look into tools like Jsoup it's a highly effective HTML parser– DEnumber50Nov 17, 2016 at 16:14
You can try to dump page source and parse it, or dump only element source.
Page source to pageSource variable (Java):
String pageSource = driver.getPageSource();
Element source to elementSource variable (Java):
WebElement element = driver.findElement(By.id("id")); String elementSource = element.getAttribute("innerHTML");