I need to write a crawler that crawls throughout exercises available on my website (1200 exercises) and save a copy of DOM in my local as I have to use that DOM to create an exercise DVD. Can anyone please suggest a way to do it?
-
2What does this have to do with testing?– SlightlyKosumiCommented Nov 16, 2016 at 8:19
-
I'm using selenium webdriver to write a crawler that's it.. and trying to figure out if there is any way to do it using java/C# or any inbuilt method in selenium.– Pankaj DubeyCommented Nov 16, 2016 at 11:57
-
I would look into tools like Jsoup it's a highly effective HTML parser– DEnumber50Commented Nov 17, 2016 at 16:14
Add a comment
|
1 Answer
You can try to dump page source and parse it, or dump only element source.
Page source to pageSource variable (Java):
String pageSource = driver.getPageSource();
Element source to elementSource variable (Java):
WebElement element = driver.findElement(By.id("id"));
String elementSource = element.getAttribute("innerHTML");