I am helping my friend with some medical records verification. In short, I have to through hundreds of medical records and pinpoint certain ones for verification. I want to be able to search through these hundreds of records. The problems is that the medical records can only be viewed in a protected and private HTML document, which means I cannot extract them to a pdf. I believe that the documents are in from a asp.net database.

My best hope is to literally have my computer take a picture of the entire webpage. The computer screenshots, scrolls down, screenshots, scrolls down, screenshots .... etc. Does anyone know any program that will automate this iterative process. Once I have a pdf, I can easily parse it into searchable text.

Any help is appreciated.

  • 1
    I'm not sure what this has to do with testing, but it's nice to hear about. – user246 May 15 '15 at 11:52

You can use Selenium Webdriver to automate this task. It can read the DOM of the HTML, and extract information from it; it controls a browser the same way you do, so anything you can do manually it can do (with a few exceptions). I would suggest looking into that rather than taking screenshots and parsing them as PDFs.

  • Agreed. Surely extracting the text from HTML elements is less effort than extracting text from a PDF of a screenshot. @Yamikuronue can you actually scroll a browser page with Selenium? – user246 May 15 '15 at 11:53
  • @user246 You shouldn't have to, since it can parse the DOM, but yes, you also can scroll the page. – Yamikuronue May 15 '15 at 12:29

Selenium 2 (webdriver) is designed to do exactly what you need. It allows you to simulate whatever real users can do with a browser: log to a website, click on buttons, fetch content of HTML elements, and analyze the text using any tools necessary. You can use different kinds of locators to identify HTML elements to process (ID, name, CSS class, link text etc).

Advantage of having text from HTML elements over images is obvious: much easier to process programmatically.

Selenium has bindings for many languages, but IMNSHO Python would be the best bet, because it also has excellent text processing features. Including regular expressions, if you want superweapons, but you can mostly avoid it by smart parsing and/or splitting. Unless you are expert in another language, of course.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.