I've used Selenium IDE to test Drupal (PHP) sites but want to create more advanced tests with loops and such. I hear Selenium WebDriver is all the rage, but when I google it, all these terms pop up and make me confused:

Selenium 2 API
(Google) WebDriver
Selenium WebDriver
Selenium Grid
Sikuli Script

I'm wondering about how all these fit together in the grand scheme of unit testing, and, more specifially, Selenium WebDriver. Like, what's the difference between them, which are comparable (do about the same thing), where do they overlap, what can x do that y can't,

  1. Can I use all of them to test PHP sites?
  2. Which could I (and should I) use in conjunction (and why) and
  3. Would I need additional software to make that work (if so, what software),
  4. When should I use what,
  5. Is Selenese something I use instead of PHP when coding tests and if so, what's the advantage of that
  6. Are there any other tools that I should use in conjunction with Selenium Webdriver?

Sorry if the question is too broad or to nooby (please critizise constructively and give me a chance to rephrase my question before handing out too many minus points---this is my first question in this forum). Of course I'm not expecting anyone to answer all of this, but please chip in if you think you can clear my confusion in any way. By the way, my programming skills are at an entry level so please keep that in mind when explaining.

  • You are correct, this is a broad question. More accurately, it could be interpreted in a broad way. JKS's approach of grouping them into categories helps to narrow the question significantly. – corsiKa Nov 16 '12 at 15:56
  • 3
    One other thing I would say is that the hallmark of a great question (as in, what separates a great question from a mediocre one) is exhibition of prior effort. So when you have the list of things that you don't know what they are, for example, a great question says "I saw all these things - here's what they appear to be from my reading about them" as well as any connections you draw from that. This helps anyone who answers understand more about what -you- understand, and avoids covering things you already know. Make sense? – corsiKa Nov 16 '12 at 15:57
  • 1
    Welcome to SQA, Johanna. I am downvoting this because of its breadth. It is a reasonable question for a beginner to ask, but in SQA we encourage questioners to be more specific and to present evidence that they have researched the question beforehand. As you learn more, please feel to follow up with additional (and more specific) questions. – user246 Nov 16 '12 at 16:49
  • Thanks! You're of course right, I should have been more clear about what I already knew. I had a hard time knowing where to take my first bite of the elephant and went a little lazy. The answers I've gotten have been great. I'll have some more time to read and reflect tomorrow and will be sure to follow up. – Johanna Nov 17 '12 at 16:11

"How to eat an elephant? One bite at a time." --- The scope of your question is so wide, that it is not possible to shows all subtle differences between different technologies you list. It would be more useful for you to focus on a single problem (e.g., interaction with Web browser) and then search for technology that helps you solve this problem. Therefore, I decided to center my answer around generic problems the listed technologies try to solve.

I posted my answer as community wiki, so others can improve it. Many of the technologies you listed I know only superficially, so I hope others will add more here.

Unit testing

Test framework enables execution of test cases automatically and reporting test results. Test framework usually provides a set of assertion predicates (e.g., isEqual) that will be used to compare actual system behavior with expected behavior (output) you define. Having tests cases with similar setup you may group them in test suites.

The family of frameworks for unit testing, i.e. testing of single components and functions, is known as xUnit. JUnit and NGUnit are implementations of such framework of Java language. phpUnit and SimpleTest would be for unit testing of PHP code pieces.

Interacting with external systems

During test execution, a test case implementation passes input data to the system under test and gets actual output data in return. Systems under tests cannot be called directly, when they expose different interfaces (e.g., SOAP). Hence, to speak to such a system, test case must use "glue", i.e., test driver that uses the protocol understood by the system. A test driver make easier for you to interact with such systems: prepare input data, parse output and make assertions on them.

  • SOAPUI: Talking to Web services (exposing SOAP or RESTful interface), Web resources (HTTP interface), JMS queues, backend services for Flash/Flex applications (AMF interface).
  • HTTPUnit, HTMLUnit: Interacting at low level with HTML Web sites and Web resources with HTTP interface
  • Selenium WebDriver, Google WebDriver: Talking to Web browser (IE, FF, Chrome).

Integration and system testing

Some of those test drivers, like HTTPUnit and WebDrivers, can be used together with xUnit frameworks to automate completely testing process. Yes, xUnit frameworks were originally designed for unit testing, but currently are often used for launching integration and system tests.

Programming Languages

Groovy is a programming language for the Java VM. If you write APIs in Groovy, you might choose to test those APIs in Groovy too. You might also choose to test Java APIs in Groovy, but there are other JVM-based languages that you could use to test Java APIs as well.

To quote its Wikipedia page, "Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots) of the software under test" and "Sikuli Script is a visual scripting API using Jython to create automation scripts."

Selenese is a set of commands for writing Selenium tests in tabular form. You can read more about it in this StackOverflow post.

  • Many thanks @dzieciou for this great answer. The only big question mark I have left now is about SoapUI - could it be used in conjunction with Selenium WebDriver or Selenium WebDriver+xUnit and would there be an advantage to that? Or are they entirely separate things, not used together? – Johanna Nov 18 '12 at 23:04
  • Using SOAP UI and Selenium would make sense if you want to access both Web site (with Selenium) and Web service from the same xUnit test case, e.g. you want to check what changed to UI after calling Web service. To call Web service you use Web service client stub generated with SOAP UI. Another case is to organize tests together and run both Web site/Selenium test cases and SOAP UI cases from the same xUnit test suite. – dzieciou Nov 19 '12 at 6:47
  • What I haven't wrote is that SOAP UI GUI enables writing complete tests (invocations, assertions) for Web service. You may think of it like Selenium IDE for Web services, but instead of recording you define data flow and its conditions. – dzieciou Nov 19 '12 at 6:50

Let me try to explain this in simple words so that we don't get confused with different jargon used with Selenium.

  • Selenium 2 API, (Google) WebDriver, Selenium WebDriver are same things. Why we call it Selenium WebDriver because it is combination of Selenium (from ThoughtWorks) and WebDriver (from Google). We call it Selenium 2 also since WebDriver provides backward compatibility to Selenium 1. So if you want to start with automation, better to start with WebDriver.

  • Selenium Grid is used for running automation in parallel mode. Suppose, we want to run a test on 3 browsers; IE, FF, Chrome, we use Grid.

  • Selenium/Webdriver supports different testing frameworks (FW). These are different Unit Testing frameworks Selenese PHPUnit, NGUnit, HtmlUnit, HttpUnit, JUnit. Depending on the application, different FW have different benefits. We should thorough study our application and then choose FW appropriately.

  • Selenium/Webdriver also supports different languages. So we can choose any language to write our script.

My personal opinion is: Selenium/Webdriver is best fit with Java language as it is written in Java and with we can use testing framework Junit or testNG.

| improve this answer | |
  • I would also add that Groovy isn't so much a testing tool but simply a java-esque scripting language. It's very easy for a Java programmer to pick it up! – corsiKa Nov 16 '12 at 15:29
  • Also, one minor thing would be that you'd use XUnit to test X language. So PHPUnit for php unit tests, JUnit for Java Unit tests, etc. – corsiKa Nov 16 '12 at 15:32
  • 1
    @corsiKa: Not necessarily. You may use JUnit+Selenium to test Web sites made in PHP, because you do not interact with PHP directly. I.e. you do not use JUnit for unit testing here. – dzieciou Nov 16 '12 at 15:34
  • @dzieciou One could also use Selenium to keep themselves logged in to StackOverflow to farm the Fanatic badge. That doesn't mean it's what it was designed to do. – corsiKa Nov 16 '12 at 15:58
  • @corsiKa: What's the actual disadvantage of using JUnit to test Web pages wrote in PHP? xUnit has "unit" in name but lot's of testers uses it for integration and system tests, e.g. combine JUnit with Fitnesse. – dzieciou Nov 16 '12 at 16:01

I'm wondering about how all these fit together in the grand scheme of unit testing, and, more specifically, Selenium WebDriver.

Webdriver is used for end to end, user acceptance testing, running in a real browser, automation. This is usually not unit or white box testing, it is black box testing.

It is used with other technologies and approaches such as

  • Selenium 2 API
  • (Google) WebDriver
  • Selenium WebDriver
  • Selenium Grid
  • Selenese
  • Watir

On exception to this is a viual based tool for browser automation:

  • Sikuli Script

Unit testing on the other hand is not about browser automation (though it can be about unit testing javascript). Products in this category:

  • JUnit
  • PHPunit
  • NGUnit
  • HtmlUnit (headless browser)
  • httpUnit (headless browser)
  • SimpleTest
  • soapUI


  • Groovy - a language for the java platform

So for a PHP developer I would look at PHPunit for unit tests and then one of the selenium based products. Usually you pick one of the main language bindings - java, python, c#, ruby, javascript, etc. to go with it.
For php I would also explore Behat and Mink as detailed at http://www.seleniumhq.org/about/platforms.jsp#programming-languages

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.