10

Background:JavaScript based automation frameworks heavily depends on asynchronous execution, promises, callbacks, anonymous functions, etc which are inherently complex and probably an overkill for a straight forward task like UI test automation where we identify objects based on some locators and perform operations.

Problem Statement: why should we choose JavaScript as a language for UI test automation considering it's fundamental asynchronous nature? What are the trade-offs of choosing it as a language for test automation?

Disclaimer: I have designed a couple of large automation frameworks using the protractor for multiple applications, but I want to understand the trade-off of choosing JavaScript as a language for UI test automation?

Note: I raised this question initially on Stack Overflow but as did not get much response so thought of this might be a better platform as focused on QA only.

Other Similar Questions: Also I have seen other close question(as remarked by Peter),but I think that question is based on the premise where we already have large test base in python/other language but I am taking one more step back where we even don't have any tests in any language.Should we even then consider JavaScript as a language for UI test automation in first place considering its inherent complexity and quirkiness for straight forward task like UI automation???

  • 1
  • or explain why your question is different – Peter M. - stands for Monica Jan 25 '17 at 15:00
  • 1
    Peter,I think that question is based on the premise where we already have large test base in python/other language but I am taking one more step back where we even don't have any tests in any language.Should we even then consider JavaScript as a language for UI test automation? – Vishal Aggarwal Jan 25 '17 at 15:17
  • 1
    Fair point. In that case it is NOT a duplicate. Personally me, I strongly prefer Python over JavaScript, and would do almost anything to avoid JavaScript. I got burned with similar language (Perl) before, and prefer not to have same experience again. – Peter M. - stands for Monica Jan 25 '17 at 17:02
  • Thanks Peter for understanding my point.I also strongly prefer Python/Ruby over javascript specially where I am dealing with only UI test automation.I do not see the point of using javaScript with its all quirks for an task like UI test automation. – Vishal Aggarwal Jan 25 '17 at 17:08
5

Yes - For Unit Testing Javascript

I've been working with this concept over the last few years and I've come to the following, multi-tier approach:

  • Unit testing on the backend. For example with Ruby or Java on the server code.
  • User acceptance testing on the UI using Selenium and a programming language such as Ruby with Capybara
  • Unit testing on the front end with Jasmine, Konacha (Mocha/Chai), etc.

So I've most recently seen the javascript testing as that third item. It mocks and stubs the actual web page with a fixed fragment of HTML for the js to work against - this is what makes it a unit, not integrated or feature test. I was resistant to this initially ("but what about if the pages changes?") until I understood this classification and that the web page change scenario would be covered by the Selenium tests.

Jasmine: A behavior-driven development framework for testing JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM. And it has a clean, obvious syntax so that you can easily write tests.

Protractor: An end-to-end test framework for Angular and AngularJS applications. Protractor runs tests against your application running in a real browser, interacting with it as a user would.

Mocha: A feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing simple and fun.

Chai: Chai is a BDD / TDD assertion library for node and the browser that can be delightfully paired with any javascript testing framework.

QUnit: QUnit is a powerful, easy-to-use JavaScript unit testing framework. It's used by the jQuery, jQuery UI and jQuery Mobile projects and is capable of testing any generic JavaScript code, including itself!

  • Thanks Michael for sharing an interesting idea.Could you please elaborate it more on how you are using JavaScript on frontend for unit testing,may be a link or couple of basic code examples will be very useful.Thanks once again this is something quite new and interesting to me. – Vishal Aggarwal Apr 29 '17 at 10:31
  • Added links to the various libraries. You'll need to read up and ask more (specific) questions as needed. Can't elaborate more in this specific question. – Michael Durrant May 2 '17 at 11:19
  • We use to specifically test the javascript code we write – Michael Durrant May 2 '17 at 11:20
  • 1
    Thanks Michael for the information however I am already aware as I am working on e2e testing using protractor and used/explored these libraries however my specific question was around unit testing from UI using JavaScript.May be I should ask that as a separate question...Anyways thanks again for giving an interesting idea to explore. – Vishal Aggarwal May 2 '17 at 11:33
  • 1
    +1 @AlanLarimer added QUnit thx! – Michael Durrant Apr 3 '18 at 19:00
4

We have recently started with writing WebDriver tests in JavaScript and I have to agree that the asynchronous execution is adding an unneeded complexity.

Pro's:

  • Same test-runner as other (front-end) unit-tests
  • Everyone knows JavaScript (e.g. front-end development with Angular, Vue or React)
  • If you use JavaScript (e.g. NodeJS) also on the back-end, sticking to one language makes sense

Con's:

  • Asynchronous execution complexity
  • Hard to Google issues as most Selenium/WebDriver stuff is in Java (adding JavaScript barely helps)

We picked JavaScript because Java is a no go as our back-end is in C#. Also experience learned me not to add more programming-languages to the stack, certainly not any the developers had no experience with. C# WebDriver issues are just as hard to Google as JavaScript. Also I think C# bindings are not on par with Java and JavaScript as they get less commits on github.

JavaScript UI test frameworks:

It is possible to hide the Async complexity with JavaScript test frameworks like:

For now we chosen to not research or use these frameworks mainly because one of our other teams decided to use plain selenium-webdriver for their tests.

With Nightwatch.JS I created a minimal test (with PageObject model) and it worked pretty well.

Some of these frameworks look promising and might be worth researching if you want to use JavaScript.

  • Thanks Neils for detailed answer however I am still not clear on your answer on few points.Are you suggesting if we are using JavaScript in front end & back end in application development then we should go for JavaScript in test automation as well? – Vishal Aggarwal Jan 26 '17 at 23:02
  • 2
    Yes, I am suggesting that. Mainly because context switching between programming languages is expensive, unless you are very proficient in a language. For me using Python would mean Googling syntax every-time I would write an UI-test and as we only write 1 or 2 UI-tests per feature (we try to test code with unit-tests mainly). I would never become very good in it. Even if it was 10 times simpler our team would be slower I think. Also our our current IDE does not support Python, so we would need to use two IDE's. – Niels van Reijmersdal Jan 27 '17 at 6:11
  • 1
    I have noticed that adding an extra programming language to a team is not a good idea, certainly not for the more junior team members. Now if you have dedicated testers, maybe it works, but still it is handier if senior developers can assist them in creating tests. It just so much handier if everyone uses a languages that they are familiar with. In the end let the team decide. They will need to maintain the tests for a long time. – Niels van Reijmersdal Jan 27 '17 at 6:14
4

As you mention, the asynchronous chaining API seen in "selenium-webdriver" NPM package and also in nightwatch, protractor, testcafe, cypress, intern and wd.js etc can be quite hard to understand.

Luckily there is a way to write E2E tests in Node and still get an API that feels intuitive and synchronous, namely http://webdriver.io

Webdriver.io is able to do this because it uses coroutines from the node-fiber package to suspend execution until the async webdriver operation has completed.

I also wrote an in-depth comparison of Javascript E2E frameworks that covers exactly this aspect of E2E testing is JS. It also has a section that discusses why JavaScript E2E is a good idea vs writing testcases in Java.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.