Data-driven approach to testing in nightwatch.js

Question

One common problem when writing some e2e tests is how to test one feature multiple times with different sets of data. In many other similar tools (TestCafe, Cypress, perhaps more) and many not-so-similar tools (Robot Framework, pytest, ...) there're ways to define data-driven approach to testing other than simply iterating over a collection. An example of this could be from TestCafe:

testData.credentials.forEach(credentials => {
    test
        .meta({ author: 'Pavel Saman', creationDate: '27/05/2020',
            env: process.env.TESTCAFE_ENV, url: baseUrl
        })
        ('Log Into User Account', async t => {

        await LogIn.logIn(credentials.username, credentials.password);

        await t
            .expect(PageMsg.pageMsg.innerText).eql(PageMsg.successfulLogIn)                  
            .expect(Selector(ProfileForm.inputObj.name.input).value).eql(credentials.name);        
    });
});

In nightwatch.js, there's this npm package nightwatch-data-driven, which looks promising, but when I tried it, it most likely doesn't work with the current version of nightwatch.js. When I follow the example on the npm page, no tests are executed and it gives "Cannot use import statement outside a module" error. Others have described the very same problem on the github page of the project. The last commit in the repository is from May last year, so the date doesn't suggest there's much activity on this project either.

Another approach is, as I mentioned, simply iterating over a collection:

const testData = require('../Resources/userData.json');

// ...
// ...

'Change User Data': (browser) => {       

    for (data of testData.userData) {

        browser.page.profileHomePage()                
            .section.input
            .fillInUserData(data.name, data.surname, data.street, data.phone, data.city, data.zip)
            .click('@saveButton');

        browser.page.success()
            .expect.element('@successEle').text.to.equal('Údaje byly uloženy!');
    };                
}

// ...

But this doesn't really look like a data-driven approach. It's true I'm driving tests based on data. But when run, reports will show I've run only a single test case. No matter how many elements I had in my data structure. That is not practical. Some other tools I know can in one way or the other parametrize test function in a way that when run on different sets of data, it will look like multiple test cases (in e.g. reports).

How do I define test cases in a data-driven approach in nightwatch.js?

To summarise what I've tried:

• searching on the web, official docs, here, ...
• nightwatch-data-driven npm package and experimenting with it
• define test cases in a similar way as I did in TestCafe

Answer 1

Nightwatch represents each spec file as an exported object mapping the test names to the test functions. To generate tests from data, therefore, you can use any means of adding properties to an object provided by JavaScript. For example, given:

[
  {
    "title": "First test",
    "url": "https://google.com"
  },
  {
    "title": "Second test",
    "url": "https://example.com/"
  }
]

you can generate the tests using e.g.:

const data = require("./data.json");

module.exports = data.reduce(function (tests, test) {
  tests[test.title] = function (browser) {
    browser.url(test.url).end();
  };
  return tests;
}, {});

and see the output:

$ npm t

> [email protected] test path/to/now-my-watch-begins
> nightwatch nightwatch.test.js


[Nightwatch Test] Test Suite
============================
ℹ Connected to localhost on port 4444 (2798ms).
  Using: firefox (77.0.1) on mac 19.5.0 platform.

Running:  First test

No assertions ran.

Running:  Second test

ℹ Connected to localhost on port 4444 (1909ms).
  Using: firefox (77.0.1) on mac 19.5.0 platform.

No assertions ran.


OK. 2 tests passed (9.144s)

Answer 2

I'll also answer my own question because after another day or two, I've read more of the documentation and found that since nightwatch version 1.3, it supports BDD syntax similar to e.g. Cypress. It's still marked as beta.

An example could be:

const testData = require('../Resources/userData.json');
const credentials = require('../Resources/credentials.json');

describe('Change User Data', function() {

    before(function(browser) {

        browser.page.popup()
            .initAndClosePopup();

        browser.page.homePage()
            .fillInCredentialsAndLogin(credentials.username, credentials.password);        

        browser.page.profileHomePage()
            .assert.elementPresent('@title')
            .expect.element('@title').text.to.equal('Můj účet');
    });

    testData.userData.forEach((data) => {

        test('Change User Data' + run, function(browser) {

            browser.page.profileHomePage()                
                .section.input
                .fillInUserData(data.name, data.surname, data.street, data.phone, data.city, data.zip)
                .click('@saveButton');

            browser.page.success()
                .expect.element('@successEle').text.to.equal('Údaje byly uloženy!');
        });
    });

    after(function(browser) {

        browser.page.logout()
            .click('@logoutButton');

        browser.page.success()
            .expect.element('@successEle').text.to.equal('Odhlášení proběhlo úspěšně!');

        browser
            .end();
    });
});

This is really readable and people might be used to writing tests in this way, so it perhaps shortens the time needed to learn this tool. There're some limitations such as no nested describe are allowed. That's is different from above-mentioned Cypress.

Answer 3

I haven't use a nightwatch.js but the error Cannot use import statement outside a module, I've encountered it though recently with nodejs. I found this really nice article, there's quite a few solution mention in there but for me adding this on my package.json help me out though. Hope this help you out too.

{
  "type": "module"
}

For the second part,

How do I define test cases in a data-driven approach in nightwatch.js?

May I ask if what exactly your data-driven approach description? I can see that in your code that you are already using this approach, but is this only specific for nightwatch or something? And is it not a good practice to just iterate over a collection? Sorry, I am also curious on this part.