1

One common problem when writing some e2e tests is how to test one feature multiple times with different sets of data. In many other similar tools (TestCafe, Cypress, perhaps more) and many not-so-similar tools (Robot Framework, pytest, ...) there're ways to define data-driven approach to testing other than simply iterating over a collection. An example of this could be from TestCafe:

testData.credentials.forEach(credentials => {
    test
        .meta({ author: 'Pavel Saman', creationDate: '27/05/2020',
            env: process.env.TESTCAFE_ENV, url: baseUrl
        })
        ('Log Into User Account', async t => {

        await LogIn.logIn(credentials.username, credentials.password);

        await t
            .expect(PageMsg.pageMsg.innerText).eql(PageMsg.successfulLogIn)                  
            .expect(Selector(ProfileForm.inputObj.name.input).value).eql(credentials.name);        
    });
});

In nightwatch.js, there's this npm package nightwatch-data-driven, which looks promising, but when I tried it, it most likely doesn't work with the current version of nightwatch.js. When I follow the example on the npm page, no tests are executed and it gives "Cannot use import statement outside a module" error. Others have described the very same problem on the github page of the project. The last commit in the repository is from May last year, so the date doesn't suggest there's much activity on this project either.

Another approach is, as I mentioned, simply iterating over a collection:

const testData = require('../Resources/userData.json');

// ...
// ...

'Change User Data': (browser) => {       

    for (data of testData.userData) {

        browser.page.profileHomePage()                
            .section.input
            .fillInUserData(data.name, data.surname, data.street, data.phone, data.city, data.zip)
            .click('@saveButton');

        browser.page.success()
            .expect.element('@successEle').text.to.equal('Údaje byly uloženy!');
    };                
}

// ...

But this doesn't really look like a data-driven approach. It's true I'm driving tests based on data. But when run, reports will show I've run only a single test case. No matter how many elements I had in my data structure. That is not practical. Some other tools I know can in one way or the other parametrize test function in a way that when run on different sets of data, it will look like multiple test cases (in e.g. reports).

How do I define test cases in a data-driven approach in nightwatch.js?

To summarise what I've tried:

  • searching on the web, official docs, here, ...
  • nightwatch-data-driven npm package and experimenting with it
  • define test cases in a similar way as I did in TestCafe
10
  • "But this doesn't really look like a data-driven approach" Why? You are driving your checking based on a set of data. Jul 8, 2020 at 22:10
  • @João Farias: Because the loop is inside the test, so it will look like only one test case everywhere (in reports, in output). In other tools I know, if they implement data-driven approach, you can parametrize the test function in one way or the other so when run, it will appeach as multiple test cases (which is closer to reality anyway).
    – pavelsaman
    Jul 9, 2020 at 4:34
  • So move the loop outside the test, use it to generate multiple tests. That's what you're doing in TestCafe. Also the error using the library isn't necessary related to the library itself; are you using imports already? If not, try using require instead.
    – jonrsharpe
    Jul 9, 2020 at 6:27
  • @jonrsharpe: have you tried it? Because I have tried it both with forEach (like in TestCafe or Cypress) and with simple for (like I have just outside of the test case), and both options give syntax error withit nightwatch. It doesn't seem like a solution. How else would you do it then?
    – pavelsaman
    Jul 9, 2020 at 8:04
  • Well it looks like you're defining tests inside an object, although you've left out the opening and closing braces, so you can't just put a random loop in an object literal. Whatever you do needs minimally to be valid JS syntax.
    – jonrsharpe
    Jul 9, 2020 at 8:09

3 Answers 3

2

Nightwatch represents each spec file as an exported object mapping the test names to the test functions. To generate tests from data, therefore, you can use any means of adding properties to an object provided by JavaScript. For example, given:

[
  {
    "title": "First test",
    "url": "https://google.com"
  },
  {
    "title": "Second test",
    "url": "https://example.com/"
  }
]

you can generate the tests using e.g.:

const data = require("./data.json");

module.exports = data.reduce(function (tests, test) {
  tests[test.title] = function (browser) {
    browser.url(test.url).end();
  };
  return tests;
}, {});

and see the output:

$ npm t

> [email protected] test path/to/now-my-watch-begins
> nightwatch nightwatch.test.js


[Nightwatch Test] Test Suite
============================
ℹ Connected to localhost on port 4444 (2798ms).
  Using: firefox (77.0.1) on mac 19.5.0 platform.

Running:  First test

No assertions ran.

Running:  Second test

ℹ Connected to localhost on port 4444 (1909ms).
  Using: firefox (77.0.1) on mac 19.5.0 platform.

No assertions ran.


OK. 2 tests passed (9.144s)
3
  • Thank you, this is a working example.
    – pavelsaman
    Jul 9, 2020 at 21:17
  • 1
    @pavelsaman no problem. Do you see what I mean now about it just being JS? Data-driven testing in dynamic languages doesn't tend to need special tooling - JS already can trivially parse JSON and build arbitrary structures.
    – jonrsharpe
    Jul 10, 2020 at 11:37
  • I never had doubts you were right, it was just that I wanted to see it because I haven't been working with JS for that long. I can see knowing JS is pretty much all one needs, and special tooling can then help radability a bit (which is why these tools exist as well I guess).
    – pavelsaman
    Jul 10, 2020 at 12:02
1

I'll also answer my own question because after another day or two, I've read more of the documentation and found that since nightwatch version 1.3, it supports BDD syntax similar to e.g. Cypress. It's still marked as beta.

An example could be:

const testData = require('../Resources/userData.json');
const credentials = require('../Resources/credentials.json');

describe('Change User Data', function() {

    before(function(browser) {

        browser.page.popup()
            .initAndClosePopup();

        browser.page.homePage()
            .fillInCredentialsAndLogin(credentials.username, credentials.password);        

        browser.page.profileHomePage()
            .assert.elementPresent('@title')
            .expect.element('@title').text.to.equal('Můj účet');
    });

    testData.userData.forEach((data) => {

        test('Change User Data' + run, function(browser) {

            browser.page.profileHomePage()                
                .section.input
                .fillInUserData(data.name, data.surname, data.street, data.phone, data.city, data.zip)
                .click('@saveButton');

            browser.page.success()
                .expect.element('@successEle').text.to.equal('Údaje byly uloženy!');
        });
    });

    after(function(browser) {

        browser.page.logout()
            .click('@logoutButton');

        browser.page.success()
            .expect.element('@successEle').text.to.equal('Odhlášení proběhlo úspěšně!');

        browser
            .end();
    });
});

This is really readable and people might be used to writing tests in this way, so it perhaps shortens the time needed to learn this tool. There're some limitations such as no nested describe are allowed. That's is different from above-mentioned Cypress.

0

I haven't use a nightwatch.js but the error Cannot use import statement outside a module, I've encountered it though recently with nodejs. I found this really nice article, there's quite a few solution mention in there but for me adding this on my package.json help me out though. Hope this help you out too.

{
  "type": "module"
}

For the second part,

How do I define test cases in a data-driven approach in nightwatch.js?

May I ask if what exactly your data-driven approach description? I can see that in your code that you are already using this approach, but is this only specific for nightwatch or something? And is it not a good practice to just iterate over a collection? Sorry, I am also curious on this part.

2
  • 1
    I'll look into the first part, thank you. As to the data-driven approach. The problem with a loop is it's inside the test case, so it will look like I've run only one TC, no matter how many times I looped over some data structure. In other tools that do implement data-driven approach, test functions are parametrized, so when run, it looks (in reports for example) that you have run multiple TC, which is closer to reality.
    – pavelsaman
    Jul 9, 2020 at 4:37
  • @pavelsaman, My point of thinking is also iteration over a collection. I see, you got a point about those test functions are being parametrized which is exactly you said that it is closer to reality. I also think it should be like that. Thank you for your explanation, I learned something new today
    – ky-chan
    Jul 9, 2020 at 6:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.