Generate test data in which format

Question

1. What is the best approach to make test data configurable ? Through XML file or properties file or any other way. But must not be hard coded.
2. What approach should be taken to run the same Test Suit multiple times with different test data ?
3. Suppose I want to test login functionality with different type of user like Admin , Customer, Manager and many more.

I want to write test in selenium. Which approach is best ? Is there different test data file for same test suit or single file of all user ?

Answer 1

There are many, many approaches, each having its pros and cons when it comes to simplicity of reading, easiness of modifying data, easiness of modifying data schema (refactoring), easiness of parsing data, type of data you want to model. If you were I would experiment. I would pickup the simplest approach (properties) to see if fits your problem. Here's the list of more approaches.

Properties file

Pros: simple

Cons: schema-less - you need to write your own parser if you want to model more complex data structures (e.g. subproperties or multi-dimensional data)

CSV

Pros: models well tabular data

Cons: may not be sufficient for more complex data structures

XML

Pros: self-descriptive, models well tree-like data

Cons: Very verbose

Spring-based XML

Pros: All pros of XML + binding for refactoring and editing from IDE

Cons: like in XML

JSON, YAML

Pros: Good trade-off between self-descriptiveness and conciness

DB

Pros: Centralized access, though everything above can be centralized (+versioned) using any versioning system

Cons: Hard to refactor, when you want to change schema, requires opening SQL client each time you want to modify it

Beans in your programming language

Pros: Tightly bound to your test code but still not hard-coded, easy to refactor together with your test scripts

Cons: Sometimes to verbose

Answer 2

First of all:There is no best approach. There are just several approaches and you have to decide which one is your favorite.

1. What is the best approach to make test data configurable ? Through XML file or properties file or any other way. But must not be hard coded.

I like it to put the test data in properties files, read the data and put them into objects. e.g.:

user.admin = user1#password1
user.manager = user2#password2

I would create a User class that is able to contain the needed data.

From my point of view, properties files are really easy to read (e.g.: Java already have a class to handle them.). With comments, some space lines and good key names they can also be easy to read and maintain.

You can give it a try, but in the end you should decide which is the best way to organize your property files and the keys in them. I do not know how the architecture of your testing framework is, so this is also a important point.

2. What approach should be taken to run the same Test Suit multiple times with different test data ?

There exist different approaches I know:

1. Create a set of different test data (multiple property fies) and made it configurable. You could have a test run configuration. You need a properties file, which contains the current test run configuration. You can set the config via Maven and/or Jenkins.
2. You can work with the power of randomess. E.g. for your accounts: users.admin = user1#password1; user2#password2; user3#password3 Now you can pick one of these accounts randomly. The negative point on this is, that theoretical can happen that several runs in a row use the same data.

For both approaches you need to log which test data was used, so that you are able to check passed and failed test cases with the correct data and that you can read the result on the right way.

Good luck!

Answer 3

Sounds like you want to do Data Binding (also called Data Driven Testing). You can parse a CSV file and have your test run as many times as there are rows in the file. Each row can have columns for username and password. You will need to add appropriate logging so that when a test 'fails' you know which iteration it failed on.

How you read in the csv file depends on your language of choice, but in general working with csv is easier than xls/xlsx.