Test users password- static or dynamic

Question

During our e2e testing, 100+ test users are created on our application. Workflow goes like this:

1. Generating a random password with faker library
2. Storing it in a variable
3. Using the variable to create the user 
4. Login as the created user and do some actions
5. Delete all the created users in the end(as part of teardown and via API)

These users will be deleted as part of cleaning up resources(teardown). From your point of view, is it better to generate passwords from faker or have a static password for all of these test users? Please share the pros and cons.

Answer 1

I would recommend taking a step back from the technical implementation of generating password values and consider what are the specific risks you're trying to mitigate with these end-to-end tests.

Is a randomly generated password actually important to the context of the test? If you are specifically testing password validation logic, then yes it might well be. If the test just needs a valid password value in order to create a user account then perhaps not.

By relying on random values generated by Faker (or any other random data generation tool), you would be introducing non-deterministic input data to your tests. Depending on the context of the tests this might be fine, or it might be problematic in ways you currently can't anticipate.

Some very limited pros/cons below.

Random Data Generation via Faker (or another lib/package)

Pros	Cons
Integration with popular test frameworks e.g. Pytest	Potential performance cost
Variation of data might expose unknown risks (e.g. validation logic)	Changes to the Faker dependency could break your tests
	Potential additional cognitive load (e.g. where does this password come from?)
	Non-deterministic

The Ministry of Testing community has good content on managing test data.

For a wider discussion on guidelines and standards for test automation see here

Answer 2

In general, there are pros and cons to both using dynamic (generated) passwords and using a static password for your test users.

Using dynamic passwords has the advantage of increased security, as it makes it much more difficult for unauthorized parties to access the test data. Additionally, dynamic passwords can reduce the risk of accidentally reusing passwords in the test environment that are also used in the production environment.

However, using dynamic passwords can also lead to complications in the testing process. For example, if you need to debug a test, you will need to store the password somewhere to be able to log in as the test user. This could potentially create a security risk, especially if the password is stored in a publicly accessible place such as a code repository. Additionally, if you generate the password with a library such as Faker, you run the risk of generating a password that is not compliant with your password policy (e.g. too short, not complex enough).

On the other hand, using a static password has the advantage of simplifying the testing process. You don't need to worry about generating the password, and there's no risk of it being too complex or not meeting the password policy. Additionally, if you use the same password for all test users, you only need to store it in one place, reducing the risk of it being lost or forgotten.

However, using a static password does have its own security risks. If the password becomes known, an unauthorized party could access the test environment and potentially cause harm. Additionally, if the password is reused in the production environment, this could lead to a security breach.

Ultimately, the decision of whether to use dynamic or static passwords will depend on your specific testing needs and the security requirements of your organization. It may be necessary to weigh the benefits and risks of each approach and make a decision based on your specific testing requirements and organizational security policies.