Test Data Management And Dependencies
This is typically the 2nd major challenge in automation. Once the organization is convinced of the value of automated tests the next challenge is data management.
In all the organizations that I have worked in, the same challenges comes up - dependencies for authentication, authorization and the specific artifacts needed in the product, such as account, site, organization, etc. It is solved differently for manual testing where either the user registers each time as part of the testing or the organization establishes test accounts with access usually set up once and then used repeatedly going forward.
Automation for test cases that have dependencies such as authentication, authorization and other artifacts creation require new approaches that are only relevant to automation.
Thus the process for testing 'after a typical login' - or other 'setup' activities are complex for automation.
Here are some of the options:
- Mocking and stubbing for the authentication or code that relies on it
- Scripts that do logins using the UI
- Permanent accounts used for test automation cases only
- Accounts that are created through an API specifically for testing
My recommendation is mocking and stubbing and/or the ability to create or use accounts specifically for automation. This can be categorized as grey box testing, where you don't know the internals but you do have appropriate hooks for testing purposes.
One aspect you will want to master is not only creating, but deleting the artifacts once the specific test has run. Without this automation you will soon create thousands of artifacts and that will become a problem. Similar to artifacts creation, you'll want to do this through API calls and not by using the UI which is slow and unreliable.
My least favored option is creating the account through the UI, e.g. the login tests. UI automation is slow, often doesn't scale and always has elements of flakiness and thus should be avoided as a dependency for running other automation. It will frequently work OK for the first couple of test cases but then as you scale testing with more cases it typically breaks down with flaky failures and slow feedback from the test suite.
