Note, this is specific to TestRail but some questions could be considered generic issues:
I have worked for a couple companies that have gone through the implementation of automation into TestRail. In each company, there has always been different practices of doing things.
When you automate a manual test case, what do you do with the manual case and what do you do with the automation case?
- example 1: Do you keep the manual test and just flag it as automated, if so do you point to the case ID in automation to have a link and add any other fields to the TestRail case? How do you keep it up to date if the test changes?
- example 2: Do you delete the manual test and import the automated test in its place?
- example 3: Do you keep both tests but flag the manual one as automated?
What do you use the reference field for?
- example 1: Do you use it to link the JIRA epic/story for the test case? how do you then link to the requirements?
- example 2: Do you use it to link to the requirements, if so, what tool do you use?
Most automated tests are WebDriver, API, etc., but do you include unit tests in TestRail?
If I have a manual test that's been automated but not under each platform, what do you do?
- example 1: Don't flag it as automated but make a note in the test stating it is automated for xyz platforms (or a separate test case field with automated platforms).
- example 2: Flag it as automated and spin up a new test case for the non-automated platforms.
- example 3: Not a concern, if it works on one platform, it should be sufficient.
If you add automated tests to testrail that do not have a corresponding test case, where do you put them?
- example 1: Use a section ID of the feature to place them where they should logically go.
- example 2: Use a section ID of the feature but then have an 'automated' folder where they go.
- example 3: Have a flag automation folder.
How do you work around being reliant on testrail IDs?
- example: if you migrate from a cloud solution to an on-premise solution, all of your IDs break. If each test case is linked to an ID, this would be thousands of code changes needed; or if a manual tester moves sections around to reorganize the group tree, it may break automation.