I'm currently writing API and UI automated tests for a website. My goal is to plug these tests in the CI workflow (Jenkins is used in this organization). The workflow would be: if there's a new commit on the SUT's repo, then trigger a build of my automated tests.
The question I ask myself is: shall I store these tests in the same repo than the SUT? Does it make a difference?
Please tell me about your experiences...
Thanks in advance.
This is a great question and one that many organizations face.
I recommend keeping the test code and the application code together because this will, at the highest level, help keep (or perhaps 'force') application and automation developers closer to each other in many other ways.
It would be 'easier' to have the repos separate, however this is a very short-term view and it is a view that promotes each group to focus on their own work only and not consider the other party's contribution as part of the same puzzle.
To make the same repo approach workable, reliable, helpful and less painful to use consider the following practices:
In the large view, avoid:
because they all push devs and testers apart and the point of testing should not be so much a defensive 'does it work / is it broken' for go / no-go decisions, instead testing should be providing immediate and valuable feedback to developers about code that needs to be fixed or improved. Companies that fully master this have moved from deploying every six months to deploying every few minutes. This is what the largest silicon valley companies do now. Netflix, Facebook, etc.
When to choose different branch or repo
If you are deploying the test scripts to a different test server to execute the steps and is executed completely independent of the source code or minimal dependency. Then it doesn't matter where you keep your code. I would suggest adding the test scripts and the required dependency together and store it under a separate branch or repo as it will ensure that we don't confuse developers with our commits and files.
When to choose, merging to master branch of SUT source code
If test is executed as a build test step in package.json or maven then add it under the same repo and branch as the product getting deployed.
It doesn't matter, from a technical perspective. You can point your trigger to any path (test repo).
The important thing is to respect the naming convention, so no one gets confused with it. I once had a developer ask me "Hey why do you have access to SUT-tests folder?" Unknowingly, I created tests inside folder with that name, but the devs would call their unit-tests folder that way.
