12

Disclaimer: this question is not related to - "Test cases in source code"

I am working on a super fast development environment where manual test cases were given no importance (thankfully things are bright on this front after some pushing) and UI automation was considered the "testing".

Automation code base (Selenium tests) has been part of of application code base. So with every application branch having a dedicated project requirement being developed, it also runs a corresponding automation tests being developed by us for that specific branch. Since we are limited number of QA we end up working in multiple branches at times. And at some point in time (no specific time again) build engineer pushes one of the application code base branch to trunk).

Now this is where problems arise -

  1. Consider we develop tests in one branch (which means application locators, some common utilities, some tests) and this branch is not yet gone to trunk, now we begin to work on new application branch and we need part of code which we developed in other branch. We are at a fix now...

  2. Since merges are not so often, when ever a branch is merged to trunk it eventually results in more conflicts for automation code which could be reduced if merges were more often. And we can not demand merge to trunk for sake of automation code base. There should be good enough application source code also working in branch before it could move to trunk.

So I suggested to take automation code away from application directory. And to keep track of application branches we could have corresponding branches in QA automation project. i.e. application branches feature1, feature2 etc will have corresponding branches feature1, feature2 etc in automation project.

Doing so will allow QA (which means us) to merge to our own trunk as and when we want, and removing current dependency of when automation code goes to application trunk.

The only hurdle which I see at this point is keeping QA automation trunk code in sync with application code trunk so that it could be used to test project code base trunk. But when we merge more often to QA then automation trunk would have tests for some features which may not yet be available in application trunk. So then -

Should we be maintaining a different branch in automation code base which corresponds to application source code trunk. And doing so we could still continue to merge to automation trunk when ever want, or

Have I got it completely wrong and there are better solutions?

2
  • Could you be a bit more specific about your branching strategy? Is your code in the trunk a type of Prod branch? Do you do any deploys from the trunk? Nov 19, 2011 at 14:19
  • yup with current set up, it is production branch and gets deployed to trunk.
    – Tarun
    Nov 19, 2011 at 18:50

4 Answers 4

8

Ok, this is exactly the situation we faced in my old gig. Allow me to offer some advice...

  • First of all, think VERY carefully before you commit to maintaining test compatibility with multiple development branches. Think about it this way: Let's say you have a team of 5 manual testers, and one main dev branch to test. Then, the SW team announces they have 3 new sub-branches and want you to test all 4 branches. You would say: "Well... I still only have 5 manual testers, so I want 15 more testers to handle the load." Automated testing is the same situation: To test more branches, you need more people.

  • If the SW team at your shop does not communicate merge status effectively, you will constantly be playing catch-up with their work. This will drive you crazy. It sounds like this is already a problem at your company.

  • With this in mind, I recommend you narrow your commitments. Commit to developing automated tests for the main dev branch only. Offer to run those tests on other branches, but don't guarantee compatibility.

2
  • definitely it's the case of over commitment with us. your pointers describe it very well.
    – Tarun
    Nov 23, 2011 at 5:20
  • I would add to this that you still need communication about merges before they happen. That way, you can run your tests against other branches to see where code updates broke your tests before they are merged in, so that you have (ideally) zero test down time after merges for test maintenance due to feature changes (because you will have made changes already). Nov 29, 2011 at 1:07
3

You've said, Selenium Tests, so I assume that all the automation code is Selenium code. In that case, handling the automation code in the same way handing application code is just wrong. Because there is no need to change the code every time if there no change in test case flow(steps) or locators.

For example, code base or db is changed not the UI. Another example is UI is redesigned but all those locators remain same. In those cases, automation will work fine even thought lot of things changed in background. This leads to another question, what if two or three teams working side by side and there is locator/UI change is those branches? How to maintain the code for that? Use Tags. If the automation code is separated by modules and locators sit in outside of the code, this is pretty easy to do. You may tag few files and do the work.

Even after all these things, there will be some issue in automating the new things. That is because, as some one said before, communication issue between two teams.

2

We keep a Selenium branch that matches the current development branch and merge the selenium branch when the development branch goes to prod. Which to me sounds like what you are doing now. We have the separate branch for two reasons. Keep the tests out of source and merging of the test code is less important. We complete all other merges first then work on the test merges last.

We are generally only working on one main sprint branch and don't really have multiple branch issues that you currently have. I think your approach will work as long as you don't need to run Trunk tests against the trunk code. I do find that the prod branch for our tests is not really that important because it doesn't represent anything real in production.

Only thing I'd be worried about is the case when you come across an serious production defect and cut a new branch from trunk to handle it. When you try to cut a new automation branch you may end up with a mixed bag as opposed to a solid instantly run-able suite against your new branch.

1

I think that you're on the right track. We have our own source control for automation of each project. Although this is mainly an easy way to keep us from having write access to the application source code (compliance requirements), it allows us to setup our automation strategy any way that we see fit. It's actually not completely uncommon for us to have another branch setup for experimental code when we're attempting to learn something different. To be honest though, lately, there hasn't been a whole lot of need to create separate branches since our switch to SpecFlow and the way that step definitions are handled with step scope. I'm sure that if we had larger high impact changes going in, we'd probably look at cutting a new branch, but right now, even a very large, 6+ month enhancement, I haven't found a need to do so.

1
  • I am thinking to propose that our tests at any point would be compatible with trunk code. Having to maintain multiple automation branches is driving us crazy
    – Tarun
    Nov 23, 2011 at 5:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.