Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

There is a complex software product, branched into Dev/Beta/Stable and having Hotfixes -- all branches migrate 'left to right' together time to time. We are writing functional tests (dozens, in future few hundred of kbytes of high level language code) and don't actually mess with original software code -- we test the executable. The question is about managing the tests on at leasts two branches.

The software consists of several components. Some of them are parallel and their updates influe only 5% of our tests code. Some are perpendicular and touch everything. The perpendicular component change is solvable via versioning the abstract communication layers.

My idea is:

  • have a universal wrapper, which recognizes the version of software and:
    • in case of component's big change the wrapper runs another tests file
    • in case of tests set change the skipIf Version >/</==/... decorators work for us
  • remove outdated tests time to time according to Skip details
  • have a single branch (trunk) in a repository

But my collegue says:

  • branch the tests
  • time to time put one branch over another, throwing away the Stable one
  • "we don't need ifs - they make code too complex"

So while we are developing several (at least Dev and Beta) branches at the same time, we would mess with folders and do copypaste. And what's wrong with if? I thought they are ok in programming...


P.S.: wanted tags: branching, code-organization, versioning, functional-testing

share|improve this question
1  
Check out similar question: sqa.stackexchange.com/questions/6641/… –  dzieciou Sep 9 '13 at 11:11
1  
Also sqa.stackexchange.com/questions/6277/… for some pros and cons of using branched/versioned code vs using in-code version or feature detection –  Kate Paulk Sep 9 '13 at 11:29

1 Answer 1

Nakiton,

Here's a quick summary of the information I gave in answers to the two questions referenced in the comments:

  • Branching is more maintenance overall, but makes sure that features which should be in version X are in version X, and work in version X as well as ensuring that test results are correct for the version. Ideally, branching should be managed along with automation machines such that the branch contains the full environment configuration instructions allowing each branch to set up and run the tests and application correctly. This is particularly important in applications with multiple licensing options and in tests against multiple operating systems.
  • In-code version detection is initially simpler, but as version differences grow can become extremely unwieldy, particularly when features are also environment-specific (such as licensing options or operating systems). Eventually you end up with either large numbers of skipIf Version calls or large chunks of almost identical code. Neither is good for readability or maintainability. In addition, your feature-version oracle is scattered throughout the codebase.
  • In code feature detection has similar advantages and disadvantages to version detection, but includes the risk that features which should be present in a version are not and the code passes because it checks against feature and not version.

The method I've seen most often is to maintain branches that roughly correspond to the minor release (in a Version.MajorRelease.MinorRelease.Build arrangement), and label each branch as it's created. Almost all script development happens against the active development branch, with some being merged to the current maintenance/beta branch if that is also employing the feature in question. Automation runs start by pulling from the appropriate branch then configuring the environment to match the branch instructions (which prevents permissions being incorrectly set due to restructuring of licensing).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.