How to handle A/B testing in automated UI tests with Appium?

Question

I'm testing native mobile apps with Appium (iOS and Android, using the Java client). We have a family of apps where some features are turned on or off depending on the particular app (like A/B testing, but that's not actually what we're doing).

How could I exclude tests from the test suite based on what features I know it's going to see in the app under test? (We start the test suite though Jenkins.)

It would be very easy to check at the beginning of each test and then bail out as appropriate, but I'm worried about how that affects statistics of how many tests actually passed.

Answer 1

Most test runners support categories or grouping. Annotate the test methods with something like @Category(A.class) now you can tell the test runner to run only tests in A.class category when run from Jenkins for a certain build parameter.

For JUnit categories look here: http://junit.org/javadoc/latest/org/junit/experimental/categories/Categories.html