JMeter to Taurus - how to handle external property files and pass/fail criteria?

Question

I've got a lot of JMeter tests setup - multiple test plans, thread groups, requests, asserts etc. We're using it for our functional API testing, as well as performance testing.

I am now investigating Taurus and particularly like the pass/fail criteria. There doesn't seem to be another tool available that offers that sort of functionality.

However, one other aspect to the JMeter tests I have is that I make extensive use of properties - for environment details, as well as thread count details. This enables me to switch between different configurations of test easily (e.g. if I want to run a quick smoke test, or a more involved functional test). I can also switch environments easily (so is needed for a CI/CD pipeline).

I'd like to keep using JMeter (as we have so much setup) for general API test development and local running. For pipeline running, switching to Taurus and using the pass/fail criteria seems good.

I have 2 questions from this:

1. I'm not sure if the external property files can be passed in like I described above? I've seen mention of included-configs but can this be switched out dynamically? Or would I need different config files for each combination of environment/test-run type?

2. With maintaining my tests in .jmx format, am I right in thinking that the pass/fail criteria I'd be able to specify in Taurus would be quite generic? I.e. relating to the entire suite and not individual scenarios etc.?

I know there's a jmx2yaml converter tool with Taurus but this didn't handle many of the properties and assertions we'd setup.

Answer 1

1. Given you will be using Taurus to run existing .jmx files you can also specify existing JMeter installation with all .properties files you want to use, they will be picked up. If you want to just pass some properties to JMeter script from Taurus you can either use properties section in YAML or pass them via -o command line argument

2. According to Pass/Fail Criteria documentation:

   [label] is a sample label. If omitted, it is applied for overall.

   so you have choice whether to limit the criteria to single sampler or transaction or to apply it to all samplers

If by "pipeline" you mean Jenkins Pipeline it might be a better idea to consider using Performance Plugin, in this case you won't have to add an "alien" Python-based wrapper tool into Java environment