I've got a lot of JMeter tests setup - multiple test plans, thread groups, requests, asserts etc. We're using it for our functional API testing, as well as performance testing.

I am now investigating Taurus and particularly like the pass/fail criteria. There doesn't seem to be another tool available that offers that sort of functionality.

However, one other aspect to the JMeter tests I have is that I make extensive use of properties - for environment details, as well as thread count details. This enables me to switch between different configurations of test easily (e.g. if I want to run a quick smoke test, or a more involved functional test). I can also switch environments easily (so is needed for a CI/CD pipeline).

I'd like to keep using JMeter (as we have so much setup) for general API test development and local running. For pipeline running, switching to Taurus and using the pass/fail criteria seems good.

I have 2 questions from this:

  1. I'm not sure if the external property files can be passed in like I described above? I've seen mention of included-configs but can this be switched out dynamically? Or would I need different config files for each combination of environment/test-run type?

  2. With maintaining my tests in .jmx format, am I right in thinking that the pass/fail criteria I'd be able to specify in Taurus would be quite generic? I.e. relating to the entire suite and not individual scenarios etc.?

I know there's a jmx2yaml converter tool with Taurus but this didn't handle many of the properties and assertions we'd setup.

  1. Given you will be using Taurus to run existing .jmx files you can also specify existing JMeter installation with all .properties files you want to use, they will be picked up. If you want to just pass some properties to JMeter script from Taurus you can either use properties section in YAML or pass them via -o command line argument

  2. According to Pass/Fail Criteria documentation:

    [label] is a sample label. If omitted, it is applied for overall.

    so you have choice whether to limit the criteria to single sampler or transaction or to apply it to all samplers

If by "pipeline" you mean Jenkins Pipeline it might be a better idea to consider using Performance Plugin, in this case you won't have to add an "alien" Python-based wrapper tool into Java environment

| improve this answer | |
  • I saw the performance plugin, but it didn't seem to offer the same level of quality gates that Taurus does. Generic pass/fail criteria should be fine I think for now. – Moorpheus Sep 8 at 10:43
  • In your first answer above, please can you give a bit more information on where the .properties files get specified. I can see there's a modules.jmeter section in bzt.rc - is it here that I refer to the .properties files? Or is it more in the case of me setting them like modules.jmeter.properties? If so, how do I refer to a file here? Thanks – Moorpheus Sep 8 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.