I've got a lot of JMeter tests setup - multiple test plans, thread groups, requests, asserts etc. We're using it for our functional API testing, as well as performance testing.
I am now investigating Taurus and particularly like the pass/fail criteria. There doesn't seem to be another tool available that offers that sort of functionality.
However, one other aspect to the JMeter tests I have is that I make extensive use of properties - for environment details, as well as thread count details. This enables me to switch between different configurations of test easily (e.g. if I want to run a quick smoke test, or a more involved functional test). I can also switch environments easily (so is needed for a CI/CD pipeline).
I'd like to keep using JMeter (as we have so much setup) for general API test development and local running. For pipeline running, switching to Taurus and using the pass/fail criteria seems good.
I have 2 questions from this:
I'm not sure if the external property files can be passed in like I described above? I've seen mention of included-configs but can this be switched out dynamically? Or would I need different config files for each combination of environment/test-run type?
With maintaining my tests in .jmx format, am I right in thinking that the pass/fail criteria I'd be able to specify in Taurus would be quite generic? I.e. relating to the entire suite and not individual scenarios etc.?
I know there's a jmx2yaml converter tool with Taurus but this didn't handle many of the properties and assertions we'd setup.