Taurus/JMeter - total time for thread group execution within pass/fail criteria

Question

I have an end-to-end scenario to include in performance testing that takes a decent length of time to execute. Between releases of the system, this time needs to be monitored to ensure it hasn't increased.

I am already taking advantage of the very useful pass/fail criteria in Taurus. However, the available criteria are quite granular (at the individual request/response level). I know Taurus does output test duration but this could include other items that I'm not interested in. I also can't see test duration supported as a metric for the pass/fail module in Taurus.

So, any suggestions on how to extract the time to complete the process and then compare this time with a pass/fail criterion? The script itself would be executed within Jenkins for what it's worth.

Answer 1

On JMeter level it would be something like:

1. Add tearDown Thread Group to your Test Plan

2. Add JSR223 Sampler to the tearDown Thread Group and put the following code into "Script" area:

   def start = vars.get('TESTSTART.MS') as long
   
   def end = System.currentTimeMillis()
   
   if ((end - start) > 5000) {
       SampleResult.setSuccessful(false)
       SampleResult.setResponseCode('999')
       SampleResult.setResponseMessage('Thread group execution took longer than 5 seconds')
   }
   

3. That's it, if your Thread Group(s) take longer than 5 seconds - the JSR223 Sampler will be marked as failed and you can catch this event with Pass/Fail Criteria as well

Explanation of the script:

• vars - stands for JMeterVariables class instance and it's used to read the value of TESTSTART.MS pre-defined variable which holds the timestamp of the test start
• SampleResult - basically an instance of SampleResult class allowing to set response code, message, body, success, etc.

More information on above and other JMeter API shorthands available for JSR223 Test Elements - Top 8 JMeter Java Classes You Should Be Using with Groovy