1

TLDR - What is the best way to review JMeter Test-Plans stored in a GitHub repo?

Currently, the code-review process that my performance-test team uses is somewhat cumbersome and of questionable effectiveness. Our team uses GitHub for version-control, and we follow the branching paradigm for check-ins, where the work done for a given story/bug/task is done in a branch of the main-line repository, then merged back into the mainline using the pull-review feature in git-hub.

This has left us with two options for reviewing changes to our test-plans: 1) Just review the raw XML in the PR diff-viewer 2) pull the un-merged branch and load each changed test-plan into a local-instance of JMeter without benefit of difference hi-lighting.

This is less than ideal - giving an effective code-review of raw XML is an issue for everyone regardless of what is being modeled. XML just isn't an easily human-readable format.

But loading the test-plans into JMeter and looking at them there is also problematic in that the information critical to any code-review, namely the exact things that changed, is lost, forcing each test-plan that changed to be reviewed in it's entirety and relying on the pull-request comments to inform the review about what changed.

As one would expect, this has led to situations where problems which could have been spotted by a code-reviewer have gone un-detected with the lamentable result of increased coding times, increased code-churn, and invalid results which have to be redone.

So my question to you, dear reader, is What is the best way to code-review JMeter Test-Plans that are stored in a git-hub repo?

1

Unfortunately JMeter .jmx files are basically XML and in case of large test plans it might be not very easy to view the diff.

You can consider using i.e.

  • Ruby-JMeter which is a Ruby DSL wrapper for JMeter tests, this way the code review will look like a code-review for "normal" Ruby project
  • or Taurus tool which supports definition of JMeter tests using simple YAML syntax so your test script would look like:

    scenarios:
      my-req: 
        requests:
        - url: http://blazedemo.com/  # url to hit
          method: GET  # request method (GET, POST, PUT, DELETE)
          label: homepage  # sampler label
    
          body: 'request-body-string'  # if present, will be used as body 
          body:  # generate query string based on parameters and request type
            param1: value1
            param2: value2
          body-file: path/to/file.txt  # this file contents will be used as post body
    
          upload-files:  # attach files to form (and enable multipart/form-data)
          - param: summaryReport  # form parameter name
            path: report.pdf  # path to file
            mime-type: application/pdf  # optional, Taurus will attempt to guess it automatically
    
          headers:  # local headers that override global
            Authentication: Token 1234567890
            Referer: http://taurus.blazemeter/docs
          think-time: 1s  # local think-time, overrides global
          timeout: 1s  # local timeout, overrides global
          content-encoding: utf-8  # content encoding (at JMeter's level), unset by default
          follow-redirects: true  # follow HTTP redirects
          random-source-ip: false  # use one of host IPs to send the request (chosen randomly).
                                   # False by default
    
          extract-regexp: {}  # explained below
          extract-jsonpath: {}  # explained below
          assert: []  # explained below
          jsr223: []  # explained below
    
0

Unfortunately, if you rely on JMeter GUI, the problem of code review is a bit inevitable.

However, one alternative is to create your JMeter scripts using purely Java, which allows you not only for easy code review, but also to use any Java feature( polymorphism e.g.) to enhance how you construct the script.

    //JMeter Engine
    StandardJMeterEngine jmeter = new StandardJMeterEngine();

    //JMeter initialization (properties, log levels, locale, etc)
    JMeterUtils.loadJMeterProperties("/path/to/your/jmeter/bin/jmeter.properties");
    JMeterUtils.initLogging();// you can comment this line out to see extra log messages of i.e. DEBUG level
    JMeterUtils.initLocale();

    // JMeter Test Plan, basic all u JOrphan HashTree
    HashTree testPlanTree = new HashTree();

    // HTTP Sampler
    HTTPSampler httpSampler = new HTTPSampler();
    httpSampler.setDomain("example.com");
    httpSampler.setPort(80);
    httpSampler.setPath("/");
    httpSampler.setMethod("GET");

    // Loop Controller
    LoopController loopController = new LoopController();
    loopController.setLoops(1);
    loopController.addTestElement(httpSampler);
    loopController.setFirst(true);
    loopController.initialize();

     // Thread Group
    ThreadGroup threadGroup = new ThreadGroup();
    threadGroup.setNumThreads(1);
    threadGroup.setRampUp(1);
    threadGroup.setSamplerController(loopController);

    // Test Plan
    TestPlan testPlan = new TestPlan("Create JMeter Script From Java Code");

    // Construct Test Plan from previously initialized elements
    testPlanTree.add("testPlan", testPlan);
    testPlanTree.add("loopController", loopController);
    testPlanTree.add("threadGroup", threadGroup);
    testPlanTree.add("httpSampler", httpSampler);

    // Run Test Plan
    jmeter.configure(testPlanTree);
    jmeter.run();

More details here.

  • That is a good idea - I have created Java-Samplers which handle making the http calls using the http-client directly, but had not considered creating the test-plans programmatically this way. I will try this and report back! – Kenneth S Aug 22 at 0:20
  • Two issues with the approach have occurred to me: 1) Running the test-plan with multiple remote clients using the JMeter master-remote feature. 2) Controlling test-plans at run-time using the bean-shell server and changing JMeter properties on the fly. The first issue may be the thorniest - how to setup the process, in code, to enlist itself with a JMeter "master" instance? I am wondering if I can create a test-plan fragment in XML that would change very little, whose only purpose is to wrap a user-defined TestPlan object that is defined in the lib/ext directory somewhere... – Kenneth S Aug 22 at 0:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.