Since you're dealing with client-server architecture, you're also dealing with network latency, spin-up times, disk latency on the server and the client system, potential I/O bottlenecks (and the testing tool itself is not exactly lightweight in I/O terms), network traffic, and numerous other factors.
Your first priority should be to limit/minimize as many variables as possible. That means isolating your application's network into its own limited-access subnet so that the application client and server systems aren't competing with the rest of your organization network.
For this kind of testing, you also want the systems on dedicated hardware if at all possible. If you're running the client or server on a virtual machine that shares resources with other VMs on the host, you're going to see a wide variation in performance times.
If you can, the system(s) should be on as close to the expected production environment as possible. If the server is expected to be load-balanced, then your test should use load balancing. If there's a specific operating system targeted, use that OS.
If, once your environment is as clean as possible, you're still seeing large variations, you could see if it's possible for the developers to add logging to capture start/end time for your targeted tasks. You may find that the test tool is adding variability to the recorded times (this isn't uncommon: all of the big box test tools I've worked with are extremely I/O heavy and can introduce significant latency issues. The OS you're using is also a factor - different operating systems have different I/O profiles).
If you absolutely can't eliminate the variability, your last option is to run your tests multiple times and report averages and modal values.
