Return to Revisions

How do you reduce the time of your regression suite?

• This may depend on what technologies the regression suite is based on. There can be technology/tool specific optimizations, like using faster database drivers, test runners, language/interpreter optimizations, or, as @siutex mentioned use Explicit Waits instead of hardcoded time delays, which tend to wait more than needed. Let's not go this way further and stay on a high-level

• The first thing to mention would probably be to "scale" your regression test suite - divide the regression test suite between CPUs or separate machines. For instance, in Python both nose and pytest test runners support parallelization between multiple CPUs or hosts (pytest-xdist)

• Reduce the effect of a network - decrease the negative performance effect of the network traffic between the components of your system. For instance, let your test database be on the same server as your web application. Look into using containers (thinking docker)

• Find and remove repetitive/redundant test "set up" and "tear down" procedures. As a practical example that I stumble upon in our Python regression test suite - oftentimes things that were executed in every test method of a class (setUp and tearDown functions) did not need to be executed for every method and could be applied once per test class only (setUpClass and tearDownClass functions)

• And, of course, profile and measure to understand the bottlenecks - detect slowest parts of your test suite and figure out why are they slow and see if you can apply optimizations.