I have a package containing the tests I want to run. There are no priority or dependency setings at the testcase or suite level.
<suite thread-count="1" name="reg 1"> <listeners> <listener class-name="my.org.project.listeners.TestcaseOrdering" /> </listeners> <test name="Regression" preserve-order="true"> <packages> <package name="my.org.testsuites" /> </packages> </test> </suite>
IMethodInterceptor. In my
intercept() implementation I re-order the tests based on my criteria (run the tests that have run the least # of times first, based on an external DB tracking test results). I store the list to return in an
ArrayList<IMethodInstance>, which guarantees that the order is preserved.
When TestNG executes the tests it does not respect the order of the list I return. It runs the test cases in a fixed order (meaning, the same each time I run). The order does not have a recognizable sequence; the methods are pulled from various class files and aren't in alpha or numeric order.
I'm stumped as to how to get TestNG to respect my order. Based on the documentation, this is the place I should be doing this re-ordering.
I discovered the issue when using Selenium Grid, but removing Grid (setting the thread count to 1) doesn't change the behavior.