While running test I use different test-suites (xml files with list of tests to execute). E.g. I need to run smoke-tests so I choose corresponding xml with simplest tests (login/logout; selecting main tabs; check that data is displayed).
But in generated allure report there are all test that are in the project (all methods with @Test
annotation). And tests that have not been executed have statuses passed
, failed
; this information misleading to conclusion that we have tested more than we actually did.
How can I fix this? I need correct information about test execution (without misleading information that there are some more tests have been executed)
The same is inside testng report; report states that all tests were executed, and some are even passed (but they actually have not even started).
P.S.: If it is not fixable, how do you deal with this (getting results for a subset of tests of your project)?