Our team currently use Selenium and C# (and NUnit) to run UI Automated tests. All tests have been manually programmed, meaning no recorders have been used.

Issue: We now have a request that these tests track their own performance (and past performance) and raise a warning when its runtime increases by x% (5% or 10%, etc).

Question: What would be the best way to accomplish this? Should we create to tool to analyze performance (performance history) of these UI and API tests from scratch or are there other tools out there we can leverage?

Blogs or stackExchange questions discussing load/performance testing usually reference three main tools for C# (NeoLoad, SilkPerformer, LoadRunnerProfessional). However, I'm not sure that what I'm being asked to do is considered performance testing (load testing) in the purest sense and therefore, not sure whether the tools mentioned above will help achieve the overall goal. They also usually separate performance/load testing from UI/API testing.

  • Please don't cross-post, see meta.stackexchange.com/q/64068/248731. – jonrsharpe Jul 23 '20 at 13:58
  • 2
    Does this answer your question? Performance Testing: Flag when a test's runtime increased by x% (Selenium C#) – Alexey R. Jul 23 '20 at 15:15
  • 2
    @Fractal the fact your question has been downvoted does not mean it is banned. It is still accepted for collecting the answers. Downvotes might mean a lot of different things and a single downvote just means that a single person does not like what or how you ask. This should not be the reason of duplicating your questions. – Alexey R. Jul 23 '20 at 15:17
  • 1
    @jonrsharpe Please don't hate me - cleaned it up a little. But you're absolutely right, cross-posting is generally discouraged. – corsiKa Jul 24 '20 at 0:17
  • 1
    Fractcal, I would consider cleaning up the question a little to make it more objective. Especially, nix the summary - you're not looking for advice (which we don't give) you're looking for a solution to a real problem you're facing at work (which we DO give.) Also, in my experience, if you're ensuring these tests run in a certain amount of time, it definitely counts as performance testing. Load testing is only one facet of performance testing. – corsiKa Jul 24 '20 at 0:23

The test results file (.trx) is an XML file which you could code a parser for. You could scrape each test name and how long it took. You would then need to save that to a database or something. Then when the next test runs you can compare the two using your script and "flag" those tests. You can decide whatever you want to do with those flags tests, like emailing the list to interested parties. I used a similar script in the past to do all my morning checks and send me an email to review when I got in each morning.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.