0

I am having a hard time finding any test frameworks or plugins for frameworks that support saving "validation data strings" as part of the test results. I find this a little surprising so I am hoping that I am just missing something. Essentially, I'd like to be able to make a call of some sort that would store debug, performance, or other data along with the test case (test method) results.

This may deviate from standard xunit but to me seems valuable and I was wondering if I need to somehow add this feature myself to an existing framework or make my own lightweight framework that supports this.

Any input much appreciated. Even a pointer to a non python framework that supports this would be interesting.

Improve this question
5
  • Do you mean logging frameworks or plugins? I would have thought any kind of logging plugin would be able to work with your testing framework. I'm not familiar with Python, but I'm doing plenty of custom logging with the testing I work with (mostly Microsoft CodedUI and load testing).
    – Kate Paulk
    Commented Feb 6, 2018 at 16:36
  • Hi Kate, thanks for your comment. I was thinking more along the lines of a way to log that is integrated with the test results in the end. In other words, if I have a test method that logs into a home page, for instance, I could log the time it takes the page to load after clicking the login button and have that log message inserted into the test results. I am thinking that I may have to write this code myself but I wanted to be sure that I wasn't missing something already out there.
    – chrismead
    Commented Feb 6, 2018 at 16:58
  • I usually handle that by using the logging framework the test framework supplies. That automatically includes the messages as part of the results.
    – Kate Paulk
    Commented Feb 6, 2018 at 17:07
  • I guess your results end up in a trx file? Those seem to have more than xunit: github.com/x97mdr/pickles/blob/master/src/Pickles/Pickles.Test/…
    – chrismead
    Commented Feb 6, 2018 at 19:25
  • Yes, they do. Some of the other frameworks I've worked with allow something similar
    – Kate Paulk
    Commented Feb 6, 2018 at 20:36

1 Answer 1

Reset to default
1

When i've written frameworks in the past i would have considered that information loggable and would have logged it to a file.

If you include a lot of information in your xunit file it can expose the limitations of XML parsers , this will slow down your CI system when displaying results and generally make the files hard to deal with.

If you can use the nose test runner there are some options for test output that will help if you use its xunit output.

    -d, --detailed-errors, --failure-detail

Add detail to error output by attempting to evaluate failed asserts

https://nose.readthedocs.io/en/latest/usage.html

Improve this answer
3
  • That is a good point about the xunit file. XUnit doesn't seem to cover non-failure info and extending it might not give good results. reflex.gforge.inria.fr/xunit.html#xunitReport
    – chrismead
    Commented Feb 6, 2018 at 19:20
  • I guess I will create an intermediate results format with more info and an xunit exporter. Thanks for your help
    – chrismead
    Commented Feb 7, 2018 at 20:57
  • At the simplistic level you can just pipe stdout to a file
    – Amias
    Commented Feb 7, 2018 at 21:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.