Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I have access to a CI Job Jenkins's JSON API. Here is how I have the sequence

  1. I crawl a CI job and look at all the builds
  2. I look for the latest test report and filter out the test reports which are older than the latest code of automation. This way I have narrowed down the number of builds.

Now I have a list of test results to compare for flakiness of test cases. I can successfully do this only if i know that the input is constant to all these tests and that only the test is different.

  1. How can I find that the input is constant when its a complex piece of code like UI of web-page (like a hash?)

  2. How can I Get that input data from Jenkins or other place? For UI automation of web-sites

  3. How can i solve this problem of verification of input being constant in a general way.

share|improve this question

1 Answer 1

How can I find that the input is constant when its a complex piece of code like UI of web-page (like a hash?)

Depending on how you use Jenkins, there may be a build number associated with the test. You can look for code changes associated with that build.

How can I Get that input data from Jenkins or other place? For UI automation of web-sites

Jenkins has a REST API. See https://wiki.jenkins-ci.org/display/JENKINS/Remote+access+API.

How can i solve this problem of verification of input being constant in a general way.

I don't know how to answer this. What kind of generality? If you version your SUT and your tests, you should be able to determine differences. Of course your test could have other dependencies whose differences are harder to classify/quantify, e.g. running against a production database backup that changes every week or running a time-dependent scheduler.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.