0

Is there a reason my automation tests always fail at night at a certain time on Jenkins while in the daytime it is more likely to pass? Is it someone accidentally cleaning the area and tapped on my devices? Or is it a network connection lost usually around that time at night? The strange thing is every time I sit in front of the devices and the computer where it runs from it passes. But if I walk away and not look at it, somehow it fails. I'm thinking someone is touching it to mess up the tests to fail. Paranoid a bit. Or is my code that stinky? But how can I explain it passes on my local machine and when I run on Jenkins if I'm watching it for every second like a hawk it passes. Everytime my eyes are not on it then it fails. Or someone doesn't want me to do this so I lose confidence? Anyone has any suggestions? What to look out for?

5
  • Define fail, do you mean the tests just wont run or your assert statements fail? Aug 24 '18 at 16:30
  • Probably when your tests run in Jenkins they are just unable to obtain a screen?
    – Alexey R.
    Aug 24 '18 at 16:48
  • They fail when fail to tap to open up an element to be clicked on. The tap to open up more elements on the screen sometimes fail. I also put in a check before this step and sometimes right before this there is a loading icon probably due to network connection problems. But other times there is no loading icon so it shouldn't be network connection problems. Aug 24 '18 at 16:53
  • 1
    Introduce taking the screenshot on fail. So that it would be more clear what does your test see exactly when encounter failure.
    – Alexey R.
    Aug 24 '18 at 17:37
  • It does take a picture, but it just shows it was on the last screen before tap. Aug 24 '18 at 18:00
0

The scenario here look somewhat strange, however believe need to capture the issues, fail reasons by putting advanced levels of reporting/loggers/screen captures and possible for check purpose video capture of the test execution.

Verify the above mentioned process consistently for a period and hopefully you would find solutions.

On the other side, recommend to look for network and test application performance during day and the night times, it may be possible during night times the build is slower as it is hit by many users or network does not have sufficient band.

Hope help, Thanks

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.