6

We are using a testing framework composed of robot-framework with ExtendedSelenium2Library that our developers are using to write acceptance tests. Our Web site has several auth options, including direct user/pass but also Google and Twitter OAuth. Devs can run these tests directly on their desktop, and they also get run via our bamboo CI setup locally and using Sauce Labs for crossbrowser.

Our attempts at testing the Google login to the site work OK on the developers' desktops but as they move into CI (elastic bamboo uses on demand instances in AWS, so IPs etc vary), Google starts responding differently with obvious antiautomation responses like this:

enter image description here

The responses start varying and not being consistent. Is anyone else doing webdriver-type automated testing of a site with Google auth, and if so how have you addressed this problem? Are there any google account settings (we're a Google Apps customer), or any ways to pre-set up a profile (any browser is OK) to ensure this functionality would really work for a customer with each build?

Improve this question
2
  • 1
    I think the plethora of anti-robot measures might indicate that Google does not want you to interact with it using robots, even if you are a Google Apps customer. Some of the Google apps have APIs for doing automation; those will probably be easier to program against than writing automation against their web pages.
    – user246
    May 16, 2016 at 18:39
  • Something specific along those lines would make a good answer.
    – Ernest Mueller
    May 16, 2016 at 21:21

2 Answers 2

Reset to default
7

We have found that dealing with third party software in CI/dev environments was a great source of intermittency/failure and pain. So in our CI environments, we always create stub services for third parties.

A stub service is basically a fake API service that you control, that behaves the same (or similar) to how the real service does.

I would recommend this method as it provides the following advantages

  • Don't have to rely on external service always being up
  • Don't receive anything unexpected (like antiautomation responses)
  • Have complete control of the stub's responses, so you can test how your service will behave if it receives something strange from the external service.
  • Don't have to have external connections to the outside world in your CI environment
Improve this answer
2
  • But half the point of testing with the service is to find changes to the service... How do you keep something like a google auth stub up to date?
    – Ernest Mueller
    May 17, 2016 at 14:40
  • 2
    Public APIs of external services shouldn't really change. Whenever they do, it usually comes with advanced notice. That being said, some do change without notice from time to time. So we have a different job in Jenkins that purely checks 3rd party services have not changed. But we don't put it in our main CI acceptance pipeline
    – user3557137
    May 17, 2016 at 16:13
4

I think the plethora of anti-robot measures might indicate that Google does not want you to interact with it using robots, even if you are a Google Apps customer. From Google support: Automated queries.

Google's Terms of Service do not allow the sending of automated queries of any sort to our system without express permission in advance from Google. Sending automated queries consumes resources and includes using any software (such as WebPosition Gold) to send automated queries to Google to determine how a website or webpage ranks in Google search results for various queries. In addition to rank checking, other types of automated access to Google without permission are also a violation of our Webmaster Guidelines and Terms of Service.

Some of the Google apps have APIs for doing automation; those will probably be easier to program against than writing automation against their web pages.

Improve this answer
3
  • I kinda meant an answer that explains what APIs and actually helps with my problem...
    – Ernest Mueller
    May 16, 2016 at 22:04
  • The link in the last paragraph will lead you to pages that describe APIs. I will leave it to someone else to list them -- it's a long list. Or perhaps you could try doing it yourself.
    – user246
    May 16, 2016 at 22:22
  • It's none of them as best as I can tell. This question is about using google oauth to log into a third party app, not a google app.
    – Ernest Mueller
    May 16, 2016 at 22:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.