2

A have given website say - https://sqa.stackexchange.com/ I need to find out if there is A/B testing being run on the websites. I have come across some tools like, - Ghostery - Conversion.com's Optimizely Chrome

Ghostery - Just scans the webpage for different things and also reports back if there any A/B testing that is running on the site. Optimizely is method which Ghostery tells us that the page is running Optimizely.

Conversion.com's Optimizely is a Chrome plugin, which checks if Optimizely is running on the webpage.

Please let me know if there are any efficient ways to find out if a webpage is conducting A/B testing and if so what elements are under the A/B testing.

  • 1
    Not sure why it was closed (except "closing mafia strikes again"). Seems to be a valid question, completely withing capabilities of browser automation tools. – Peter M. - stands for Monica Mar 21 '17 at 13:26
  • 1
    why you want to know this? Maybe knowing the reason might help to guide the search for the solution. – Peter M. - stands for Monica Mar 21 '17 at 13:27
  • This is needed sometimes to know what A/B testing your competitors are using, or for a QA test point of view to find out what experiments are being run a page or for a QA interview purposes. – WiredTheories Mar 21 '17 at 14:34
  • 1
    @AllPigsREqual - Depending on how the A/B tests are implemented you maybe be able to see the experiments in your local or session storage of your browser. Even with this though you still won't know what "A/B testing your competitors are using" unless you know the name of the experiments and the possible variants involved. My guess is neither of these will be good enough for competitors or interview purposes. – Chris Kenst Mar 21 '17 at 16:06
  • 1
    "How to find out if a website is performing A/B Testing" - the same way you can find out about any kind of testing on a particular website - ask the site owner. – Joe Strazzere Mar 21 '17 at 19:13
3

There is no certain way from a black box perspective, especially if the tests are not just 50/50 binary switches. For example if a test is being run at 5% / 95% (as we do in my company) that it will only show up, on average, once every 20 runs.

Given the above, there are a few things I can think of that you can do:

  1. If the ab test is performed via URL parameters that you can see, e.g. ?&AbTest=show_details:true

  2. Visit the page / workflow multiple times and observe any differences. This may require a tool, one that does screenshot comparisons such as GhostInspector

  3. Create a simple test in seleniumIDE and run it repeatedly to observe changes

  4. Write selenium based tests in a programming languages and run multiple times to see if different results are ever located.

  5. Thinking outside of the box (whether black or white) - ask the company if they are doing this!

  6. Look in network tools for page source, network tab and cookies and look for anything containing the name ab, abtest ab_test, ab_, etc.

1

I agree with some of the earlier comments the best way to know is to either ask or observe (which is much harder). If you know what A/B testing tool the site is using it's possible to do some research within documentation to help understand how things are possible stored. (Optimizely for example.)

From a browser standpoint you can look for a few things (this will depend on how the A/B tests are implemented):

  1. If the tests are stored in localStorage then they could possibly appear under DevTools > Application > localStorage for that particular page. Or for that matter Session Storage.
  2. Look for a possible URL parameter (Optimizely uses a URL parameter for opting out like http://www.example.com/?optimizely_opt_out=true)
  3. Use a visual comparison tool like Applitools to take a screen shot of the page and then run several tests to determine if anything on the page changes. Changes are usually recorded as failed tests.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.