5

When we execute the same set of end-to-end browser-automated tests on BrowserStack, we notice a dramatic slowdown - tests are much slower than if executed locally and there are occasional failures because of timing issues. We try to address the failures by adding explicit waits, which overall make the tests less readable and the test flow less understandable and clean.

What are the general guidelines to improve on end-to-end testing execution speed?

Our stack includes Protractor/WebDriverJS; we use grunt tasks to start a local build server and expose it to BrowserStack via ngrok.

  • 1
    "By adding explicit waits, the tests become less readable and the test flow less understandable and clean;" from code review point of view, I guess? – Yu Zhang Jul 10 '17 at 21:11
  • 1
    @YuZhang correct, from readability point of view. Protractor itself is advertised as "You no longer need to add waits and sleeps to your test. Protractor can automatically execute the next step in your test the moment the webpage finishes pending tasks, so you don’t have to worry about waiting for your test and webpage to sync.". But, in reality, we still have to add waits here and there.. Thanks. – alecxe Jul 10 '17 at 21:13
6

I discovered the following points while speeding up my protractor(selenium) tests:

  • Run tests in a headless browser(from version 59, chrome can be run
    headless natively
    )

  • Run tests as direct connection(directConnect: true)

  • Don't log off & reuse user session between tests with the same base state(Home Page) wherever possible.

  • Make short & simple to the point tests with one and only one assertion only.

  • Run tests in parallel(using shardTestFiles/maxInstances)

  • Disable angular/CSS animations, it will help in faster page loads

    $('body').allowAnimations(false);

  • remove all explicit waits & reduce implicit waits to a minimum and supplement it with conditional waits(using expected conditions)/fluent waits(with short polling intervals) carefully

Custom Fluent wait Code Example:

browser.manage().timeouts().implicitlyWait(0);
browser.wait(function () {
    browser.sleep(1000);
    return element(by.id('id1')).isDisplayed()
    .then(
        function (isDisplayed) { 
            return isDisplayed; 
        }, 
        function (error) { 
            return false 
        });
}, 20 * 1000);
  • Remove redundant test steps in between tests(this is the biggest time saver)

  • Don't use UI tests to create/setup test data(use APIs to seed data), use UI tests to validate end-user business scenarios only.

  • Don't use shared(like with developers/business analysts/not even with manual testers)/or production replica(heavy) database .Use dedicated light QA automation database with carefully designed baseline data only.

  • Restore/refresh database before runs to keep it clean & light.

  • While designing test cases, eliminate unnecessary long UI steps and let the tests focus only on test validations with least page navigations.

  • 1. How to disable animation? 2. I tried removing implicit and I found myself ending up writing too much code for every click and senkeys etc. How did you do , do you mind sharing? What was your approach? – paul Jun 23 '18 at 14:54
  • 1
    would you mind posting it as a separate question so that all can contribute and provide detailed answers? – Vishal Aggarwal Jun 23 '18 at 15:02
  • You are right? I have added sqa.stackexchange.com/questions/34405/… here – paul Jun 23 '18 at 16:20
2

From my point of view here are some general guidelines to improve end-to-end testing execution speed for GUI automation.

1- Build a solid Object Repository by using unique attributes/properties to define test objects.

2-Devide test script into small test functions that are simple (Do not re-implement complex business).

3- Define correct points where to add Sync functions (moving to new page, waiting to object appear/disappear , waiting for property change,.....etc).

4- Use Dynamic waiting not implicit, for the correct behavior / property of a test object.

5- Modify the Driver Time out (Selenium) to be 1 : 5 seconds no more, because each time we use FindElement() Function the Driver will wait this time until object loaded in DOM or throw exception.

[Conclusion] - Design your Test Script by a way to get benefit of tool speed and also Synchronized with Application performance.

Example of Selenium/C# Dynamic wait for object to be loaded in DOM and displayed on GUI.

Calling for bellow Function:

Public IwebElement TestObject;
TestObject.Exists(5); // where 5 is time out by seconds.

public static bool Exists(this IWebElement element,int timeOut)
    {
        //GlobalVars.Test.Browser is IwebDriver Object
         bool isExist = false;

        // check that element is not null;
        if (element == null)
        {
            return false;
        }

        // check the element existance.
        try
        {               

            GlobalVars.Test.Browser.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(0);
            WebDriverWait wait = new WebDriverWait(GlobalVars.Test.Browser, TimeSpan.FromSeconds((double)timeOut));
            wait.IgnoreExceptionTypes(typeof(StaleElementReferenceException));
            isExist = wait.Until<bool>((d) => { try { return element.Displayed; } catch (Exception ex) { return false; } });

            return isExist;
        }
        catch (Exception ex)
        {
            return false;
        }
        finally
        {
            GlobalVars.Test.Browser.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(GlobalVars.Test.BrowserTimeOut);
        }   

    }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.