Balancing readability and reliability in end-to-end testing

Question

One of the things that often helps to make tests more reliable and to reduce flakiness is Explicit Waits - waiting for an element to be visible or clickable before interacting with it, waiting for a page url to change etc.

But, on the other hand, adding explicit waits makes the codebase larger, less readable and the flow of the tests less explicit and understandable.

What are some of the things we can do to keep tests readable and clean but also reliable and not flaky?

Our context: Protractor/WebDriverJS, Jasmine, using Page Objects

Answer 1

For readability and reliability:

• The codebase being larger is not an issue for me. All that gets changes is the way certain finds wait, plus a few short methods to: wait for page to download, wait for page to render, wait for js to finish, wait for element to appear, etc.

• Tests that are less readable. Also not an issue for me. I'm likely to use something like wait_for(el) or wait_for(el,time) instead of find(el) and that is easy to understand to me.

• Workflow that seems less readable and understandable. Also not an issue for me, code that is there to do the find being changed to wait_for is still reasonably understandable to me.

So back to the title:

Balancing readability and reliability in end-to-end testing

Reliability is important. If wait strategies are needed to make the tests reliable then they are needed and should be used. The two things I would do to try and make the tests readable are:

1. Make sure that the various wait strategies are extracted to helpers. Name them well, e.g. wait_for(el), wait_for_page_load, wait_for_render, wait_for_js, etc. wait_for(el) should be a wrapper around the regular find you use for element(s).
2. Use wait helpers consistently throughout the application.

Answer 2

[Previous Answer]

This answer is how to speed up your end to end testing and also keep your test reliable by synchronizing scripts with application server performance

The remaining part from above answer , Is how to make your script more readable. Please find bellow some guidelines:

1. Use a shared object repository , not a private repository.
2. Design the repository to be organized as following structure [Browser.Page.TestObject.Action()] so when scripting your code will be more organized and readable.
3. Implement explicit wait functions for behavior/ properties of test objects and pages, with readable meaning [WaitUntilEnabled , WaitChildItemsCountGreaterThan , WaitUntilHidden , Exists]
4. Implement Functions to handle sepcifice action or property for tables, drop down lists,...etc [RowCount,ColumnsCount,CellChildElement,Select,ScrollDown]
5. Last but not least , use C# selenium extensions , or any language extensions mechanism to extends IWebElement and IWebDriver classs.

Bellow are 2 Examples for [Objects Repository and Extensions]

Object Repository

 namespace Framework
{

    public class StackExchangeObjectRepo
    {

        #region Constructors

        public StackExchangeObjectRepo(IWebDriver contextTestObject)
        {

            pgMain = new pgMainNode(contextTestObject);           

        }

        #endregion

        #region Test Objects

        public pgMainNode pgMain { get; private set; }

        #endregion

        #region Pages Classes

        public class pgMainNode
        {
            #region Constructors

            public pgMainNode(IWebDriver parent)
            {
                PageFactory.InitElements(parent, this);
            }

            #endregion

            #region Test Objects

            [FindsBy(How = How.XPath, Using = "______")]
            public IWebElement lnkUserName;

            [FindsBy(How = How.XPath, Using = "______")]
            public IWebElement lnkLogOut;

            [FindsBy(How = How.XPath, Using = "______")]
            public IWebElement lnkAdminLogOut;

            #endregion

        }

        #endregion

    }

Extensions

   namespace Framework
{

    public static class Extensions
    {

        /*
        GlobalVars.Test.Browser is an IWebDriver Class
        */

        /// <summary>
        /// This function will wait untile ready state is Completeted then check that element is displayed.
        /// </summary>
        /// <param name="element"></param>
        /// <returns></returns>
        public static bool Exists(this IWebElement element)
        {
            bool isExist = false;

            // check that element is not null;
            if (element == null)
            {
                return false;
            }

            // check the element existance.
            try
            {

                GlobalVars.Test.Browser.Sync();
                GlobalVars.Test.Browser.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(1);
                WebDriverWait wait = new WebDriverWait(GlobalVars.Test.Browser, TimeSpan.FromSeconds(0));
                wait.IgnoreExceptionTypes(typeof(StaleElementReferenceException), typeof(NoSuchElementException));
                isExist = wait.Until<bool>((d) => { try { return element.Displayed; } catch (Exception) { return false; } });

                return isExist;
            }
            catch (Exception)
            {
                return false;
            }
            finally
            {
                GlobalVars.Test.Browser.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(GlobalVars.Test.BrowserTimeOut);
            }

        }
        /// <summary>
         /// This function will check existance of an element then wait untile it Displayed = false Or Time Out is Over
         /// </summary>
         /// <param name="el"></param>
         /// <returns></returns>
        public static bool WaitUntilHidden(this IWebElement el, int timeout)
        {
            bool tempRes = true;

            try
            {

                if (Exists(el))
                {
                    GlobalVars.Test.Browser.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(0);
                    WebDriverWait wait = new WebDriverWait(GlobalVars.Test.Browser, TimeSpan.FromSeconds((double)timeout));
                    wait.IgnoreExceptionTypes(typeof(StaleElementReferenceException));
                    tempRes = wait.Until<bool>((d) => { try { return !el.Displayed; } catch (Exception ex) { return false; } });

                }

                return tempRes;
            }
            catch (Exception ex)
            {
                return false;
            }
            finally
            {
                GlobalVars.Test.Browser.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(GlobalVars.Test.BrowserTimeOut);
            }

        }
    }
}

Calling Example to show Readability of your Test

  // Calling Example:
public class Test
{

    StackExchangeObjectRepo StackExchange = new StackExchangeObjectRepo(ParentWebDriver);

    public void Testfunction()
    {
       if(StackExchange.pgMain.lnkUserName.Exists()) 
           StackExchange.pgMain.lnkUserName.Click();
    }
}

Answer 3

I completely disagree with your premise that explicit wait decrease readability of my code.

I wrote a custom wrapper for wait, which not only enforces explicit wait (which as you also noticed decreases flakiness), but it also increases readability by wrapping additional helper activities to the necessary actions.

Years ago, I was trained to understand any programming project as developing a custom domain-specific high-level language to solve problems in the domain area. This is not any different. With competent programming (and investing some in proper names and modularization), your code is both more readable and more reliable.

Of course it will take several attempts, refactoring, but it is just an old-school competent programming.

As always, when you squeeze the complexity balloon, complexity will flow somewhere else. So with such custom high-level language, sometimes you need to descend to lower levels to accomplish some functionality. But again, if such actions are common, just refactor them into wrapper method.

Pareto principle says you can get 80% of the functionality with just 20% of actions. Refactor wisely.

Answer 4

As you have written, the explicit wait will make the test more reliable but it will also make sure all test will pass.On contrary, I do not think codebase become larger an issue as our priority is to write efficient test, and explicit wait does not make code less readable if you follow proper naming conventions.

Answer 5

I don't see how readability and reliability conflict each other in anyway. Research Clean Code and refactoring. Test-code is code and should follow the clean-code guidelines if you ask me if you want it readable.

There are more ways to make your code more readable. Every method (including your wait's and such) longer than 5 lines should be refactored into even smaller functions with good descriptive names. This to improve readability.

I would hide waits with some methods like this simple example:

class loginPage {
  public login(userObject) {
    waitAndtypeUsername(user.username);
    waitAndtypePassword(user.password);
    waitAndSumbit();
  }

  private waitForMethods...(argument) {
    // contains selector, wait code, focus code, type or click code.
    // probably around 5-10 lines max, or else make more methods.
  }
}

Maybe you can remove the wait from the methods names if you always wait, only add extra information if it helps readability. I would keep the public pageObjects methods on the top and the private methods under those.

“Clean code always looks like it was written by someone who cares.”

I don't see how more code makes it less readable if you practise clean-code. Reliability is some code you hide in your pageObjects. Your tests read clean by using pageObjects the right way. Your main pageObject methods should also read clean. They should be short and read like well written prose.

“The ratio of time spent reading (code) versus writing is well over 10 to 1 ... (therefore) making it easy to read makes it easier to write.”\

I would really urge every developer (also test-automation-developers!) to either read the Clean Code book or watch the video training course if you have problems with unreadable, unmaintainable, rigid or fragile code.