What are some good open source UI Testing tools for testing Windows GUI?

Question

I need a list of some good tools which are open source and meant for automating the UI testing of Windows GUI. Any help will certainly be appreciated.

Answer 1

Sikilu:

Sikuli automates anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to a GUI's internal or source code.

Answer 2

TestStack White: (partial open-source)

White is an open-source framework for automating rich client applications based on Win32, WinForms, WPF, Silverlight and SWT (Java) platforms. It is .NET based and does not require the use of any proprietary scripting languages. Tests/automation programs using White can be written with whatever .NET language, IDE and tools you are already using. White provides a consistent object-oriented API, hiding the complexity of Microsoft's UIAutomation library (on which White is based) and windows messages.

With the free Visual Studio Express you can create automated tests with the .Net framework. See my example console C# app code to automate notepad.exe below:

using System;
using TestStack.White;
using TestStack.White.InputDevices;
using TestStack.White.UIItems.Finders;
using TestStack.White.UIItems.MenuItems;
using TestStack.White.UIItems.WindowItems;

namespace WhiteTest
{
    class Program {
        static void Main(string[] args) {
            Tests tests = new Tests();
            tests.Notepad();
        }
    }

    class Tests {
        public void Notepad() {
            // Arrange
            Application app = Application.Launch("notepad.exe");
            Window window = app.GetWindow("Untitled - Notepad");

            // Act
            var box = window.Get(SearchCriteria.ByClassName("Edit"));
            Keyboard.Instance.Send("test", box);

            window.MenuBar.MenuItem("File", "Save As...").Click();
            var filename = window.Get(SearchCriteria.ByClassName("Edit"));
            Keyboard.Instance.Send(DateTime.Now.ToString("yyyyMMddHHmmssffff") + "test.txt", filename);
            window.Get(SearchCriteria.ByText("Save")).Click();

            app.Kill();

            // Assert 
            //  file is created in Documents folder
        }
    }
}

Answer 3

This is a newer answer to this question, so give the state of things as of Feb 2019... I would say Appium, Microsoft is deprecating their Coded UI Testing tools and suggests using Appium + WinAppDriver.

From here:

Coded UI Test for automated UI-driven functional testing is deprecated. Visual Studio 2019 is the last version where Coded UI Test will be available. We recommend using Selenium for testing web apps and Appium with WinAppDriver for testing desktop and UWP apps.

Answer 4

One option would be to use robot framework with the autoit library. Robot is a keyword driven acceptance testing framework, which allows you to write your tests in a very readable, english-like format. Robot files are plain text so they integrate well with version control, and there is a robot format plugin for jenkins to support continuous integration. Robot also has a nice mechanism for tagging tests, and generates both xml and html reports and logs.

Answer 5

I'd like to add FlaUI to the list. It is basically a complete rewrite of White with more flexibility, more features, built-in caching and such.

Answer 6

Katalon Studio supports Windows 10 application testing. From v 7.0 (currently in beta).

I belive it is Winium based.

More info here: https://docs.katalon.com/katalon-studio/docs/introduction-desktop-app-testing.html#windows-execution-type.

Answer 7

There's more than one project on codeplex.com for automating the UI testing of Windows GUI, among all the other stuff there.

I have been using "UIAutomation Powershell Extensions" https://uiautomation.codeplex.com/

Answer 8

Twin is a tool for automating Windows graphical applications.The design is based on the web automation tool Selenium/WebDriver, and it can be used in similar ways. Grid 2.0 from webdriver was designed with TWIN in mind, so you can test against multiple virtual machines with different OS installed and let grid 2.0 install the last version of the AUT and route the tests to the correct VM. You can explore and download it from here: https://code.google.com/archive/p/twin/

Answer 9

I have worked on SilkTest and it is good. Also, as @Ankit mentioned there are many good UI testing tools, you can find here: http://en.wikipedia.org/wiki/List_of_GUI_testing_tools

I would like to add another one named Watir. Used it in one project for data scraping long back and found it helpful.

Watir (Web Application Testing in Ruby, pronounced water), is an open-source family of Ruby libraries for automating web browsers.[1][2][3][4] It drives Internet Explorer, Firefox, Chrome, Opera and Safari, and is available as a RubyGems gem.[4][5] Watir was primarily developed by Bret Pettichord and Paul Rogers. (Source: Wikipedia - https://en.wikipedia.org/wiki/Watir)

Answer 10

I've used both TestStack White (for controlling a windows app) - which was simple and straightforward to use - and Appium (for controlling Android).

Out of the two, I'd probably go for Appium if the app dev environment wasn't C# - I'm using Specflow as a BDD environment on top of Appium (ironically, I'm developing my android test frameworks in C#) and it's a great combination.

Answer 11

I can strongly recommend considering using the python robot framework plugin, (with or without using robot framework), to give a GUI test automation tool where you have the following capabilities:

• Use the same test scripts for testing on any of Windows, OSX, Linux (possibly even Android either via an emulator or by mirroring the device screen)
• Use with Robot Framework projects (https://robotframework.org), Robocorp projects (https://robocorp.com) or with pure Python projects.
• Open/Close or give focus to an App
• Find a specific image or text on screen or in a given region
• Wait for specific events
• Snapshot screen or region
• Mouse Actions including move, position, hover, click, right click, double click, drag & drop, etc.
• Keyboard actions including typing, special characters (including modifiers), pasting text, etc.
• Search for matches to an image or mask onscreen or within a region with a given similarity score.
• Capture & OCR text from an image captured from a region (using Tesseract)

I have even successfully used it on a virtual screen, (essential for CI/CT/CD workflows). The one thing to be aware of is that you need to consider screen sizes & resolutions either by having multiple target image directories & selecting based on the resolution or by using a fixed screen size & resolution.

To the best of my knowledge all of the tools involved are free & Open Source.