The Sikuli website says:

Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots).

Under what circumstances is Sikuli a better technology for test automation than Selenium or Watir, which depend on symbolic identifiers or paths for referencing UI elements? Please consider not only test creation but also test maintenance.

  • While I don't use Sikuli for test automation, I have found that a Sikuli script that reproduces a bug can be run in less time than it takes for a Dev to follow a set of steps to reproduce the same bug. Of course, YMMV.
    – user867
    May 11, 2012 at 2:13
  • 1
    @user867, 4 years have passed since you asked this question. Could you please share your experience in using these tools? What was your choice? And what were the reasons for that choice? May 5, 2016 at 9:13
  • I asked the question and user867 provided a comment. I ended up not using Sikuli.
    – user246
    May 5, 2016 at 13:13
  • For testing web application, there is an alternative to Sikuli: Kantu Browser - like Sikuli it uses screenshots instead of selectors, but it directly works inside Google Chromium.
    – Bobby231
    Feb 22, 2017 at 14:03

16 Answers 16


I can come up with some ideas:

  • You are testing for devs who don't know how to give their elements IDs or use other APIs, and won't go back and fix things in a timely fashion. This is accumulating technical debt - better to get the devs to do it right - but, e.g., on legacy apps designed without a thought toward testability, screenshot testing might be the only reasonable option for creating UI tests. In fact, this tool might solve an upcoming problem for me along those lines . . .

  • You are testing specifically to make sure that the UI stays reasonably similar in appearance. Normal UI testing doesn't care what the button looks like, and couldn't catch if the image used got deleted by accident or swapped with an old button image. I've seen other screenshot testing used for these purposes on major apps.

  • If you can get really accurate screenshots from the PM, you could make your UI tests before the IDs have been assigned to the elements and also ensure that the final UI matches PM specs, automatically.

As long as you can assign screenshots to variables and reuse them (so you only have one screenshot of the "Okay" button in the entire test suite), maintenance wouldn't be too bad IMO. It might even be easier if IDs are still changing and the UI is in extreme flux. I worked on a project once where devs kept removing IDs by accident, and I had to wait literally weeks before I got a reliable identifier quite regularly. But UI elements will always have an image, by definition, meaning a 1-minute test fix rather than a 3-week wait where I can't run that test automatically and need to run it manually, or a 15-minute to 1-hour fix to code a work-around function and test it.

The main difficulty I can think of would arise when there could be two or more similar pieces of UI (e.g., two "Okay" buttons, one on an error and one on a standard configuration screen), but even IDs can have similar issues (do you want "dynamicMenuOption11" or "dynamicMenuOption12"?), and you could probably fix this most of the time just by taking a bigger screenshot (e.g., get the edge of the "Apply" button or error dialogue to identify the right button). That's an easier fix than coding the ID in, checking other values, and so on, IMO.

Edit: I do think Tarun brings up a good point about how well Sikuli tolerates changes. The video claims some sort of "best-fit" matching, but how tolerant is that? And, if it is too tolerant, could it produce false actions and false passes?

  • 1
    The tolerance is adjustable, but it's default value is '0.7.' I've found that is a little too tolerant for the application I've been testing, but one of the longstanding bugs in the application I've been testing is that it's got very poor visual contrast. Make of that what you will.
    – user867
    May 11, 2012 at 2:05
  • What about selecting random value from Dropdown List with scroll bar where you can see only 10% of elements of the list? With Selenium you can get items count, generate random value according to the quantity of elements and select item by index in list. What about Sikuli? May 5, 2016 at 9:25
  • What about performance? It seems to me that it needs less computations to find element by xPath (in Selenium) than by finding subimage on fullscreen (in Sikuli) May 5, 2016 at 9:35

I have actually used Sikuli and believe it is the future for QA automation. The fuzzy logic on image compare is pure genius. You can crank it up to be an exact match or turn it down to lesser. I have ran the same test on multiple browsers/OSes and resolutions without too many issues.

It is way more robust than the old x,y testing of yesteryear. I created an image.sikuli script that hosts all my images and I just reuse them through out.

The app I am tasked with testing does not play well with QTP or TestComplete, I am unable to see elements. This seems to be more and more common with applications that are using embedded components. Sikuli bridges that gap, and I find it easier to use than the others (although the IDE is not very good). It does not have the record and playback that some of the other tools promote, but again doing that never produces a repeatable robust automation.

Selenium also does not do ANY image compare. Although the test may pass, it may look like crap on IE 9. Sikuli can at least do a high level compare on elements, and buttons to make sure they are rendering correctly.

  • 2
    What about selecting random value from Dropdown List with scroll bar where you can see only 10% of elements of the list? With Selenium you can get items count, generate random value according to the quantity of elements and select item by index in list. What about Sikuli? May 5, 2016 at 9:25
  • What about performance? It seems to me that it needs less computations to find element by xPath (in Selenium) than by finding subimage on fullscreen (in Sikuli) May 5, 2016 at 9:36

In my opinion, none!

When you are using images to locate elements then you are in trouble when someone comes along and decides to change how the application looks or the position of an element on the screen. You will build a fragile test suite and will be forever updating the images that you have used.

One concern would be that IE and firefox don't always render css in the same way and subtle differences would cause your test to fail, so you would need to be checking for different images in your test. This is really an overhead that you could save yourself from doing.

Selenium and Watir on the other hand doesn't care what the element looks like, instead you can find it using many different techniques (id, name, class...) that allow you to build a reliable test suite.

  • 1
    That is my impression too, but I thought I might be missing something. People use visual cues to understand how user interfaces are organized, e.g. all the text fields and buttons within a box are probably part of the same form, and the highlighted ones are probably mandatory fields. If Sikuli could work with visual cues, I think it could be as powerful as Selenium/Watir. Perhaps that is where the project is headed.
    – user246
    Jun 7, 2011 at 14:43
  • Have to agree that i see little use for this. Screenshot comparisons are notoriously difficult for computers, but very easy for us humans.
    – Ardesco
    Jun 30, 2011 at 11:20
  • 7
    I think some people overlook how to really use Sikuli. You don't take a screenshot of whole (web)page & work with it. That's not the point. There are screenshot tools you can use then manually compare or run some tool to compare screenshot to reference image. The way you use Sikuli is to define "atomic" UI elements (i.e. button on page, a link, some text) typically w/o dependency on layout of other elements and save cropped screenshots of them as reference images. Then compare them against what you see on screen at runtime. So you build a repository of graphic UI elements vs web UI locators.
    – David
    Jul 13, 2012 at 22:49
  • 6
    had to break up comment due to length. So you do Sikuli tests like can you click this button on screen (doe it exist anywhere on screen)? And you can make it relative to some other elements on screen as in button must be next to these items, etc. How atomic you make it depends on your needs. Fully atomic means it can be anywhere and not relative to some other elements. Using these techniques properly, changes in page layout won't necessarily break a test. You just have to update UI elements when changed (e.g. button look) and page layout only if using relativity of elements to each other.
    – David
    Jul 13, 2012 at 22:52
  • 1
    I have seen problems with Sikuli tests when screen resolution changes, desktop theme, number of colors, even RDP settings. If the system you run tests on needs to change - OS, or whatever, this is something to consider. The biggest concern I have is that the project does not seem to be active anymore so if you plan to use it to write new tests consider how that fits in with your overall testing tools.
    – jtreser
    Oct 21, 2012 at 16:16

I am not sure it's as easy as saying Sikuli is a better technology for test automation than Selenium or Watir - it's just an alternative.

On my current project, the front end test team were struggling to test an application using flexmonkium. They had the skills and experience to write tests in flexmonkium, but our application needed to be recompiled with the flexmonkium libraries, otherwise the tests simply couldn't hook into the application. It was not going to be possible to make this happen for quite a while, so the team needed another approach. We choose Sikuli, since it didn't need any hooks into the application under test - it simply works off what is on screen.

Sikuli has drawbacks (slow, unpredictable, cross browser issues etc.), but in some scenarios, it may be a good choice. You need to make up your own mind.


Sikuli is useful whenever you're automating something that's not web based.

I really like Selenium and Watir, but they're restricted to web technologies, and even though these are probably the most common nowadays, there are still plenty of companies who use desktop apps. For these, I find Sikuli and its fuzzy matches to be quite useful.

Bear in mind that the IDE is not great -- and that you'll need to use it to generate the screenshots and adjust matching precision and position.

  • +1, I was just going to comment on this. Nobody else seems to mention that Sikuli and like tools work on desktop and native mobile apps (mobile via emulator) whereas Selenium and Watir are web only.
    – David
    Jul 13, 2012 at 22:38
  • 2
    One other comment, you're not restricted to using the Sikuli IDE to generate screenshots nor setting precision and positioning. The IDE like Selenium IDE is for throwaway code/tests, and for beginners. There is a robust Java and Python API you can use to build your tests. And you can also manually create your reference image screenshots, just save as PNG format, or call code to dynamically convert from source format to PNG before passing to Sikuli.
    – David
    Jul 13, 2012 at 22:40
  • Yeah, I believe my comment was a bit too deterministic about needing the IDE for the screenshots and adjustments. I still find the IDE useful for those adjustments, though.
    – cbl
    Jul 14, 2012 at 16:28

I have used it for 1 day. I think it is easier to use and faster to learn. Also you have the benefits of image compare. (Selenium does not have this). This said, I think You could use it for Flash automation. (I never managed to do this in a good way with selenium).

Furthermore you don't have to learn autoIt or another scripting for desktop automation.

So it has some advantages. But in my opinion you are better of learning selenium and autoit, and maybe use sikuli as a plugin.

When starting from scratch with little programming experience and you need to have fast results. (or for demo purposes). You could use it, especially when you need desktop automation / scripting as well.

When you only need webscripting you could use imacros as well, since this can be picked up quickly.

  • I don't see how it's better learning AutoIt. Sikuli and AutoIt do about the same things GUI automation of web and desktop apps. Only difference is Sikuli is image based and AutoIt is UI component identifier based. But I have seen cases where AutoIt can't identify a desktop (not browser) UI element, so you'd have to resort to Sikuli, or commercial test tool, or make do with keyboard shortcuts or mouse movements to compensate for not being able to control some UI element.
    – David
    Jul 13, 2012 at 22:44

I've used Sikuli several times and my big win was for testing memory leaks on our mobile app (using the simulator) that i could not have found manually. Also running with Sikuli allows us to bypass budget restrictions :) Additionally we have customized apps, so i like that i can use Sikuli to make sure the right colour is applied in the right place - trust me i forget which client gets which label or which colour where, so i love it for that.

It's pretty easy to learn (IMHO) - i have no coding background and i am able to use it. The user forums rock - they will help and provide answers in a very short timeframe.

  • Can you explain how you used Sikuli to test memory leaks?
    – user246
    Aug 30, 2011 at 19:32
  • in the iOS simulator there is a "simulate low memory warning" so i had a simple workflow that i had done as proof of concept for running Sikuli on iOS simulator so i added a function and then called it throughout the script... then looped it 100 times :) worked like a charm Aug 30, 2011 at 19:57

My company has the standard-issue (at least these days) automation buzzword stack of Maven, Selenium, Jenkins, blah-blah and so on. I use Sikuli. What it takes them a week to automate I automate in a couple hours.

Sikuli can automate things Selenium simply cannot, such as Flash or iPhone sims. Selenium can check if a video file is there, but Sikuli can play it and 'watch' it. I see, especially with web pages, Selenium automators constantly battling page loads with all kinds of wait-this-and-wait-that, leaving the script sitting there for long periods doing nothing and at other times failing out while a page is still loading. Sikuli has event handlers (onChange(), onVanish(), etc.) which if used with any cleverness will handle the tests dynamic to the connection simulated, literally waiting like a user for things to happen and then acting on them.

Same with finding correct objects to test. A couple parameters passed into a loop and Sikuli will go 'find' the object to test just like a user, and likewise know when to give up. I see Selenium folks resorting to passing database hacks to find the same object - hardly an automated user-acceptance test!

Granted, to use the tool successfully Python skills are needed, but everyone knows Java or Ruby given they are the current buzzword-shiny things managers are chasing. And Python is nothing like those languages. And of course there is the risk of images changing - which is nowhere near as bad as you think (when iOS7 actually launches, I've got about a half-day of work updating images). But when used appropriately, my experience leads me to concluding Sikuli smokes Selenium.


Here some points about I have discovered:

  1. Sikuli can not select by 100% text match. So it is extremely difficult to use it in financial applications cause there are not too many different GUI elements, but lots of similar buttons, lists, tables... So how could you select specific string (by its contents) in table with 1000+ rows (only 25 are in a visible area, the question about lists & tables) with tool that works by text recognition. Even if you will find the way to scroll to the item you need here comes the problem to distinguish row with "Economic Community of West African States" from "Economic Community of East African States". With maximum recognition strength 0,99 they are still the same.

  2. Sikuli is unpredictable. Some GUI elements can change their skins, colors, images (here is an example of transparency in windows GUI, so the autotest was broken just because of changed background):

The same object still different colors

  1. Sikuli is fast to learn. Need just half of a day to create the first test (navigation through several windows, selecting values in drop down lists, generating report)

  2. Sikuli tests look like comix. In native Sikuli IDE test looks like comix, someone can like that but I prefer text instructions :)

Sikuli test looks like this in Sikuli IDE

  1. Sikuli is good for testing UI with high requirements in usability. E.g. mobile applications, simple web applications where the icons should be easy to distinguish, easy to find. If Sikuli (with default recognition strength) will make a mistake in finding icon, field etc. that means that (most likely) design of them is not that good (so even user could mismatch GUI items).

When Sikuli (image recognition) is better than Selenium (objects description)

  1. Need to check changes in interface
  2. Working with graphical interface (icons, buttons with images...) that should be very clear to user
  3. There is no easy access to objects properties (e.g. remote desktop. I'm using it in this case)

I have used Sikuli and found it to be a great help in testing WPF.NET GUI applications (which do not comprise standard window instances with window handles like other Window apps do) and are otherwise very hard to test. I think Watir and Selenium are primarily for testing browser based applications, whereas Sikuli can also be used to test non-browser GUI applications.


I think putting all your money on image recognition only tests might get you into trouble as soon as the look/layout of a site changes and you didn't or couldn't make your screenshot collection smart enough to still work - in that case all your test cases would need to be repaired. I'd try a combination of both - Sikuli as well as Selenium are available as Java libraries - combine it with JUnit and you're set. Write as much as possible in Selenium but use Sikuli for visual confirmation or for hard to implement stuff. (don't know about web design much, but in Java software with some test software only tools like Sikuli managed to do drag & drop properly)

  • This. Test automation is all about efficiency, and using each tool for what it's most suited for can be much more efficient than trying to find a one-size-fits-all solution.
    – user867
    Oct 24, 2012 at 4:55

I am using Selenium in my project for functional testing and because Selenium doesn't support GUI testing so for that I am using Sikuli to capture GUI defects so for me both are complimenting each other if used properly. Both sikuli and selenium has its pros n cons. It depends on which kind of web application you are testing and objective of testing. Sikuli has its limitation to capture functional defetcs on multi browser testing as all browsers renders differently. And Selenium don't has any feature to capture GUI defects. Only thing we capture a screenshot and check manually GUI defects. So instead of chechking manually I am using sikuli in our project. As requirement is to test multi-lingual, multi browser testing so all functional things I am covering with Selenium using Java language in Eclipse IDE. And capturing snapshots, then using Sikuli to find GUI defects. And its easy to integrate Sikuli with java because Sikuli itself is a JAR file which we can easily set in our eclipse buid path.


I just watched their video and answering this on basis of certain assumption I made, Sikuli is better when -

Tests is only on one browser, one operating system, one resolution, one (every thing else you could think of) OR if you are ready to develop different Sikuli scripts for each configuration.

Sikuli system some how reminds me of old time automation tool when UI operations would be tied to screen coordinates. And then we moved to better element locators for good reasons.

Though if you see the intro video of Sikuli for mac, I feel Sikuli is better off for these task then browser basesd UI automation in Agile environment.

  • It's not quite as bad as you make it sound. It depends on the application you want to test, of course, but most of the time Sikuli is only trying to match small sections of the screen, so resolution is rarely a problem, and the different appearences of browsers and operating systems can be compensated for with sufficiently robust test scripts.
    – user867
    May 11, 2012 at 2:10
  • Agree wit user867. Screen resolution only comes into play when you want to specifically test against something like a browser window and page rendering at 1024x768. If you're matching/testing against a button or other graphic element, it's size will always be fixed regardless of the screen size, unless someone changed the zoom level of the desktop, etc. And doesn't matter about position of element on screen, Sikuli knows how to find it anywhere on the screen.
    – David
    Jul 13, 2012 at 22:36

Sikuli is useful when there is no easy access to a GUI's internal or source code. Selenium or Watir identifies web elements by ID, Name, XPath etc. on the contrary Sikuli finds the elements by image/screenshot which is easier.


I prefer Sikuli for quick tests of a specific functionality/defect, and then Selenium for regression/catch-all E2E tests.

Some examples of my usage


  • The functionality of questionnaire. Checks for UI text based on ids, and then runs through all permutations of the questionnaire to check that all behavior and page redirects are as expected.

  • Run through --suite=all at the end of each story to make sure nothing was broken elsewhere in the app. It's quick and runs in the background, and can test in multiple browsers at the same time.

  • Coding is done in Protractor, which compliments our AngularJS app.

  • Seems more stable and refined to me, with much better documentation and resources online if something were to act up.

  • Minor point, but it's much easier to show "real" work is being done in Javascript as opposed to the Sikuli style of code. Devs and higher ups see it as more real engineering being done.


  • Quick metrics for running through a process as a real user would. I can very quickly put together a script that will use Python to keep track of how long the script is running at different points, then have my mouse taken over and the app gone through like a real person would. Not milliseconds like Selenium.

  • Sikuli isn't just for screenshots! On a previous project when I did the above for my manager, I used regions instead of screenshots. It helped since our lists we needed to click on were pulled from an api, and the source changed frequently. Made that script useful regardless of the tile it was choosing (I just had it pick the first in the list)

  • If there's a front end ready but identifiers aren't in place yet/at all, Sikuli doesn't care and just goes to work. You don't need access to the code at all, which works well if you're distant from the developers.

  • We once had a defect that was easy enough to reproduce, but the only issue it caused was that our grids overlapped and just looked wrong. Sikuli made it easier to look for a fuzzy image of the overlapping and check if it was still doing that each build.

  • Minor, but Python is a very easy language to learn, and is supported natively by Sikuli. Your tests can be a hybrid of the click(picture1.png) and Python to have a wide range of functionality.

I personally think Selenium is better for test maintenance, but Sikuli makes test creation a breeze. Also, Sikuli better mimics how a user would go through the app since it takes over the mouse rather than running in the background. Selenium strikes me as much better for larger work and a professional setting. Time and a place for both, I suppose.

  • There is a Python API for Selenium too.
    – user246
    May 18, 2015 at 19:47

I use Selenium (through CodeCeption) and it works well when I can target some IDs or CSS rules.

But we have Canvas & Flash games which expose nothing for testing. The only interface is the GUI !

These games have "events" to which my test has to respond correctly to resume the main "game" (Yep, the DEV didn't thought of test-ability!)

For these, I use Sikuli. It uses Python for scripts and it is possible to disable the display of image AND get him ask a name for each taken screenshot.

For handling "events" I use simple "try catch" constructs so my test can run through a game happily.

The longest part is reverse engineering since, of course, games aren't documented.

I have the same speed scripting in CodeCeption than Sikuli, if it takes more time in Sikuli, this is only due because I can't force these developers to provide interfaces for testing etc.

For the website, I ask IDs and classes on key elements (form elements, any interractable, outputs, ... )

If I fire Sikuli, someone did something wrong :-D

If you didn't pay attention to circumstances above :

  • Flash / HTML5 canvas
  • Moving HTML fragments that can't be identified easily (structural pattern, ID attribute, CSS selector, ...)

A side feature is the automatic generation of the HTML file describing the procedure. Project managers understand them ! (Ther is a lot of images...)