UI Test framework from the ground up

Question

I have explored some test automation tools and I thought I'd go with WebDriverJS (Selenium/JavaScript using Node.js) with Mocha as the test framework.

I have been able to write out scripts that do what I want them to do (ie. open browser, login, do X), and it has made my regression testing a lot easier and more efficient, but I consider it a "hybrid" approach. It does a bunch of actions for me, but it still requires me to run the script, look at the results, and pass/fail my test cases. This is obviously not scalable as the company grows.

The rest of my team are "non-technical"; they do not know much about automation nor do they have any interest in learning or teaching me, including my manager. I am able to run the scripts locally, but I am having trouble hooking it into something remote.

There are two basic points that I want to accomplish:

• I want to be able to run a very basic regression of core test cases remotely, and allow my non-technical QA team members to be able to click a button that says "Run Regression" or something of that sort
• Once that test has ran, I want to be able to get a nice human-readable report of the stats regarding the test run (Pass/Fail status, execution time, etc).

What are the steps I can take to accomplish these two goals? My company uses jenkins to build/deploy code, so I would imagine that is the place that my coworkers can find this "Run Regression" button.

Answer 1

You can do it, but it's not "simple" or "quick". I have done one that was really sweet, but it took awhile and a team of people. HTML result output, gui test case selection and kick off, robust testing with error handling and step skipping on fails.

Unit test framework will give you a code interface to execute script files from and get a result. I trained non-tech testers to do this part just fine. I also had to encapsulate Selenium under a simplistic framework that intellisense prompting helped testers to write each test step. If they don't need to write/modify the tests and you are doing that, you can just use the framework through an IDE like eclipse/visual studio and there is a play button to kick off the tests from.

It should tie into CI just fine so long as you ensure it's setup properly for Jenkins to read and checked into a project/solution that is setup on CI. I personally recommend a separate project/solution than the standard code.

If you get it all tied into CI you can likely create a simple integration with the API and tie it into a webpage with a button on it. You can also likely utilize the results through Jenkins to avoid extra custom report work, but that might require training on Jenkins then.

Essentially this is a fully custom designed thing and you need to figure out all the components you want and how much effort you want to put into it. Once you know that feel free to ask on here again about specific points where you might want feedback, but this one question is very generic right now, so the answer is yes you can, but it's custom code and integration depending on the users, tooling, and the project with which you are automating.

Answer 2

You have Jenkins. Now you need to do to add your unit tests and your UI test as Jenkins projects. No need even to "press a button" - Jenkins can run your test based on hour, or based on checkins.

Jenkins generates pretty good web accessible info:

• dashboard showing status of all projects ("weather dashboard") and
• reports - at least easy-to-distinguish "All clear", "Cloudy" and "Fail" reports.

Jenkins will even tell you how many and which test failed. If it is not enough, or not obvious, you can create a wiki page in your corporate wiki (you do use wiki for knowledge capture, right?) with examples how good ("blue ball") and failed ("red ball") test looks like. Humans are pretty good at seeing similar patterns.

Anything tricky and non-obvious ("most test passed but some test failed") will have to be analyzed by human, and automating it is a waste of time.