JUnit-friendly framework for production monitoring

Question

My team deploys code to production every week or so. We have a large set of post-deployment validations, e.g.:

• New log lines are appended to Tomcat log (ssh to machine, tail -f the log)
• Database table has more than 100 new rows in a minute (ssh to machine, run some sql commands)
• A GET call with a given parameter returns the right value (browser or wget)

The tests are done manually, and takes 2-4 agonizing hours.

Since we have a Jenkins CI system, I would like to automate these tests to small units which produce JUnit XMLS which can be analyzed by the Jenkins web UI.

Is there a test suite capable of easily ssh-ing servers with JUnit output?

Answer 1

1. Use JUnit to handle the running of the tests from the Jenkins server.
2. Use JDBC to query the DB for new rows.
3. Change up the Tomcat servers so they send logging events back to the Jenkins serverJUnit test. You'll need to write a little socket listener for the JUnit test to gather the Tomcat events (Writing an LoggingEvent listener isn't hard. I'm not brilliant coder and I got the event listener with Observer-Observables working in less than a day.)
4. Java has tons of http(s) GET/POST capabilities so wget shouldn't be hard to implement either.

I don't know of a testing framework that will let you abstract away all the boilerplate code of building JDBC connections, SSH output, etc. Maybe Spring does that kind of abstraction.

Have you looked at JBehave? It's an acceptance testing framework that sits on top of JUnit.

Answer 2

Using the Jenkins server is a great idea. You just need to create Maven projects that run basic JUnit or TestNG tests. The JUnit or TestNG tests should accomplish your tasks and provide assertions. At least, that is how I would do it. You just use the post-build action in Jenkins to find your JUnit output .xml files (requires a special plugin to do that).