3

First, please leave your stones down: I'm a windows-oriented developer and I have very limited knowledge in testing tools world - especially in those that works on multiple platforms.

I have been assigned to write from scratch a c++ application, which should work as a service (daemon for Linux) and reads some data from a COM port. I have full access to SVN server and I can tweak it as needed. The test units are written using boost library.

My question

is there any tool (preferable open source) which can called after an SVN commit to trigger repository update from SVN, compilation and if all went well, to run the unit tests on each platform? All testing platforms will be installed as virtual machines - in case it matters. Writing batch/Java/Perl/Ruby/* scripts is not a problem if needed.

Update: - also, not a problem if we have to call the testing script manually (the one that triggers the tests to be ran on all platforms).

Improve this question
4
  • My first advice would be to consider making a prototype of key functionality in Python. You might find out that Python version is fast enough and you don't need C++ at all. And if you do, remember Brooks law: "Plan to throw away prototype - you will throw version 1 anyway".
    – Peter M. - stands for Monica
    Commented Aug 20, 2014 at 15:55
  • SVN hooks reference: svnbook.red-bean.com/en/1.8/svn.ref.reposhooks.html
    – Peter M. - stands for Monica
    Commented Aug 20, 2014 at 15:57
  • @PeterMasiar - my project will be in C++ and there's no way I can avoid that. The testing script (or any other helper script) can be written in whatever language I choose.
    – dcg
    Commented Aug 20, 2014 at 21:36
  • OK you may still try to sell developing fast pilot research in Python to try out whole stack of technologies and API with core functionality. Or not. As I said, Brooks law applies. If you never implemented something similar, you will throw version 1.
    – Peter M. - stands for Monica
    Commented Aug 20, 2014 at 22:48

2 Answers 2

Reset to default
2

To comment on exactly how to write that...sure you can "code" whatever you want to code, but if you are looking for simplicity and repeatability with minimal work...

I would recommend Sikuli as it will use image recognition so it won't matter what VM it is as long as it kicks it off the same way. It can also execute any test just like you would manually. You can check updates with SVN through command line and put that in a batch/cron file/job and then kick off Sikuli via command line execution to go and kick off all the tests. This includes the remoting to the other vms.

The reason I say Sikuli is that since most of what you are doing is stable OS and third party/FOSS tools the interfaces will not likely change. That will provide script stability. Sikuli you can record images that match the interfaces and create the script once and voila. Regardless of your VM connections changing between linux/windows/mac it's all there. And if you open an application to kick off the unit tests manually you can do the exact same steps with Sikuli. http://www.sikuli.org/

Otherwise enjoy a programming activity where you can figure out all the little nuances of each test kick off on each vm and string them all together. Either way Python should be adequate for your utility as Peter mentions and is a language for Sikuli as well.

Improve this answer
4
  • Your answer gives advice for a testing tool, but I think the question is about a tool that automatically kicks off testing, which presumably would be tool-agnostic: "My question is - is there any tool (preferable open source) which can called after an SVN commit to trigger repository update from SVN, compilation and if all went well, to run the unit tests on each platform?"
    – Bryan Oakley
    Commented Aug 20, 2014 at 17:01
  • Yes. The tool mentioned could kick everything off as it only runs on a visual interface (not to user, but graphics rendered). It can also be kicked off command line. So then you create a push for new code and when the files in a particular folder are modified kick off the test. Of course CI would be better, but I assumed that since he had "multiple" environments and code bases that they were not all compatible to be integrated together into CI. Could setup multiple CI's though...
    – mutt
    Commented Aug 20, 2014 at 17:10
  • @Bryan Oakley - If he can use a CI tool I totally agree with you that it would be better cause that's what it's designed for...however, if not I think the setup with Sikuli would be pretty fun to try...
    – mutt
    Commented Aug 20, 2014 at 17:19
  • sikuli is useful for what I need. There is no UI in my application. Only command line and some COM and sockets activity.
    – dcg
    Commented Aug 22, 2014 at 15:31
6

What you want is called an continuous integration (CI) server. A very popular one is called Jenkins (jenkins-ci.org), though there are others both open source and commercial.

You can configure it to watch for checkins, and to build a project. If the build succeeds, it can trigger tests to be run. If the tests pass, you can trigger other jobs such as automatic deployment, uploading to a server, etc.

Jenkins works as a standalone thing, or can work in a master/slave configuration. The slaves can be on any of the major OSs. So, for example, you could automatically build on windows, linux and MacOS simultaneously.

Improve this answer
2
  • I checked their website, it looks interesting - but it is still not clear for me if it's possible to run the tests on more than one platform at the same time and collect the results.
    – dcg
    Commented Aug 22, 2014 at 15:29
  • @dog: yes, you can run tests on more than one platform at the same time. You configure how many concurrent jobs it supports, from 1 to whatever your system can handle. Each job will have its own separate reports, but you can create a job that consolidates the reports after they all finish.
    – Bryan Oakley
    Commented Aug 22, 2014 at 15:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.