Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I work on a team which is basically all backend. Functional and unit testing is achieved by python or perl scripts or sometimes bash scripts. But if we need to load or performance test our applications, we currently have a few scripts that generate linear load and we write some scripts that would print out some stuff to console e.g., a script will generate 1000 emails queued for a certain daemon and a couple of scripts will output process snapshot (ps), memory usage, etc. to the console and we monitor those. We can obviously write a multi-threaded script specific to each load test and something that will monitor results but that would be a very specific approach and would require us to do that for each test. Basically the reasons to use a generic tool that can be tailored for specific use.

What I am looking for and unable to find so far, is a backend tool that will help us generate load with multiple threads and can be used to collect console output and aggregate it over time to generate a tabular or a graphical result to be analyzed. There are a lot of tools available for web testing, but I am looking for something similar for backend.

We work with Perl and Python, so that is a critical part. And we are mostly working on linux or a mac. Also preferred is a open source solution.

share|improve this question
add comment

2 Answers

up vote 1 down vote accepted

Given that you're primary Linux and prefer OSS, have you tried solutions such as soap/loadUI or JMeter. I know that there's some features of soapUI that require a professional license, but, it is reasonable for most companies. One of the features with this that I believe would suit you well would be the ability to call specific tests/suites from command line. For this, you wouldn't really need to modify the code behind the tool so much as set up your scripts and call them.

I also stumbled on this answer from SO that lists a few python solutions that I've heard of, but never tried.

If you need to monitor stats from your servers, is there a specific OS that you're using? Do you already have any tools that are maybe running on your production servers that do this that you could also use on your non-production servers?

share|improve this answer
    
Lyndon, thanks for the links. LoadUI looks good. As for JMeter, from what I saw it only has Java apis unless I missed something. We are a perl/python shop so adding another language is not practical. We are also thinking of adding features to our in-house tool loader.io –  Suchit Parikh Dec 13 '12 at 18:24
add comment

Visual Studio's load testing can do what you are asking. It requires a license for Visual Studio Ultimate. Even though it is geared towards web testing, it can actually execute any unit test as well.

The way it works is you create a LoadTest and that LoadTest can execute any number of scenarios and each scenario can execute any number of tests. At the Scenario level you can specify how many threads you want to use to execute the tests and how you want to execute the tests (randomly pick them, run them in order, specify percentages - i.e. Test1 runs 5% of the time, Test2 runs 95%).

If you need to generate a lot of load (more than you could generate from a single client) you can set up a controller and multiple agents to spread the tests out over multiple clients.

After the run is over it also will produce pretty graphs, charts and summary and allows you to export to excel where you can create additional pivot tables and charts if you want to. If the servers are windows servers you can also monitor performance counters very easily on them.

share|improve this answer
    
Sam, Visual Studio might do the job in general. Except for me, I am either working on a linux virtual box, a remote linux server or a Mac (in short not a windows friendly place). Also we are an open source company, so licensed products are the last thing on the list. I added the specifics to the question itself. –  Suchit Parikh Dec 12 '12 at 23:46
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.