Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I'm about to develop a performance testing tool for a server side app and was thinking to measure the following things:

Hardware status monitoring via SNMP, Statistics about response status and respose time

My questions are:

  1. Is it viable to check for things like well-formed xml or valid json strings?
  2. What other things to test to be present or to monitor/track, etc?
share|improve this question
    
What does checking for well-formed XML and json have to do with performance? –  Joe Strazzere Sep 5 '12 at 10:57
add comment

2 Answers

up vote 2 down vote accepted

user1108631, welcome to SQA. Depending on how the terms are used and who you are talking to, performance tests can be a different beast from stress tests. For brevity, I will only refer to performance tests, but I think my answer is applicable to both.

It is hard to answer your first question without knowing more about the server. If a performance test does nothing else, it needs to measure whether (and how quickly) the server responds under load. Of course it is also important that the responses are correct. In order to keep your test resource requirements within your budget, your performance test may need to accept a lower standard of correctness than your functional tests would. For example, instead of checking everything about a response, you may need to be satisfied that the response contains the expected status code. You may need to experiment to find the right balance between keeping your performance test manageable and detecting the kinds of problems you expect to arise. If the first thing that goes wrong when your server is under load is that it starts returning poorly formed XML/JSON, you may need to check for that. Otherwise, it may not be worth the expense.

Regarding your second question, you can think about two kinds of metrics a performance should capture: capacity metrics and diagnostics. Capacity metrics describe the system's capacity, e.g. number of concurrent users, responses per second, duration between request and response, and so on. (The specifics depend on the service your server provides.) Diagnostics are measurements that help you understand why things go wrong, e.g. memory footprint, number of concurrent database connections, CPU load, and anything else you can think of that's constrained by hardware or by configuration settings.

share|improve this answer
add comment

In addition to the things you mentioned, I would want to know the following:

  • Failure Rates as the request rate increases.
  • Response times as invalid responses are submitted. (This would catch if your exception processing is taking up more time than handling a "normal" request).

Analyzing the responses for well-formed XML/JSON may be too much processing to do if you're only using one machine to test with. You wouldn't want to run the risk of skewing your results because your local CPU couldn't keep with processing the server requests/responses AND making sure that the responses are well-formed.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.