Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Say I want to compare the performance of Java's StringBuilder.toString() with new String(char[]) for strings from one to 3000 characters in length. I do know beforehand how long the resulting string will be. How many times should I try each length, and how can I collect the results into usable information? I'm mostly interested in execution time, but if there are any memory usage quirks I'd like to know about them also.

share|improve this question
1  
Have you thought that through? I don't think char[].toString() does what you think it does. Here's my example: i.stack.imgur.com/8i2Dd.png - That's entirely tangential to how you would test something like it, of course, to which I can't add much to u246's answer! –  corsiKa Mar 15 '13 at 15:52
    
Agh, you're right. Edited the question. –  Randall Bohn Mar 15 '13 at 20:13
add comment

2 Answers

You can use a micro-benchmarking framewrok like Caliper.

share|improve this answer
add comment

You didn't say how much precision you need -- do you just want to know which is faster, or do you want to know how much faster?

Here are some factors that interfere with benchmarks like that:

  • Clock granularity. I assume you will use something (System.currentTimeMillis, System.nanotime(), or a stop watch or whatever) to measure the time interval. The more coarse the granularity, the more samples you need to take. If this were the only confounder in your measurements (it isn't), you could derive a sample size from the granularity and the amount of precision you need.
  • JVM behavior. The Hotspot compiler will convert code paths between bytecodes and machine code depending on how frequently they're used. The optimizer has to observe things for a while before it converts code. If your benchmark ends too soon, you will miss out on the optimization.
  • Other activity on the machine. There are other things running on the computer besides your benchmark.

In any case, I suggest figuring it out empirically. Write the benchmark with a sample size of N and write down the results. Repeat with larger multiples of N, e.g. 1000, 10000, 1000000, etc., until the results converge to as much precision as you need.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.