Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It only takes a minute to sign up.
Sign up to join this community
I understand that a higher standard deviation means that there is a variation in the response time, and vice versa for a low standard deviation.
But what is the context of the actual value? Is it in relation to the min/max values? In the screenshot below there's std dev of 7.83, how should that interpreted?
In simple words, its how the value (on an avg.) has deviated from the Average value. Regardless of the context, I would say you can safely ignore this value given the min/max value stay within acceptable SLA; if not then higher SD points to skewing of numbers towards either the min (negative number) or the max (positive integer) value.
According to Wikipedia, Standard Deviation "is a measure that is used to quantify the amount of variation or dispersion of a set of data values."
In performance testing context, it give some information about the consistency of the response time (RT). In your example, the average RT is 145, and Std Dev is 7.8, this means that 67% of the time, the RT was in the range of 141 and 149. (avg - SD/2, avg + SD/2) This is relatively consistent, when compared to some of your other data samples (example, that first one has a pretty wide range).
For performance testing, I prefer to use either the median, or better the 95th percentile point to characterize the performance. I find it easer to say "95% of our users see response within 16 seconds" rather than "the average response is 6.3 seconds with a Std Dev of 8.3 seconds" The former statement is usually easier for a wide range of stakeholders to understand.
Standard deviation (SD) describes how steep is the bulge of the normal distribution curve. It means that around 66% of values are within SD around the mean. Small SD means it is a narrow steep hump. Big SD means hump spreads widely around the mean.
I highly recommend Larry Gonick's "Cartoon guide to Statistics" (or any other of his "Cartoon guide to X" introductions to other sciences or history). By far most fun into to statistics.
It is in relation to response times of all samplers.
Additional material:
