Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I'm looking for a good, in-depth resource for load testing REST APIs and web services that goes well beyond "beginner load testing". I'm wanting a technology-agnostic resource that focuses on the goals and techniques, and maybe suggests tools to accomplish those techniques - but not something that is tied to a particular stack or toolset. If the resource includes related testing, like performance and scalability for web apps & web services, that's a bonus.

To be clear: I'm not looking for articles on how to load test, tutorials, or tool recommendations; we're already doing basic load testing and getting value from it. I want something in-depth to help identify holes in our current approach (if any) and improve what we are already doing, plus wanting to round out my own knowledge on the topic. A quick look on Amazon isn't getting me anything that sounds promising, so I thought I'd turn to the experts.

Any recommendations? Is there a clear "best in class" resource for this?

share|improve this question
    
Is the a specific approach or technique you are using that you might like some opinions on that you might share? –  Dan Snell May 30 '13 at 17:51
    
Nope, I'm looking for general information. We're getting a lot of value from our load testing, and I'm trying to get ideas about how we can best improve what we are doing to get more value out of it. I'm hoping to avoid digging through resources tailored to a relatively specific use case to extract the generally useful bits, though it looks like that might be the only option. I'm really surprised that there doesn't seem to be a good book on this subject - load testing web services seems to be a hot thing these days. –  Ethel Evans Jun 4 '13 at 0:53
add comment

2 Answers

up vote 1 down vote accepted

I can suggest a few resources based on the Microsoft stack, i.e. using Visual Studio load testing tool:

http://vsptqrg.codeplex.com/

You aren't really tied to any of the tools, and you can do it with free tools as well, for example with Fiddler.

The general approach would be:

a) gather the usage profile - i.e. how many times each api is hit in the unit of time (an hour for ex) b) write tests that simulate hitting each API c) set a goal for your load test

  • if you want to find out what the bottleneck is, try an incremental load (or if you want to be really fancy, a binary search goal)
  • if you want to do stress testing or to analyze response time under load, set a constant load

d) configure your load test to run each individual test in such a way that you mimic the real usage profile; this is where it gets tricky and you many need to use a $$$ tool - because a single computer might not be able to generate enough traffic you may need to use several coordinated load agents e) run the test and monitor the systems under test (there's a whole theory on what to monitor); professional tools like Visual Studio will choose a base set with what to monitor for you - they will also suggest thresholds

Full Disclosure: I work for Microsoft

share|improve this answer
add comment

As per your query,

"I'm wanting a technology-agnostic resource that focuses on the goals and techniques, and maybe suggests tools to accomplish those techniques - but not something that is tied to a particular stack or toolset."

..the first thing that came to my mind was the way Netflix approaches their testing. Specifically, I thought of the Chao Monkey and some of their approaches to testing their web services. They use many different technologies to test their services and they tend to open source a lot of it. You need to have some infrastructure to take advantage of it (since they are using the AWS, you would need to adapt it to your needs)

You asked "Any recommendations? Is there a clear "best in class" resource for this?": I would say that Netflix is best in class and suggest looking at the whole Netflix infrastructure (start with the blog and and follow links to Github, etc). Since they are responsible for over 30% of all the traffic on the Internet, they know how to deal with Web services from a resiliency, performance, load, functionality, fail-over, load balancing, etc.

Your mileage might vary since it depends on the project you're apply this to. However, I think the goals, techniques, and tools to accomplish your testing can be found on their developer and technical blog pages.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.