What are the best approaches and what concerns are there about performance testing a web application that is hosted in the cloud?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
I have no idea about best approaches, but I can think of a few concerns to get discussion started:
Oh deja vu ...
I think that Larry Elison summed it up nicely when he stated that Cloud computing is just, servers and the internet. Video is here -> http://www.youtube.com/watch?v=UOEFXaWHppE
Performance testing "Cloud" applications should be tested as though you would test any existing web application. Specifically for performance you would want to testing a single instance and then how it scales out when you apply dynamic scaling to see if you get a linear performance improvement or not.
Apart from that the same performance testing tools and approaches and skills would apply, i.e. Loadrunner et.al.
Another challenge though would be that you won't have access to the performance logs, counters or configuration options that you would normally use when performance tesitng and tuning.
There is a company called CloudHarmony.com that benchmarks and publishes all their information online for all the different cloud providers.
They basically run performance tests all the time, so it will save you the work of testing each provider.
Your question is about testing a specific web app, in which case, the normal rules apply. The easiest, try out a tool like PageSpeed to give you feedback on the website.
I think the key to your question is "web application". So we should not be directly concerned with which cloud you use or even latency.
Performance testing can mean different things to different people. I see it as:
You need to know how the application performs in the view of users. Slow pages lower your revenue and customer satisfaction. To see the direct correlation, see these great performance testing statistics. Metrics like rendering time or response time are what your users care about. These measurements are what performance testing tools provide for you.
Secondly, and just as importantly, you should load test your entire site. Create realistic test scenarios/scripts that take actions on your site like actual users. Be especially sure to interact with your application in ways that stress your database because it is the most common bottleneck. Turn on the slow query log and review it to find any expensive joins or just bad SQL. Ramp up the load test volume from a small number to a large number. This allows you to see at what load the application starts to slow down or generate errors. Look for steep curves in response times that indicate "hitting a wall" for system performance. This can point you to a configuration limitation such as running out of database connections. Good cloud load testing tools include: LoadStorm, Browsermob, and Soasta.
Performance testing for web applications should focus on the user perspective. Be sensitive to errors and outliers of average response times.
Hope that helps. Lots of good resources on the same performance-testing.org site. Blogs, downloads, tool lists, articles, and industry news.
From my perspective, dynamic scaling has to be tested in advance to ensure that everything is good. The infrastructure's support cloud provider has to be used to get apple-to-apple comparison. If you test environment is not same as production, the integrity of data poses serious questions.