Is there any fundamental difference in the way applications/services hosted in the cloud are to be tested versus a traditional web based application? It looks essentially the same to me except the way these applications manage large loads (elastic clouds) and high availability.
I think that Larry Elison summed it up nicely when he stated that Cloud computing is just, servers and the internet. Video is here -> http://www.youtube.com/watch?v=UOEFXaWHppE
Testing "Cloud" applications should be tested as though you would test any existing web application with a few added test cases to test the additional "Cloud" features, if you application has them:
Dynamic scaling You would want to test that the application can dynamically scale up and down, with no loss of data and end user connectivity.
Automated provisioning For apps that provision new services automatically when a new user signs up, you would want to test this process, as well as the reverse when a user leaves the service.
Device Synchronisation If the service is like Dropbox, or iCloud, then there may be device to device synchronisation issues that need to be tested, particularly recovery situations when a sync is disrupted and incomplete and needs to be restarted.
This really depends.
No if you have created virtual machines on the cloud and have moved your application onto those virtual machines and placed these behind the a load balancer. You might choose to check that the load balancer functions as expected, and the performance when you place you application under load to ensure you have the correct setting in place on the load balancer.
Yes if you have embraced the cloud technologies and built your application to deliver high availability and scalability. Your application components will be distributed across machines and you will find that testing a distributed system is best done by testing each of the components first in isolation through unit tests and then a smaller set of integrated tests to prove that the deployed application works as intended.
I found this nice article by Matt Heusser which nicely sums up software testing in the cloud.
Providing the link here to share with all those who are looking for some advice.
In other words (a short summary):
Cloud computing has made available two very useful technologies to the masses
- Distributed computing
This creates new opportunities for testers to test quickly and efficiently. The challenges would be test around the edges of these services. Some of things to look are
- Elastic clouds (automatic provisioning of computing power and memory)
- Disaster recovery
- Synchronization (across devices and services)
SoftTest Webinar: Testing a SaaS (Software as a Service) Platform in an Agile World http://www.blog.sogeti.ie/2010/04/softtest-webinar-testing-saas-software.html
Important takeaway from the presentation - Application related security (SQL Injections, Cross Side Scripting), Infrastructure related (Disaster recovery mechanism, Latency (Network, Data Transfer, Load balancing, Remote Monitoring)). Related Read - http://highscalability.com/blog/2009/7/25/latency-is-everywhere-and-it-costs-you-sales-how-to-crush-it.html
SaaS Testing Overview - Foundation http://www.slideshare.net/ramgarg/saa-s-testingoverviewppt-5072671
Please check CloudTest Lite - cloud based free tool for performance testing web apps (http://www.soasta.com/info-center/press-releases/cloudtest-lite-edition-announcement/)
Developing Test Expertise in cloud based apps is different from traditional test approach. Particularly, How much load cloud can scale up is the key.
Hope it Helps.
Assuming that a 'regular' web application is one that runs on bare metal or a virtualized infrastructure with tight SLAs (two nines and better) and doesn't requires scale or elasticity, then yes there is a big difference.
Cloud VMs have an MTBF measured in months, so all components that your application depends on, computes, networks, storage, are error prone. Testing a cloud application thus becomes the problem of testing if your application behaves properly when any of these components fails. For the application to intelligently respond to failures cross-cuts into its functionality so fundamentally, cloud applications need to be tested differently.
The second big difference injected by the highly unreliable components (but highly reliable infrastructure: please note that difference) is that there needs to be automation to deploy these components. That automation is integral to the functionality of the application and can exhibit very complex 'convergence' behavior. You suddenly need to test for idempotency of configuration processes, or forward progress on convergence. Those are attributes that small fixed asset applications never had to include in their operational QA.
So, yes, big difference in QA approach between cloud applications and non-cloud applications. And clouds can be private or public, it is the MTBF characteristics of the components, elasticity, and scale that make the difference. Scalability is different from scale, and scalability does introduce a new functionality requirement, but testing for it is very different if you have 50 servers or 5000 servers.