Performance / Load Testing: Where can I start?

Question

My Application is a cloud-based solution.

And I need to test its performance with many concurrent users that should be within the minimum market viability for concurrent users.

I do not have a clear performance benchmark for the system, like it should handle xxxx number of concurrent users in x seconds or less, or it should be able to upload x number of Docs. x seconds or less.

Where can I start ?

Answer 1

If you are going to run the tests on a dedicated machine, use JMeter.

If you want to have a lightweight solution for running tests both locally & remotely - take a look at http://locust.io/, it is a very powerful & lightweight solution, which does not eat RAM in the same way that JMeter does.

Answer 2

I can suggest "JMeter" as the tool for performance testing of cloud based applications. It can record and playback the tests. You can set the number of threads i.e. users and many other things.

You can start initially with a 10 user load and then 50 then maybe a hundred. This way you can generate graphs which can help you in analysing the performance of the application i.e. how it behaves with different user loads. Also it can let you test the stress level of the application in that how many concurrent users it can support.

Answer 3

I think you asked two questions.

How many concurrent users do I need to support for minimum market viability?

There is no universal answer for that. For a brand-new website in an niche market, 5 concurrent users may be enough. For the insurance enrollment website for the Affordable Care Act, the minimum number could be in hundreds of thousands.

It is possible that your product owner may be able to give you that number. More likely, here is what really happens: you deploy the site to production and watch whether it breaks. If it does, someone works on making it faster. That can be embarrassing for a national government or a major bank, but for a little start-up, it is a perfectly reasonable way to do things.

On the other hand, you really might want to find out about performance problems before you deploy to production.

Starting without any guidelines, how do I measure performance?

This is kind of like regression testing: you start by deciding on a set of paths through the system. If you are testing an existing site, you may be able to use the access logs to figure out which actions happen the most. If you are testing a new site, you don't have any users, so you have to guess.

Here are some criteria you can use:

• Choose paths that all or most users are likely to use every time they visit the site. For example, a YouTube user is likely to search for videos and view videos every time they use the site. They are unlikely to change their preferences every time.
• Choose paths that you expect to be server-intensive. If you know an action requires the server to do a lot of work, include. If you know an action requires locking a resource, include that too.
• Choose paths that need to be fast. We would like for every action to happen instantaneously, but some actions are more important to be fast than others.

Some test plans call for many users performing the same action at the same time, whereas others call for a random mix of actions. The former is easier to write than the latter, but in production, the actions will be random, so you should write your test that way too.