In load testing, what is a 1% test to assess demand?

Question

This document from Google on designing for scale states:

In order to accurately assess demand, you can run a 1% test to determine how much traffic will flow to your new service.

Googling for 'What is a 1% test' doesn't give any useful results. What is a 1% test and how does it allow assessment of demand?

Answer 1

You're taking the words out of context:

You should test at the maximum load that you expect to encounter. In order to accurately assess demand, you can run a 1% test to determine how much traffic will flow to your new service.

With regards to this assess demand recommendation, it's more about setting up the environment of proper size.

For example, you anticipate 1000 hourly active users. The guys from Google recommend initially kick off a 1% test, i.e. run 10 users for an hour and measure the throughput (incoming/outgoing traffic)

Once you get the figure how much traffic will 10 users consume you would be able to multiply it by 100 and put the numbers to Google Cloud Platform Pricing Calculator to ensure enough capacity in terms of bandwidth.

Going forward there are several types of performance testing which are normally being executed against the applications:

1. Load Testing - putting the system under anticipated load to ensure that it behaves normally, response times are within the acceptable range, etc. For example 1000 users for 1 hour
2. Soak Testing - basically the same as point 1 but for a prolonged period of time, i.e. overnight or over the weekend. It allows detecting the majority of memory leaks or problems with disk space.
3. Stress Testing - locating the breaking point of the application or the first bottleneck. Start with 1 virtual user and gradually increase the load until response time starts exceeding the acceptable threshold or errors start occurring (whatever comes the first). This way you will have the insights regarding how much load your application is capable of handling providing normal response times and what is the component which is gonna break first

Answer 2

This sounds like Canary testing, where you select a subset of users and try out new software with them. The idea is that you get to minimize risks by not flipping everyone onto the new version at once. You can observe how this group gets on and decide to roll out further if it looks good.

You can miss some scaling issues because you won't have the load.

There is no silver bullet with testing; you need to assess your risks and ensure your tests cover your project's specific requirements.