I would like to know if there is any acceptable ratio for test size data size compared to production data size? If my prod data is 200TB would a test data size of 20TB be adequate? Does such a consideration exist?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
There are several factors to consider when selecting test data.
For example, if your 200TB of production data is essentially equivalent, then you could potentially calculate a statistical sample from the total equivalent population. If not you could group your 200TB of production data into smaller equivalent subsets and calculate sample sizes for each subset population of data. However, there could always be outliers and if those aren't correctly identified then you could be stepping over some pretty big holes. Also, you might want to identify any failure indicators such as production data that has been historically problematic.
Ultimately, there is no magic formula. Part of the challenge as a tester is to determine the appropriate test data, and the appropriate amount of test data that will provide confidence that the feature is well tested and has a low risk of failure with production data in the wild.
As Bj Rollison says, it depends on the nature of the data.
My view is that good test data meets these criteria:
The short version is that there is no hard rule for how much test data is enough. If the 20TB sample you take from your 200TB production database doesn't cover a particular feature, you've got a hole that will likely become a regression magnet.
Adding to the previous responses, it also depends on the kind of testing you are trying to perform.
Performance or load testing: Figure out the scale of your test environment compared to the production e.g. If your production has 100 application servers and 10 database servers to handle your 200 TB data with given set of server configuration, you might be able to achieve your load and performance testing with 10 application servers and 1 db server with 20TB of data (these numbers also depend on the configs of your servers).
In case of functional testing, you might want to cover various use cases with as less data as possible.
No, there is no apriori acceptable ratio. Here is why: without knowing the production data's probability distribution, you cannot determine how big your representative sample needs to be.
A way to proceed is to decide on a way to measure a sample's distribution and then measure progressively bigger samples until the distribution converges.
If you can't do that experiment, you may need to just choose the largest sample you can afford to work with, and make it clear to everyone why you made that choice.