It really depends on the type of company you are, or the products you are testing. It also goes to your testing approach. Are you basing your tests on the data available, or are you creating data required by your tests.
IMHO, the most effective way is to get a copy of production data, and perform analysis on it.
- De-duplicate the data.
- Perform "equivalence partitioning" on the data to identify test data records that, whilst different test exactly the same thing.
Then use that analysis to generate a base set of data that covers the production scenarios, then on top of that, add any additional data that you need for your test scenarios.
The main difference between regular test data, vs production data is that testers and developers tend to use the system the way it was designed, where real users will tend to leave fields blank, miss key or insert junk data in. So it is a lot "dirtier" than most test data suites.
This is easy enough to uncover if you pull it in to a spreadsheet and run a few macros on it.
So to completely answer your question:
- Testers absolutely must have access to analyse production data.
- You may have data privacy issues that prevent it being used in raw form in testing environments the you should look in to requiring obfuscated and anonymised data and invest in tools to do that.
- If testers can't be trusted with the data, I sure as heck wouldn't trust developers with it :-)
- For highly sensitive data which can't be obfuscated, and raw data was required. Such as general ledger information linked to stock trades, in highly complex financial and regulatory instruments (where the dollar values and individual, sequenced entries are critical to the testing), then we would treat the test environments with the same data protection and privacy controls as any production environment.