My employer runs some of our systems on physical machines with attached hard drives. I am charged with performance-testing those systems. For cost reasons, I've been asked to test those systems running on virtual machines (using Xen) attached to a SAN. This is clearly not an apples-to-apples comparison. Some systems use a lot of disk I/O, and so the SAN issue is especially worrisome. Rather than responding with "can't be done" or "not reliable", I want to recommend what is possible.
Here are some things that come to mind or that I've found with Google searches:
- Measure SAN speed vs. hard drive and calculate a ratio
- Borrow a physical machine long enough to run a benchmark, do the same with a virtual machine and calculate a ratio
- Even if you can't predict absolute performance on physical machines, you may be able to predict relative performance (i.e. whether the candidate release will be faster or slower than what's currently in production)
- Measure multiple times at different times of day to mitigate resource contention issues, i.e. conflicts with other virtual machines running on the same physical machine or with other clients using the SAN.
Are there other things you can do to mitigate differences between physical machine performance and virtual machine performance in an environment similar to mine? I am particularly interested in actual experiences rather than educated guesses.