Hot answers tagged data-warehouse-testing
I agree there's not much out there - I took on my first (and so far, last) data warehouse testing project a couple of years ago and found it pretty hard to find good advice. I haven't responded so far, as I think my experience having done just one project is fairly slight so I was waiting to see if you got more useful responses. Some good resources: Karen ...
It is not possible to do exhaustive testing in the context of data warehousing. I had been into ETL testing. There was a corner case bug in which if there is a lock created on one of the tables in the source database, the migration of data to the destination DB failed. Is it possible for any tester to imagine a scenario where table gets locked automatically ...
There is a lot of overlap between ETL and DB testing, where both need similar types of test data to stress-test operations with what will represent reality later. See also: http://www.iri.com/blog/test-data/test-data-management-test-data-needs-assessment/
I believe this article gives a good overview of the different data warehouse testing that should be applied: http://quality-gates.com/?p=1284 We are using this method in almost every data warehouse project.
[Disclaimer: This product is created by my company] We have a product call QuerySurge that allows you to fully compare data in the source tables with the data loaded into the data warehouse. Compared to other approaches, this seems to be the best automated method for validation.
There are lots and lots of tests that can be carried out in ETL and data warehouse testing. It helps if you have a set of prioritised requirements as to what you want to try and test. The usual 3 'functional' checks are: Have you checked the data coming in is ok? (garbage in garbage out) Often times using the information from data profiling and looking ...
Updating to reflect exit criteria for testing. You can look at MSDN article - Adventures with Testing BI/DW Application:On a crusade to find the Holy Grail - http://msdn.microsoft.com/en-us/library/gg248101.aspx Critical areas to focus are Validating the ETL Scenarios for full, delta pull Testing with Production Data for multiple regions, multiple data ...
Starts the project with run the jobs manually. check all the source data is loaded into target or not. Write Sql queries to compare source and target data. review the column mapping document. check surrogate key is generating unique value or not. check surrogate key value is not null. chcek primary key value is not null. check the count of source and target ...
I found this to be a useful, though high level article about data validation. It's more about how to approach the testing than specific tests, but it does contain some specific tests and has a lot of useful information: http://msdn.microsoft.com/en-us/library/gg261774.aspx
Only top voted, non community-wiki answers of a minimum length are eligible