What are the primary testing techniques/methods to validate the every data point in the rows of records on source side to Target. How do we handle when there are millions of records to be validated during both initial and incremental ETL process.

1 Answer 1


You may want to look for MapReduce techniques, in tools such as Hadoop.

MapReduce breaks sets of data for distributed computing (mapping) and summarizes (reducing) results in a distributed fashion as well, incrementally.

Reference: https://www.tutorialspoint.com/hadoop/hadoop_mapreduce.htm

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.