In our project, related to Big Data.
On daily basis we receive 20 to 30 xmlXML
files and using hive/pig scriptshive/pig scripts
the XML are loaded to database. The files are placed outside HadoopHadoop
environment. sometimesSometimes they millions of records
Currently we are manually validating XML to XSD XSD to Table Structure spot check few XML records with table data
The ask from business team is to get:
1. XML to XSD
2. XSD to Table Structure
3. spot check few XML records with table data
The ask from business team is to get:
1). Total count of records from XML files
2). Generate distinct records from a column
3). See wetherwhether we can automate XML to table data validation
Is there a way to do it using any automation tools.
Thank you Pratibha?