My application has a daily task which exports certain activities to CSV files. In the ideal world, admins would check those files daily to make sure they are sane and the application is behaving as expected. In the real world, that won't (always) happen, so I'd like to create some sort of automation framework where (ideally) business users can build validation rules to be applied against the CSV.


Scenario "Participants can only create messages with their own ID":
For file 'Messages_{Date}':
    Assert column 'User role' matches one of ['Admin', 'Participant']
    If 'User role' = 'Participant'
        Assert column 'User id' matches JSON.parse(message).user

I wonder if a BDD tool like Jasmine can help with making these test cases clear and reporting any issues to both developers and business (power-)users, and also perhaps handle some things like "This report was broken already yesterday so don't send a new email today".

Am I approaching this wrong? Is this kind of data-level validation a bad idea? What tools do people use for this kind of thing?

  • Theoretically I could do this with DB queries rather than with the CSV exports if that were better supported by the tools available. – Michael Kopinsky Dec 1 '14 at 20:18
  • Are those CSV files consumed then by a known app? Or only by a human? – dzieciou Mar 11 '16 at 15:00

Data Validation is a very important part of any testing strategy for an application that is dependent on data. There are two things you should do:

  1. Validate data in your test environment
    1. You will have to have a way to generate real world scenarios for your application, hopefully your functional testing will already have this covered.
    2. Based on the known data set generated by your testing, create data validation tests that make sense using your technology of choice. You need to test the CSV since it is a specific deliverable of the application, but you may also want to test data in the database.
  2. Create something to inform your business users
    1. I would lean towards building reports using something like SSRS or Cognos than providing test case results.

I try to avoid testing data directly in production, and instead try to test any ETL processes thoroughly enough in a test environment to be confident that there will not be any data issues. This can be quite a challenge, depending on your application's design.

  • A lot of the scenarios that we'd want to be testing are affected significantly by admin configuration in our prod environment. Things like "Participants in arm 80 should only receive one of the following messages". I could theoretically sync that config back to my test or staging environment, but that's just one more thing to keep in sync. – Michael Kopinsky Dec 2 '14 at 14:36
  • The things I want to communicate to users are not test case results - they are actual data. I'm actually leaning toward this right now - currently I have a report which essentially is "Show all messages sent by the system"; I will probably add a summary report which is a simple pivot table-like tabulation which is easier for them to review. – Michael Kopinsky Dec 2 '14 at 14:52

Two observations that might lead to a different solution:

  1. "Certain Activities" sounds like a simple log file
  2. CSV files are pretty much XLS spreadsheets

My suggestion: create a macro that grabs the file, creates a range, and then loops through each row in the 'User Role' column to verify the output. On a "failure" the system can send an email to an Admin.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.