Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

This is going to be a bit long/wordy/verbose, but I've been looking at it for a while and I think that I'm bad at explaining things because people seem to get confused about what I'm asking.

You know how, when you add the [DataSource(...)] attribute to a test, the test will perform multiple runs, and then show as multiple executions in the test runner (MTM or VS)? I need to reproduce that kind of behaviour, while still being able to use a datasource.

Maybe showing what I'm trying to do might make more sense.

Let's say I am testing a webservice that has an Add(int,int) method and I want to test a number of different adds, so I have a TestMethod that looks like this:

[TestMethod]
public void CanAdd()
{
    var clientProxy = this.getClientProxy();
    Assert.AreEqual(2, clientProxy.Add(1, 1));
}

now, I add the datafile, a csv that looks like this:

arg1,arg2,expected
1,1,2
0,1,1
-1,1,0

I import that into my test,

[TestMethod]
[DataSource(...)]
[DeploymentItem(...)]
public void CanAdd()
{
    var clientProxy = this.getClientProxy();
    Assert.AreEqual(
        (int)TestContext.DataRow["expected"], 
        clientProxy.Add(
            (int)TestContext.DataRow["arg1"], 
            (int)TestContext.DataRow["arg2"]));
}

Now, I'll get 3 runs of my test, one for each data row.

This all works fine... but the problem is that I must test multiple (configurable) proxy locations. And it has to work from a test run kicked off from MTM.

What it would ideally look like would be:

[TestMethod]
[DataSource(...)]
[DeploymentItem(...)]
public void CanAdd()
{
    /*  in this scenario, getClientProxy would update the client proxy 
        with the address of the deployed location of the service
    */
    var clientProxy = this.getClientProxy(
        TestContext.DataRow["deployedLocation"].ToString()); 

    Assert.AreEqual(
        (int)TestContext.DataRow["expected"], 
        clientProxy.Add(
            (int)TestContext.DataRow["arg1"], 
            (int)TestContext.DataRow["arg2"]));
}

And, if I had 2 deployed services (which are identical outside of their physical location), then I'd have 6 test runs.

This doesn't work, and here's why:

  • Can't put the location of the service into the datasource because:
    • It'll be different depending on who's running the test(s)
    • If each person running the test(s) manually changed the datasource, they'd have to do a checkin and a rebuild.
  • Can't have another datasource attribute

Basically, I want to have a data driven test, where the data would be like this:

location,arg1,arg2,expected
http://www.location1.com/service.svc,1,1,2
http://www.location1.com/service.svc,0,1,1
http://www.location1.com/service.svc,-1,1,0
http://www.location2.com/service.svc,1,1,2
http://www.location2.com/service.svc,0,1,1
http://www.location2.com/service.svc,-1,1,0

except that I can't actually put the location into the datasource.

Has anyone come across this kind of behaviour? Any ideas on a workaround? I'm currently looking at creating an aspect that will call the testmethod for n locations passed into the attribute (similar to this) but I don't know if it'll work as I expect.

share|improve this question
    
Can't you just use two CSV files? One for location, which could be used by multiple people so long as they have the same name for the file (although that would be messy). Keeping the other for the function values should be ok. I've used multiple Datasource files in the past with success. –  MichaelF Aug 5 '11 at 19:22
    
@MichaelF: Yes, I can have 1 of the datasources (the service locations) as a file that is a deployment item listed in the user's .testsettings for the run, and that would work, but the problem is with reporting the results of the test: You still only get the number of test runs as you do values in the attributed datasource. –  SnOrfus Aug 5 '11 at 19:37
    
@MichaelF: Why is that important? Well, if you have 10 tests, and each of them fails trying to perform against one specific deployed location, it's immediately obvious what is causing the failure. Why it needs to be configurable is because the run might want to only test against 1 specific deployed location. The actual scenario is a web service in an Azure Web Role, hosted in the devfabric, or on a live azure account. We want to be able to run local as much as possible to cut cost, but live/both during specific times in the dev cycle (or live only, for perf-like tests for eg). –  SnOrfus Aug 5 '11 at 19:50
    
Well that's more understandable, I did not grok that from your posting since it seemed more important to you to be able to run the test but provide flexibility for locations. I've done many tests depending on data source rows, but you could also do tests dependent on iterations or determine the test iterations by some other method. –  MichaelF Aug 8 '11 at 13:25
add comment

2 Answers

I was able to accomplish this by writing a custom TestClassExtensionAttribute and then calling a custom ITestMethodInvoker from the TestExtensionExecution.

I answered a similar question on SO about this: http://stackoverflow.com/a/14673329/573218

I have posted the code on GitHub and am working on cleaning it up a bit: https://github.com/johnkoerner/MSTestLooper

share|improve this answer
add comment

If I understand you situation correctly you have a data-driven test, and you want to be able to run that data-driven test using the arguments in the 'datafile' against one or more services with varying locations.

One approach may be to read in a file that lists the desired services (location) at runtime into an array, then loop through each element (service) calling your CanAdd() method.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.