Environment parameters for VSTS Cloud-based load tests

Question

We're developing a web service using Azure and VSTS and I'd like to run some load tests as part of the CI/CD process.

However, I'm struggling with being able to set the service url for our various environments as part of the build/release definitions i.e. myservice-dev.azure-api.net, myservice-test.azure-api.net etc

For our unit tests I'm able to declare custom parameters and overrides using the .runsettings file but I don't believe I have access to something similar for performance and load tests.

My Google-foo has failed me so I'm worried that I'm approaching this entirely wrong.

Any suggestions would be appreciated!

Answer 1

Having raised the question on the MS developer forum the suggestion was to use a token replacement task (there are a number of extensions on the marketplace). This makes it easy to declare and override variables in your code with those in your build definition.

Its not an ideal solution as it makes running things locally a bit more painful but it does work.

You can add a Replace token task (for example: https://marketplace.visualstudio.com/items?itemName=qetza.replacetokens ) to replace the Context Parameter value. Steps as below

1. Add Replace Token task, and specify the target files
2. Assign a variable (for example: website) to ContextParameters in .loadtests file ContextParameter Name="WebServer" Value="#{website}#"
3. Define the variable in Variables tab

https://developercommunity.visualstudio.com/content/problem/123486/unable-to-override-context-parameter-in-cloud-load.html

Alternatively, it seems like the suggestion is to have multiple .loadtests in your project for each environment with the .loadtest itself acting as the config. Then in VSTS you would select which loadtest to run as part of the build/release definition.

Answer 2

Hack 1: Create a datasource like CSV called site.Csv and you can do a string replace into that CSV before the load tests runs in Azure devops release pipeline step.

Hack 2: you can create a SQL data source and have a seperate step to insert the site into that before the load test runs as above.

Hack3.: plugins may be the answer too, you can creatr plugins to read from anywhere so if you wanted to read from persistence store you can code the implementation and do the string replacent as before.