Authentication and load testing using JMeter

Question

Looking to get advice on how to approach load testing using JMeter. My question is more about best practices since I am totally new to this tool.

I am confortable working with samplers, view results and thread groups components, was able to send requests and make assertions, get tokens, create variables etc... but I cannot figure out how to approach the following scenario for say 100 users concurrently:

• send login request
• use the returned token to talk to call the API endpoints
• logout

The problem occurs only when running the test for more than one user concurrently, not getting any issues running my thread solely.

When executed concurrently, the first thread group instance always passes the auth step and runs a few requests, but then as soon as other instances get triggered, things go awry (e.g. an instance requests a token (not refresh) while the token exists already, or tries to hit an endpoint when logged out...)

How do you organize your API testing?

If I am planning to test 1000 users hitting the API concurrently, do I really need to provide 1000 accounts to authenticate them, or are there any tips to follow?

For the same application, I've developed automated tests using js/ jasmine / protractor. Is there any chance I can reuse my scenarios in Jmeter? Any library or plugin that may help integrate existing js tests into Jmeter?

Also, is recording my interactions with the app using the recorder the best way to approach load testing SPA applications?

EDIT: I am not required to use JMeter, just gave it a try since it is widely used by QA professionals, I am open to suggestions on tools and ideas how to load and stress test our REST Api / AngularJS app. I've tried Locust.io a load testing Python framework, and i liked it a lot, but again I am trying to stick with Java and JS based tools since those are the only technologies we are using to build and test this app, and some people of my team aren't used to nor interested in Python.

Answer 1

Are you positive you have parametrized your tokens appropriately? Using the debug sampler you can look at your requests and confirm that the token is being updated for each new "login".

Regarding the number of logins, that depends on your API. Does the API treat the same user as part of the same session? When you login a second time with the same user do you get a new token or a cached token?

If you are getting a cached token, then yes - you are going to need 1000 logins. (Sorry! This is typically a huge stumbling block with load/performance testing - data setup.)

I've never used protractor, but I doubt you can reuse anything from JMeter. If you pull open the .jmx file you will see it is just an XML file that structures the calls to the methods of the JMeter libraries. You'd have to dive into those libraries to get the actual scripted actions that are taking place when JMeter runs.

With SPAs you have to pay particular attention to the web service requests that are being made. I've run into several occasions where just making an update request does not faithfully simulate a user experience.

When a user goes to a page and gets a list of items, the prior operation was some sort of "get". You may know exactly what you want to "update", but skipping all of those "get" operations is going to give you potential false positives. Recording SHOULD capture all of that traffic - but don't assume. Post update, does your SPA refresh the content of a table or page?

You'll likely need to manually create "update" request to force the web service / server to respond to your simulation as though you are a real user. What I am saying is do NOT strip down your recordings to the bare minimum if you are looking to gauge performance. If you are purely looking to generate load, this is sometimes okay - but make sure you document this in your final report. People often interchange the terms performance test and load test. They are not the same thing.

Finally, recording isn't going to create any parameterization, you will more than likely have to write at least one BeanShell PreProcessor somewhere. Typically this happens with date values that need to be updated to sometime other than the value that was captured when you recorded traffic.

Answer 2

The main idea of load testing is simulating real users as close as possible. If you need to test some form of web API - you need to mimic real life API usage. Therefore when it comes to load testing each JMeter thread (virtual user) must represent a real user so you will need to use different credentials for each virtual user.

With regards to re-using your JavaScript-based tests, my expectation is that they are very resource intensive so it is highly unlikely you will be able to create the main load with them. Common practise is creating the main load using JMeter's HTTP Request samplers and run UI tests in parallel to measure client-side performance using a separate Thread Group with 1 (or several) threads.

You can kick off any external program using JMeter's OS Process Sampler or check out WebDriver Sampler plugin which provides JMeter integration with Selenium, you can write the code in a variety of supported languages, JavaScript is default.