Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I am looking into automating some performance testing and am struggling to find the answer to the above question.

If I create a test using Selenium and the IE WebDriver, which loads a page and waits for some event to happen (and element being populated via AJAX for instance), will the time recorded closely match what a user would record if they sat there with a stopwatch (allowing for user reaction time, etc)

share|improve this question

3 Answers 3

When you wait for some event to happen such as an element being populated via AJAX, you aren't literally waiting; you are polling. By adjusting your polling frequency, you can control the interval between when an event occurs and when Selenium detects it.

That said, Selenium might be the wrong tool for the job, but the question isn't detailed enough for me to tell.

share|improve this answer

If truth be told, Selenium will execute and assert pass/fail really fast (you cannot imitate a user's patience with Selenium) for an event such as checking and/or waiting for an element to load/populate. Besides, depending on your application, the polling might be faster/slower than a typical user's actions since it goes into the domain of fuzzy logic and may involve checking of multiple conditions that may or may not evaluate to true/false as fast as that for a human user. Also, you cannot account for any user-devised optimizations from past experiences in interacting with your website.

share|improve this answer

Maybe.

It will reflect approximately* the time a user's browser would spend waiting for AJAX, assuming the load on your servers is the same as the load the user encounters, and assuming that the user has a CPU and RAM configuration comparable to the one on the computer running the test, and assuming the user is on the same speed of connection you're using to test, with the same amount of network traffic.

*Data is only approximate due to the polling frequency as discussed by user246

There's some more information in this blog post about taking averages of response times to account for variance in any of the above:

In standard benchmarks, it is common to see 90th percentile response times used. The benchmark may specify that the 90th percentile response time of a transaction should be within x seconds. This means that only 10% of the transactions have a response time higher than x seconds and can therefore be a meaningful measure. For web applications, the requirements are usually even higher – after all, if 10% of your users are dissatisfied with the site performance, that could be a significant number of users. Therefore, it is common to see 95th percentile used for SLAs in web applications.

A word of caution – web page response times can vary dramatically if measured at the last mile (i.e. real users computers that are connected via cable or DSL to the internet).

There's also some information here about how to measure response time:

For the analysis of server-side problems measuring at the server-side is enough. We however have to be aware that this does not reflect the response times of our end users. It is a purely technical metric for optimizing the way we create content and service requests. The prerequisite to meaningful measurements is that we separate different transaction types properly.

So really, the question is what are you measuring? For server-side optimization, you probably want to isolate each AJAX request, run that with load using a load-testing tool, and see at what point it slows down beyond your acceptable standards as described in your SLA. Client-side optimization can only go so far when you don't control the client.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.