We have given a task to a third party company to develop a website from scratch for us.

What kind of measurable goals can be put in place in order to get quality assurance from the development. Furthermore, what performance criteria could be set for testing?

What is the general approach for these two questions?

  • I'm new to this forum, please let me know if anything needs editing/changing or further specifying/clarification.
    – 3kstc
    Jan 12 '18 at 3:50
  • Plenty of help you can get by Googling, it is hard to make any practical suggestions, it is a bit too general.
    – Yu Zhang
    Jan 12 '18 at 5:47
  • @YuZhang I tried Googling, I couldn't find something with decent quality. Any suggestions are welcome.
    – 3kstc
    Jan 12 '18 at 6:03
  • The question is: what is it that you actually trying to solve? You ask for something that would work everywhere. That doesn't exist. My website might be OK with 2 requests per day. Yours might need to support 2K per minute. That will influence your criteria in a big way.
    – Ray Oei
    Jan 12 '18 at 14:33
  • Do you want to do "QA" to check that the 3rd party delivers what you have asked or do you want to check that it works the way you intended? Might look the same thing, but it often isn't! And how do they work? Are you able to get regular demos on what they build or is it big bang at the end? This will all influence your questions on what to measure and to what purpose....
    – Ray Oei
    Jan 12 '18 at 14:34

In ideal world you should have NFRs which describe in details what is expected and what are the acceptable ranges for different metrics and pass them along with other (functional) requirements to the company. The underlying reason is that performance needs to be checked on early stages as it will appear that your application needs to scale and the company designed it in an unscalable way - they will need to re-implement its core from scratch.

Normally people are interested in answers to the following questions:

  1. Will my application support X users providing reasonable response time where "reasonable" may vary depending on your application nature - Load Testing
  2. Will my application support prolonged load (there are no memory leaks, it doesn't run out of free space, there is no performance degradation over time) - Soak Testing
  3. What is the maximum amount of users my application can handle before breaking (or response time starts exceeding acceptable) - Stress Testing

The metrics mostly depend on your application nature and your NFRs, the most commonly used performance metrics for web applications are:

  • Response Time (average/min/max/median/90%)
  • Throughput (number of requests/transactions/bytes per second)
  • Resources consumption (CPU, RAM, Swap, Disk, etc.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.