Non-Functional requirements specification

Question

I'm working on a college software project. The whole purpose of the software is to test the application. The teacher is focusing on non-functional requirements, so we have to present metrics, indicators for reliability, usability, performance and maintainability. The thing is, this application will only run in locally and is designed only for one user. I'm having problems coming up with metrics and indicators, considering I don't have a point of reference. I see many examples but I'm sure the teacher will ask why I picked the numbers I did. I asked him, but he told me I'll just have to do some research, which I did, but couldn't manage to apply them to this case.

Do you have any links that provide this kind of information?

Answer 1

Welcome to the evil realm of nonfunctional requirements.

The core information your teacher gave you is, in other words, "No idea, go figure something out, and I will decide if I agree with it." And this is what happens most of the time in real world projects.

Focusing on your information: one user, local environment, we already have some valuable points to analyse. What you need are use cases. What are your users suppose to be doing with the program, where is the focus of its use?

Examples:

Reliability:

The program doesn't crash, if if you do X. X is your metric here, which you can brak down to e.g. "normal stuff", "extra stuff (running other programs in background, chaning focus,..), "evil things" (you want to leave these things out, because if a user decides to run the program on a laptop in a marathon race, blindfold, jumping on one leg and having pi calculated in the background to the 10^20000 decimal).

Usability:

The user can work with the program, ..X. X as metric again: "...and is able to use all functions after no more than 3 clicks/windowchanges"(simple), "...and a new user is able to complete the program within a certain timeframe of x minutes" (evil)

Performance:

Loading time for the program(<1min), time between changes when the user is interacting with it(<2s), and so on.

Maintainability:

How should updates, bugfixes be handled? How often? (Like: once a year, updated must not require a full reinstall, ...)

It all depends on the scenario you whish to use the program in. Your other issue, defending the metrics, requires some kind of evidence that you did not make these values up. My suggestion is to conduct a field analysis of those who should later use the program(assuming the user is at all in the scope of the program's design). Take times for performance issues, confront users with "this was 5s, are you sure you are satisified with this speed just for a scroll down action?". Just watch out that you are using realistic values: 0,01s reaction time is nice, but is it necessary for the aim of the program?

Good luck!

Answer 2

For me, the baseline is always a comparison to some other system.

The other system need not be software. It might be implemented by, say, people wielding pencils and paper forms. If that exists, you can measure its response time, availability, reliability, error rate, and so on.

The other system need not exist. I can always posit an imaginary people/pencils/paper system and (at some risk, given the increased challenge of measuring an imaginary system) use that as a baseline.

Answer 3

"we have to present metrics, indicators for reliability, usability, performance and maintainability."

Reliability is the ability to get the same result over and over. To get a reliability metric, perhaps your project knows the expected results of the application being tested, and then records the actual result from running that same scenario X number of times. Then it can tell you "the expected result matched the actual result Y times out of X runs", and the context for each run.

Performance is usually thought of in the case of online applications when used by many users...but it can also be thought of simply in terms of system resources being used at different times. If you go to the Task Manager there is a tab for performance which gives you CPU usage information, and the Processes tab gives you memory usage information per process. If there's a way for your project to capture that information at certain times both while the application is running and while it is not running (so you have a comparison), then you have performance metrics. Response time is also a performance metric - try measuring how long it takes from the time the user does something to the time that action is actually completed.

Maintainability as far as I know is just a measure of how quickly/easily a programmer can make a change to the application. The only way I can think of as to how to get maintainability metrics in your project would be to have a way for you to log what you do and how long it takes for you to do it, so that information can be reported on later.

Usability is kind of broad and may depend on what you have access to. It might include availability, which would be a measure of the extent to which the application is up and running (if it crashes or simply is not accessible that affects the availability metric and you'll want to have a log of how much time it is unavailable for). And/or your teacher may be looking for some internal metrics...For example maybe Cyclomatic and/or Structural Complexity. You could make your project "crawl" through the application's code to determine the depth and width and number of paths for each control or function point. On the other hand, if you do not have access to the code for the application being tested then do you at least have usability requirements? - It may just be a matter of logging how many (and which) usability requirements are actually met.

There can, of course, be a lot more to each of these topics, but I hope that provides an overview that helps know how to apply these to your project. Good luck!

Answer 4

Sounds like your professor wants code coverage metrics. Your question doesn't include what language you're working with, but Code Climate works with Ruby, JS, or PHP (and it's free for open source projects).

These code coverage tools usually measure complexity, smells, regressions, and overall test coverage. They seem like good metrics for your project.

Whatever language you're using, I'm sure you could find a solid, open source code coverage tool for it. Good luck.

Answer 5

I have an example for you. Performance requirement; for a web application, how long it takes the application to perform a given function. If you open your browser's developer tools, you will come across a range of performance measuring tools that can generate lots of numbers and graphs for you. https://www.google.com/search?q=firefox+developer+tools

You should also find out the hardware specifications of your users' PCs and run the tests on a machine with similar specs.

To come up with some numbers for what are acceptable response times, you could run a simple survey with your "users". "What is the maximum time you would wait for the application to perform xyz function?"