8

There have been several questions lately along the lines of "Is X a functional or non-functional type of testing?". Although I understand the distinction, it's never been an important distinction for me in my career. Either I was on a team that only did one type of testing, or I have been the only tester on the team and responsible for all types of testing (and I didn't make a big distinction between functional and other types of tests).

What are the pros and cons of separating functional tests from non-functional tests? Are the concerns usually more about dividing work in a team, organizing test suites, applying different policies for different types of testing, or what?

| improve this question | | | | |
  • I'd really like to hear more from people who do understand why this distinction is sometimes made. Clearly it's important to some people, or we wouldn't be getting questions asking which category X testing fits into (where X is accessibility, installation, etc.). – Ethel Evans Jul 18 '11 at 22:41
  • Wonder if it has become "more important" and "less important" tests now a days :-| – Tarun Jul 19 '11 at 14:25
  • @Bharat who is this quoting? – jonrsharpe May 3 at 7:41
4

Why separating functional/non-functional aspects of a system is important for me as a tester

I make such a distinction at the very beginning of a project (either during drafting the strategy or implementing it). It helps in making non-functional or 'implicit' (another word for overlooked, forgotten, or ignored completely) requirements sufficiently 'explicit' to everyone involved. This enables me to ascertain and communicate about where (and how much) we are focussing immediately and how we plan to cater to the critical non functional aspects of the system over a period of time (why I say over a period is in the elaboration of non-functional requirements).

This is certainly good; since as "headlights" of a project you want to illuminate everything that lies ahead.

Few examples/benefits of making such distinction and then including it in your plan could be

  • As a tester I do not write unit tests (written by devs as a part of TDD) but since unit tests cater to non-functional requirements like testability,maintainability or extensibility; I can always make it a part of my strategy and point to the correct reference or to its lack thereof.
  • Similarly, I am not a security expert but I might highlight security or vulnerability as legitimate concerns and let experts handle that.

Some elaboration on non-functional requirements

Non-functional requirements are largely cross cutting and implicit, and once we pay heed to them; it has to be a continuous effort through the project lifetime. Mike Cohn calls these "constraints" and says these are similar to taxes which we have to pay in every agile sprint in this blog.

I largely derive the functional/non-functional distinction from Brian Marick's testing quadrants which has been elaborated in the book Agile Testing by Lisa Crispin and Janet Gregory as well.

For me business facing tests are mostly functional and technology facing tests make up for the non-functional requirements.

| improve this answer | | | | |
  • I think you are getting close to what I'm trying to understand, but I'm having a hard time pulling it out from the rest of this post. Could you reorganize your post so it answers the explicit questions I asked first, or with bullet points, or in some other way separates them from additional discussion? I'm interested in the discussion also, but I'm having trouble understanding what is a direct answer and what is essentially elaboration. – Ethel Evans Jul 20 '11 at 19:51
  • 1
    @Ethel ..updated/re-arranged the content a bit. I hope now it makes some sense :-) – Rajneesh Jul 21 '11 at 16:12
5

We categorize things because we believe we need to treat the categories differently. It is hard to express an opinion about why separating functional and non-functional tests is important without knowing how one would use the answer. Important for what purpose? Sometimes people ask such questions in these forums because they did not have an answer in a job interview or a classroom. In such circumstances, the question may seem more important than it really is. Perhaps that is the case here.

It seems to me that "non-functional requirement" is just a clumsy way to say "implied requirement" or "unspoken requirement" or "goes-without-saying requirement".

It may seem important to distinguish functional and non-functional testing for budgeting purposes. Some non-functional requirements are treated differently because they have become specializations, e.g. internationalization testing and security testing. Sometimes specializations are outsourced to other groups or other companies because it is too expensive or time-consuming to maintain expertise in-house.

It seems to me that the test plan for a non-functional requirement must be made explicit just as it is for functional requirements. Once the test plan is explicit, the dichotomy does not matter.

| improve this answer | | | | |
  • 1
    This is exactly the way that I've always looked at it as well, although I haven't seen the term used since the last binder sized Business Requirements Document that I received. I've found that in agile-, context-driven environments that the distinction tends to be removed entirely – Lyndon Vrooman Jul 19 '11 at 10:03
1

I'd see it as more of a tracking issue, I haven't done much non-functional testing (at least I wouldn't call it that) although I could look back and probably say we covered that in some other kind of testing. Mostly when working on a project I look at the Functional Requirements and start basing tasks/tests/checks off that, anything not in the requirements might be considered non-functional but I've never called it that.

If you go by Wikipedia, we've probably all done it: http://en.wikipedia.org/wiki/Non-functional_testing

Although I never quite grasped their definition of behavior for Requirements either. http://en.wikipedia.org/wiki/Non-functional_requirements

| improve this answer | | | | |
1

Any time you group tests into a suite of tests you are likely grouping along some category. For example, we have several suites of functional tests that get ran against every daily build. We also have other tests for security, battery, reliability (non-functional categories). Some of these suites get ran less frequently.

Although we don't necessarily differentiate between structural, functional, non-functional, or behavioral tests explicitly, these are categories that help us model our suites of tests so we can design our tests on specific focus areas.

| improve this answer | | | | |
0

I haven't seen it as an important distinction - perhaps because my test teams are small, and are responsible for all the testing on their projects. I don't have specialization to the level of "this one does functional testing" and "that one does non-functional testing".

It's good to understand the depth and breadth of possible testing though, as you decide what you will test for a given project, and what you won't test.

| improve this answer | | | | |
0

It's important to make sure both are covered.

Separating them helps ensure that.

It's not so much about the distinction and definition of each being different - others address that - it's more the simple fact that the separation approach makes sure each are approached correctly for their purpose.

| improve this answer | | | | |
0

Every software testing company performs functional and non-functional testing in projects. The main reason for the separation of functional and non-functional testing is that coverage should be done as per the requirement of the project.

Functional testing ensures that the requirements or specifications are properly satisfied by the application. Whereas Non-functional testing is a type of software testing that is performed to verify the non-functional requirements of the application. It verifies whether the behavior of the system is as per the requirement or not. It tests all the aspects which are not tested in functional testing.

| improve this answer | | | | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.