Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

You can find quite an interesting panoply of open source operating systems to choose from. Being open source you can have access to a lot of their code, build systems, bug tracking, etc.

As of this time I didn't came across a document that plans or shows the design of stability testing for an OS. It would be useful for me to know about: what test suite they use, what benchmarks, what stress tests, do they have some stability test plans released to the public, etc?

I used here stability testing as an umbrella term for:

Can someone provide some guidelines on how open source OS stability testing is done?

share|improve this question
    
Perhaps you could spur some targeted answers by including your definition of "stability testing"? –  Joe Strazzere Jul 9 '13 at 11:34
    
@JoeStrazzere Thanks! I gave it a shot. –  Eduard Florinescu Jul 9 '13 at 12:24

1 Answer 1

up vote 1 down vote accepted

Short version

Not necessarily.

Long version

The idea

First thing to realize is what open-source means for software: the basic idea is that the source code is available. In more specific terms, someone has decided (for a new project or a running one) to share the code, i.e. every technical detail with the whole world. Furthermore, since there's technically nothing that can stop people from compiling and/or fully using such code it typically also means that the software is free (as in free beer) to use.

That's about the idea of open source. It has nothing to do with planning or organization of human resources.

The community

So in practice, and if the project is successful and interesting enough, the above will also mean that there will be people who will actually (try to) use it, and there will be people that will look at that code, study it, analyse it, play with it and talk about it, and eventually send bugfixes back to the author.

From this list of activities, at least three involve, or are a form of what is known as software testing.

So vaguely said, the quality in FOSS comes from:

  • lot of discussion before design and coding
  • perpetual code peer review
  • feedback from people trying to play with the the SW
  • feedback from people trying to use the SW in real situations
  • feedback from people trying to help develop the SW (maybe just fix their use case)
  • sometimes even feedback from professionals paid by companies that have their business built on FOSS technologies
  • no deadline tyranny
  • and also quality of the original code base and architecture

much more than creation and execution of testing plans (perfectly ballanced to match the needs of the focus group).

The strategy

Other thing to consider is, that each planning must start with a strategy. For commercial products, strategy is money and money is strategy: Who will run it? How will they run it? How much will get back at us? How much support requests we'll be able to handle? How many different hardware platforms even exist?

The strategy part is really a tough call since it usually happens before anything else, and the decisions taken here will influence almost every aspect of the whole project, and are extremely expensive to change. Therefore great amount of information needs to be gathered, which costs additional resources.

No need to say that any reasonable test planning needs to copy the strategy. (You're not going to test the brand new .NET-based accounting software with Powershell-powered installation routines on Win 2000, are you?)

Since most open source starts either as "scratching someone's own itch", without much focus on how much other people have it, it's virtually impossible to take this step, but it's also much less important, since more users can't scratch itch any better than the developer can themselves. If the scratcher is not sharp enough, they can help sharpening (send a patch).

The conclusion

Just in case you got that feeling: I'm not trying to say that there cannot be a highly organized group of professionals proceeding with a rigorous and well-thought testing plan on an open source. It definitely can and many such teams definitely do exist (I think Red Hat might have group like that for Fedora). But just because they are open-sourcing the code, it does not mean that they are willing to open-source their know-how.

At the end of the day, what counts most are bug reports, test cases and patches.

Operating systems are no exception here.

share|improve this answer
2  
More specific to the question: for an open-source OS, there isn't a specific thing called a stability test, but lots of people (with a vested interest in stability) test it in their own environment. –  user246 Jul 10 '13 at 0:37
    
@user246 Can you give a link with an example of other organization running some stability tests on some open-source OSs? –  Eduard Florinescu Jul 10 '13 at 6:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.