2

We have a distributed application which should be monitored. There are some modules which collect external data (disk usage, free disk space, CPU utilization, memory utilization, etc) and some internal data (number of nodes online, number of files in storage systems, number if users online, etc).

The main contradictory point in this question is that to check if these modules are working we essentially need to create the modules for testing framework, which duplicate the functionality of the modules under test. There are some additional functions, for example, we can change some data to check if the collecting module reacts to that (create large file, get one node offline, etc), but in general it is the same functionality as the tested module plus more. It seems like a waste of resources to create tests which are more complex than the actual module itself and fully include its functionality.

So what is the best approach to testing such modules?

Improve this question
6
  • 1
    There are no "best practices", there are only good practices in context.
    – Bruce McLeod
    Commented Aug 21, 2012 at 22:11
  • Welcome to SQA, Ivan Ivanyuk. I am not sure I understand the question. You have an application, and you have monitoring modules. You are concerned that your testing strategy involves duplicating "the same module for testing framework". Do you mean (1) duplicating all or parts of the application or (2) duplicating the testing modules? I assume you meant (1), but your use of the phrase "same module" is throwing me off.
    – user246
    Commented Aug 22, 2012 at 1:21
  • @user246, yes, I meant case(1). Edited question to express it more clearly.
    – Ivan Ivanyuk
    Commented Aug 22, 2012 at 5:14
  • It might help to make this question more specific. Choose a monitor for a single value (e.g. disk usage) and describe why you believe you need to duplicate the application in order to test that monitor. That may help uncover your assumptions, which will help us suggest alternatives.
    – user246
    Commented Aug 22, 2012 at 14:03
  • @user246, well, the question was meant to be general :-) to cover basic principles of testing such modules.
    – Ivan Ivanyuk
    Commented Aug 23, 2012 at 6:56

1 Answer 1

Reset to default
4

The question is what your test objectives are: What exactly do you want to test?

I see two options here. Let's take the monitoring of free disk space as an example. By your tests, do you want to ensure that you can actually correctly measure the free disk space, or do you want to ensure that the graphs and statistics based on the free disk space data are correct?

If you really want to test that the procedure that determines free disk space works, you will most probably have to actually let it work on a device, and that will be a lot of test effort (maybe you could try to create a device emulation to help you with that).

But I suspect that determining free disk space (and the like) would be just a system call for you, or you maybe use a third-party library for it. In that case, if you trust the system call or library to be working, you lose nothing by just mocking that system call and generating some controlled mocked free disk space data. The tests then focus on checking what your monitoring modules do with this data: Does the data have correct timestamps? Are moving averages computed correctly? Do the graphs work? etc

Improve this answer
2
  • Well, in our case it's a mix of both objectives. For example we need to know free disk space on every fs which mounted in specific folder for balancing disk load. So we need to check that this system call is used correctly and returns all relevant data; and then to check that results are correctly processed and transferred to our monitoring database.
    – Ivan Ivanyuk
    Commented Aug 22, 2012 at 7:35
  • 1
    For checking whether the system call is used correctly, you could mock it. I'm not sure about checking whether it "returns all relevant data". You should know what data it returns. You can then check (again, via Mocks) whether that returned data is handled correctly.
    – Joachim Hofer
    Commented Aug 22, 2012 at 10:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.