What's the best way to test log4net logging?

Question

What's the best way to test log4net logging?

As logging is to some degree an interface to our component, I want to make sure it keeps working as it can be configured in a few different ways.

Should we write a custom appender or is there one out there that we can use already?

Answer 1

Your question brings these considerations to mind:

• Does the component invoke the logging system at the appropriate times?

This is hard to determine and impossible to determine automatically because you cannot predict where things will go wrong. It is likely that additional logging will appear as a result of bugs discovered during testing.

• Is the appropriate data logged?

There are two parts to this. First, do the log messages reflect what the developer intended? You must create the conditions that cause the component to log each kind of message. This part may be suitable for automation. In some cases, the cost of creating those conditions may overweigh its value. One often uses mock objects in these cases, although mock objects are not a panacea. Second, having generated the message, you must decide whether it contains the intended data. Automating this may require parsing, and again, you must decide whether the cost of implementing and maintaining the parsing code is justified.

Second, you must decide whether you agree with the developer's intentions. Does the message contain the appropriate amount of data? This is difficult to automate and is perhaps more suitable for a manual audit.

• Does the component honor the logging instructions in the database?

I do not know whether log4net has APIs for interrogating its configuration. If it does, I recommend using them to determine whether the component honors the instructions from the database. If not, you may need to infer an answer based on its behavior. For example, if the database specifies logging to a certain file, you can check whether that file exists and is non-empty. This sounds as if it can be automated.

• Do the logging messages strike the appropriate balance of clarity and economy?

This is a judgement call that cannot be automated.

• Does the log file satisfy any requirements of downstream tools?

Will someone want to parse the log messages to look for usage patterns or urgent problems? If so, are the log messages in a format amendable to parsing?

Finally, to be candid, I believe the logging system is best tested by developers because they will be the primary consumers of the logs. When I test a product, I am primarily concerned that (1) the logging system will not put too much burden on the machine, (2) changing the log level will not cause the software to malfunction, and (3) the logging system honors its configuration. There are rarely any hard requirements around logging messages and so it is difficult to have an objective discussion with a developer about whether the way in which something is logged is right or wrong.