wiki:LoggingBestPractice

Logging Best Practice

A simple logging class already exists at TI12-security/trunk/python/NDG/Log.py, however far greater functionality and flexibility can be achieved by useing the python  logging package. All developers should familiarise themselves with this package document: in particular read the introduction because understanding the class-level pages is difficult without it. There is also a good introductory article  here. The discussion below assumes familiarity with the basics.

The key advantages of using the logging package is that when plugging packages together (such as dx and security) messages can be sent to different loggers in a logger heirarchy allowing decisions on how different messages are logged to be made by the application (script, webservice, etc) rather than the low-level module generating the message.

This could include sending log messages to a remote logging service. The logging package provides several remote logging handlers: SMTPHandler, SysLogHandler? and HTTPHandler. Alternatively we could implement our own logging handler to interact with a custom logging service.

Guidelines and requirements

All modules that wish to use the logging infrastructure should include the following lines:

import logging
logger.getLogger(__name__)

This ensures that the logger heirarchy will always reflect the package heirarchy. We would expect to end up with loggers such as "ndg.discovery.server" and "ndg.csml.parser".

Subsequent logging should be done by calling methods on the logger object, rather than the functions provided in the logging package (e.g. logging.debug()). The latter will send messages to the root logger, thus bypassing any package-level filtering.

Any script that imports a module using logging must ensure that at least one handler is installed into the logging heirarchy. The minimum code to do this is

logging.basicConfig()

# Or alternatively sending a message to the root logger
# will automatically install a handler
logging.info('Initialising logging')

Scripts could also set more sophisticated logging requirements such as:

root = logging.getLogger()
rootHandler = logging.StreamHandler()
rootHandler.setLevel(logging.ERROR)
root.addHandler(rootHandler)

csmlLogger = logging.getLogger('ndg.csml')
csmlLogger.setLevel(logging.DEBUG)
csmlLogHandler = logging.FileHandler('./log/csml.log')
csmlLogger.addHandler(csmlLogHandler)

csmlLogger.warn('Sent just to logfile')
csmlLogger.error('Sent to stdout and logfile')

This would log all messages from the ndg.csml package (and sub-packages) to "./log/csml.log" and also log all ERROR and CRITICAL messages to sys.stdout (i.e. ERROR/CRITICAL messages from ndg.csml get logged twice)

Note that the logging level can be set on loggers or handlers, however in the instance above doing root.setLevel(logging.ERROR) would not have the desired effect because loggers will not filter messages that have been passed up the logger heirarchy.