Login

Twisted protocol for receiving logging module messages over a socket

Author:
afternoon
Posted:
October 8, 2008
Language:
Python
Version:
1.0
Tags:
logging twisted
Score:
2 (after 2 ratings)

When using Python's logging module in a concurrent environment (e.g. mod_python), messages get dropped by the standard file-based handlers. The SocketHandler allows you to send logging messages to a remote socket. This snippet provides code for listening for such messages and writing them out to a log file. The final log file is configured as a standard logging file-based handler.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
"""A Twisted receiver for messages sent by Python logging's SocketHandler.

The format used by SocketHandler is a 4-byte length record followed by a pickle
containing the log data.

To start the receiver, use bin/logreceiver.tac::

    twistd --python=bin/logreceiver.tac

"""
from struct import unpack
from cPickle import loads

from logging import makeLogRecord, getLogger
from logging.config import fileConfig
from logging.handlers import DEFAULT_TCP_LOGGING_PORT

from twisted.application.service import Application
from twisted.application.internet import TCPServer
from twisted.internet.protocol import Protocol, Factory

from django.conf import settings


class Logging(Protocol):
    def __init__(self):
        self.data = "" # definitely must be bytes, not unicode
        self.slen = None

    def dataReceived(self, data):
        """Handle data from the log sender."""
        self.data += data

        # grab the length field from the first 4 bytes of the message
        if not self.slen and len(self.data) >= 4:
            self.slen = unpack(">L", self.data[:4])[0]

        # handle a chunk (be careful in case we have data from the next chunk)
        if self.slen and len(self.data) >= self.slen + 4:
            self.handle_chunk(self.data[4:self.slen + 4])
            self.data = self.data[self.slen + 4:]
            self.slen = None

    def handle_chunk(self, chunk):
        record = makeLogRecord(loads(chunk))
        logger = getLogger(record.name)
        logger.handle(record)


class LoggingFactory(Factory):
    protocol = Logging


fileConfig(getattr(settings, "LOG_RECEIVER_CONFIG_FILE", "logging.ini"))
log = getLogger("myapp")
log.debug("Started log receiver")

service = TCPServer(DEFAULT_TCP_LOGGING_PORT, LoggingFactory())
application = Application("Log Receiver")
service.setServiceParent(application)

More like this

  1. MODPYTHON logging by adroffner 7 years ago
  2. log django exceptions to file by vaibhav-jain 5 months ago
  3. A view for downloading attachment by achimnol 5 years, 7 months ago
  4. Logging solution for mod_python/FCGI by mikeivanov 7 years, 6 months ago
  5. Logging Middleware by Magus 7 years, 6 months ago

Comments

Please login first before commenting.