Log Handlers dictate how the log entries are handled. For example FileHandler would allow us to pipe our logs to a file. HTTP Handler makes it possible to send the logs over HTTP to a remote server. We can write our own log handlers if we need to customize the way our logs are processed. Writing a custom handler is pretty simple. We have to subclass it from logging.Handler class and must define the emit method. This method is called with each log record so we can process it.
Here’s an exmaple of a custom log handler which POSTs the logs to a remote server using the popular requests library.
|
1 2 3 4 5 |
class RequestsHandler(Handler): def emit(self, record): log_entry = self.format(record) return requests.post('http://example.com:8080/', log_entry, headers={"Content-type": "application/json"}).content |
Now we have our custom handler that will post the logs to another server. What if we want to send the message in a specific format? We write our custom formatter. A formatter has a format method which gets the record. We can take the record and return a message formatted according our need. Here’s an example for logstash format:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
class LogstashFormatter(Formatter): def __init__(self, task_name=None): self.task_name = task_name super(LogstashFormatter, self).__init__() def format(self, record): data = {'@message': record.msg, '@timestamp': datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S.%fZ'), '@type': 'IronWorker'} if self.task_name: data['@task_name'] = self.task_name return json.dumps(data) |
So we have got ourselves a custom log handler and a formatter. But how do we use them? Here’s a short code snippet:
|
1 2 3 4 5 6 7 8 |
def __init_logging(self): self.logger = logging.getLogger(self.logger_name.upper()) self.logger.setLevel(self.logging_level) handler = RequestsHandler() formatter = LogstashFormatter(self.logger_name.upper()) handler.setFormatter(formatter) self.logger.addHandler(handler) |
6 replies on “Python: Writing custom log handler and formatter”
how can i create custom logger function like we have default functions –
i want to create simple like below
logger.success(“message”)
I had to add Handler.__init__(self) to the __init__() method of RequestsHandler to get this to work:
class RequestsHandler(Handler):
def __init__(self):
Handler.__init__(self)
def emit(self, record):
log_entry = self.format(record)
return requests.post(‘http://example.com:8080/’,
log_entry, headers={“Content-type”: “application/json”}).content
Probably because of different Python versions.
Same here! Thanks for the solution!
For me, I used “
super(RequestsHandler, self).__init__()” in the __init__() of RequestsHandlerI’m trying to implement a similar behaviour in on elf my projects where I’m sending logs from one flask app (call it A) to another one (call it B) then in app B it actually call an was lambda function which stores the logs to a remote database. The reason to do it in this way I want to show my students how can we send logs over a remote server. Everything is working except one issue, upon running my first application and complete workflow it should add one log entry to database (because I have only one log statement in my app) but it’s creating the same log entry (except the process id) twice in the database, what can be wrong?
[…] you write your custom handler and formatter, these instances are already provided to you. Masnun, in this article , gives great examples about how you do […]