🌱 Quentin Pleplé

Project: Color Apache Logs

May 16th, 2014

This is a tool to color Apache logs to make them more readable.

tail -f /var/log/apache2/access_log | python color-apache-logs.py

Here you can see, from left to right:

  • in gray, the IP adress of the visitor, when it’s not a bot
  • the response code and method, in green for 2XX responses, yellow for 3XX and 4XX responses and red for 5XX responses
  • the response time in milliseconds, in green if below 400ms, yellow between 400ms and 750ms and red above 750ms
  • the URI queried in white
  • the referrer in gray

To tell Apache to log the response times, we have to define a new LogFormat in the VirtualHost section. I created a new combined_response_time log format by appending %D (response time in microseconds) to the classic combined log format. See Apache’s documentation on custom log formats for more information.

<VirtualHost *:80>
    ...

    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" %D" combined_response_time
    CustomLog /var/log/apache2/access_log combined_response_time
</VirtualHost>

Here is the tool. Feel free to change the thresholds or the way fields are displayed.

# apache-logs-colored.py

import sys, re
from termcolor import colored

# %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" %D

e = re.compile("""
    (?P<ip>[^ ]*)
    \
    (?P<logname>[^ ]*)
    \
    (?P<user>[^ ]*)
    \ \[
    (?P<datetime>[^\]]*)
    \]\ "
    (?P<method>[^ ]*)
    \
    (?P<path>[^ ]*)
    \
    (?P<protocol>[^\"]*)
    \"\
    (?P<response_code>[^ ]*)
    \
    (?P<response_size>[^ ]*)
    \ \"
    (?P<referer>[^\"]*)
    \" \ \"
    (?P<user_agent>[^\"]*)
    \"
    (?:
        \
        (?P<response_time>[^ ]*)
    )?
""", re.VERBOSE)

def print_req(req):
    if 'bot' in req['user_agent']:
        print colored('          [bot]', 'cyan'),
    else:
        print colored('{ip:>15}'.format(**req) , 'grey'),

    colors = {'2': 'green', '3': 'yellow', '4': 'yellow', '5': 'red'}
    color = colors[req['response_code'][0]]
    print colored("{response_code:<3} {method:<4}".format(**req), color),

    if 'response_time' in req and req['response_time']:
        ms = int(req['response_time']) / 1000
        color = 'green' if ms < 400 else ('yellow' if ms < 750 else 'red')
        print colored('{:>4}'.format(ms), color),
    else:
        print '    ',

    print req['path'],

    if req['referer'] != '-':
        print colored(req['referer'], 'grey'),

    print

while True:
    line = sys.stdin.readline()
    if line == '':
            break
    m = e.match(line)
    if not m:
        print colored('NOT MATCHED: ' + line, 'red')
        continue
    req = m.groupdict()
    print_req(req)

You may need to install package termcolor:

pip install termcolor

Comments