Due to compliance requirements in the financials industry we needed to log every request a user made to our system, the action taken (view function) and response from the server.
I found a lot of other logging solution bit most revolved around debugging and DB query logging. I needed to be able to tell what a user did while being logged in as much detail as I could with out tracking the mouse pointer position on screen.
So I created this (, my first ,) middleware. Its very simple really. keeping track of a request, view_func and response object in a single model called Record (models.py file included in the code).
The fields I used are optimized to what I intend to show in the UI I am planning for this model. Depending on how you use the doc string of your views they can be tapped to explain to the user what each request/func/responce group in a session is meant to do.
There were a few gotcha's: 1. I only care about authenticated requests. So I added the 'quest.user.is_authenticated()' test. 2. I did not care about the favicon request so I skipped them. 2. The actual login request is not authenticated while the response is. This caused the process_response/view to look for a record that is not there. So I added the 'except ObjectDoesNotExist' to skip this case.
I added one bell: Logging a full HTML reply is wasteful and mostly useless. I added two values in the setting files. LOGALL_LOG_HTML_RESPONSE to toggle if we want to log them or not. And LOGALL_HTML_START to describe what a full HTML starts with. Personally I use the first few characters of my base.html template that all the rest of my templates expend. I simplified the code to the left for readability.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 | #settings.py
#Should we log full HTML responses?
LOGALL_LOG_HTML_RESPONSE = True
# If we how do we recognized a full HTML response
LOGALL_HTML_START = "<!DOCTYPE html"
#logAll.models.py
from django.db import models
from django.contrib.auth.models import User
class Record(models.Model):
"""
Basic log record describing all user interaction with the UI.
Will be propagated by a middle ware.
This will be one BIG DB table!
"""
created_at = models.DateTimeField(auto_now_add = True)
sessionId = models.CharField(max_length=256)
requestUser = models.ForeignKey(User)
requestPath = models.TextField()
requestQueryString = models.TextField()
requestVars = models.TextField()
requestMethod = models.CharField(max_length=4)
requestSecure = models.BooleanField(default=False)
requestAjax = models.BooleanField(default=False)
requestMETA = models.TextField(null=True, blank=True)
requestAddress = models.IPAddressField()
viewFunction = models.CharField(max_length=256)
viewDocString = models.TextField(null=True, blank=True)
viewArgs = models.TextField()
responseCode = models.CharField(max_length=3)
responseContent = models.TextField()
#logAll.middleware.py
import time
import simplejson as json
from django.core.exceptions import ObjectDoesNotExist
from logAll.models import Record
from settings import LOGALL_LOG_HTML_RESPONSE, LOGALL_HTML_START
class LogAllMiddleware(object):
def process_request(self,request):
# Only log requests of authinticate users
try:
if not request.user.is_authenticated():
return None
except AttributeError:
return None
# Skip favicon requests cause I do not care about them
if request.path =="/favicon.ico":
return None
newRecord = Record(
created_at = str(time.time()),
sessionId = request.session.session_key,
requestUser = request.user,
requestPath = request.path,
requestQueryString = request.META["QUERY_STRING"],
requestVars = json.dumps(request.REQUEST.__dict__),
requestMethod = request.method,
requestSecure = request.is_secure(),
requestAjax = request.is_ajax(),
requestMETA = request.META.__str__(),
requestAddress = request.META["REMOTE_ADDR"],
)
newRecord.save()
return None
def process_view(self, request, view_func, view_args, view_kwargs):
try:
if not request.user.is_authenticated():
return None
except AttributeError:
return None
# Fix the issue with the authrization request
try:
theRecord = Record.objects.get(
sessionId = request.session.session_key,
requestUser = request.user,
requestPath = request.path,
requestMethod = request.method,
requestSecure = request.is_secure(),
requestAjax = request.is_ajax(),
requestMETA = request.META.__str__()
)
theRecord.viewFunction = view_func.func_name
theRecord.viewDocString = view_func.func_doc
theRecord.viewArgs = json.dumps(view_kwargs)
theRecord.save()
except ObjectDoesNotExist:
pass
return None
def process_response(self, request, response):
# Only log autherized requests
try:
if not request.user.is_authenticated():
return response
except AttributeError:
return response
# Skip favicon requests cause I do not care about them
if request.path =="/favicon.ico":
return response
# Fix the issue with the authorization request
try:
theRecord = Record.objects.get(
sessionId = request.session.session_key,
requestUser = request.user,
requestPath = request.path,
requestMethod = request.method,
requestSecure = request.is_secure(),
requestAjax = request.is_ajax(),
requestMETA = request.META.__str__()
)
theRecord.responseCode = response.status_code
# Decidce wether we want to log the a full html response
# as this will probabaly will take a LOT of space.
#
# In my case most of the replies I want to catch happen
# to be plain text ajax replies
if LOGALL_LOG_HTML_RESPONSE:
# IF set to true then log the respoce regardless
theRecord.responseContent = response.content
elif response.content.startswith(LOGALL_HTML_START):
theRecord.responseContent = "FULL HTML RESPONSE"
else:
theRecord.responseContent = response.content
theRecord.save()
except ObjectDoesNotExist:
pass
return response
|
More like this
- Template tag - list punctuation for a list of items by shapiromatron 10 months ago
- JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 10 months, 1 week ago
- Serializer factory with Django Rest Framework by julio 1 year, 5 months ago
- Image compression before saving the new model / work with JPG, PNG by Schleidens 1 year, 5 months ago
- Help text hyperlinks by sa2812 1 year, 6 months ago
Comments
Wont you log request datetime?
#
line 150 to 156 with the code block that begins if LOGALL_LOG_HTML_RESPONSE:` looks a little strange, in fact it may as well look like this...
at the moment regardless of what
LOGALL_LOG_HTML_RESPONSE
is your code just like my example above callstheRecord.responseContent = response.content
anyway.I don't think this is what you wanted...
#
Please login first before commenting.