Persistent connection to PostgreSQL database

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# Custom DB backend postgresql_psycopg2 based
# implements persistent database connection using thread local storage
from threading import local

from django.db.backends.postgresql_psycopg2.base import DatabaseError, \
    DatabaseWrapper as BaseDatabaseWrapper, IntegrityError
from psycopg2 import OperationalError

threadlocal = local()

class DatabaseWrapper(BaseDatabaseWrapper):
    def _cursor(self, *args, **kwargs):
        if hasattr(threadlocal, 'connection') and threadlocal.connection is \
            not None and self.connection is None:
            try: # Check if connection is alive
                threadlocal.connection.cursor().execute('SELECT 1')
            except OperationalError: # The connection is not working, need reconnect
                threadlocal.connection = None
            else:
                self.connection = threadlocal.connection
        cursor = super(DatabaseWrapper, self)._cursor(*args, **kwargs)
        if (not hasattr(threadlocal, 'connection') or threadlocal.connection \
             is None) and self.connection is not None:
            threadlocal.connection = self.connection
        return cursor

    def close(self):
        if self.connection is not None:
            self.connection.commit()
            self.connection = None

More like this

  1. Custom db backend for statement_timeout support (postgresql_psycopg2) by gonz 3 years, 1 month ago
  2. localsettings by elpaso66 3 years, 2 months ago
  3. Server Side Cursors for Django's psycopg2 Backend by ryanbutterfield 2 years, 2 months ago
  4. django-techblog fields.py for use with postgresql by dhmorgan 2 years, 10 months ago
  5. PostgreSQL ON DELETE CASCADE by mjt 3 years, 6 months ago

Comments

adamcik (on August 30, 2009):

Love the idea, however pyscopg2.pool.PersistentConnectionPool might be of interest if one wishes to make this snippet thread safe :-)

See the pyscopg2 git-repo for the pool code for more details.

#

mike_tk (on August 31, 2009):

I have the pyscopg2.pool.PersistentConnectionPool code ready too, but not published yet. I've added note about threads safety.

#

ripper (on May 7, 2010):

This snippet more then doubled my speed while making huge amounts of posts to my django app but in its current state leaves idle connections that dont go away in postgres. I have crashed many test servers with this code by filling up postgres with huge amounts of idle connections that dont go away =)

#

mike_tk (on October 29, 2010):

@ripper

Probably because you use it in threaded mode. The number of connections should not goes higher than processes amount * threads in each process.

I never had such kind of problems with apache mod_wsgi in daemon mode.

#

(Forgotten your password?)