Internal error - database connection problem with one environment

[STZ-SW]

when I called my trac this morning I saw the following error message: "Trac detected an internal error:"

Traceback (most recent call last):
  File "/var/lib/python-support/python2.5/trac/web/main.py", line 387, in dispatch_request
    dispatcher.dispatch(req)
  File "/var/lib/python-support/python2.5/trac/web/main.py", line 191, in dispatch
    chosen_handler = self._pre_process_request(req, chosen_handler)
  File "/var/lib/python-support/python2.5/trac/web/main.py", line 263, in _pre_process_request
    chosen_handler = f.pre_process_request(req, chosen_handler)
  File "/var/lib/python-support/python2.5/trac/versioncontrol/api.py", line 73, in pre_process_request
    self.get_repository(req.authname) # triggers a sync if applicable
  File "/var/lib/python-support/python2.5/trac/versioncontrol/api.py", line 101, in get_repository
    repos = self._connector.get_repository(rtype, rdir, authname)
  File "/var/lib/python-support/python2.5/trac/versioncontrol/svn_fs.py", line 260, in get_repository
    crepos = CachedRepository(self.env.get_db_cnx(), repos, None, self.log)
  File "/var/lib/python-support/python2.5/trac/versioncontrol/cache.py", line 34, in __init__
    self.sync()
  File "/var/lib/python-support/python2.5/trac/versioncontrol/cache.py", line 53, in sync
    cursor = self.db.cursor()
  File "/var/lib/python-support/python2.5/trac/db/util.py", line 78, in cursor
    return IterableCursor(self.cnx.cursor())
  File "/var/lib/python-support/python2.5/trac/db/util.py", line 78, in cursor
    return IterableCursor(self.cnx.cursor())
  File "/usr/lib/python2.5/site-packages/pyPgSQL/PgSQL.py", line 2599, in cursor
    return Cursor(self, name, isRefCursor)
  File "/usr/lib/python2.5/site-packages/pyPgSQL/PgSQL.py", line 2718, in __init__
    self.conn._Connection__setupTransaction()
  File "/usr/lib/python2.5/site-packages/pyPgSQL/PgSQL.py", line 2510, in __setupTransaction
    self.conn.query("BEGIN WORK")
OperationalError: no connection to the server

At a first glance this looks like a database connection error. But only one environment is affected. The other ones work fine as before. So it can't be a database error, can it?

Resync didn't help and upgrade is not required (trac says).

[STZ-SW]

I forgot to say:

A more detailed error description is missing.

[Christian Boos]

Can you connect to that db? Are you running out of available connections (see also #4987)?

Also, have a look at #4984, it might be a similar issue (though the error message is different here).

[STZ-SW]

Replying to cboos:

Can you connect to that db? Are you running out of available connections (see also #4987)?

The db was fine and there were enough connections left.

Also, have a look at #4984, it might be a similar issue (though the error message is different here).

You are right could be the same problem!

[RedM@…]

We are having this issue as well. Our Trac server has been running great for about a year, but just recently started having this issue. We have 7 environments running on tracd (with postgres) and have seen this error on two different environments; the error occurs about once a week. Restarting tracd (and not touching postgres) temporarily solves the problem.

We've made a couple changes around the time we started having the issue:

• I setup the email2trac (v 0.10) script on our server. That includes installing the Microsoft SMTP server and running a batch file (that pushes the emails into email2trac) every ten minutes using Scheduled Tasks.
• In their infinite wisdom, our IT Dept installed Norton Ghost server on our Trac server (basicly repurposed to do both). I'm really suspicious that this has something to do with this new error.

Maybe there's some issue with running out of network bandwidth? Seems strange since Trac should be talking to Postgres through the loopback, but maybe it's still an issue?

Thanks for such a great product. We use it heavily for development and our Help Desk has recently switched from using Email to communicating completely through Trac.

Some configuration details: We are running Trac 0.10.3, Postgres 8.1.4-1, Python 2.4, pyPgSQL 2.5.1 all on the same Windows Server 2003 running in a virtual machine under VMWare.

[bryan@…]

I'm getting the same error. It looks to me as if the connection pool is closing idle connections down but not opening new ones if there are none available.

[Christian Boos]

Also, consider using psycopg2 instead of pyPgSQL.

[Christian Boos]

The db connection pool code has been overhauled in recent versions of Trac (0.11.1? - 0.11.2 for sure).

So if anyone still see this issue:

• when using psycopg2 instead of pyPgSQL
• when using Trac 0.11.2 or higher

then feel free to reopen this ticket, with all the informations that could allow us to reproduce and troubleshoot the issue.