Excessive time taken by notification can lead to db timeout?

[pranavbhasin@…]

All database activites are taking a long time - creation of a ticket always times out though the ticket gets saved properly and now I am consistently getting this error while hitting any page that makes a database call.

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 406, in dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 183, in dispatch
    req.perm = PermissionCache(self.env, req.authname)
  File "/usr/lib/python2.4/site-packages/trac/perm.py", line 263, in __init__
    self.perms = PermissionSystem(env).get_user_permissions(username)
  File "/usr/lib/python2.4/site-packages/trac/perm.py", line 227, in get_user_permissions
    for perm in self.store.get_user_permissions(username):
  File "/usr/lib/python2.4/site-packages/trac/perm.py", line 109, in get_user_permissions
    db = self.env.get_db_cnx()
  File "/usr/lib/python2.4/site-packages/trac/env.py", line 203, in get_db_cnx
    return DatabaseManager(self).get_connection()
  File "/usr/lib/python2.4/site-packages/trac/db/api.py", line 76, in get_connection
    return self._cnx_pool.get_cnx(self.timeout or None)
  File "/usr/lib/python2.4/site-packages/trac/db/pool.py", line 107, in get_cnx
    raise TimeoutError('Unable to get database '
TimeoutError: Unable to get database connection within 20 seconds

[Emmanuel Blot]

Which version of Trac do you use?

[pranavbhasin@…]

Here is the environment information: Trac 0.10 Sqlite 3.3.6 Pysqlite 2.3.3 Python 2.4.3

It appears to me that the database connection is not being closed at the end of each request, which causes the page not to refresh after the data has been saved and eventually causes the database to run out of connections.

[pranavbhasin@…]

Found the issue: The problem was with the smtp server not being reachable.

[Christian Boos]

Same thing happened on t.e.o recently. I think we should be able to cope with this situation, for increased robustness.

There should be a mechanism in place so that requests don't get stuck forever waiting for delivery. Looks like Python 2.6 allows to specify a ​timeout in SMTP(). Could we find some workaround for older versions?

[eric.descargues@…]

We have the same problem and an Apache server restart doesn't correct the problem.

Traceback (most recent call last):
  File "D:\trac\Python24\lib\site-packages\trac\web\main.py", line 406, in dispatch_request
    dispatcher.dispatch(req)
  File "D:\trac\Python24\lib\site-packages\trac\web\main.py", line 191, in dispatch
    chosen_handler = self._pre_process_request(req, chosen_handler)
  File "D:\trac\Python24\lib\site-packages\trac\web\main.py", line 263, in _pre_process_request
    chosen_handler = f.pre_process_request(req, chosen_handler)
  File "D:\trac\Python24\lib\site-packages\trac\versioncontrol\api.py", line 73, in pre_process_request
    self.get_repository(req.authname).sync()
  File "D:\trac\Python24\lib\site-packages\trac\versioncontrol\api.py", line 96, in get_repository
    db = self.env.get_db_cnx() # prevent possible deadlock, see #4465
  File "D:\trac\Python24\lib\site-packages\trac\env.py", line 203, in get_db_cnx
    return DatabaseManager(self).get_connection()
  File "D:\trac\Python24\lib\site-packages\trac\db\api.py", line 76, in get_connection
    return self._cnx_pool.get_cnx(self.timeout or None)
  File "D:\trac\Python24\lib\site-packages\trac\db\pool.py", line 107, in get_cnx
    raise TimeoutError('Unable to get database '
TimeoutError: Unable to get database connection within 20 seconds

Do you suggest a workaround ?

[ebray]

Undoing anonymous butchering. Setting priority and severity back to sane values—it's clear that the original issue in this ticket is due to notification problems. This isn't the only ticket addressing this problem either.

[osimons]

Not really a notification problem, it is a db pool problem. But, as notifications may take a long time (and even longer with smtp errors), it may hog the connection for longer than expected - eventually exhausting the default process-wide pool of 10 connections and locking the process.

Hopefully improved by recent change [7537].

[Christian Boos]

I'm not sure we still have db connection active when we're about to send the notification, but if so, it's something we should fix.

[Christian Boos]

Regarding comment:4, not sure we can do anything before Python 2.6.

For the more general problem, see also #9111.