Edgewall Software
Modify

Opened 12 years ago

Last modified 3 years ago

#6219 new defect

Excessive time taken by notification can lead to db timeout?

Reported by: pranavbhasin@… Owned by:
Priority: normal Milestone: next-major-releases
Component: notification Version:
Severity: major Keywords: smtp pool python26
Cc: Branch:
Release Notes:
API Changes:

Description (last modified by Emmanuel Blot)

All database activites are taking a long time - creation of a ticket always times out though the ticket gets saved properly and now I am consistently getting this error while hitting any page that makes a database call.

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 406, in dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 183, in dispatch
    req.perm = PermissionCache(self.env, req.authname)
  File "/usr/lib/python2.4/site-packages/trac/perm.py", line 263, in __init__
    self.perms = PermissionSystem(env).get_user_permissions(username)
  File "/usr/lib/python2.4/site-packages/trac/perm.py", line 227, in get_user_permissions
    for perm in self.store.get_user_permissions(username):
  File "/usr/lib/python2.4/site-packages/trac/perm.py", line 109, in get_user_permissions
    db = self.env.get_db_cnx()
  File "/usr/lib/python2.4/site-packages/trac/env.py", line 203, in get_db_cnx
    return DatabaseManager(self).get_connection()
  File "/usr/lib/python2.4/site-packages/trac/db/api.py", line 76, in get_connection
    return self._cnx_pool.get_cnx(self.timeout or None)
  File "/usr/lib/python2.4/site-packages/trac/db/pool.py", line 107, in get_cnx
    raise TimeoutError('Unable to get database '
TimeoutError: Unable to get database connection within 20 seconds

Attachments (0)

Change History (17)

comment:1 by Emmanuel Blot, 12 years ago

Description: modified (diff)
Keywords: needinfo added

Which version of Trac do you use?

comment:2 by pranavbhasin@…, 12 years ago

Here is the environment information: Trac 0.10 Sqlite 3.3.6 Pysqlite 2.3.3 Python 2.4.3

It appears to me that the database connection is not being closed at the end of each request, which causes the page not to refresh after the data has been saved and eventually causes the database to run out of connections.

comment:3 by pranavbhasin@…, 12 years ago

Resolution: worksforme
Status: newclosed

Found the issue: The problem was with the smtp server not being reachable.

comment:4 by Christian Boos, 12 years ago

Component: generalnotification
Keywords: smtp added; needinfo removed
Milestone: 0.12
Resolution: worksforme
Status: closedreopened

Same thing happened on t.e.o recently. I think we should be able to cope with this situation, for increased robustness.

There should be a mechanism in place so that requests don't get stuck forever waiting for delivery. Looks like Python 2.6 allows to specify a timeout in SMTP(). Could we find some workaround for older versions?

comment:5 by eric.descargues@…, 12 years ago

Priority: normalhighest
Severity: normalblocker

We have the same problem and an Apache server restart doesn't correct the problem.

Traceback (most recent call last):
  File "D:\trac\Python24\lib\site-packages\trac\web\main.py", line 406, in dispatch_request
    dispatcher.dispatch(req)
  File "D:\trac\Python24\lib\site-packages\trac\web\main.py", line 191, in dispatch
    chosen_handler = self._pre_process_request(req, chosen_handler)
  File "D:\trac\Python24\lib\site-packages\trac\web\main.py", line 263, in _pre_process_request
    chosen_handler = f.pre_process_request(req, chosen_handler)
  File "D:\trac\Python24\lib\site-packages\trac\versioncontrol\api.py", line 73, in pre_process_request
    self.get_repository(req.authname).sync()
  File "D:\trac\Python24\lib\site-packages\trac\versioncontrol\api.py", line 96, in get_repository
    db = self.env.get_db_cnx() # prevent possible deadlock, see #4465
  File "D:\trac\Python24\lib\site-packages\trac\env.py", line 203, in get_db_cnx
    return DatabaseManager(self).get_connection()
  File "D:\trac\Python24\lib\site-packages\trac\db\api.py", line 76, in get_connection
    return self._cnx_pool.get_cnx(self.timeout or None)
  File "D:\trac\Python24\lib\site-packages\trac\db\pool.py", line 107, in get_cnx
    raise TimeoutError('Unable to get database '
TimeoutError: Unable to get database connection within 20 seconds

Do you suggest a workaround ?

Last edited 3 years ago by Jun Omae (previous) (diff)

comment:6 by anonymous, 11 years ago

Resolution: fixed
Status: reopenedclosed

comment:7 by ebray <hyugaricdeau@…>, 11 years ago

Resolution: fixed
Status: closedreopened

comment:8 by anonymous, 11 years ago

Owner: changed from Jonas Borgström to Test
Status: reopenednew

comment:9 by anonymous, 11 years ago

Owner: changed from Test to anonymous
Status: newassigned

comment:10 by anonymous, 11 years ago

Resolution: fixed
Status: assignedclosed

comment:11 by ebray, 11 years ago

Priority: highestnormal
Resolution: fixed
Severity: blockernormal
Status: closedreopened

Undoing anonymous butchering. Setting priority and severity back to sane values—it's clear that the original issue in this ticket is due to notification problems. This isn't the only ticket addressing this problem either.

comment:12 by osimons, 11 years ago

Keywords: pool added

Not really a notification problem, it is a db pool problem. But, as notifications may take a long time (and even longer with smtp errors), it may hog the connection for longer than expected - eventually exhausting the default process-wide pool of 10 connections and locking the process.

Hopefully improved by recent change [7537].

comment:13 by Christian Boos, 11 years ago

Keywords: verify added
Summary: Error on trac setup on RHEL5 with sqlliteexcessive time taken by notification can lead to db timeout?

I'm not sure we still have db connection active when we're about to send the notification, but if so, it's something we should fix.

comment:14 by anonymous, 10 years ago

Summary: excessive time taken by notification can lead to db timeout?Excessive time taken by notification can lead to db timeout?

comment:15 by Christian Boos, 9 years ago

Owner: changed from anonymous to Christian Boos
Severity: normalmajor
Status: reopenednew

Regarding comment:4, not sure we can do anything before Python 2.6.

For the more general problem, see also #9111.

comment:16 by Christian Boos, 9 years ago

Keywords: python26 added; verify removed

comment:17 by Ryan J Ollos, 4 years ago

Owner: Christian Boos removed

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The ticket will remain with no owner.
The ticket will be disowned. Next status will be 'new'.
as The resolution will be set. Next status will be 'closed'.
The owner will be changed from (none) to anonymous. Next status will be 'assigned'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.