MemoryError in sqlite_backend

[robin@…]

Sorry if this is a duplicate. I looked at the other MemoryError? tickets and none of them seemed to match very closely. We've periodically had MemoryError? issues with our Trac over the past week since we went live with it. When it happens the following traceback is returned to the browser, and the traceback below that was found in the apache error log.

As you can see from the tracebacks we are using Python 2.4, the FastCGI interface and Trac 0.11rc1. Other relevant versions are, SQLite: 3.2.8, pysqlite: 2.4.1, Genshi: 0.5dev-r698. (I've just updated Genshi to r852 to see if it helps any.)

Also, we are getting slammed by the googlebot at the moment. I've updated our robots.txt to try and limit that a bit but they won't pick up the new file for a few more hours.

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/api.py",
line 339, in send_error
    'text/html')
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/chrome.py",
line 684, in render_template
    data = self.populate_data(req, data)
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/chrome.py",
line 592, in populate_data
    d['chrome'].update(req.chrome)
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/api.py",
line 168, in __getattr__
    value = self.callbacks[name](self)
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/chrome.py",
line 460, in prepare_request
    for category, name, text in contributor.get_navigation_items(req):
  File "build/bdist.linux-i686/egg/tractags/web_ui.py", line 60, in
get_navigation_items
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/api.py",
line 168, in __getattr__
    value = self.callbacks[name](self)
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/main.py",
line 254, in _get_perm
    return PermissionCache(self.env, self.authenticate(req))
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/main.py",
line 131, in authenticate
    authname = authenticator.authenticate(req)
  File "build/bdist.macosx-10.3-fat/egg/acct_mgr/web_ui.py", line 354, in wrap
  File "build/bdist.macosx-10.3-fat/egg/acct_mgr/web_ui.py", line 365,
in authenticate
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/auth.py",
line 71, in authenticate
    authname = self._get_name_for_cookie(req, req.incookie['trac_auth'])
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/auth.py",
line 181, in _get_name_for_cookie
    db = self.env.get_db_cnx()
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/env.py",
line 251, in get_db_cnx
    return DatabaseManager(self).get_connection()
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/db/api.py",
line 76, in get_connection
    return self._cnx_pool.get_cnx(self.timeout or None)
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/db/pool.py",
line 101, in get_cnx
    cnx = self._connector.get_connection(**self._kwargs)
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/db/sqlite_backend.py",
line 125, in get_connection
    return SQLiteConnection(path, params)
  File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/db/sqlite_backend.py",
line 176, in __init__
    timeout=timeout)
MemoryError

[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr: Traceback (most recent call last):
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/_fcgi.py", line 570, in run
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     protocolStatus, appStatus = self.server.handler(self)
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/_fcgi.py", line 1256, in handler
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     result = self.application(environ, start_response)
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/main.py", line 388, in dispatch_request
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     return _dispatch_request(req, env, env_error)
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/web/main.py", line 445, in _dispatch_reques
t
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     traceback = get_last_traceback()
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/site-packages/Trac-0.11rc1-py2.4.egg/trac/util/__init__.py", line 139, in get_last_tr
aceback
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     traceback.print_exc(file=tb)
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/traceback.py", line 212, in print_exc
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     print_exception(etype, value, tb, limit, file)
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/traceback.py", line 125, in print_exception
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     print_tb(tb, limit, file)
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/traceback.py", line 69, in print_tb
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     line = linecache.getline(filename, lineno)
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/linecache.py", line 14, in getline
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     lines = getlines(filename)
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/linecache.py", line 40, in getlines
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     return updatecache(filename)
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:   File "/usr/lib/python2.4/linecache.py", line 101, in updatecache
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr:     lines = fp.readlines()
[Fri May 23 09:42:08 2008] [warn] mod_fcgid: stderr: MemoryError

[robin@…]

FYI, swtiching to the newer Genshi does seem to have helped. For the most part the process in-memory size stays fairly steady, although there are a few pages that can still result in the process growing very large. And for the record, another thing that helped was switching away from mod_fcgid and using mod_wsgi instead.

[cboos]

Thanks for the feedback, I think we can safely close this one as duplicate of #6614.