Memory Errors

[alexbcoles@…]

After a period of use, I am getting Memory Errors running TRAC as tracd. This is tough to reproduce, because it only seems to happen sporadically. It seems though (anecdotally) that extensive browsing through the SVN repository seems to cause this to happen more often.

Traceback (most recent call last):
  File "/users/home/alexbcoles/python/lib/python2.4/site-packages/Trac-0.11dev_r6265-py2.4.egg/trac/web/api.py", line 341, in send_error
    'text/html')
  File "/users/home/alexbcoles/python/lib/python2.4/site-packages/Trac-0.11dev_r6265-py2.4.egg/trac/web/chrome.py", line 590, in render_template
    data = self.populate_data(req, data)
  File "/users/home/alexbcoles/python/lib/python2.4/site-packages/Trac-0.11dev_r6265-py2.4.egg/trac/web/chrome.py", line 498, in populate_data
    d['chrome'].update(req.chrome)
  File "/users/home/alexbcoles/python/lib/python2.4/site-packages/Trac-0.11dev_r6265-py2.4.egg/trac/web/api.py", line 170, in __getattr__
    value = self.callbacks[name](self)
  File "/users/home/alexbcoles/python/lib/python2.4/site-packages/Trac-0.11dev_r6265-py2.4.egg/trac/util/compat.py", line 129, in newfunc
    return func_(*(args + fargs), **dict(kwargs, **fkwargs))
  File "/users/home/alexbcoles/python/lib/python2.4/site-packages/Trac-0.11dev_r6265-py2.4.egg/trac/web/chrome.py", line 377, in prepare_request
    for category, name, text in contributor.get_navigation_items(req):
  File "/users/home/alexbcoles/python/lib/python2.4/site-packages/Trac-0.11dev_r6265-py2.4.egg/trac/wiki/web_ui.py", line 78, in get_navigation_items
    if 'WIKI_VIEW' in req.perm('wiki'):
  File "/users/home/alexbcoles/python/lib/python2.4/site-packages/Trac-0.11dev_r6265-py2.4.egg/trac/perm.py", line 519, in has_permission
    return self._has_permission(action, resource)
  File "/users/home/alexbcoles/python/lib/python2.4/site-packages/Trac-0.11dev_r6265-py2.4.egg/trac/perm.py", line 532, in _has_permission
    self._cache[key] = decision
MemoryError

[alexbcoles@…]

I forgot to attribute this bug to myself.

[athomas]

This is strange, as the permission cache (that is triggering this) is a transient object with a lifetime of a single request. Also, the new permission system isn't currently being used for the repository subsystem. It's possible that there's a leak somewhere that's causing this.

Can you give us some more information? How much memory does your system have? What SVN bindings are you using?

[alexbcoles@…]

I am running this on a shared host, rather than my own box. As such, its difficult to give you a figure on the available memory - only the memory I am using for a user process.

The server is running Solaris 10. Version of Python is 2.4.4.

As I don't have root access, I am using my own Python and PYTHONPATH, created with the aid of virtual-python.py. Although I am not root user, I am using my own Python copied over with virtual-python.py.

Which version of the SVN bindings? I am using my host's libsvn and svn (symbollically linked from /usr/local to my home directory) in place, built for my host's version of Subversion - 1.4.4.

[cboos]

Are you using a scoped repository? If so, it may be related to #5213.

[alexbcoles@…]

No, I am using one repository - the whole repository (i.e. the scope is /, the root of the repository) - per Trac project.

[cboos]

See #G166 - for now try to remove the _speedups.so C extension and see if that improves your memory usage.

[cboos]

#G166 fix helped and after that, I've also tested intensive "log" and browsing queries... the memory usage, while big, stabilizes.

So I think it's safe to say that at least in the general case, there's no leak. The occasional memory error could be trapped more cleanly and presented as such instead of showing a random backtrace.

[cboos]

Follow-up to cboos:

The occasional memory error could be trapped more cleanly and presented as such instead of showing a random backtrace.

Fixed in r6327.

[ilias@…]

can you provide a piece of code which can be used to detect if there's a leak?

I am not sure if I can use the code mentioned in genshi:#166

see #6549