Berkeley DB error while opening 'nodes' table for filesystem

[anonymous]

Oups…

Trac detected an internal error:

("Berkeley DB error while opening 'nodes' table for filesystem /var/svn/edgewall.com/trac/db:\nCannot allocate memory", 160029)

If you think this really should work and you can reproduce it. Then you should consider to report this problem to the Trac team.

Go to ​http://trac.edgewall.com/ and there you create a new ticket where you describe the problem, how to reproduce it and don't forget to include the python traceback found below.
Python traceback

Traceback (most recent call last):

File "/usr/lib/python2.1/site-packages/trac/trac.py", line 203, in main

real_main()

File "/usr/lib/python2.1/site-packages/trac/trac.py", line 143, in real_main

rep = repos.svn_repos_open(repos_dir, pool)

SubversionException?: ("Berkeley DB error while opening 'nodes' table for filesystem /var/svn/edgewall.com/trac/db:\nCannot allocate memory", 160029)

[jonas]

Something messed up (probably bdb or svn). Somehow a lock file was left behind in the repository. I had to run svnadmin recover on the repository to get it up and running again.

Judging by the web-server log, no one was trying to modify the repository
at the time. This was strange….

I'm closing this bug, but will do some more investigation…

[vittorio]

i got this after indexing my 3 trac-projects with htdig. the other 2 remained intact.
svnadmin recover got it up running again too.
i didnt access svn while indexing.

Traceback (most recent call last):
  File "/usr/lib/python2.1/site-packages/trac/core.py", line 419, in cgi_start
    real_cgi_start()
  File "/usr/lib/python2.1/site-packages/trac/core.py", line 407, in real_cgi_start
    module = module_factory(args, env, database, req)
  File "/usr/lib/python2.1/site-packages/trac/core.py", line 152, in module_factory
    pool, rep, fs_ptr = open_svn_repos(repos_dir)
  File "/usr/lib/python2.1/site-packages/trac/core.py", line 317, in open_svn_repos
    rep = repos.svn_repos_open(repos_dir, pool)
SubversionException: ("Berkeley DB error while opening 'nodes' table for filesystem /var/lib/svn/repos/test/db:\nCannot allocate memory", 160029)

trac 0.7-pre trunk, 23.april
debian woody on vmware 4.5
subversion from www.backports.org subversion-1.0.1
python 2.1
sqlit-2.8.13 pysqlit-0.5 clearsilver-0.9.7 Silvercity-0.9.5

i will try to reproduce, when its reproducible will try if subversion-1.0.2 does fix this.

[vittorio]

i did a 2nd indexing with htdig resulting in the same error on another repository. but this time i wasnt able to recover the repository!

after installing subversion 1.0.2 from www.backports.org the system seems to be stable!
there was no error after 5 times indexing (big) repositories ( also while checkin to stress more)

closeing the bug therefore. either its a problem with trac+subversion 1.0.1 (or python bindings) or the 1.0.1 backport.

i will index and stress more and reopen the bug if theres a problem again.

[vittorio]

on next indexing, same problem again :/ (same traceback)

and cant recover:

vmdebian:/var/lib/svn/repos$ svnadmin recover /var/lib/svn/repos/test
Please wait; recovering the repository may take some time...
svn: DB_RUNRECOVERY: Fatal error, run database recovery

[vittorio]

although i get DB_RUNRECOVERY fatal error on every svn recover, the repository just works without a problem. all files have www-data:www-data ownership. if i do a svnadmin dump, svnadmin load, the DB_RUNRECOVERY error disappears on svnadmin recover. this is probably a subversion issue.
to make sure the repo locking problem isnt related to permission problems, i did a chown -R www-data:www-data on all repos, but repos still get locked when indexing with htdig.
as jonas pointed out on #trac that:

1. the server could be under heavy load so apache times out and kills trac.cgi.
2. Python segfaults for some reason.

i started alot computational hungry tasks so the average load was > 20, but this didnt triggered the problem earlier. maybe thats not the right test because htdig will slow down too, so stressing apache would be better probably. investigating more …

[remi2402@…]

In my case, I was able to reproduce the error twice in the same conditions :

• set up svn using "svn create"
• start svnserve
• installing/configuring trac-admin for the repository (at this point, still no error)
• one checkout and one commit
• go to the trac-browser to see the error

In my case, svnadmin recover did the trick, but I only have 2 revisions on my svn tree.

Hope this helps

Rémi

[jonas]

remi2402@…, this sounds like svnserve and trac.cgi are run by different unix users.
This will create file permission problems that will result in the same error message.

​http://svnbook.red-bean.com/svnbook/book.html#svn-ch-6-sect-5

The above chapter describes how to configure permissions for multiple repository access methods (both svnserve and httpd/trac.cgi)