trac locks database and causes potential corruption when using apache on windows

[anonymous]

Here's the description from one of our developers…

Trac is causing Subversion database corruption and I had to disable it. Even though Trac only reads database, every read operation locks it, like it's going to do transaction. That doesn't jive well with normal svn client that uses svn_dav Apache module to access database. In the end, two different access methods cause DB corruption (we didn't lose anything with nightly backup and incremental dump on commit). I don't expect to re-activate Trac soon (soon=weeks or months).

Edit: This is not a Trac bug, but a Subversion one. Using svn 1.1, and particularly with FSFS, it is no longer an issue. Edit2: Also, to clarify, it was never an issue of corruption, but rather a mere inconvenience due to loss of state in berkeley db. No data has ever been reported lost.

[trac@…]

This sounds like a Subversion bug, not a Trac bug. Nothing that Trac should be able to do should cause DB corruption.

Although it may help with the changes to access the repository via the RA interface rather than the filesystem one.

[anonymous]

More comments… It's not that causes corruption, it's that locks the database and never exists. Apache is running on Windows 2000 server (svn repository is on same box). We noticed that when lock happens happens we have multiple python.exe processes running. Worst of all nothing can kill these processes and we have to reboot server.

[pythonista@…]

I've saw this twice with trac 0.6.x, also under Apache / Win2K Server - locked Subversion, multiple Python processes hung up on the server. Although it may purely be coincidence, I haven't had an issue since upgrading to 0.7.x.

[joel@…]

I ran into this two times the last week. Running svnadmin recover solved the problem the first time, but i disabled trac when it happened again.

Is this a trac or svn-problem?

[anonymous]

Since we disabled Trac (2+ weeks ago) 0 lockups.

[anonymous]

We have the same problem…..

[anonymous]

I've completely disabled trac here, too. It's not only related to windows as it seems to be implied. Indeed I run trac on a 'debian stable'

[joel@…]

After some testing and experimenting i've came to the following conclusion: This is probably NOT a trac/svn-problem! It is probably just a permission error on the svn db.

We had problem with several people writing to the repository using ​file:///, but when wo put them all in the svn grop and chmodded the db directory g+s that problem solved. I thought this might apply to trac too, so I tried it and guess what? Adding the apache-user to the svn group has resulted in no more lockups so far.

[anonymous]

And you think this will help us on Windows platform? Permission setting is well known issue/solution on Linux/Unix systems. We are all talking about Windows servers here.

[anonymous]

www-data is in the svn group, and the repository permission are set according to subversion recommendations (g+s)… and I had (and still have) locks from time to time.

[bje@…]

We are running trac and apache here. The primary problem we encountered was setting permissions up correctly, but since trac runs as the apache user we have no problem with concurrent access between trac and apache.

It is useful to set your directories as g+s, but you must also set a umask of 002 or similar for every process accessing those directories via group access, since DB4 will create new files periodically. If this happens with a umask of 022, the process which caused the creation will exit cleanly, but the next process to attempt to read the database via group permissions will cause DB4 to lock the database and complain loudly about a lack of permissions.

We here now have a umask of 002 set for the apache process, which neatly covers both davsvn and trac access.

— bje

[anonymous]

Thanks. Nice explanation. Too bad it doesn't cover Windows platform, since there are no umask and g+s there.

[rocky]

It appears this issue has been boiled down to a windows-only issue at this point. The reality here is that its most likely a fault in the svn/python support, but until this is confirmed this ticket can remain open.

[Christopher Lenz]

0.7.2 is cancelled

[per.norman@…]

We have the same problem on Trac 0.7 (reported as #519).

[anonymous]

I had this problem quite a bit with SVN & ViewCVS (which also uses the python bindings, but after upgrading the python bindings to 1.0.6 I have only had one lockup in 3 weeks where I used to them almost daily before.

Please check your python bindings and upgrade - it may solve your problems too!

[claude@…]

We have been running Trac 0.7.1 on a Windows XP machine for over a month here and got a few lock ups. It is believed that they occur after the server is shutdown and restarted.

I had to go through the SVN documented procedure to unlock it:

​http://svnbook.red-bean.com/svnbook-1.0/apb.html#svn-ap-b-sect-1.2.1

This operation always succeeded so far. Hopefully, it will continue to succeed.

Now, is this related to the issue described in this ticket? I do not know.

[daniel]

This subversion issue should be fixed in subversion 1.1, especially when using FSFS.

​http://web.mit.edu/ghudson/info/fsfs

[daniel]

Review this later.

[Christopher Lenz]

Don't close as anonymous or without a comment.

[jbolla]

I've also had this occur while running the latest version of trac (8.0) and Subversion under apache on windows xp pro. trac/subversion/apache are all the latest builds as of writing this.

[jbolla]

since switching from using svnserve to the apache protocol for accessing our repository, i have not had the lockup issue.

[anonymous]

Using trac 8.0 and svn 1.1.1 (bdb) on w2k3/IIS6.0. No problem in using WebSVN and trac (timeline) one after another, but svn+ssh/svnserve locks up the repo immediately. Luckily, recovery is always possible (I do have backups ;) ).

Just putting this here for reference purposes. If migration to fsfs saves the day, I'll bring an update here.

[igor]

Update regarding "Using trac 8.0 and svn 1.1.1 (bdb) on w2k3/IIS6.0"

Moving from bdb to fsfs solved my locking problems too. So, the solution seems to either get rid of svnserve (at least, when using 1.1.1 or earlier), or get rid of bdb.

[anonymous]

I saw it today. I am running SVN 1.1.3 and trac 0.8.1 on XP.

[anonymous]

I posted a possible explanation for this on the Trac mailing list. You may view it on ​Gmane .

— Roland

[Christopher Lenz]

Just found a piece of code in the ​ViewCVS vclib that might help here:

    # Register a handler for SIGTERM so we can have a chance to
    # cleanup.  If ViewCVS takes too long to start generating CGI
    # output, Apache will grow impatient and SIGTERM it.  While we
    # don't mind getting told to bail, we want to gracefully close the
    # repository before we bail.
    def _sigterm_handler(signum, frame, self=self):
      self._close()
      sys.exit(-1)
    signal.signal(signal.SIGTERM, _sigterm_handler)

[anonymous]

Although this code may useful on Unix systems, it won't help on Windows installations.

[Arthur]

I've just had this occur on a Linux server (RedHat 7.3).

Clicked on 'browse source' and Trac reported the DB error.

Subversion is usually accessed by svnserve, so subversion Apache DAV is not running. Nobody was performing a commit at the time. Haven't had a single SVN database corruption before (repository ~ 6 months old).

Trac 0.8.1 Subversion 1.1.3 sqlite 2.8.16

[anonymous]

There is a thread on comp.apache.devel indicating some problematic behavior in Apache. See it on ​GMane

[Christopher Lenz]

As noted on the ​mailing list thread mentioned above:

Christopher Lenz <cmlenz <at> ...> writes:

> [...]
> So, if I understand you correctly, there's not much Trac can do to fix the 
> problem, right? Maybe if you would use mod_python, the problem would 
> either "go away", or we might at least have an option to fix it by 
> registering some kind of cleanup handler or something.

Yes, I think Trac can't do much here. Maybe this will force us to switch
to use mod_python.

Regards, Roland

As we're not aware of how or even whether this issue can be fixed in Trac, there's no foreseeable way we can resolve it for 0.9.

[matthew.donald@…]

Sould this ticket be closed as "wontfix"?

If its a Subversion problem and it can't be fixed in Trac, maybe it should just be closed.

I should note in passing that this is locking bug #10000 for DB2/¾. Sleepycat seems to have a whole series of locking and concurrent update problems which have existed for years. I gave up and now avoid using any version of their database these days

[Armin Ronacher <armin.ronacher@…>]

I would say close as "invalid" since it's not Trac related. Normally it helps to create a fsfs repository to solve that problem.

[Christian Boos]

My take on this would be to close this issue as worksforme. It was a real issue with TracSubversion, mostly related to the use of the bdb backend for svn and the CGI frontend for Trac, so I feel a bit uneasy with closing this as invalid.

Now, bdb doesn't seem to be used so much anymore and the use of Trac with CGI has diminished as well because the alternative frontends are so much faster…