Really large commit breaks timeline and source browser

[trac@…]

I recieved the following error suddenly when looking at my timeline:

Repository checkins event provider (ChangesetModule) failed:

SubversionException: ("Can't set position pointer in file '/mnt/cifs/fp/svn/randd/db/revs/13332': Value too large for defined data type", 75)

which was then immeadiatly followed by this in the source browser:

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 356, in dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 224, in dispatch
    resp = chosen_handler.process_request(req)
  File "/usr/lib/python2.4/site-packages/trac/versioncontrol/web_ui/browser.py", line 131, in process_request
    self._render_directory(req, repos, node, rev)
  File "/usr/lib/python2.4/site-packages/trac/versioncontrol/web_ui/browser.py", line 156, in _render_directory
    changes = get_changes(self.env, repos, [i['rev'] for i in info])
  File "/usr/lib/python2.4/site-packages/trac/versioncontrol/web_ui/util.py", line 37, in get_changes
    changeset = repos.get_changeset(rev)
  File "/usr/lib/python2.4/site-packages/trac/versioncontrol/cache.py", line 41, in get_changeset
    self.sync()
  File "/usr/lib/python2.4/site-packages/trac/versioncontrol/cache.py", line 98, in sync
    for path,kind,action,base_path,base_rev in changeset.get_changes():
  File "/usr/lib/python2.4/site-packages/trac/versioncontrol/svn_fs.py", line 645, in get_changes
    repos.svn_repos_replay(root, e_ptr, e_baton, pool())
  File "/usr/local/lib/svn-python/libsvn/repos.py", line 239, in svn_repos_replay
    return apply(_repos.svn_repos_replay, args)
  File "/usr/local/lib/svn-python/svn/repos.py", line 113, in delete_entry
    if _fs.is_dir(self._get_root(parent_baton[2]), base_path):
  File "/usr/local/lib/svn-python/libsvn/fs.py", line 352, in svn_fs_is_dir
    return apply(_fs.svn_fs_is_dir, args)
SubversionException: ('Found malformed header in revision file', 160004)

Further investigation of the repository discovered that in Changeset 13332 referenced in the first error 15209 files (4GB of data) had been added.

Since this is a research institue this is not uncommon (yay for SVN handling this), but should Trac be able to handle this? Our other SVN viewer - ​Insurrection - and our ​desktop ​clients did not choke on it.

I have started a resync to see if that helps:

• Trac 0.10
• SVN 1.4.0
• Ubuntu Linux

[cboos]

Well, hm, with /mnt/cifs/fp/... it seems that you're really looking for trouble.

Are your other SVN readers also accessing the repository from this CIFS mounted path, or are there running on the same machine that is hosting the repository?

[Liam Clancy (metafeather) <trac@…>]

The Issurrection viewer is on the same machine, the others are clients, although both have repository browsers and can return information for before and after this changeset, ad browse the tree.

I havn't had any problems so far thorough the use of of the cifs mount despite largish commits. The server is well maintianed and on a Gigabit network to the main SVN repos - unfortunately that running windows and is out of my control.

I could setup a read-only svk or svn mirror but would like an idea about this error before shifting around so much data.

[anonymous]

Replying to Liam Clancy (metafeather) <trac@metafeather.com>:

The Insurrection viewer is on the same machine, the others are clients,

So, to be sure we understand each other: Insurrection is on the same machine as the repository, i.e. the Windows machine, right?

The desktop clients you're talking about are probably not looking directly in the repository, but probably talking to a server process through some protocol (http, svn, ...). Where's that server process running? Probably also on the windows machine?

I could setup a read-only svk or svn mirror but would like an idea about this error before shifting around so much data.

Yes, that's the solution we suggest for now, waiting for #493.

For the problem itself, I'm afraid there's not much we can do about that.

[Liam Clancy (metafeather) <trac@…>]

The trac-admin resync ook a few hours to run but everything is back to normal in both the timeline and source browser views.

No other changes where made, so I guess this can be regarded as a vindication of sorts.

(You don't quite realise how easy the UI is untill you have to use some of the alternatives.)

The 1st error was recieved during the resync but this did not stop the process, and all changesets are viewable both before and after, which may indicate a difference in behaviour/error handling.

[Liam Clancy (metafeather) <trac@…>]

Actually turns out that this was resolved by our SVN admin who has now split the repository, and so the size of the revision is now zero.

So somewhere there is still lurking a bug in handling a commit of 15000 odd files.

[cboos]

Well, yes, but as I explained above, I don't see what we can do to fix this, besides implementing #493, so I'm closing this as duplicate.

(I assume my hypothesis in the comment above were correct, as you didn't answer to my clarification questions)