[patch] Resync handles changeset intervals

[cboos@…]

This was suggested by Juanma Barranquero on the Trac mailing list Message 2162

This feature is useful for fixing formatting errors in one changeset, without having to resync the whole repository.

This is useful while waiting for #781.

Some explanations:

• resync 1000
  will resync the repository starting at revision 1000
• resync 1000:1010
  will resync the repository starting at revision 1000 ending at revision 1010
• resync :1010
  will resync the repository starting at revision 1 ending at revision 1010
• resync
  will resync the whole repository, as usual

[cboos@…]

patch that implements the proposed feature

[Juanma Barranquero]

Patch adapted to changes in [1379].

[Juanma Barranquero]

I've adapted this very useful patch to change [1379] on trunk.

BTW, it'd be really nice if this was accepted. Even if someday there is an interface to modify svn:log messages from Trac, resync'ing can be necessary, and this patch makes it much more bearable for large repositories, with just a little bookkeeping on the user's part.

[cboos@…]

If Christopher is ok, I'll apply that patch.

I'll also have a try to adapt it to the new versioncontrol API.

[Christopher Lenz <cmlenz@…>]

Sure, go ahead.

Note that the version control stuff is on my branch, so you'd have to provide a patch at this point.

[anonymous]

The patch is non-valid again, after [1413].

[mgood <trac matt-good net>]

Oops, cboos, log in before you accept a ticket.

[Manuzhai]

I would like to get this feature in soon so I can get on with #2382.

I've worked on a new patch which adds to parameters to the cached repository's .sync() method. It then becomes .sync(self, first = None, last = None).

Now, I've tried to run the existing tests on this patch, but that's not working so well. Three of the tests throw an Exception because the new code uses self.repos.rev_older_than(), which doesn't seem to exist on the mock repository. Not sure why this is, though.

Also, I'm afraid this patch in its current state is incompatible with PostgreSQL, because it seems to lack the INSERT OR REPLACE syntax. This could be implemented in rules for the PostgreSQL database, but I'm not sure that would work. Alternatively, the Python code could be expanded to switch on existence of the changeset metadata, but I think that would require an extra rev_older_than() for every changeset, making it slower still.

Any ideas on how we could clean this up?

(BTW: I was hoping this would go into trunk instead of vc-refactoring, since it seems fairly independent of what's going on in there.)

[Manuzhai]

New patch to do the main work for this in a different place.

[Manuzhai]

Sorry, wrong file. Here's the diff.

[vyt@…]

I try limited-sync.diff with trac 0.9, but trac-admin still resync whole repository. Please, tell me, where is complete patch for trac 0.9 ?

[cboos]

No one updated this patch for 0.9, AFAIK.

As part of the vc-refactoring, I'll try to adapt the resync support, so that it will be possible to implement #2482, #2382 and this one.

[cmlenz]

cboos, what does this enhancement request have to do with the vc-refactoring branch?

I'd really prefer that we keep the branch focussed on enabling pluggable VC system support.

[cboos]

The pending changes I have on the vc-refactoring branch are about a refactoring of the repository cache.

Hm, maybe that could be done as a refactoring step 2, as it's unfinished and will take some time to finalize.

Let's first merge the step 1, pluggable VC system support.

I'll sync the branch and temporarily remove a few things (tag support and such), in order to reach quickly a merge candidate.

[vyt@…]

Patch against 0.9 stable

[vyt@…]

Attached patch is against 0.9 stable, works for me. Note that syntax

trac-admin project resync 3:

doesn't work but

trac-admin project resync 3:-1

works fine.

[vyt@…]

About attached limited-sync.diff - it's cool but contain small error in line 84: condition should be "and not" instead of "or". Otherwise resync doesn't stop at specified last.

while current_rev is not None and not self.repos.rev_older_than(last, current_rev):

[cboos]

I'll try to integrate this once the vc-refactoring branch is merged ... which should be imminent, now.

Vitaly, would you mind testing the branch and give some feedback?

[vyt@…]

I do checkout as described in TracMercurial and everything seems to be ok. I test it on my test box, and old environment (created with 0.9 release) works fine. SVN renames, changes and deletion shown in Trac correctly. trac-admin project resync also works fine.

What I should test especially ?

[markus]

Ok, the vc-refactoring branch has been merged into trunk some time ago. What about integrating this patch into current (0.10dev) trunk? The ticket is scheduled for 1.0... :(

[cboos]

I have a few things to finish before restarting the vc-refactoring but that feature will then be part of it.

[cboos]

Post-poning.

[markus]

Just curious: If the VcRefactoring gets postponed to, say, 1.0, does this ticket have to wait then as well?

[cboos]

I did some small progresses this W.E. on the VcRefactoring front (the new cache). The goal is to have TracBrowser and TracChangeset operate mainly (if not exclusively) on the cache. So the resync will really become a critical and central operation. I'm not even sure at this point if partial resync would still work, as in some possible implementations, the cached node data will depend on what was there in the previous revision... so if you rebuild only [m:n], then n+1 might become non-sense...

If the partial resync wanted here is mainly for getting back the updated commit messages, I think a dedicated command will likely be more appropriate than the "complete" resync (something like changeset message [from:[to]]). If the resync is needed because the repository was rebuilt to omit a few last erroneous commits (i.e. the "obliterate" thing), then a truncate n operation could be done, and the automatic sync check will do the incremental resync from n...

OTOH, while waiting for the above vaporware to condensate, if someone wants to refresh the above patch for 0.10.1, he would be welcome ;)

[markus]

Thanks for the information. I finally read your VcRefactoring proposal as well and it seems like #781 does no longer depend on this ticket then.

I don't think we need an update of the patch above if we're going to split the resync command in two (or more) separate commands.

[cboos]

Replying to markus:

Thanks for the information. I finally read your VcRefactoring proposal ... I don't think we need an update of the patch above if we're going to split the resync command in two (or more) separate commands.

Well, most of this is at the brainstorming stage ;)

Concerning #781, it's more dependant on changes happening at how the resources in general (a chanhgeset in this case) are stored. I hope to find time to write down a TracDev/Proposals/DataModel one of these days, as I think I finally found a plausible way to implement the TracObjectModelProposal.

[markus]

Replying to cboos:

I hope to find time to write down a TracDev/Proposals/DataModel one of these days, as I think I finally found a plausible way to implement the TracObjectModelProposal.

Great, I'm looking forward to this one. As I have some spare time the next weeks I planned to create a plugin for #781, but wanted to have this ticket and #2348 "fixed" before I begin. Now that you'll come up with a new proposal (which sounds great) it seems like I have to wait a few more weeks/months before I can start with the plugin. :-(

Anyway, enough off-topic. We should switch to #781, I guess.

[cboos]

Will be part of the version control admin panel in 0.12.

[cboos]

Well, there's already #4797 and #2382 for this. Resyncing a range doesn't sound that useful anyway.