Edgewall Software

Opened 11 years ago

Closed 10 years ago

Last modified 10 years ago

#11219 closed defect (fixed)

SubversionMergePropertyDiffRenderer is slow

Reported by: ebouaziz@… Owned by: Jun Omae
Priority: normal Milestone: 0.12.6
Component: version control/changeset view Version:
Severity: major Keywords: performance svn mergeinfo
Cc: Jun Omae, mpotter@… Branch:
Release Notes:

Reduce execution time of rendering svn:mergeinfo in changeset view.

API Changes:
Internal Changes:


The repository has around 40000 revisions, and a lot of mergeinfo on the trunk.

When viewing a changeset that affects mergeinfo on the trunk, it takes 20 seconds to render the changes. render_property_diff() loops on every revision contained in the new mergeinfo property and makes one or more database requests for each (_get_node_revs()), which makes thousands in this case. The displayed information as a result is something like (with links):

Property svn:mergeinfo changed
/sandboxes/t6823 (added)	merged: 38954-38956

Note that svn returns the same information in 0.1 second

$ svn diff -c 38957 svn://svn.xxx.yyy/sdk/
Property changes on: trunk
Modified: svn:mergeinfo
   Merged /sandboxes/t6823:r38954-38956

Sadly, it seams that the versioncontrol API does not allow to fetch property diffs, only file diffs.

This is with Trac 1.1.2dev and Postgres 9.1

Attachments (0)

Change History (10)

comment:1 by Christian Boos, 11 years ago

Keywords: performance svn mergeinfo added
Milestone: next-stable-1.0.x
Priority: lownormal
Severity: normalmajor

Yeah, sorry for that, the code in that area leaves much to be desired w.r.t. performance. We already have #8459, but here you narrowed the problem somehow.

If you would follow-up on your analysis and produce a patch fixing this, you'd become an instant Trac-hero ;-)

comment:2 by ebouaziz@…, 11 years ago

I though that maybe we could get the information from svn through


but for some reason, has_node() in svn_fs.py returns False. I tried to look into the python-subversion module to find why, but I don't even find the actual code for fs.check_path(), it all wrapped etc, so I'm stuck.

comment:3 by Jun Omae, 11 years ago

I've tried to reduce time for _get_node_revs() in log:jomae.git:ticket11219/1.0-stable.

After the changes, the method search only the revisions in differences between svn:mergeinfo properties for each path. Also, the method divides single revision range into small multiple ranges and search the ranges in node_change records.


comment:4 by Jun Omae, 11 years ago

jomae.git:ticket11219/1.0-stable is very slow on MySQL…. Reworked at jomae.git:ticket11219.2/1.0-stable.

Timing of render_property_diff for svn:mergeinfo with [11654] in a clone of http://svn.edgewall.com/repos/trac.

Repository type Database before after
direct-svnfs 1.203752s 0.106252s
svn sqlite 0.368621s 0.031234s
svn postgres 0.182075s 0.023396s
svn mysql 4.226078s 0.043819s
Last edited 11 years ago by Jun Omae (previous) (diff)

comment:5 by Jun Omae, 11 years ago

Cc: Jun Omae added

comment:6 by Mark Potter <mpotter@…>, 10 years ago

Cc: mpotter@… added

comment:7 by Jun Omae, 10 years ago

Milestone: next-stable-1.0.x0.12.6
Owner: set to Jun Omae
Status: newassigned

Reworked in jomae.git@t11219.2_0.12 ([4938066f/jomae.git]), again.

In our case, svn:mergeinfo has 25 lines and node_change table has about 150,000 records. Viewing a changeset is very slow. I want to fix it on 0.12-stable.

Last edited 10 years ago by Jun Omae (previous) (diff)

in reply to:  7 comment:8 by Jun Omae, 10 years ago

Reworked in jomae.git@t11219.2_0.12 ([4938066f/jomae.git]), again.

Committed in [12959] and merged in [12960-12961].

I have a idea to reduce calls and time of _get_node_revs(). I try it for both this ticket and #8459.

comment:9 by Jun Omae, 10 years ago

Release Notes: modified (diff)
Resolution: fixed
Status: assignedclosed

Comitted in [13015,13018] and merged in [13016-13017,13019-13020].

comment:10 by ebouaziz@…, 10 years ago

Thanks for this fix! For information, it now takes only 2 seconds to display the mergeinfo properties on a repository with 50000+ revisions (Postgres 9.1)

Modify Ticket

Change Properties
Set your email in Preferences
as closed The owner will remain Jun Omae.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Jun Omae to the specified user.

Add Comment

E-mail address and name can be saved in the Preferences .
Note: See TracTickets for help on using tickets.