#11219 closed defect (fixed)
SubversionMergePropertyDiffRenderer is slow — at Version 9
Reported by: | Owned by: | Jun Omae | |
---|---|---|---|
Priority: | normal | Milestone: | 0.12.6 |
Component: | version control/changeset view | Version: | |
Severity: | major | Keywords: | performance svn mergeinfo |
Cc: | Jun Omae, mpotter@… | Branch: | |
Release Notes: |
Reduce execution time of rendering |
||
API Changes: | |||
Internal Changes: |
Description
The repository has around 40000 revisions, and a lot of mergeinfo on the trunk.
When viewing a changeset that affects mergeinfo on the trunk, it takes 20 seconds to render the changes. render_property_diff()
loops on every revision contained in the new mergeinfo property and makes one or more database requests for each (_get_node_revs()
), which makes thousands in this case. The displayed information as a result is something like (with links):
trunk Property svn:mergeinfo changed /sandboxes/t6823 (added) merged: 38954-38956
Note that svn returns the same information in 0.1 second
$ svn diff -c 38957 svn://svn.xxx.yyy/sdk/ (...) Property changes on: trunk ___________________________________________________________________ Modified: svn:mergeinfo Merged /sandboxes/t6823:r38954-38956
Sadly, it seams that the versioncontrol API does not allow to fetch property diffs, only file diffs.
This is with Trac 1.1.2dev and Postgres 9.1
Change History (9)
comment:1 by , 11 years ago
Keywords: | performance svn mergeinfo added |
---|---|
Milestone: | → next-stable-1.0.x |
Priority: | low → normal |
Severity: | normal → major |
comment:2 by , 11 years ago
I though that maybe we could get the information from svn through
repos.get_changes(old_context.resource.realm, old_context.resource.version, new_context.resource.realm, new_context.resource.version)
but for some reason, has_node()
in svn_fs.py
returns False. I tried to look into the python-subversion module to find why, but I don't even find the actual code for fs.check_path()
, it all wrapped etc, so I'm stuck.
comment:3 by , 11 years ago
I've tried to reduce time for _get_node_revs()
in log:jomae.git:ticket11219/1.0-stable.
After the changes, the method search only the revisions in differences between svn:mergeinfo
properties for each path. Also, the method divides single revision range into small multiple ranges and search the ranges in node_change
records.
Thoughts?
comment:4 by , 11 years ago
jomae.git:ticket11219/1.0-stable is very slow on MySQL…. Reworked at jomae.git:ticket11219.2/1.0-stable.
Timing of render_property_diff
for svn:mergeinfo
with [11654] in a clone of http://svn.edgewall.com/repos/trac.
Repository type | Database | before | after |
---|---|---|---|
direct-svnfs | 1.203752s | 0.106252s | |
svn | sqlite | 0.368621s | 0.031234s |
svn | postgres | 0.182075s | 0.023396s |
svn | mysql | 4.226078s | 0.043819s |
comment:5 by , 11 years ago
Cc: | added |
---|
comment:6 by , 10 years ago
Cc: | added |
---|
follow-up: 8 comment:7 by , 10 years ago
Milestone: | next-stable-1.0.x → 0.12.6 |
---|---|
Owner: | set to |
Status: | new → assigned |
Reworked in jomae.git@t11219.2_0.12 ([4938066f/jomae.git]), again.
In our case, svn:mergeinfo
has 25 lines and node_change
table has about 150,000 records. Viewing a changeset is very slow. I want to fix it on 0.12-stable.
comment:8 by , 10 years ago
Reworked in jomae.git@t11219.2_0.12 ([4938066f/jomae.git]), again.
Committed in [12959] and merged in [12960-12961].
I have a idea to reduce calls and time of _get_node_revs()
. I try it for both this ticket and #8459.
comment:9 by , 10 years ago
Release Notes: | modified (diff) |
---|---|
Resolution: | → fixed |
Status: | assigned → closed |
Comitted in [13015,13018] and merged in [13016-13017,13019-13020].
Yeah, sorry for that, the code in that area leaves much to be desired w.r.t. performance. We already have #8459, but here you narrowed the problem somehow.
If you would follow-up on your analysis and produce a patch fixing this, you'd become an instant Trac-hero ;-)