Version controls diffs of large text files kill the system
|Reported by:||anonymous||Owned by:|
|Component:||version control/changeset view||Version:||0.12.2|
|Severity:||major||Keywords:||diff performance genshi|
Description (last modified by )
We have some large text files (they are some data files we deal with) and when you try to view "last revision" or a revision with them it bombs (i.e. the service's cpu usage goes to 100%, and things come to a screeching halt).
Looks like the issue is in: versioncontrol/web_ui/changeset.py
_content_changes method tries to do a diff of anything that isn't a binary. This is fine for most text files, unless they are big (like even 5 to 10 megs seems a bit much for the diffing method used). Anyway, I simply added the following to
if (len(old_content) > 100000 or len(new_content) > 100000): return None
(Sorry if it isn't proper pep8)
It looks like you have a max_diff_bytes but that only effects the diff results. Ideally you would want a threshold before you have the diff results.
Change History (11)
comment:1 Changed 5 years ago by
|Component:||general → version control/changeset view|
|Priority:||normal → high|
|Severity:||normal → major|