Opened 13 years ago
Last modified 6 months ago
#10267 new defect
Version controls diffs of large text files kill the system — at Initial Version
| Reported by: | anonymous | Owned by: | |
|---|---|---|---|
| Priority: | high | Milestone: | next-dev-1.7.x |
| Component: | version control/changeset view | Version: | 0.12.2 |
| Severity: | major | Keywords: | diff performance genshi |
| Cc: | Branch: | ||
| Release Notes: | |||
| API Changes: | |||
| Internal Changes: | |||
Description
We have some large text files (they are some data files we deal with) and when you try to view "last revision" or a revision with them it bombs (i.e. the service's cpu usage goes to 100%, and things come to a screeching halt).
Looks like the issue is in: versioncontrol/web_ui/changeset.py
The _content_changes method tries to do a diff of anything that isn't a binary. This is fine for most text files, unless they are big (like even 5 to 10 megs seems a bit much for the diffing method used). Anyway, I simply added the following to _content_changes:
if (len(old_content) > 100000 or len(new_content) > 100000):
return None
(Sorry if it isn't proper pep8)
It looks like you have a max_diff_bytes but that only effects the diff results. Ideally you would want a threshold before you have the diff results.
