Edgewall Software
Modify

Opened 2 years ago

Last modified 4 months ago

#13129 new defect

trac-admin resync leaks memory

Reported by: turnopil@… Owned by:
Priority: high Milestone: next-stable-1.4.x
Component: version control Version: 1.2.3
Severity: normal Keywords: svn memory
Cc: Branch:
Release Notes:
API Changes:
Internal Changes:

Description

I am using a trac installation 1.2.3 version. Was installed via easy_install

subversion-python27 Version: 1.9.7
python27-psycopg2   Version: 2.5.1
subversion          Version: 1.9.7

I'm trying to run a resync in our environment. Unfortunately this fails apparently due to running out of memory. The repository our project uses has over 100k revisions, and my estimation is that trac-admin resync leaks about 40 MB of memory per 1000 revisions. But after 90K revisions he used all 8 gigabytes of RAM.

-bash: fork: Cannot allocate memory
PID   USER     PR NI VIRT  RES  SHR  S %CPU %MEM TIME+    COMMAND
401   root     20 0  0     0      0  D 8.3  0.0  0:02.91  kswapd0
10228 root     30 10 7804m 7.0g 1168 D 3.6  93.7 11:59.68 trac-admin
9736  postgres 20 0  328m  133m 132m S 0.7  1.7  0:00.56  postmaster

We are using a postgres 9.6 backend.

Attachments (0)

Change History (6)

comment:1 by Jun Omae, 2 years ago

Keywords: resync removed

I think the issue is the same of #13112. Please try to use 1.2-stable or apply [16835] to Trac 1.2.3.

in reply to:  1 comment:2 by turnopil@…, 2 years ago

Replying to Jun Omae:

I think the issue is the same of #13112. Please try to use 1.2-stable or apply [16835] to Trac 1.2.3.

Than you! I will try

comment:3 by turnopil@…, 2 years ago

Confirmed. Memory leaks still exists but 50% less.

Is it planned to solve this problem completely? Or it's technically impossible?

comment:4 by Jun Omae, 2 years ago

Thanks for the feedback.

I spent more time to investigate other memory leaks on #13112, however I couldn't find. I guess that Subversion Python bindings has the causes of the memory leaks, but no evidence.

comment:5 by Ryan J Ollos, 15 months ago

Milestone: next-stable-1.2.xnext-stable-1.4.x

comment:6 by Jun Omae, 4 months ago

The root cause is that svn.delta.make_editor() leads ref-count leak, see comment:6:ticket:13356 for the details.

I'm trying to create a fix for the leak in Subversion Python bindings, but it is not finished.

Workaround:

  • tracopt/versioncontrol/svn/svn_fs.py

    diff --git a/tracopt/versioncontrol/svn/svn_fs.py b/tracopt/versioncontrol/svn/svn_fs.py
    index 3285f63d2..1129442b1 100644
    a b class SubversionRepository(Repository):  
    736736            deltas = sorted(((_from_svn(path), kind, change)
    737737                             for path, kind, change in editor.deltas),
    738738                            key=lambda entry: entry[0])
     739
     740            # Workaround for ref-count leak due to delta.make_editor()
     741            editor.__dict__.clear()
     742
    739743            for path, kind, change in deltas:
    740744                old_node = new_node = None
    741745                if change != Changeset.ADD:
    class SubversionChangeset(Changeset):  
    10861090            changes.append([path, kind, action, base_path, base_rev])
    10871091            idx += 1
    10881092
     1093        # Workaround for ref-count leak due to delta.make_editor()
     1094        editor.__dict__.clear()
     1095
    10891096        moves = []
    10901097        # a MOVE is a COPY whose `base_path` corresponds to a `new_path`
    10911098        # which has been deleted

Memory profile:

  • Python 2.7.12
  • Trac 1.0-stable
  • Subversion 1.9.7
  • Mirror of Trac Subversion repository (latest rev r17519)
Increment on repos.sync()
Before the patch 194.1 MiB
After the patch 24.1 MiB

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The ticket will remain with no owner.
The ticket will be disowned.
as The resolution will be set. Next status will be 'closed'.
The owner will be changed from (none) to anonymous. Next status will be 'assigned'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.