Historian leaves the git log instance open

[spam.martin.schulze@…]

How to reproduce

Setup a Trac instance with bitten on Apache. Add a git repository to Trac and a Build Configuration for it to Bitten.

After some succesful builds and enough commits to the repository each visit of the bitten-slave should leave open several git log processes in the system.

System information

• Arch Linux x64
• Python 2.7.9

I already posted this error on the Bitten bug tracker (​http://bitten.edgewall.org/ticket/836) but the problem is not in Bitten.

When the bitten-slave visits my Trac instance the function get_historian in PyGIT gets called.

​A commenter suggested to add --no-pager to the git command which solved the problem for him and for me as well.

I would suggest to add this change to the PyGIT module.

[Jun Omae]

Strange. Trac uses a pipe for stdout of git command. The git command doesn't spawn pager if the stdout is not a tty. Adding --no-pager option must have no effect.

    def log_pipe(self, *cmd_args):
        return self.__pipe('log', stdout=PIPE, *cmd_args)

[Martin Schulze <spam.martin.schulze@…>]

Ok, I take back that --no-pager fixes the problem. I added the option at the wrong place and git errored out which solved the problem of non-closing git processes but broke the functionality (without me noticing).

This means the problem still persists.

[Jun Omae]

Could you please try the following patch?

tracopt/versioncontrol/git/PyGIT.py

@@ -963 +963 @@
             except KeyError:
                 next_path[:] = [path]
                 return gen.next()
-        yield historian
 
-        if p:
-            p[0].stdout.close()
-            terminate(p[0])
-            p[0].wait()
+        try:
+            yield historian
+        finally:
+            if p:
+                p[0].stdout.close()
+                terminate(p[0])
+                p[0].wait()
 
     def last_change(self, sha, path, historian=None):
         if historian is not None:

[Jun Omae]

Reproduced with the following (from ​bitten:source:trunk/bitten/queue.py@:76-80#L74):

>>> from trac.env import open_environment
>>> env = open_environment(...)
>>> repos = env.get_repository('pygit2')
>>> node = repos.get_node('/')
>>> for entry in node.get_entries():
...   break
...
>>>

After the above, run ps xf:

$ ps xf
  PID TTY      STAT   TIME COMMAND
 2073 ?        S      1:08 sshd: jun66j5@pts/1
 2074 pts/1    Ss     0:05  \_ -bash
 1752 pts/1    Tl     0:23      \_ vim
26846 pts/1    S+     0:00      \_ make python=25-1.0 clean Trac.egg-info start-python
27083 pts/1    S+     0:00          \_ python
27090 pts/1    S+     0:00              \_ git --git-dir=/home/jun66j5/var/git/pygit2 log --pretty=format:%n%H --name-status 324a8d5....
 1772 ?        S      0:35 sshd: jun66j5@pts/0
 1773 pts/0    Ss     0:03  \_ -bash
19523 pts/0    Tl     1:01      \_ vim
27102 pts/0    R+     0:00      \_ ps xf

[Martin Schulze <spam.martin.schulze@…>]

The patch seems to solve the problem. I can find no accumulating git processes after applying it.

[Jun Omae]

Committed in [13844] and merged to trunk in [13845].

[anders@…]

I was the one who suggested —no-pager and I did exactly this mistake myself, —no-pager in the wrong place. Everything became 45 years old (1970-01-01) because the plugin was broken.

Back-porting the patch suggested above into my setup with Trac 0.12 and the Git plugin apparently fixes it for me too. Very good, thanks!

Replying to Martin Schulze <spam.martin.schulze@…>:

Ok, I take back that --no-pager fixes the problem. I added the option at the wrong place and git errored out which solved the problem of non-closing git processes but broke the functionality (without me noticing).

This means the problem still persists.