Enhancing TracMercurial plugin to use new server-command protocol

[Joongi Kim <me@…>]

I suggest to use Mercurial's new command server protocol for the TracMercurial plugin.

After recent releases of Mercurial such as 1.8 and 1.9, TracMercurial plugin did not work properly until several patches were delivered, mainly due to Mercurial's internal API changes.

Now Mercurial 1.9 provides ​a stable command-server protocol. It is a fairly simple protocol using pipes to send length + data streams, and features the same command set to that of the console command line.

TracMercurial is for showing the timeline (prev/next and file list) and browsing the repository. It could be implemented using a few commands such as log, tip, heads, tags, diff, status --change, etc.

If we combine this enhancement with a CachedRepository implementation (ref #8417), the performance problem will be minized.

I think this approach will significantly reduce the efforts for adapting to a new version of Mercurial, and make Trac users to escape from hassle of upgrading.

[Remy Blank]

I don't know. I rather like the current implementation of the plugin. Adapting to API changes is often relatively straightforward, and performance is good. I don't see the "stable" command protocol as a compelling enough reason to accept a performance reduction.

Then again, I'm not the author of the plugin, so I'll leave the decision to Christian.

[Christian Boos]

The big downside I see is that it will probably not be possible to use the same tricks to speed up the display of the last revision for directory entries, which is where most of the complexity of the plugin reside (r7744, [10512-10514]). We could have an option to not display the last revision for a folder that could be set for big repositories. That would probably be acceptable, as it's the interface you also get with hg serve or BitBucket, but it's a step backward. Perhaps by implementing #8417 (with VcRefactoring#NewRepositoryCache or similar) we could also avoid this limitation.

Using the command server could also be not completely trivial from Trac (potentially short-lived multi-processes, each potentially multi-threaded). We don't want to create a command server process for every call, and we probably want to have one per repository and access it in a thread-safe way. Yet we shouldn't keep those command servers running forever…

OTOH, on the plus side:

• better isolation from problems related to memory usage
• stable API
• possibility to change the license to BSD (as the GitPlugin recently did)

So the plan could be:

1. simplify the plugin to the point where only a few simple high-level operations from Mercurial could be used
   1. make it possible to not show version information for folders
   2. work on #8417 and see if we can find a way to retrieve the version info for folders from the cached data
2. see if we can find an effective and robust way to work with command servers
3. replace the direct API calls with indirect ones delegating to the hg command server … or to the direct API again (as a GPLv3+ plugin for the new BSD plugin ;-) )

[Peter Suter]

Replying to Christian Boos:

If someone manages to create a Mercurial plugin working with ​​Mercurial:PythonHglib (#10411), that could fit besides the svn and git support

This reminded me, a while ago I experimented with this minimal, very naive Mercurial connector​ using ​Mercurial:PythonHglib. It's likely not very good / fast and incomplete / incorrect in some places (for sure at least similarly incomplete as TracGit), but the basics seem to be working, so I'm attaching it just in case someone is interested in experimenting further or in even fully implementing the plan from comment:2.

[Peter Suter]

I updated the attached patch​ adding some tests adapted from Git, although one of them still fails.

[Peter Suter]

I updated the attached patch​ again, all tests should pass now. (Although I really still don't understand the conventions in this system.)

11 years ago:

As I don't think that part of the API will stay as is for long, I only documented it minimally

Aha… :)

replace the direct API calls with indirect ones delegating to the hg command server … or to the direct API again

Are there already any specific ideas how this would work? Would it be similar to the ​th:TracPygit2Plugin approach? It seems to somehow ​wrap tracopt.versioncontrol.git.git_fs.GitConnector. comment:4:ticket:10606 hints at another similar long term plan?

[Christian Boos]

I'll test and integrate.

If performance is an issue, we can always say "install TracMercurial", but it will be nice to have out-of-the-box support for small repositories.

[Ryan J Ollos]

OT: I'm reminded by #12270, would it make sense to publish TracMercurial to PyPI?

[Ryan J Ollos]

Replying to Ryan J Ollos:

OT: I'm reminded by #12270, would it make sense to publish TracMercurial to PyPI?

⇒ #12710.

[Jun Omae]

I noticed mercurial/util.py invokes os.stat_float_times(False) since Mercurial 3.7. As the result, os.stat() and os.path.getmtime() return int values as timestamp. File modification check in Trac core reduces from microseconds-resolution to seconds-resolution if TracMercurial is enabled.

# disable Python's problematic floating point timestamps (issue4836)
# (Python hypocritically says you shouldn't change this behavior in
# libraries, and sure enough Mercurial is not a library.)
os.stat_float_times(False)

at ​https://www.mercurial-scm.org/repo/hg/annotate/3.7/mercurial/util.py#l116

[Peter Suter]

os.stat_float_times(False) does ​not get called anymore since Mercurial 4.6. Also discussed in comment:11:ticket:13049