Edgewall Software

Opened 13 years ago

Last modified 4 years ago

#10411 new enhancement

Enhancing TracMercurial plugin to use new server-command protocol

Reported by: Joongi Kim <me@…> Owned by:
Priority: normal Milestone: plugin - mercurial
Component: plugin/mercurial Version: 0.12dev
Severity: normal Keywords: mercurial command server
Cc: Branch:
Release Notes:
API Changes:
Internal Changes:

Description (last modified by Peter Suter)

I suggest to use Mercurial's new command server protocol for the TracMercurial plugin.

After recent releases of Mercurial such as 1.8 and 1.9, TracMercurial plugin did not work properly until several patches were delivered, mainly due to Mercurial's internal API changes.

Now Mercurial 1.9 provides a stable command-server protocol. It is a fairly simple protocol using pipes to send length + data streams, and features the same command set to that of the console command line.

TracMercurial is for showing the timeline (prev/next and file list) and browsing the repository. It could be implemented using a few commands such as log, tip, heads, tags, diff, status --change, etc.

If we combine this enhancement with a CachedRepository implementation (ref #8417), the performance problem will be minized.

I think this approach will significantly reduce the efforts for adapting to a new version of Mercurial, and make Trac users to escape from hassle of upgrading.

Attachments (1)

T10411-hglib.diff (35.8 KB ) - added by Peter Suter 7 years ago.

Download all attachments as: .zip

Change History (13)

comment:1 by Remy Blank, 13 years ago

I don't know. I rather like the current implementation of the plugin. Adapting to API changes is often relatively straightforward, and performance is good. I don't see the "stable" command protocol as a compelling enough reason to accept a performance reduction.

Then again, I'm not the author of the plugin, so I'll leave the decision to Christian.

comment:2 by Christian Boos, 13 years ago

The big downside I see is that it will probably not be possible to use the same tricks to speed up the display of the last revision for directory entries, which is where most of the complexity of the plugin reside (r7744, [10512-10514]). We could have an option to not display the last revision for a folder that could be set for big repositories. That would probably be acceptable, as it's the interface you also get with hg serve or BitBucket, but it's a step backward. Perhaps by implementing #8417 (with VcRefactoring#NewRepositoryCache or similar) we could also avoid this limitation.

Using the command server could also be not completely trivial from Trac (potentially short-lived multi-processes, each potentially multi-threaded). We don't want to create a command server process for every call, and we probably want to have one per repository and access it in a thread-safe way. Yet we shouldn't keep those command servers running forever…

OTOH, on the plus side:

  • better isolation from problems related to memory usage
  • stable API
  • possibility to change the license to BSD (as the GitPlugin recently did)

So the plan could be:

  1. simplify the plugin to the point where only a few simple high-level operations from Mercurial could be used
    1. make it possible to not show version information for folders
    2. work on #8417 and see if we can find a way to retrieve the version info for folders from the cached data
  2. see if we can find an effective and robust way to work with command servers
  3. replace the direct API calls with indirect ones delegating to the hg command server … or to the direct API again (as a GPLv3+ plugin for the new BSD plugin ;-) )

comment:3 by Peter Suter, 7 years ago

Replying to Christian Boos:

If someone manages to create a Mercurial plugin working with ​Mercurial:PythonHglib (#10411), that could fit besides the svn and git support

This reminded me, a while ago I experimented with this minimal, very naive Mercurial connector using Mercurial:PythonHglib. It's likely not very good / fast and incomplete / incorrect in some places (for sure at least similarly incomplete as TracGit), but the basics seem to be working, so I'm attaching it just in case someone is interested in experimenting further or in even fully implementing the plan from comment:2.

comment:4 by Peter Suter, 7 years ago

I updated the attached patch adding some tests adapted from Git, although one of them still fails.

by Peter Suter, 7 years ago

Attachment: T10411-hglib.diff added

in reply to:  4 comment:5 by Peter Suter, 7 years ago

I updated the attached patch again, all tests should pass now. (Although I really still don't understand the conventions in this system.)

11 years ago:

As I don't think that part of the API will stay as is for long, I only documented it minimally

Aha… :)

replace the direct API calls with indirect ones delegating to the hg command server … or to the direct API again

Are there already any specific ideas how this would work? Would it be similar to the th:TracPygit2Plugin approach? It seems to somehow wrap tracopt.versioncontrol.git.git_fs.GitConnector. comment:4:ticket:10606 hints at another similar long term plan?

Last edited 7 years ago by Peter Suter (previous) (diff)

comment:6 by Peter Suter, 7 years ago

Description: modified (diff)

comment:7 by Christian Boos, 7 years ago

Owner: set to Christian Boos
Status: newassigned

I'll test and integrate.

If performance is an issue, we can always say "install TracMercurial", but it will be nice to have out-of-the-box support for small repositories.

comment:8 by Ryan J Ollos, 7 years ago

OT: I'm reminded by #12270, would it make sense to publish TracMercurial to PyPI?

in reply to:  8 comment:9 by Ryan J Ollos, 7 years ago

Replying to Ryan J Ollos:

OT: I'm reminded by #12270, would it make sense to publish TracMercurial to PyPI?


comment:10 by Jun Omae, 7 years ago

I noticed mercurial/util.py invokes os.stat_float_times(False) since Mercurial 3.7. As the result, os.stat() and os.path.getmtime() return int values as timestamp. File modification check in Trac core reduces from microseconds-resolution to seconds-resolution if TracMercurial is enabled.

# disable Python's problematic floating point timestamps (issue4836)
# (Python hypocritically says you shouldn't change this behavior in
# libraries, and sure enough Mercurial is not a library.)

at https://www.mercurial-scm.org/repo/hg/annotate/3.7/mercurial/util.py#l116

comment:11 by Peter Suter, 6 years ago

os.stat_float_times(False) does not get called anymore since Mercurial 4.6. Also discussed in comment:11:ticket:13049

comment:12 by Ryan J Ollos, 4 years ago

Owner: Christian Boos removed
Status: assignednew

Modify Ticket

Change Properties
Set your email in Preferences
as new The ticket will remain with no owner.
The ticket will be disowned.
as The resolution will be set. Next status will be 'closed'.
The owner will be changed from (none) to anonymous. Next status will be 'assigned'.

Add Comment

E-mail address and name can be saved in the Preferences .
Note: See TracTickets for help on using tickets.