Edgewall Software
Modify

Opened 6 years ago

Last modified 6 months ago

#10411 assigned enhancement

Enhancing TracMercurial plugin to use new server-command protocol

Reported by: Joongi Kim <me@…> Owned by: Christian Boos
Priority: normal Milestone: plugin - mercurial
Component: plugin/mercurial Version: 0.12dev
Severity: normal Keywords: mercurial command server
Cc:
Release Notes:
API Changes:

Description (last modified by Peter Suter)

I suggest to use Mercurial's new command server protocol for the TracMercurial plugin.

After recent releases of Mercurial such as 1.8 and 1.9, TracMercurial plugin did not work properly until several patches were delivered, mainly due to Mercurial's internal API changes.

Now Mercurial 1.9 provides a stable command-server protocol. It is a fairly simple protocol using pipes to send length + data streams, and features the same command set to that of the console command line.

TracMercurial is for showing the timeline (prev/next and file list) and browsing the repository. It could be implemented using a few commands such as log, tip, heads, tags, diff, status --change, etc.

If we combine this enhancement with a CachedRepository implementation (ref #8417), the performance problem will be minized.

I think this approach will significantly reduce the efforts for adapting to a new version of Mercurial, and make Trac users to escape from hassle of upgrading.

Attachments (1)

T10411-hglib.diff (35.8 KB ) - added by Peter Suter 10 months ago.

Download all attachments as: .zip

Change History (11)

comment:1 Changed 6 years ago by Remy Blank

I don't know. I rather like the current implementation of the plugin. Adapting to API changes is often relatively straightforward, and performance is good. I don't see the "stable" command protocol as a compelling enough reason to accept a performance reduction.

Then again, I'm not the author of the plugin, so I'll leave the decision to Christian.

comment:2 Changed 6 years ago by Christian Boos

The big downside I see is that it will probably not be possible to use the same tricks to speed up the display of the last revision for directory entries, which is where most of the complexity of the plugin reside (r7744, [10512-10514]). We could have an option to not display the last revision for a folder that could be set for big repositories. That would probably be acceptable, as it's the interface you also get with hg serve or BitBucket, but it's a step backward. Perhaps by implementing #8417 (with VcRefactoring#NewRepositoryCache or similar) we could also avoid this limitation.

Using the command server could also be not completely trivial from Trac (potentially short-lived multi-processes, each potentially multi-threaded). We don't want to create a command server process for every call, and we probably want to have one per repository and access it in a thread-safe way. Yet we shouldn't keep those command servers running forever…

OTOH, on the plus side:

  • better isolation from problems related to memory usage
  • stable API
  • possibility to change the license to BSD (as the GitPlugin recently did)

So the plan could be:

  1. simplify the plugin to the point where only a few simple high-level operations from Mercurial could be used
    1. make it possible to not show version information for folders
    2. work on #8417 and see if we can find a way to retrieve the version info for folders from the cached data
  2. see if we can find an effective and robust way to work with command servers
  3. replace the direct API calls with indirect ones delegating to the hg command server … or to the direct API again (as a GPLv3+ plugin for the new BSD plugin ;-) )

comment:3 Changed 10 months ago by Peter Suter

Replying to Christian Boos:

If someone manages to create a Mercurial plugin working with ​Mercurial:PythonHglib (#10411), that could fit besides the svn and git support

This reminded me, a while ago I experimented with this minimal, very naive Mercurial connector using Mercurial:PythonHglib. It's likely not very good / fast and incomplete / incorrect in some places (for sure at least similarly incomplete as TracGit), but the basics seem to be working, so I'm attaching it just in case someone is interested in experimenting further or in even fully implementing the plan from comment:2.

comment:4 Changed 10 months ago by Peter Suter

I updated the attached patch adding some tests adapted from Git, although one of them still fails.

Changed 10 months ago by Peter Suter

Attachment: T10411-hglib.diff added

comment:5 in reply to:  4 Changed 10 months ago by Peter Suter

I updated the attached patch again, all tests should pass now. (Although I really still don't understand the conventions in this system.)

11 years ago:

As I don't think that part of the API will stay as is for long, I only documented it minimally

Aha… :)

replace the direct API calls with indirect ones delegating to the hg command server … or to the direct API again

Are there already any specific ideas how this would work? Would it be similar to the th:TracPygit2Plugin approach? It seems to somehow wrap tracopt.versioncontrol.git.git_fs.GitConnector. comment:4:ticket:10606 hints at another similar long term plan?

Last edited 10 months ago by Peter Suter (previous) (diff)

comment:6 Changed 10 months ago by Peter Suter

Description: modified (diff)

comment:7 Changed 10 months ago by Christian Boos

Owner: set to Christian Boos
Status: newassigned

I'll test and integrate.

If performance is an issue, we can always say "install TracMercurial", but it will be nice to have out-of-the-box support for small repositories.

comment:8 Changed 10 months ago by Ryan J Ollos

OT: I'm reminded by #12270, would it make sense to publish TracMercurial to PyPI?

comment:9 in reply to:  8 Changed 10 months ago by Ryan J Ollos

Replying to Ryan J Ollos:

OT: I'm reminded by #12270, would it make sense to publish TracMercurial to PyPI?

#12710.

comment:10 Changed 6 months ago by Jun Omae

I noticed mercurial/util.py invokes os.stat_float_times(False) since Mercurial 3.7. As the result, os.stat() and os.path.getmtime() return int values as timestamp. File modification check in Trac core reduces from microseconds-resolution to seconds-resolution if TracMercurial is enabled.

# disable Python's problematic floating point timestamps (issue4836)
# (Python hypocritically says you shouldn't change this behavior in
# libraries, and sure enough Mercurial is not a library.)
os.stat_float_times(False)

at https://www.mercurial-scm.org/repo/hg/annotate/3.7/mercurial/util.py#l116

Modify Ticket

Change Properties
Set your email in Preferences
Action
as assigned The owner will remain Christian Boos.
The ticket will be disowned.
as The resolution will be set.
to The owner will be changed from Christian Boos to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .

 
Note: See TracTickets for help on using tickets.