Edgewall Software

Opened 9 years ago

Closed 8 years ago

Last modified 8 years ago

#10437 closed enhancement (fixed)

Adding Pygments syntax highlighting for new file type does not work as documented

Reported by: kriegaex@… Owned by: kriegaex@…
Priority: normal Milestone: 1.0
Component: rendering Version: 0.12-stable
Severity: normal Keywords: pygments, mimetype, MIME type, syntax highlighting, syntax coloring
Cc: Emmanuel Blot Branch:
Release Notes:

Mimetype associations can now be extended or overriden by regexps.

API Changes:
Internal Changes:


I failed for hours in adding syntax highlighting to Trac for a new file type I had added to my local Pygments installation before and which worked well with pygmentize on the console. The file type does not have shebang, but must be detected by a file name pattern (I need globbing, i.e. pattern matching for it because there is no fixed file extension).

The documentation talks about setting mime_map and pygments_modes in the mimeviewer section of trac.ini, but this either did nothing or led to errors in Trac, stating that Pygmentize did not know the specified MIME type (I forgot the exact error message).

What I ended up doing were two things not mentioned in the documentation at all:

  • I added a new (self-invented) MIME type text/x-something to /etc/mime.types.
  • I explicitly set the MIME type via svn propset svn:mime-type text/x-something my_file for each relevant file in my SVN repository.

Maybe there is another solution, but if so, the documentation does not mention it. What it does describe is incomplete at best and outright wrong in my opinion. So I request the following:

  • Please correct and extend the documentation, adding a step-by-step tutorial for adding a new file type for syntax highlighting to Trac via Pygments.
  • Please enable Trac to let Pygments determine the file type directly by its own file name patterns, because this works nicely in pygmentize. This would let most users get rid of having to modify mime.types and enable to correctly highlight files without a MIME type explicitly set in SVN. The advantages are clear: no MIME type fiddling, no trac.ini editing.

Attachments (1)

mime_map_patterns.trac.patch (4.5 KB ) - added by Alexander Kriegisch <kriegaex@…> 9 years ago.
Add file name regex matching to Trac for MIME type identification

Download all attachments as: .zip

Change History (13)

in reply to:  description comment:1 by Alexander Kriegisch <kriegaex@…>, 9 years ago

I found the error message I had before: "Exception: No Pygments lexer found for mime-type 'text/x-something'."

comment:3 by Remy Blank, 9 years ago

Milestone: 0.12.3next-major-0.1X

That's to invasive for 0.12.3, and probably even for 0.12-stable.

Note that setting svn:mime-type is only half a solution, as it won't work with Mercurial or Git repositories.

comment:4 by Alexander Kriegisch <kriegaex@…>, 9 years ago

I know that setting svn:mime-type is a workaround rather than a solution. This is why I wanted it fixed in Trac in the first place. ;-)

And why is it too invasive? You just need to

  • call an external porgam (a.k.a. black box) in a slightly different way and
  • fix the (semi-)documented behaviour in the product for users who prefer to solve this in Trac rathen than Pygments plus document it better,

don't you?

For my personal needs, though, I would be glad to receive a quick fix as a patch and test it for you, then wait for the next minor or major update, whichever seems more appropriate to you. All I want is get rid of those SVN MIME types and let Pygments do its own work.

by Alexander Kriegisch <kriegaex@…>, 9 years ago

Add file name regex matching to Trac for MIME type identification

comment:5 by Alexander Kriegisch <kriegaex@…>, 9 years ago

As you can see, with my (unfortunately very limited) Python skills I extended mimeview/api.py in a fairly non-invasive way by adding a new config option to trac.ini, section [mimeviewer]. The option is named mime_map_patterns and contains a list of key-value mappings from MIME types to file name regex patterns. This is much more powerful than just literal file extension matches. The patch contains two examples, see there for more info.

BTW, the original location of the patch is http://freetz.org/browser/trunk/tools/developer/mime_map_patterns.trac.patch.

As for my claim about the patch being non-invasive: Look where it is applied. It only kicks into action *after* the other MIME type matching mechanisms have failed. Thus, the patch is backward-compatible with what Trac does today and should be safe to be applied to the 0.12.x branch - no need to wait for the next major version IMHO.

comment:6 by Christian Boos, 8 years ago

See also #5533.

in reply to:  6 comment:7 by Alexander@…, 8 years ago

The patch in this ticket has been rotting since October 2011, so I forked the Trac Git repository, committed the patch both in trunk and in branch 0.12-stable and created two corresponding requests:

I hope this makes it easier for Trac's maintainers to integrate the new setting upstream.

comment:8 by Christian Boos, 8 years ago

Milestone: next-major-releases1.0
Type: defectenhancement

Thanks for the reminder! Yes, providing a pull request (or just a link to the proper changeset or branch in a git repository) indeed helps, though we don't use github directly for merging the requests.

As for the proposed change itself, I made a few modifications:

  • TracDev/CodingStyle cleanups (mostly line length)
  • improve docstring of mime_map_patterns, make clear that ':' is simply used to separate the MIME type from the regexp
  • ignore the mapping and issue a warning when there's an error in the regexp
  • as specifying associations with regexps can be very powerful and much more precise than the suffix matching done by mime_map, it feels a waste to do it as a last resort; doing it first allows one to override a mapping (e.g. override '.txt' → text/plain with text/x-cmake:CMakeLists\.txt)

Finally, 0.12.x is really in feature freeze now, so the feature can only make it for 1.0 (or later).

Integrating one extra docstring for 1.0 is a minor breach to the string freeze, but as most config options are still untranslated, adding one more won't hurt…

See log:cboos.git:ticket10437-mime_map_patterns-override.

Last edited 8 years ago by Christian Boos (previous) (diff)

comment:9 by Christian Boos, 8 years ago

Cc: Emmanuel Blot added

comment:10 by Christian Boos, 8 years ago

Resolution: fixed
Status: newclosed

Applied in r11229. Thanks Alexander!

comment:11 by Christian Boos, 8 years ago

Owner: set to kriegaex@…

comment:12 by Christian Boos, 8 years ago

Release Notes: modified (diff)

Modify Ticket

Change Properties
Set your email in Preferences
as closed The owner will remain kriegaex@….
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from kriegaex@… to the specified user.

Add Comment

E-mail address and name can be saved in the Preferences .
Note: See TracTickets for help on using tickets.