Add support for additional mime types using pygments get_all_lexers

[loop]

i contributed an actionscript lexer to the pygments project. so it would be nice to add it also to trac as default with a following line in api.py​ :

'text/x-actionscript':      ['as'],

[Christian Boos]

Well, it would be nicer if we could do such things from the pygments renderer automatically, using the information provided by pygments.lexers.get_all_lexers. Related to #3332.

[Remy Blank]

Related to #5828.

[Christian Boos]

Tickets #5613, #8570 were closed as duplicates.

We should try to fix this for 0.12.

[Christian Boos]

Maybe for 0.12.1.

[Thijs Triemstra <lists@…>]

Renaming the ticket because it's not actionscript-specific anymore.

[Thijs Triemstra <lists@…>]

#7847 was closed as a duplicate.

[Christian Boos]

#10725 was closed as duplicate.

Time to do something about this bug ;-)

[Peter Suter]

Adds pygments.lexers.get_all_lexers info to MIME_MAP

[Peter Suter]

Patch attachment:T5533_pygments_lexer_mimetypes.diff​​ seems simple enough, am I missing something here?

[Christian Boos]

Replying to psuter:

Patch attachment:T5533_pygments_lexer_mimetypes.diff​​ seems simple enough, am I missing something here?

There's a possibility to miss an initial lookup with this approach…

A Mimeview.get_mimetype() may happen first, for a repository file or an attachment, and the Pygments.get_quality_ratio(mimetype) call happens then, after the mimetype has been (incorrectly) detected.

A refresh (forced refresh for an attachment) will do… but still, maybe we can think about an improved solution.

[Christian Boos]

For example:

• [72aed033/cboos.git] 1.0dev: add IHTMLPreviewRenderer.get_extra_mimetypes method for augmenting the known mimetypes.

[Christian Boos]

Ok with my proposed fix?

[Remy Blank]

Looks good, yes.

[Peter Suter]

Sorry, I meant to check this. Yes, looks good.

The comment describing the new optional get_extra_mimetypes() method on the IHTMLPreviewRenderer interface could maybe be slightly improved. Currently it says Returning nothing is fine, but return None would not work.

More generally, how would a plugin know what the already known mime types are? Should it care? What about conflicts, priorities etc.? (Maybe out of scope for now.)

Also it doesn't really describe what the effect will be of adding additional mimetypes, and why this is related to HTML preview rendering.

Or maybe these details fit better in TracDev instead.

Targeting milestone 1.0 seems a bit risky for a plugin interface addition.

[Christian Boos]

Replying to psuter:

Sorry, I meant to check this. Yes, looks good.

The comment describing the new optional get_extra_mimetypes() method on the IHTMLPreviewRenderer interface could maybe be slightly improved. Currently it says Returning nothing is fine, but return None would not work.

Ah yes, forgot the usual "... or []" when iterating on providers.

More generally, how would a plugin know what the already known mime types are? Should it care? What about conflicts, priorities etc.? (Maybe out of scope for now.)

Well, if that's really a concern (i.e. a plugin wants only to add some types and not override them), then it can simply test via Mimeview(self.env).get_mimetype('file.keyword'). If that's not enough and access to that component's _mime_map is needed, we should indeed fix that with a better API (#3332 or something like that…).

Also it doesn't really describe what the effect will be of adding additional mimetypes, and why this is related to HTML preview rendering.

I've edited the docstring in this direction. It's true it has little to do with "HTML preview", except for the fact that IHTMLPreviewRenderer is what PygmentsRenderer implements and that IHTMLPreviewRenderer is already flagged as "to be merged in IContentConverter with an improved API", so this seems the best interim solution.

Or maybe these details fit better in TracDev instead.

Targeting milestone 1.0 seems a bit risky for a plugin interface addition.

Well, compared to the benefits this feature provides, I think this implementation is "good enough" and worth including in 1.0. I wanted to implement it since a very long time, and always failed because I didn't find a "clean" enough way to do it (short of bringing the whole #3332 topic on the table again). Your simple approach with add_mimetype motivated me to find a middle ground.

[Christian Boos]

Committed in r11228.

See log:cboos.git:ticket5533-get-extra-mimetypes-r11227 for the steps.

[Ryan J Ollos <ryan.j.ollos@…>]

I was just looking through the changeset [11228#file1] and noticed that the name variable is not used in get_extra_mimetypes (pygments.py), so wondering if it was intended to return name as the second entry in the tuple rather than aliases.

[Christian Boos]

Note that after this change, the PatchRenderer wouldn't handle the #!diff WikiProcessor anymore (ticket:10840#comment:12). Fixed in r11319 for 1.0.1.

[Ryan J Ollos]

Replying to Ryan J Ollos <ryan.j.ollos@…>:

I was just looking through the changeset [11228#file1] and noticed that the name variable is not used in get_extra_mimetypes (pygments.py), so wondering if it was intended to return name as the second entry in the tuple rather than aliases.

Removed unused expression in [13185:13186].