pygments_renderer should support passing options to lexers

[Tim Hatch]

I decided to split this out of #5613 because that was originally about allowing short names as wiki processors. Someone brought this up on IRC as well today, so I decided to give it a shot. I'm thinking the main place to set options should be in trac.ini, but I'd like more input (do we need a WebAdmin page for them? a way to set them as part of a wiki processor would also be great, I think, especially in the case of 'startinline' for php. I'm going to attach a first implementation for 0.11-dev, followed by an adaptation for 0.10's tracpygments.

I would like feedback, particularly on whether this is something other people are interested in, and whether it should be specified somehow for wiki processors too.

[Tim Hatch]

First version, against r5537 in 0.11-dev

[Tim Hatch]

First version, against r2367 for tracpygments

[bpat1434]

I'm all for this; however, trac.ini might get bloated with this. Perhaps the ability to create a new lexer.ini file in the conf directory (or the trac root for global configurations) which holds the lexer options. Something along the lines of:

lexer.csharp.unicodelevel = basic
lexer.delphi.turbopascal = disabled
lexer.php.startinline = enabled
lexer.php.disabledmodules = stripos,get_func_args,preg_match
lexer.rst.handlecodeblocks = disabled

[Christian Boos]

Tim, how is this now handled at the Pygments level, is there a way to setup an .ini file with the options there?

Also, I think it would be nice to be able to pass the options in code blocks (e.g. #!cs lexer.csharp.unicodelevel=basic).

[Tim Hatch]

There's no current method for setting up Pygments global settings, they need to be passed to the instantiation as these patches show. I'm against having a new lexer ini, but recognize that having a [mimeviewer] pygments_options can get unwieldy for lots of options.

How about something like this (basically removing the lexer. prefix and placing it in the main trac.ini):

[pygments_options]
*.stripnl = false
csharp.unicodelevel = basic
delphi.turbopascal = disabled
php.startinline = enabled
php.disabledmodules = stripos,get_func_args,preg_match
rst.handlecodeblocks = disabled

The only thing this doesn't satisfy is that some case normalization will need to be done on the lexer aliases used as ini keys. Putting each option in a dedicated line allows commas in them (my listoption solution above didn't). I'm also +1 on having a way to set them for code blocks (the real case where this shines is whether php starts inline or not).

[Remy Blank]

The idea in comment:3 looks nice. Any chance to get this into 0.12?

[Christian Boos]

A [pygments] section would indeed be better than a [mimeviewer] pygments_options option.

Feel free to move back when there's a patch.

[jappie@…]

I'd also like the ability to pass options in code-blocks. I'd prefer the following syntax:

#!php startinline=true

Because we define a code-block for the processor php, I see no need to explicitly state it again (like #!php lexer.php.startinline=true or #!php php.startinline=true).

[Ryan J Ollos]

#1932 closed as a duplicate. The request in #1932 will be satisfied by setting the startsinline option of ​PhpLexer.

[Ryan J Ollos]

Replying to jappie@…:

I'd also like the ability to pass options in code-blocks. I'd prefer the following syntax:

#!php startinline=true

Because we define a code-block for the processor php, I see no need to explicitly state it again (like #!php lexer.php.startinline=true or #!php php.startinline=true).

In Trac, MIME Type WikiProcessors support some keyword arguments that are known by formatter.py: lineno, marks, id​. However, there doesn't seem to be a way for Trac to know the options supported by the Lexers. The Lexer classes don't declare the valid options. Instances of the Lexer classes will consume valid options as keyword arguments and ignore invalid options.

The only way I can see to allow Pygments Lexer options to be used as keyword arguments for MIME WikiProcessor is to consider all unknown keyword arguments to be lexer options. I think that is probably okay though since the Lexers will just ignore any unknown options. We could also see if Pygments is willing to consider an enhancement in which they specify the valid option names as a class attribute.

[Ryan J Ollos]

Proposed changes in log:rjollos.git:t5654_pygments_options. I'll add tests before committing.

DONE Add documentation for pygments-lexer options to 1.1/TracSyntaxColoring.

[Ryan J Ollos]

Committed to trunk in [14207:14208]. Edited 1.1/TracSyntaxColoring@5.