Syntax highlighting with Colorer

[daniel]

A mimeviewer plugin for Colorer.

See: ​http://colorer.sourceforge.net/

[cmlenz@…]

Depends on Colorer implementing reading its input from stdin:

​http://sourceforge.net/forum/forum.php?thread_id=1073629&forum_id=110043

[Christopher Lenz]

Can't target this until the colorer package has incorporated the required features.

[anonymous]

why do not use temp file ?
write to temp file then run colorer then delete temp file

[anonymous]

Colorer-take5.beta3 supports reading from stdin
command line is

  colorer.exe -h -dc -tphp

where php is sort of file

​http://sourceforge.net/project/showfiles.php?group_id=34855&package_id=27292

[Jonas Borgström]

This can wait until 0.9.

[Nathah Miller <nxmiller@…>]

windows version:

# -*- coding: iso8859-1 -*-
#
# Copyright (C) 2004 Edgewall Software
# Copyright (C) 2004 Nathan Miller <nxmiller@yahoo.com>
#
# Trac is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# Trac is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
#
# Author: Nathan Miller <nxmiller@yahoo.com>
#
# Syntax highlighting module, using colorer
#

import re
import sys
import os
import random

from trac.util import NaivePopen

supported_types = [
    (1, 'text/plain',         'txt'),
    (1, 'text/html',          'html'),
    (1, 'text/css',           'css'),
    (1, 'text/xml',           'xml'),
    (1, 'text/xsl',           'xls'),
    (1, 'text/x-perl',        'perl'),
    (1, 'text/x-php',         'php'),
    (1, 'text/x-python',      'py'),
    (1, 'text/x-diff',        'diff'),
    (1, 'text/x-javascript',  'js'),
    (1, 'text/x-csrc',        'c'),
    (1, 'text/x-chdr',        'h'),
    (1, 'text/x-c++src',      'cpp'),
    (1, 'text/x-c++hdr',      'hpp'),
    (1, 'text/x-java',        'java'),
    (1, 'text/x-idl',         'idl'),
    (1, 'text/x-asm',         'asm'),
    (1, 'text/x-sql',         'sql'),
    (1, 'application/x-sh',   'sh'),
    # much more rare languages....
    ]

types = {}
for p,t,s in supported_types:
    types[t] = s


def display(data, mimetype, filename, env):

    try:
        ext = types[mimetype]
    except KeyError:
        raise Exception, "Colorer doesn't support %s" % mimetype

    # if filename is defined, highlight according to filename, not to mimetype 
    if filename is None:
        filename = 'tmp.' + ext

    tmpfname = 'c:/temp/tmp%08X.%s' % (random.randint(0,sys.maxint), filename)

    f = open(tmpfname, 'wb')
    f.write(data)
    f.close()

    cmdline = env.get_config('mimeviewer', 'colorer_path', 'C:\\bin\\Colorer\\bin\\colorer.exe')
    cmdline += ' -h -dc -eiUTF-8 -eoUTF-8 ' + tmpfname
    np = NaivePopen(cmdline)

    os.unlink(tmpfname)

    if np.errorlevel:
        err = 'Running (%s) failed: %s, %s.' % (cmdline, np.errorlevel, np.err)
        raise Exception, err
    odata = re.sub("^\xEF\xBB\xBF","",np.out)

    return '<div class="code-block">' + odata + '</div>'

[Matthew Good]

#3691 has been marked as a duplicate.

[Christian Boos]

I think we should have Colorer support out of the box, as we have for enscript and SilverCity. It's not as if we currently had a bright solution for TracSyntaxColoring… for example we advise to downgrade the SilverCity packages to 0.9.5, and enscript is aging and not practical on Windows.

Ideally, there should be Python bindings for Colorer, as invoking an .exe for each code snippet to be rendered can become costly (as that's currently the case for enscript).

There's also the possibility to provide a cache for code block rendering, that will ideally benefit any renderer, like a md5 hash of (code block content + rendering options) → cached html fragment.

[Christian Boos]

Replying to mgood:

#3691 has been marked as a duplicate.

And there's an updated patch there that people can use with 0.10, in the interim.

[Matthew Good]

Replying to cboos:

Replying to mgood:

#3691 has been marked as a duplicate.

And there's an updated patch there that people can use with 0.10, in the interim.

Well, the "patch" requires some work for people to use since it hardcodes paths to "colorer.exe" and a temp directory, and the creation of temp files is not particularly safe.

[Christopher Lenz]

I see no reason why this shouldn't be implemented using a separate plugin. Ideally, at some point we could even move out the other highlighters into plugins (IMHO).

[Christian Boos]

Well, I don't see why the buggy and unsupported enscript should be prefered to the more capable Colorer, so yes, we should remove support for all highlighters altogether.

[Christopher Lenz]

Replying to cboos:

Well, I don't see why the buggy and unsupported enscript should be prefered to the more capable Colorer, so yes, we should remove support for all highlighters altogether.

I'm not sure whether that's supposed to be sarcastic or something. “Removing support for highlighters" is not what I said, nor what I meant. Note that colorer is a somewhat “exotic” tool that (very?) few people have installed. There are no Python bindings for it, so you need to call it over the command-line (or you could use ctypes I guess). So colorer really isn't the perfect solution for all things syntax highlighting in Trac.

Also note that there's a good reason why we have the enscript, silvercity, and php mimeviewers in the core: history. Those modules were added before we had a plugin system. There was no other way to add highlighters without adding those modules to Trac directly.

[Christian Boos]

My suggestion was to move the enscript, silvercity and php mimeviewers out of the core trac.mimeview package. We could possibly still bundle them in a set of "default" plugins (e.g. in this case /tracext/mimeview)