Edgewall Software
Modify

Opened 10 years ago

Closed 10 years ago

Last modified 6 years ago

#9288 closed defect (fixed)

syntax highlighting for javascript code broken in 0.12b1

Reported by: anonymous Owned by: Remy Blank
Priority: normal Milestone: 0.12
Component: rendering Version: 0.12b1
Severity: normal Keywords:
Cc: hvr@… Branch:
Release Notes:
API Changes:
Internal Changes:

Description

TracSyntaxColoring for #!js broken with Trac 0.12b1, see

function buildFixesList(wenjians) {
  var rootUL = document.createElement("ul");
  var topLI = document.createElement("li");
  return 0;
}

Attachments (0)

Change History (13)

comment:1 by Remy Blank, 10 years ago

Milestone: 0.12
Owner: set to Remy Blank

This doesn't seem to be a regression in Trac, as the same happens on the Bitten Trac. Maybe due to a Pygments upgrade? Strangely, the JavaScript snippet is rendered with the PlainTextRenderer.

I'll look into it.

comment:2 by Carsten Klein <carsten.klein@…>, 10 years ago

When getting a lexer manually from pygments, e.g. get_lexer_by_name('js') it will return the correct lexer:

Output by Javascript Lexer is

>>> for token in lexer.get_tokens(content):
...     print token
...
(Token.Keyword.Declaration, u'function')
(Token.Text, u' ')
(Token.Name.Other, u'test')
(Token.Punctuation, u'()')
[...]

for an input of

function test()
{
}

What I also notices is that you no longer can use the long name for that language, e.g. 'javascript', it will then fail will an unknown processor error:

{{{#!javascript
function test()
{}
}}}

will yield

function test()
{}

So maybe it actually does not try to query pygments available lexers in the first place, so this should actually be a regression in trac, don't you think?

comment:3 by Carsten Klein <carsten.klein@…>, 10 years ago

Just noticed that, also PHP and other processors are broken as well.

Perhaps the class names defined in STANDARD_TYPES have changed, or the trac CSS no longer includes these style classes?

comment:4 by Carsten Klein <carsten.klein@…>, 10 years ago

In mimeview.api there was a slight change to MODE_RE:

From

# Simple builtin autodetection from the content using a regexp
MODE_RE = re.compile(
    r"#!.+?env (\w+)|"                       # look for shebang with env
    r"#!(?:[/\w.-_]+/)?(\w+)|"               # look for regular shebang
    r"-\*-\s*(?:mode:\s*)?([\w+-]+)\s*-\*-|" # look for Emacs' -*- mode -*-
    r"vim:.*?(?:syntax|filetype|ft)=(\w+)"   # look for VIM's syntax=<n>
    )

to

MODE_RE = re.compile(r"""
      \#!.+?env\s+(\w+)                     # 1. look for shebang with env
    | \#!(?:[/\w.-_]+/)?(\w+)               # 2. look for regular shebang
    | -\*-\s*(?:mode:\s*)?([\w+-]+)\s*-\*-  # 3. look for Emacs' -*- mode -*-
    | vim:.*?(?:syntax|filetype|ft)=(\w+)   # 4. look for VIM's syntax=<n>
    """, re.VERBOSE)

Perhaps this is the problem w/ different versions of python?

in reply to:  3 comment:5 by Carsten Klein <carsten.klein@…>, 10 years ago

Replying to Carsten Klein <carsten.klein@…>:

Just noticed that, also PHP and other processors are broken as well.

Perhaps the class names defined in STANDARD_TYPES have changed, or the trac CSS no longer includes these style classes?

However, PHP will be lexed by the pygments lexer alright, except that it will not be displayed correctly:

function test()
{
  var $a = 2;
}

comment:6 by hvr@…, 10 years ago

Cc: hvr@… added

comment:7 by Remy Blank, 10 years ago

It seems that [8617], while using the more standard application/javascript MIME type instead of text/x-javascript, also broke JavaScript rendering with Pygments, as it doesn't recognize the former.

I have opened [pygments:ticket:504 #504] with Pygments to request adding the application/javascript MIME type.

In the meantime, we could work around that with the following patch:

  • trac/mimeview/pygments.py

    diff --git a/trac/mimeview/pygments.py b/trac/mimeview/pygments.py
    a b  
    186186            name = aliases and aliases[0] or lexname
    187187            for mimetype in mimetypes:
    188188                self._types[mimetype] = (name, self.QUALITY_RATIO)
     189
     190        # Pygments currently doesn't know application/javascript
     191        if 'application/javascript' not in self._types:
     192            js_entry = self._types.get('text/javascript')
     193            if js_entry:
     194                self._types['application/javascript'] = js_entry
     195
    189196        self._types.update(
    190197            Mimeview(self.env).configured_modes_mapping('pygments')
    191198        )

Thoughts?

comment:8 by Christian Boos, 10 years ago

'text/javascript' or 'text/x-javascript'?

in reply to:  8 comment:9 by Remy Blank, 10 years ago

Replying to cboos:

'text/javascript' or 'text/x-javascript'?

It doesn't matter, both are recognized by Pygments. See the lexer documentation for Pygments, more specifically JavascriptLexer.

comment:10 by Carsten Klein <carsten.klein@…>, 10 years ago

Actually it is always text/javascript. At least that is what modern browsers expect it to be when declaring script sections in the html.

in reply to:  10 comment:11 by anonymous, 10 years ago

Replying to Carsten Klein <carsten.klein@…>:

Actually it is always text/javascript. At least that is what modern browsers expect it to be when declaring script sections in the html.

from what I've read, the only "modern" webbrowser which requires type="text/javascript" is IE… ;)

and it doesn't help that even the HTML5 spec mentions text/javascript :(

comment:12 by Remy Blank, 10 years ago

Resolution: fixed
Status: newclosed

We do use text/javascript as a MIME type for <script> tags. This ticket is about syntax highlighting of .js files, which was actually broken.

Anyway, patch applied in [9639].

in reply to:  7 comment:13 by Ryan J Ollos, 6 years ago

Replying to rblank:

It seems that [8617], while using the more standard application/javascript MIME type instead of text/x-javascript, also broke JavaScript rendering with Pygments, as it doesn't recognize the former.

I have opened [pygments:ticket:504 #504] with Pygments to request adding the application/javascript MIME type.

The issue was fixed in Pygments 1.4. I modified the source code comment in [13183:13184] so that it will be clear that the code from [9639] can be removed if we ever drop support for Pygments < 1.4.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Remy Blank.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Remy Blank to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.