Error page fails with non-ASCII chars in code

[trac-ja@…]

In 0.11, Trac can show error details in Error page if user granted TRAC_ADMIN permission.

In this error page, we can show stacktraces with their source. However, if the source includes non-ASCII chars, these are not decode to unicode and Trac fails with unicode decode error. native stacktrace.

[trac-ja@…]

patch against source:trunk@6635​.

[Christian Boos]

In get_lines_from_file, we could even guess the charset by looking at the first line (match = re.search(r'-\*-\s+coding:\s+([\w-]+)\s+-\*-', lines[0])) and use match.group(1) if available as second argument for to_unicode.

[trac-ja@…]

Replying to cboos:

In get_lines_from_file, we could even guess the charset by looking at the first line (match = re.search(r'-\*-\s+coding:\s+([\w-]+)\s+-\*-', lines[0])) and use match.group(1) if available as second argument for to_unicode.

Patch updated.

I use the regex coding[=:]\s*([-\w.]+) for detecting a encoding in the Macro's source file from ​http://docs.python.org/ref/encodings.html.

[trac-ja@…]

Patch against source:trunk@6692​

[Christian Boos]

Yes, even better! Thanks.

[elescargot@…]

Hi,

I've got this problem (some pages won't show - "new ticket" and "roadmap") after upgrading to trac 0.11 from trac 0.9.6. Instead of the page, there is the following text :

" Traceback (most recent call last):

File "C:\Python25\lib\site-packages\trac\web\api.py", line 339, in send_error

'text/html')

File "C:\Python25\lib\site-packages\trac\web\chrome.py", line 704, in render_template

return stream.render(method, doctype=doctype)

File "C:\Python25\Lib\site-packages\genshi\core.py", line 154, in render

return encode(generator, method=method, encoding=encoding)

File "C:\Python25\Lib\site-packages\genshi\output.py", line 45, in encode

output = u.join(list(iterator))

File "C:\Python25\Lib\site-packages\genshi\output.py", line 274, in call

for kind, data, pos in stream:

File "C:\Python25\Lib\site-packages\genshi\output.py", line 519, in call

for kind, data, pos in stream:

File "C:\Python25\Lib\site-packages\genshi\output.py", line 688, in call

text = mjoin(textbuf, escape_quotes=False)

File "C:\Python25\Lib\site-packages\genshi\core.py", line 379, in join

for item in seq]))

File "C:\Python25\Lib\site-packages\genshi\core.py", line 405, in escape

text = unicode(text).replace('&', '&') \

UnicodeDecodeError: 'ascii' codec can't decode byte 0x82 in position 1002: ordinal not in range(128) "

If this is really the same problem, could you explain how to use your patch?

Thanks

[elescargot@…]

If needed, here is my configuration :

• win XP
• trac 0.11 b2
• Apache 2.2
• Subversion 1.4.6
• Python 2.5

I think that's all… Please help!

[Christian Boos]

Replying to trac-ja@i-act.co.jp:

… Patch updated.

Patch verified and applied in r6828, thanks!

For the others having commented on this ticket (in particular comment:4), the error is different. It happens as soon as a string containing non-ascii characters makes its way to Genshi: something that normally should not happen, as we're careful to pass unicode objects around and coerce strings to unicode when needed. Try disabling all plugins and see if this still happen, then reintroduce plugins one by one in order to find the problematic one.

[elescargot@…]

Replying to cboos:

Replying to trac-ja@i-act.co.jp:

… Patch updated.

Patch verified and applied in r6828, thanks!

For the others having commented on this ticket (in particular comment:4), the error is different. It happens as soon as a string containing non-ascii characters makes its way to Genshi: something that normally should not happen, as we're careful to pass unicode objects around and coerce strings to unicode when needed. Try disabling all plugins and see if this still happen, then reintroduce plugins one by one in order to find the problematic one.

The problem is that I have no plugins… So I can't see where does the problem come from. Do you have any other idea?

Thanks,

Eleonore

[Christian Boos]

Ok, so first upgrade to r6828: when logged in with an user having the TRAC_ADMIN permission, you'll now be able to see a "rich" traceback, showing the values of the local variables. In the last frame:

File "C:\Python25\Lib\site-packages\genshi\core.py", line 405, in escape
    text = unicode(text).replace('&', '&') \

Look for the value of text, this will give you an hint about where the problematic text comes from.

[anonymous]

Well, when I copy the modified lines in my file trac/util/_init.py_, I still have the same problem.

Moreover, I've got another problem (maybe linked to the first) : the pages I can see in trac are in raw HTML, there is no button… seems that the template cannot be loaded. If you want I can show you a printscreen (but I don't know how to post images here).

I've made a mistake in my configuration, the server isn't on win xp but win 2003 rc2.

Thanks again for helping.

Eleonore

[elescargot@…]

In fact it seems that my problem is the same that the one mentioned in ticket number #3908, as far as the unicode problem is concerned. But I haven't seen any solution…

Eleonore

[Christian Boos]

Eleonore, so please update #3908 with some details about your setup (if you're using MySQL, check also the requirements described in the MySqlDb page, in particular the encoding needed for the collation type).

[anonymous]

Traceback (most recent call last):

File "/usr/local/python2.5/lib/python2.5/site-packages/Trac-0.11dev_r6895-py2.5.egg/trac/web/api.py", line 339, in send_error

'text/html')

File "/usr/local/python2.5/lib/python2.5/site-packages/Trac-0.11dev_r6895-py2.5.egg/trac/web/chrome.py", line 723, in render_template

return stream.render(method, doctype=doctype)

File "/usr/local/python2.5/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/core.py", line 154, in render

return encode(generator, method=method, encoding=encoding)

File "/usr/local/python2.5/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 45, in encode

output = u.join(list(iterator))

File "/usr/local/python2.5/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 274, in call

for kind, data, pos in stream:

File "/usr/local/python2.5/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 519, in call

for kind, data, pos in stream:

File "/usr/local/python2.5/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/output.py", line 688, in call

text = mjoin(textbuf, escape_quotes=False)

File "/usr/local/python2.5/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/core.py", line 379, in join

for item in seq]))

File "/usr/local/python2.5/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg/genshi/core.py", line 405, in escape

text = unicode(text).replace('&', '&') \

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 994: ordinal not in range(128)

[Christian Boos]

See also #7959.

[Christian Boos]

See also complementary fix [7929] (for 0.11.4).