Changeset contents doesn't encoded to UTF8

[anonymous]

I use trac trunk from 21 april 2005.

In changeset html output (and in diff format) content doesn't encoded to UTF8 as it does in browser module.

[anonymous]

BTW, Changeset.py also should check for DISP_MAX_FILE_SIZE in html output (not in diff format).

[Christopher Lenz]

Fixed in [1660].

(I don't think there's a good way we can limit the size of the diff HTML output at this point.)

[vyt@…]

Thank you very much!

[vyt@…]

I tried this changes and it doesn't work.

[vyt@…]

Patch for correct encoding in changeset content

[vyt@…]

Attached patch work for me. It also affect mimeview/api.py I think what charset detection should be common code for Trac.

[Christopher Lenz]

Should be fixed in [1696].

[vyt@…]

Yes, it works. But… It think what this code

charset = mimeview.get_charset(old_node.content_type) or default_charset
old_content = util.to_utf8(old_content, charset)

in many places in Trac could be simpler. Charset detection should be inside util.to_utf8().

[Christopher Lenz]

We don't want to add a dependency between trac.util and the trac.versioncontrol layer. That's why I didn't apply your patch directly.

[vyt@…]

In other words, mimeview.get_charset() will be different for different scm backends ?

[Christopher Lenz]

No, the trac.mimeview package shouldn't know that the trac.versioncontrol package even exists. And get_charset should also work for things like attachments, of course.

[anonymous]

This is still broken for me in 0.10.3.1. Here is a screenshot (sorry I can't provide access to the trac website directly):

​http://www.mukund.org/tmp/bad-display.png

I have set the default encoding to utf-8, but it still shows bad characters instead of the actual UTF-8 in various languages (japanese, hindi, tamil, etc.). In other programs such as editors, and on the website (it's a web project), I get the actual UTF-8 displayed correctly.

[anonymous]

Closing as invalid as this now works for me. Very very strange. Maybe the default encoding didn't take effect immediately. I have no idea how this started working again. As you can see from the screenshot, it was broken.

[Christian Boos]

Replying to anonymous:

Maybe the default encoding didn't take effect immediately.

The change in the default encoding took effect immediately, but the changeset pages are cached by the browser. So the change in encoding was seen only after the page was forcibly reloaded or the diff options changed.

[Christian Boos]

(re-closing as fixed)