broken messages on trac-admin

[trac-ja@…]

trac-admin shows broken messages on non UTF-8 console
(ex. CMD.EXE). The inputs are normally done, invalid-cases
are only a display.

There is very simple cause. Even what charset the console is,
display trac-admin with UTF-8.

When this patch is applied, it will improve:

trac/scripts/admin.py

@@ -25 +25 @@
 import time
 import traceback
 import urllib
+import locale
 
 import trac
 from trac import perm, util, db_default
@@ -198 +199 @@
         return [a for a in words if a.startswith (text)]
 
     def print_listing(self, headers, data, sep=' ', decor=True):
+        (cons_locale, cons_charset) = locale.getdefaultlocale()
         ldata = list(data)
         if decor:
             ldata.insert(0, headers)
@@ -218 +220 @@
                     sp = sep
                 if cnum + 1 == ncols:
                     sp = '' # No separator after last column
+                if ldata[rnum][cnum]:
+                    if cons_charset:
+                        pdata = unicode(ldata[rnum][cnum], 'utf-8').encode(cons_charset)
+                    else:
+                        pdata = ldata[rnum][cnum]
+                else:
+                    pdata = ''
                 print ('%%-%ds%s' % (colw[cnum], sp)) \
-                      % (ldata[rnum][cnum] or ''),
+                      % (pdata),
             print
             if rnum == 0 and decor:
                 print ''.join(['-' for x in

[cmlenz]

There are still problems with incorrect column width for strings with multi-byte characters, as we're counting bytes in print_listing(), not characters. IMHO we should tackle this on the unicode branch.

[eblot]

See also #859

[trac-ja@…]

Excuse me that the answer is delayed.

I think, this problem has three faces:

1. Character-based wrap/cut.
2. Byte-based wrap/cut escaping multi-byte charcter's border, for RSS output. (<description> element allows lines shorter than 80 bytes.)
3. Griph-based wrap/cut, for trac-admin's display and other monospace displays.

1 resolves by unicode branch.

2 is not pointed out, yet.

3 is pointed by cmlenz on this ticket.

Python's string object knows number of characters (by length
of unicode string) and number of bytes (by length of utf-8 or
other encoded string), but doesn't know colums by griph-size.

Later version-2.4, Python has unicodedata.east_asian_width()
method implements ​East Asian Width
that is Unicode's Specification. It's not impletemted Python-2.3
before. However, we can use that to resolve this problem
(w/ dismissing Python-2.3 environments... ).

(Note: docutils-0.4 uses above.)

I will try to write patches, but I need time to think/write,
because these are very large and difficult features. Do you know
more simple methods? (I don't know Python very much.)

[trac-ja@…]

Memo:

I found good library.

​http://city.plala.jp/download/eawidth/ .

[jonas]

I've fixed this in the unicode branch. It's not in trunk yet.

[cboos]

This must have been fixed by r3024.
Please test and reopen if there are still issues with this.