Edgewall Software
Modify

Ticket #4717 (new defect)

Opened 4 years ago

Last modified 4 months ago

wrong table cell width with multibyte in ticket notify

Reported by: xuefer@… Owned by: cboos
Priority: normal Milestone: 0.12.1
Component: general Version: 0.10.3
Severity: minor Keywords: notification unicode
Cc: shunichi.goto@…, jun66j5@…

Description

Expected:

Priority:  major                                 |    Component:  Others
Keywords:  tester xxxxxxxxxxxxx                  |     Username:  abc

Unxpected:

Priority:  major                                 |    Component:  Others
Keywords:  tester xxxxxxxxxxxxx                        |     Username:  abc

where xxxxxxxxxxxxx is 7 Chinese characters, in 14 'w' width

Attachments

4717.diff (4.3 KB) - added by jomae 4 months ago.
broken-table-cell-width.png (15.5 KB) - added by jomae 4 months ago.
fixed-table-cell-width.png (15.5 KB) - added by jomae 4 months ago.

Change History

comment:1 Changed 4 years ago by mgood

  • Status changed from new to closed
  • Resolution set to wontfix
  • Milestone 0.10.4 deleted

So, you're saying that the characters in question are displayed in a font such that each character is wider than an ascii character. For plaintext mail formatting we expect the emails to be displayed in a fixed-width font so that all characters are the same width. If you can, configure your mail client to use a font where the characters will all be the same width. If you can't do that, I'm afraid there's no way to work around that for the plaintext mails. HTML email formatting has been requested, and would enable better formatting with variable-width fonts. See #2625.

comment:2 Changed 4 years ago by xuefer@…

  • Status changed from closed to reopened
  • Resolution wontfix deleted

i'm sure using fixed-width font, and each Chinese character is in the same width as 2 ascii, which IS expected, even u use fixed-width font in html. i agree that using html table can solve the problem, because browser calc the width correctly, but not trac. that's why i ask tach to fix itself, calc the text width, not character count (count of code point)

if u know php, there is  mb_strwidth to do the job.

i'm not familar with python, but it seems there's:

east_asian_width( unichr) 

Returns the east asian width assigned to the Unicode character unichr as string. New in version 2.4. 

One shall never suppose 1 Chinese char is same with as 1 ascii char, in fixed-width font.

comment:3 Changed 4 years ago by cboos

  • Keywords notification unicode added
  • Component changed from ticket system to general
  • Severity changed from normal to minor
  • Milestone set to 0.12

east_asian_width(unichr) - New in version 2.4

Maybe we can bump the Python requirement in the next version of Trac?

(btw, mgood, tee from itertools is also new in 2.4)

comment:4 follow-up: ↓ 5 Changed 20 months ago by cboos

  • Owner changed from jonas to cboos
  • Status changed from reopened to new

Note for self: reuse this code

    from unicodedata import east_asian_width as eaw
    _colwidth = lambda s: sum([eaw(c) in 'WF' and 2 or 1 for c in s])

(from Shun-ichi Goto on Mercurial-devel, used with permission)

where xxxxxxxxxxxxx is 7 Chinese characters, in 14 'w' width

maybe a real example would help...

comment:5 in reply to: ↑ 4 Changed 20 months ago by Shun-ichi Goto <shunichi.goto@…>

  • Cc shunichi.goto@… added

Replying to cboos:

where xxxxxxxxxxxxx is 7 Chinese characters, in 14 'w' width

maybe a real example would help...

For example in Japanese, the text "日本語" (means "Japanese") has 3 characters and occupies 6 column width.

>>> len(u'日本語')
3
>>> _colwidth(u'日本語')
6

Expected (left:

Priority:  major                                 |    Component:  Others
Keywords:  tester 日本語                         |     Username:  abc

Unwanted (extra 3 spaces):

Priority:  major                                 |    Component:  Others
Keywords:  tester 日本語                            |     Username:  abc

comment:6 follow-up: ↓ 7 Changed 20 months ago by cboos

Note that it doesn't look like browsers are respecting this 3 asian characters -> 6 column width convention, at least when using the default fixed font, as can be seen in the above example. Same goes for Thunderbird when displaying the notification mail for comment:5.

comment:7 in reply to: ↑ 6 Changed 20 months ago by Shun-ichi Goto <shunichi.goto@…>

Replying to cboos:

Note that it doesn't look like browsers are respecting this 3 asian characters -> 6 column width convention, at least when using the default fixed font, as can be seen in the above example.

I guess it is issue of renderer of application on using multiple fonts (for ASCII and Japanese). For example, if you use MSゴシック (msgothic) font on windows, you will see good looking because it is a fixed-width font having both ASCII and Japanese glyphs.

Changed 4 months ago by jomae

Changed 4 months ago by jomae

Changed 4 months ago by jomae

comment:8 Changed 4 months ago by jomae

  • Cc jun66j5@… added

I created a patch 4717.diff for the issue using unicodedata.east_asian_width.

New [notification] ambiguous_char_width option is like  ambiwidth option of Vim. CJK (Chinese, Japanese and Korean) users need this option and specify 'double'. Default of the option is 'single', the most users don't need to specify the option.

If you want to know about Ambiguous Characters, visit  http://www.unicode.org/reports/tr11/.

Trac 0.12dev-r9679:

Trac 0.12dev-r9679 with the patch:

comment:9 Changed 4 months ago by cboos

  • Milestone changed from next-major-0.1X to 0.12.1

Thanks for the patch, it looks good except for a few details:

  • from unicodedata import east_asian_width should be among the first block of imports, as it's a module coming from stock Python
  • I didn't count, but it looks like you have some lines > 79 (single width ;-) ) chars; please check our TracDev/CodingStyle

Finally, I think the setting makes sense in a locally deployed Trac instance, where most users are either CJK ones or not. But ultimately this should become a user setting, in the notification preferences (#4056).

View

Add a comment

Modify Ticket

Change Properties
<Author field>
Action
as new
as The resolution will be set. Next status will be 'closed'
to The owner will be changed from cboos. Next status will be 'new'
The owner will be changed from cboos to anonymous. Next status will be 'assigned'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.