Edgewall Software
Modify

Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#11350 closed defect (fixed)

Subject of notification email too long - add limit

Reported by: trac-trac@… Owned by: Ryan J Ollos
Priority: normal Milestone: 1.0.2
Component: notification Version: 1.0
Severity: normal Keywords: batch-modify notification
Cc: Branch:
Release Notes:

The subject of batch ticket email notifications is limited to 75 characters.

API Changes:

The function shorten_line in trac.util.text was corrected to return a string less than or equal to maxlen characters.

Internal Changes:

Description

I received a Trac notification email with a subject line like this:

[Pidgin] Batch modify: #4933, #5329, #5583, #5696, #6521, #6751,
 #6881, #6988, #7110, #7145, #7304, #7626, #7999, #8023, #8040,
 #8075, #8135, #8201, #8220, #8238, #8321, #8323, #8341, #8369,
 #8371, #8382, #8388, #8414, #8467, #8507, #8547, #8564, #8599,
 #8630, #8647, #8656, #8680, #8689, #8699, #8701, #8714, #8753,
 #8778, #8788, #8829, #8854, #8919, #8967, #8984, #8993, #9009,
 #9020, #9035, #9043, #9077, #9099, #9110, #9136, #9144, #9163,
 #9167, #9185, #9189, #9194, #9195, #9201, #9225, #9230, #9237,
...

The total length of the subject line was 5546 characters. Such long subject lines are not well-handled by email clients. Eg. Thunderbird doesn't allow the message to be viewed without a workaround. Trac should impose some hard limit on the length of the subject line for all emails. Subjects longer than that should be truncated with "…" appended.

For this specific email (batch-modification) I'd also suggest not listing all ticket numbers in the subject line if there are more than X of them.

Attachments (0)

Change History (12)

comment:1 by Jun Omae, 7 years ago

Keywords: batch-modify notification added; email removed
Milestone: next-stable-1.0.x

comment:2 by Ryan J Ollos, 7 years ago

Milestone: next-stable-1.0.x1.0.2
Owner: set to Ryan J Ollos
Status: newassigned

I noticed some strange behavior of trac.util.text:shorten_line.

>>> from trac.util.text import shorten_line
>>> sl = shorten_line('0123456789', 10)
>>> print sl
'0123456789 ...'
>>> len(sl)
14
>>> sl = shorten_line('0123456789', 9)
>>> print sl
012345678 ...
>>> len(sl)
13
  • When the length of the input text is maxlen, the text is still shortened.
  • The shortened text does not have a length less than or equal to maxlen.

I'm considering the following patch:

  • trac/util/text.py

    diff --git a/trac/util/text.py b/trac/util/text.py
    index bd1f65a..c2c63ad 100644
    a b def shorten_line(text, maxlen=75):  
    418418    This tries to be (a bit) clever and attempts to find a proper word
    419419    boundary for doing so.
    420420    """
    421     if len(text or '') < maxlen:
     421    if len(text or '') <= maxlen:
    422422        return text
    423     cut = max(text.rfind(' ', 0, maxlen), text.rfind('\n', 0, maxlen))
     423    suffix = ' ...'
     424    maxtextlen = maxlen - len(suffix)
     425    cut = max(text.rfind(' ', 0, maxtextlen), text.rfind('\n', 0, maxtextlen))
    424426    if cut < 0:
    425         cut = maxlen
    426     return text[:cut] + ' ...'
     427        cut = maxtextlen
     428    return text[:cut] + suffix
    427429
    428430
    429431class UnicodeTextWrapper(textwrap.TextWrapper):

Which results in:

>>> from trac.util.text import shorten_line
>>> sl = shorten_line('0123456789', 10)
>>> print sl
0123456789
>>> len(sl)
10
>>> sl = shorten_line('0123456789', 9)
>>> print sl
01234 ...
>>> len(sl)
9

I'll look more closely at existing places where shorten_line is used before making the changes. Typically I wouldn't suggest making changes to the behavior of the public API like this, but the behavior of shorten_line seems rather poor and I think we should fix it.

Unit tests would be added before committing changes to shorten_line.

As for the primary fix needed for this ticket, after a quick search I found SO:1592291, which seems to suggest that we limit the length of the subject of a notification email to 78 characters.

Last edited 7 years ago by Ryan J Ollos (previous) (diff)

in reply to:  2 comment:3 by Ryan J Ollos, 7 years ago

Replying to rjollos:

[…]

  • When the length of the input text is maxlen, the text is still shortened.
  • The shortened text is not length maxlen

The characteristics of the function have existed since it was introduced in [249].

in reply to:  2 ; comment:4 by Peter Suter, 7 years ago

  • When the length of the input text is maxlen, the text is still shortened.
  • The shortened text is not length maxlen

I agree, that's not what I would have expected.

I'll look more closely at existing places where shorten_line is used before making the changes. Typically I wouldn't suggest making changes to the behavior of the public API like this, but the behavior of shorten_line seems rather poor and I think we should fix it.

Right, we should be careful not to break public API's needlessly. But here I can't even find any caller that explicitly specifies the maxlen. Probably all that's wanted is just a reasonably short message, without any specific requirements that need to be preserved. So it seems fine to improve the behavior as long as it stays reasonably similar.

As for the primary fix needed for this ticket, after a quick search I found SO:1592291, which seems to suggest that we limit the length of the subject of a notification email to 78 characters.

I would maybe even just go with the default of 75. 78 seems more like an approximate recommendation than an exact standard. For example someone in that discussion mentions Outlook will cut subjects >77.

Any idea if / how encodings affect this limit in practice?

Would a simple change in NotifyEmail to headers['Subject'] = shorten_line(self.subject) make sense?

in reply to:  4 ; comment:5 by Jun Omae, 7 years ago

Would a simple change in NotifyEmail to headers['Subject'] = shorten_line(self.subject) make sense?

No. The limitation is very very short for CJK users. I don't think NotifyEmail should have any limitations.

I think we should cut only subject of batch-modify notification.

The following is real example. The subject has 68 characters (104 bytes) and 5 lines in mime encoding.

[tracpath] #1873: ExcelDownloadPlugin を Excel 2007 形式でエクスポートできるようにする
Subject: =?utf-8?q?=5Btracpath=5D_=231873=3A_ExcelDownloadPlugin_?=
	=?utf-8?q?=E3=82=92_Excel_2007_=E5=BD=A2=E5=BC=8F=E3=81=A7?=
	=?utf-8?q?=E3=82=A8=E3=82=AF=E3=82=B9=E3=83=9D=E3=83=BC=E3=83=88?=
	=?utf-8?q?=E3=81=A7=E3=81=8D=E3=82=8B=E3=82=88=E3=81=86=E3=81=AB?=
	=?utf-8?q?=E3=81=99=E3=82=8B?=

in reply to:  5 comment:6 by Peter Suter, 7 years ago

No. The limitation is very very short for CJK users. I don't think NotifyEmail should have any limitations.

Ah, I suspected that. Thank you for the confirmation.

The following is real example. The subject has 68 characters

So shorten_line with maxlen=75 would leave it unchanged? (As mime encoding happens later.)


(By the way, I got Re: [The Trac Project] #11164: Git persistent cache not refreshed on changeset added notification from another process recently (118 characters) and Thunderbird seemed to be fine with it.)

in reply to:  4 comment:7 by Ryan J Ollos, 7 years ago

Replying to psuter:

Would a simple change in NotifyEmail to headers['Subject'] = shorten_line(self.subject) make sense?

That's the change I was going to propose, however I hadn't considered potential problems with languages other than English. I guess that is still unclear after your comment:6. Shortening simply by searching for Ascii whitespace and linefeed characters seems problematic for non-Latin languages based on the very little I know about how some non-Latin languages separate words. At a minimum, it seems shorten_line should truncate on Unicode whitespace.

If we shorten the subject for all emails, we might want to put the full Summary in the body of the email. Another consideration is that we would probably want the shortened summary to be the same before and after the Re: is prefixed in trunk/trac/ticket/notification.py@10285:171#L161, otherwise email clients might not group the first and subsequent messages (i.e. it wouldn't treat Word Word Word ... and Re: Word Word ... as the same message).

Just fixing this for Batch notifications is looking quite a bit easier than the alternative!

comment:8 by Ryan J Ollos, 7 years ago

Proposed changes can be found in log:rjollos.git:t11350.

comment:9 by trac-trac@…, 7 years ago

Does this fix affect emails other than batch notification? I'd still suggest that there should be some hard limit for all emails - probably over 75 characters, but under 5546.

comment:10 by Ryan J Ollos, 7 years ago

The proposed change does not shorten the summary for emails other than batch notification.

Trac allows configuring a limit on the length of the ticket description and comments, but not on the ticket summary: TracIni#ticket-section. A related possibility would be to add a [ticket] max_summary_size parameter.

Last edited 7 years ago by Ryan J Ollos (previous) (diff)

comment:11 by Ryan J Ollos, 7 years ago

API Changes: modified (diff)
Release Notes: modified (diff)
Resolution: fixed
Status: assignedclosed

Committed to 1.0-stable in [12237:12239] and merged to trunk in [12240].

In the future I'll look into limiting the max size of the summary for all notifications, or adding an option to limit the max size of the summary.

Version 0, edited 7 years ago by Ryan J Ollos (next)

in reply to:  11 comment:12 by Ryan J Ollos, 7 years ago

Replying to rjollos:

In the future I'll look into limiting the max size of the summary for all notifications, or adding an option to limit the max size of the ticket summary.

See #11472.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Ryan J Ollos.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Ryan J Ollos to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.