Edgewall Software

Opened 12 years ago

Last modified 6 years ago

#9085 new enhancement

Notification system silently drops failures to unicode email addresses

Reported by: John Hampton Owned by:
Priority: normal Milestone: next-major-releases
Component: notification Version:
Severity: major Keywords: unicode email addresses
Cc: raphael.schmid@…, Thijs Triemstra Branch:
Release Notes:
API Changes:
Internal Changes:


See #9082 for some background.

Currently, email addresses with unicode in them are silently discarded when sending ticket updates. While not trying to deliver the email is correct, it's a little misleading. If the user enters a unicode email address in the CC list, then it appears to get added (indeed it does) but they will never receive any emails. Unfortunately, there is no indication as to why the email will never be delivered. An admin can look in the log and see the email addresses which have been discarded, but they first have to know what they are looking for. Additionally, the same thing occurs for addresses in the smtp_always_cc and smtp_always_bcc fields.

We could show a warning indicating that the email wasn't delivered to the problem addresses. The problem with the CC list, is that the invalid address is put on the CC list, and so anytime an update occurred, the warning would be shown. An administrator would then need to clean out the CClist. The problem is even more complicated if a user has the invalid email address in their preferences and simply puts their username on the CC list (or as the owner/reporter/updater of a ticket). An admin would then need to clear out the email address in the user's session.

I think the solution is to warn when an email can't be delivered, as well as stricter input validation. Input validation would need to be in:

when entering email address
CC list
would need to differentiate between entering an email address and a username. This might be really hard when taking into account use_short_addr and smtp_default_domain
smtp_always_cc and smtp_always_bcc
Don't know how to properly validate these, or whether it's worth it. Populating these with invalid values should be readily apparent to the admin.

Attachments (0)

Change History (10)

comment:1 by Raphael J. Schmid <raphael.schmid@…>, 12 years ago

Cc: raphael.schmid@… added

comment:2 by Christian Boos, 12 years ago

Milestone: 0.13next-major-0.1X

Milestone 0.13 only when there's a patch or a strong commitment to implement it for that release.

comment:3 by Thijs Triemstra, 11 years ago

Cc: Thijs Triemstra added
Severity: normalmajor

comment:4 by Thijs Triemstra, 11 years ago

Added this to the TracDev/Proposals/EmailValidation proposal.

comment:5 by Carsten Klein <carsten.klein@…>, 11 years ago

Thanks for the input.

Feel free to comment/review the notice I have provided under discussion/Additional Requirements and Non Requirements.

comment:6 by Carsten Klein <carsten.klein@…>, 11 years ago

Yet, however, I fear that the discard of the email addresses is an MTA related one.

Do you use the smtp or the sendmail based mailer?

Does your MTA support unicode addresses with umlauts etc?

comment:7 by Carsten Klein <carsten.klein@…>, 11 years ago

Reading through http://en.wikipedia.org/wiki/Email_address it seems that UTF-8 support in smtp headers is still in the works.

Thus, malformed e-mail addresses should definitely be prevented from entering into the system.

For smtp_always_cc and smtp_always_bcc I would definitely log a warning in the log files, for CC the ticket system must be adjusted to support this.

In the preferences module this must also be prevented for now.

+1 from me for this for the next major release or earlier if this can be done in a reasonable amount of time

comment:8 by Carsten Klein <carsten.klein@…>, 11 years ago

Alas, the regular expression for validating email addresses on input is rather complex, see http://www.ex-parrot.com/pdw/Mail-RFC822-Address.html.

comment:9 by Carsten Klein <carsten.klein@…>, 11 years ago

I just had a good look at the rfc822 module.

The rfc822.parseaddr() function will parse addresses according to RFC822. Based on some arbitrary unicode input including umlauts I found that it will encode these using the \x.. notation.

However, and due to standard set forth in RFC822 nearly all input goes through, which basically also means that we cannot use this for pre-validation of a given address but we might use this for encoding the addresses to a format that is more compliant with existing mailers, for as long as they do not support the unicode character set natively in the headers.

But see for yourselves…

# -*- coding: utf-8 -*-

from rfc822 import parseaddr as p

e = ['öäü@üöö.cöm', 'Foo Bar <foo (grandmaster of all evil)@bar.com>', '"does this make any sense"@@not.at.all']

for a in e:
  print p(a)

comment:10 by Ryan J Ollos, 7 years ago

Owner: John Hampton removed

Modify Ticket

Change Properties
Set your email in Preferences
as new The ticket will remain with no owner.
The ticket will be disowned.
as The resolution will be set. Next status will be 'closed'.
The owner will be changed from (none) to anonymous. Next status will be 'assigned'.

Add Comment

E-mail address and name can be saved in the Preferences .
Note: See TracTickets for help on using tickets.