Edgewall Software

Opened 13 years ago

Last modified 3 years ago

#4270 new defect

Ticket notification emails contain wiki formatting

Reported by: chris.alex.thomas@… Owned by:
Priority: normal Milestone: next-major-releases
Component: notification Version: 0.10.2
Severity: normal Keywords: notification wiki_to_text
Cc: chris.alex.thomas@…, trac@…, banduwgs@…, daniel.oconnor@…, jetcat@…, hju@… Branch:
Release Notes:
API Changes:



when I send an email of a ticket, through the CC list, I find that it sends the wiki formatting as well, since it's a text email, the normal text should be displayed and the wiki formatter should strip out the formatting before the email is sent.

In order to get acceptable results from the ticket display, an example is like thus

  1. an item CamelCase
  2. another item

but in the email, instead of seeing what I have above, I see this

1. an item with !CamelCase[[br]]
2. another item[[br]]

so of course, people reading the emails, are wondering what all the [[br]] are for and it makes some emails kinda unreadable.

this should be remedied

Attachments (1)

notifyformatter.py (20.0 KB ) - added by d_dorothy@… 13 years ago.
proposed NotifyFormatter class (and related one line class and convenience function)

Download all attachments as: .zip

Change History (20)

comment:1 by anonymous, 13 years ago

Cc: chris.alex.thomas@… added
Component: generalticket system

sorry jonas, incorrectly assigned the component and forgot my email address from the CC

comment:2 by Christian Boos, 13 years ago

Milestone: 1.0

Yes, a TextFormatter for wiki content would be useful in that case.

As a bonus, the links should be replaced by footnotes, e.g.

 1. an item with CamelCase (1)
 2. another item

(1) http://example.org/trac/wiki/CamelCase

comment:3 by Emmanuel Blot, 13 years ago

Keywords: notification added

comment:4 by d_dorothy@…, 13 years ago

I came up with a simple patch for now until something fancier could be done (I do not have lots of experience with python regular expressions)

To do this right you need to make a class that inherits from Formatter and knows how to parse through the text using regex. I could not get this to work b/c of my lack of python regex knowledge.

--- notification.py	Wed Nov 29 16:40:02 2006
+++ notification.py	Wed Nov 29 16:34:58 2006
@@ -276,12 +276,42 @@
         if self.user_name:
-            self.server.login(self.user_name, self.password)
+            self.server.login(self.user_name, self.password)
+    def removeWikiTags(self, text):
+        import re 
+        #replace [[br]] with new line
+        text = text.replace("[[br]]", "\n")
+        text = text.replace("[[BR]]", "\n")
+        from trac.wiki.api import WikiSystem
+        #return re.sub(WikiSystem(self.env).rules, self.replace, text)
+        #Ignore wiki-macros
+        r = re.compile(r"\[\[.*\]\]")
+        m = r.search(text)
+        while m:
+            if m.end() < len(text):
+                text = text[:m.start()] + text[m.end():]
+            else:
+                text = text[:m.start()]
+            m = r.search(text)
+        #Ignore bold and italic
+        r = re.compile("''+")
+        tmp = r.split(text)
+        text = ""
+        for m in tmp:
+            text += m
+        return text
     def send(self, torcpts, ccrcpts, mime_headers={}):
         from email.MIMEText import MIMEText
         from email.Utils import formatdate, formataddr
-        body = self.hdf.render(self.template_name)
+        body = self.removeWikiTags(self.hdf.render(self.template_name))
         projname = self.config.get('project', 'name')
         public_cc = self.config.getbool('notification', 'use_public_cc')
         headers = {}

If this helps let me know.

comment:5 by d_dorothy@…, 13 years ago

I have made some classes that do this right. Except for tables. Not sure how best to do these and make them readable in plain text. I just left some of the wiki formatting there to help with readability in plain text.

Also my unit test is sort of system dependent. I looked into the TracDev/UnitTests documentation but did not quite understand how to do some of the stuff. (sorry for my limited python understanding, I am working on this)

Also any ideas on where exatly this code should be put? Should it be placed directly in notification.py or should it be a separate file referenced by notification.py?

I think that it will be able to be plugged in similar to my previous post, though I have not tested this.

body = NotifyFormatter(env).format(self.hdf.render(self.template_name))

I will attach my file.

by d_dorothy@…, 13 years ago

Attachment: notifyformatter.py added

proposed NotifyFormatter class (and related one line class and convenience function)

comment:6 by chris.alex.thomas@…, 13 years ago


isnt the easiest way to solve this problem to simply NOT reformat the mail before it's sent? to simply take the text that gets reformatted by the wiki and sending the email of the data which is input into the wiki reprocessor instead of sending the output of the wiki reprocessor?

I understand that sometimes the simplest way is sometimes the worst design, because perhaps the flow of the code would be broken by "hacking" this to work and sometimes this can only happen whilst maintaining the great design, is not by hacking, but a redesign of that section of code, which has implications for other code as well.

but it sounds the easiest way. Anyone with a clue know whether this idea can fly?

comment:7 by d_dorothy@…, 13 years ago

I looked into what you asked about. I went further back and tried to track a notification from the beginning.

The ticket change is stored in the db with the wiki syntax in it and the notification mechanism appears to just suck this data up to generate the email header and body. This means that if you do not run the text retrieved from the database through a formatting mechanism you will get the output that is currently seen. For this reason you must do something to format it.

It is possible to just strip out the Wiki formatting (using regular expressions) and return it in plain text but that does not always lend itself to readability in the text format (think about tables, lists, code blocks, etc.).

If I have missed an important detail please point out where I am wrong so I can provide you with better answers.

comment:8 by Emmanuel Blot, 13 years ago

See also #4516

comment:9 by trac@…, 13 years ago

Cc: trac@… added

Another option is to render it into HTML and send it as a multipart MIME with the HTML (found this ticket because a user [complained to / asked] me today).

in reply to:  9 comment:10 by Markus Tacker <m@…>, 13 years ago

Replying to trac@revragnarok.com:

Another option is to render it into HTML and send it as a multipart MIME with the HTML (found this ticket because a user [complained to / asked] me today).

See #2625 for this.

in reply to:  5 ; comment:11 by anonymous, 12 years ago

Cc: banduwgs@… added

Replying to d_dorothy@bellsouth.net:

I think that it will be able to be plugged in similar to my previous post, though I have not tested this.

Did anyone tested this? I couldn't make it work with Trac 0.10.4 yet. Or else is there any better workaround for this? I could not locate such a thing on Trac Users.

I will give more debugging details in a separate email.


in reply to:  11 comment:12 by anonymous, 12 years ago

Replying to anonymous:

I will give more debugging details in a separate email.

This is what I get:

2007-06-13 12:14:16,365 Trac[web_ui] ERROR: Failure sending notification on change to ticket #77: global name 'NotifyFormatter' is not defined
Traceback (most recent call last):
 File "D:\Python24\Lib\site-packages\trac\ticket\web_ui.py", line 562, in _do_save
 tn.notify(ticket, newticket=False, modtime=now)
 File "D:\Python24\Lib\site-packages\trac\ticket\notification.py", line 129, in notify
 NotifyEmail.notify(self, ticket.id, subject)
 File "D:\Python24\lib\site-packages\trac\notification.py", line 216, in notify
 Notify.notify(self, resid)
 File "D:\Python24\lib\site-packages\trac\notification.py", line 115, in notify
 self.send(torcpts, ccrcpts)
 File "D:\Python24\Lib\site-packages\trac\ticket\notification.py", line 275, in send
 NotifyEmail.send(self, torcpts, ccrcpts, hdrs)
 File "D:\Python24\lib\site-packages\trac\notification.py", line 293, in send
 body = NotifyFormatter(env).format(self.hdf.render(self.template_name))
NameError: global name 'NotifyFormatter' is not defined

What I did simply was

  1. Copied notifyformatter.py to D:\Python24\lib\site-packages\trac\.
  2. Modified D:\Python24\lib\site-packages\trac\notification.py as given in comment:ticket:4270:5

I noticed that .pyc or .pyo files for notifyformatter is not created.

What I have missed here?


comment:13 by daniel.oconnor@…, 12 years ago

Cc: daniel.oconnor@… added

comment:14 by Christian Boos, 12 years ago

Component: ticket systemwiki system
Milestone: 1.00.12
Summary: Ticket CC emails contain wiki formattingTicket notification emails contain wiki formatting

#7095 was closed as duplicate (it was focusing on the InterWiki links - see comment:2 for how the links could be handled).

comment:15 by anonymous, 11 years ago

Component: wiki systemnotification

comment:16 by jetcat@…, 11 years ago

Cc: jetcat@… added

comment:17 by hju@…, 11 years ago

Cc: hju@… added

comment:18 by Ryan J Ollos, 5 years ago

Owner: Jonas Borgström removed

comment:19 by Peter Suter, 3 years ago

Keywords: wiki_to_text added

Modify Ticket

Change Properties
Set your email in Preferences
as new The ticket will remain with no owner.
The ticket will be disowned. Next status will be 'new'.
as The resolution will be set. Next status will be 'closed'.
The owner will be changed from (none) to anonymous. Next status will be 'assigned'.

Add Comment

E-mail address and name can be saved in the Preferences .
Note: See TracTickets for help on using tickets.