Genshi cannot handle encoded exception message when a Wiki Macros is failed

[t2y <tetsuya.morimoto@…>]

I encountered this bug after my Trac was migrated 0.12 to 1.0.2dev_r12278. My Trac has many reports including Wiki Macros, and then a macro is failed for some reason like this. Our database is postgresql and it returns the encoded error message including Japanese.

2013-12-01 11:35:59,458 Trac[formatter] ERROR: Macro TicketQuery(version=v4.0,order=time,time=today..,col=id|summary|component|status|version|milestone|priority|time|reporter|qa_contact) failed:
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0.2dev_r12278-py2.6.egg/trac/wiki/formatter.py", line 766, in _macro_formatter
    return macro.ensure_inline(macro.process(args))
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0.2dev_r12278-py2.6.egg/trac/wiki/formatter.py", line 357, in process
    text = self.processor(text)
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0.2dev_r12278-py2.6.egg/trac/wiki/formatter.py", line 344, in _macro_processor
    text)
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0.2dev_r12278-py2.6.egg/trac/ticket/query.py", line 1332, in expand_macro
    tickets = query.execute(req)
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0.2dev_r12278-py2.6.egg/trac/ticket/query.py", line 302, in execute
    self.num_items = self._count(sql, args)
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0.2dev_r12278-py2.6.egg/trac/ticket/query.py", line 283, in _count
    % sql, args)[0][0]
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0.2dev_r12278-py2.6.egg/trac/db/api.py", line 122, in execute
    return db.execute(query, params)
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0.2dev_r12278-py2.6.egg/trac/db/util.py", line 121, in execute
    cursor.execute(query, params)
  File "/usr/local/lib/python2.6/dist-packages/Trac-1.0.2dev_r12278-py2.6.egg/trac/db/util.py", line 65, in execute
    return self.cursor.execute(sql_escape_percent(sql), args)
ProgrammingError: 列参照"status"は曖昧です
LINE 1: SELECT COUNT(*) FROM (SELECT t.status AS status,t.summary AS...

Currently, Genshi expects the unicode string, but _macro_formatter() function doesn't consider the string type when an exception happened.

• trac/wiki/formatter.py

   757     def _macro_formatter(self, match, fullmatch, macro, only_inline=False):      
   758         name = fullmatch.group('macroname')                                      
   759         if name.lower() == 'br':                                                 
   760             return '<br />'                                                      
   761         if name and name[-1] == '?': # Macro?() shortcut for MacroList(Macro)    
   762             args = name[:-1] or '*'                                              
   763         else:                                                                    
   764             args = fullmatch.group('macroargs')                                  
   765         try:                                                                     
   766             return macro.ensure_inline(macro.process(args))                      
   767         except Exception, e:                                                     
   768             self.env.log.error('Macro %s(%s) failed: %s' %                       
   769                     (name, args, exception_to_unicode(e, traceback=True)))       
   770             return system_message('Error: Macro %s(%s) failed' % (name, args),   
   771                                   e)
  

[t2y <tetsuya.morimoto@…>]

patch to decode the exception

[t2y <tetsuya.morimoto@…>]

Applying attached patch works (handles the error appropriately) for my Trac.

[Jun Omae]

The ProgrammingError is caused by changes in #11140. I reopened it, now.

[t2y <tetsuya.morimoto@…>]

Replying to jomae:

The ProgrammingError is caused by changes in #11140. I reopened it, now.

The TicketQuery Macro could show correctly by adding format=table.

[Jun Omae]

Thanks. The issue leads TracError with Genshi UnicodeDecodeError error in the user's browser.

2013-12-05 17:13:55,168 Trac[main] WARNING: [192.168.11.17] HTTPInternalError: 500 Trac Error (Genshi UnicodeDecodeError error while rendering template (unknown template location))

The patch looks good and would make wiki rendering robust. Also we should apply to 0.12-stable. I'll push it and unit tests.

[Jun Omae]

Proposed changes can be found in log:jomae.git:ticket11385_0.12.6dev for 0.12-stable and log:jomae.git:ticket11385 for 1.0-stable.

[Ryan J Ollos]

I tested the 1.0-stable branch and it works well.

One trivial thing is to reorder the imports Component, implements, TracError → Component, TracError, implements, as the latter seems to be our normal convention in the codebase.

[Jun Omae]

Sure. Thanks for your testing and reviewing!

[Jun Omae]

Fixed in [12295] and merged in [12296,12297].

[t2y <tetsuya.morimoto@…>]

After applying r12296 to my Trac, I confirmed this bug was fixed. Thank you jomae!

[Jun Omae]

Thanks for the feedback. The ProgrammingError, root cause of the issue, has been fixed in [12299].