Edgewall Software
Modify

Opened 11 years ago

Closed 9 years ago

#3024 closed defect (fixed)

Cannot render (or upgrade) some wiki pages.

Reported by: tjb@… Owned by: Christian Boos
Priority: high Milestone: 0.10
Component: wiki system Version: 0.9.4
Severity: normal Keywords: unicode
Cc: pacopablo@…, tjb@…, matt_tricks@…
Release Notes:
API Changes:

Description

When trying to access the TracReports wiki page, we see the following error

Python Traceback

Traceback (most recent call last):
  File "/usr/lib/python2.3/site-packages/trac/web/main.py", line 300, in dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.3/site-packages/trac/web/main.py", line 176, in dispatch
    resp = chosen_handler.process_request(req)
  File "/usr/lib/python2.3/site-packages/trac/wiki/web_ui.py", line 117, in process_request
    self._render_view(req, db, page)
  File "/usr/lib/python2.3/site-packages/trac/wiki/web_ui.py", line 364, in _render_view
    req.hdf['wiki.page_html'] = wiki_to_html(page.text, self.env, req)
  File "/usr/lib/python2.3/site-packages/trac/wiki/formatter.py", line 826, in wiki_to_html
    return Markup(out.getvalue())
  File "/usr/lib/python2.3/StringIO.py", line 203, in getvalue
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 63: ordinal not in range(128)

searchHighlight()

This is apparently another unicode problem, and is consistent across our trac installations. It also occurs on some other wiki pages.

At the same time, trac-admin wiki upgrade fails when trying to upgrade TracReports, and gets no further.

It reports an error when trying to decode a character - I'll look for the details and attache them when I can.

Our current trac version is 3130, and we are running on Suse Enterprise Linux 9, Apache 2.0.49, and mod-python.

Attachments (0)

Change History (13)

comment:1 Changed 11 years ago by Christian Boos

Keywords: unicode added
Milestone: 0.10
Owner: changed from Jonas Borgström to Christian Boos
Priority: normalhigh

I think I know why. From StringIO.py:

The StringIO object can accept either Unicode or 8-bit strings, but mixing the two may take some care. If both are used, 8-bit strings that cannot be interpreted as 7-bit ASCII (that use the 8th bit) will cause a UnicodeError to be raised when getvalue() is called.

You certainly have a macro which still produces output as str objects.

Could you try the following patch?

Index: formatter.py
===================================================================
--- formatter.py	(revision 3118)
+++ formatter.py	(working copy)
@@ -26,7 +26,7 @@
 from trac.core import *
 from trac.mimeview import *
 from trac.wiki.api import WikiSystem, IWikiChangeListener, IWikiMacroProvider
-from trac.util import shorten_line
+from trac.util import shorten_line, to_unicode
 from trac.util.markup import escape, Markup, Element, html
 
 __all__ = ['wiki_to_html', 'wiki_to_oneliner', 'wiki_to_outline', 'Formatter' ]
@@ -91,7 +91,8 @@
             if self.name in list(macro_provider.get_macros()):
                 self.env.log.debug('Executing Wiki macro %s by provider %s'
                                    % (self.name, macro_provider))
-                return macro_provider.render_macro(req, self.name, text)
+                return to_unicode(macro_provider.render_macro(req, self.name,
+                                                              text))
 
     def _mimeview_processor(self, req, text):
         return Mimeview(self.env).render(req, self.name, text)

comment:2 Changed 11 years ago by pacopablo

Cc: pacopablo@… added

I ran into this error too. You patch fixed it cboos. However, your last change [3138] broke format_datetime. You forgot to specify the module name when calling getlocale. Here is the patch

Index: util/__init__.py
===================================================================
--- util/__init__.py    (revision 3138)
+++ util/__init__.py    (working copy)
@@ -208,7 +208,7 @@
             t = time.localtime(int(t))
 
     text = time.strftime(format, t)
-    lc_time_encoding = sys.platform != 'win32' and getlocale(locale.LC_TIME)[1]
+    lc_time_encoding = sys.platform != 'win32' and locale.getlocale(locale.LC_TIME)[1]
     encoding = lc_time_encoding or locale.getpreferredencoding()
     return unicode(text, encoding, 'replace')

comment:3 Changed 11 years ago by tjb@…

Cc: tjb@… added

I updated to [3139], and applied both of the above patches.
The result is still the same, with an identical traceback to the original.
Running trac-admin wiki upgrade still fails as well.

comment:4 Changed 11 years ago by Christian Boos

Status: newassigned

Please upgrade to [3141] and try again…

comment:5 Changed 11 years ago by tjb@…

I upgraded to 3141, and had effectively no change. Some of the line numbers have changed, but the stack trace is the same.

I am leaving work in 30 minutes, and won't be back for 7 days - therefore cannot do any more testing in the short term. Sorry - I'll check the ticket from time to time at home.

comment:6 Changed 11 years ago by Christian Boos

Would be nice to post at least the stack trace, if you can.

comment:7 Changed 11 years ago by tjb@…

Here it is

Python Traceback

Traceback (most recent call last):
  File "/usr/lib/python2.3/site-packages/trac/web/main.py", line 301, in dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.3/site-packages/trac/web/main.py", line 177, in dispatch
    resp = chosen_handler.process_request(req)
  File "/usr/lib/python2.3/site-packages/trac/wiki/web_ui.py", line 117, in process_request
    self._render_view(req, db, page)
  File "/usr/lib/python2.3/site-packages/trac/wiki/web_ui.py", line 364, in _render_view
    req.hdf['wiki.page_html'] = wiki_to_html(page.text, self.env, req)
  File "/usr/lib/python2.3/site-packages/trac/wiki/formatter.py", line 827, in wiki_to_html
    return Markup(out.getvalue())
  File "/usr/lib/python2.3/StringIO.py", line 203, in getvalue
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 63: ordinal not in range(128)

comment:8 Changed 11 years ago by tjb@…

I updated to [3222] today, tried the wiki upgrade, and it works. The wiki pages display normally as well. I don't know what might have changed to fix this over the last week, my skim of the timeline showed nothing obvious.

The return to_unicode(…) patch is still in place.

I will leave it to you to close this if you are happy - thanks.

comment:9 Changed 11 years ago by Christian Boos

Resolution: fixed
Status: assignedclosed

I'm quite sure it was [3141] which fixed it… maybe a wrong install step on your part at that time. Well, never mind, now it works :)

The patch adding to_unicode() in _macro_processor is not necessary anymore, as this is now done at a later stage (in process()). You should revert it.

I think we can close this one now.

comment:10 Changed 10 years ago by anonymous

Cc: matt_tricks@… added
Milestone: 0.100.11
Resolution: fixed
Status: closedreopened

Hi guys,

I'm still seeing this in the latest 0.11dev subversion code.

Any ideas?

I also get UnicodeDecodeErrors in my wiki nder some of the help pages. Perhaps my setup is wrong but I can't for the life of me think how.

See my comments under {3908} for more details including my version info.

Cheers, Matt

comment:11 Changed 10 years ago by anonymous

Err, of course I meant to link to the ticket number: #3908

comment:12 in reply to:  11 Changed 10 years ago by anonymous

Got this error when trying to access the TracReports wiki page

comment:13 in reply to:  10 Changed 9 years ago by Christian Boos

Milestone: 0.11.10.10
Resolution: fixed
Status: reopenedclosed

Replying to anonymous:

Hi guys, … See my comments under {3908} for more details including my version info.

So no need to keep this one reopened as well. Restoring status for 0.10.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Christian Boos.
The resolution will be deleted.
to The owner will be changed from Christian Boos to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .

 
Note: See TracTickets for help on using tickets.