Edgewall Software
Modify

Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#12536 closed defect (fixed)

TypeError is raised while rendering wiki text on Genshi 0.6 with speedups

Reported by: Jun Omae Owned by: Jun Omae
Priority: normal Milestone: 1.0.13
Component: wiki system Version:
Severity: normal Keywords:
Cc: Branch:
Release Notes:

Add workaround for genshi.core.escape raising TypeError due to genshi:#439.

API Changes:
Internal Changes:

Description (last modified by Jun Omae)

Wiki including [[search:|""]] leads a TypeError on Genshi 0.6 with speedups. Root cause of this issue is genshi.core.escape(). I got the same issue on my production environment.

>>> import genshi
>>> genshi.__version__
'0.6'
>>> from genshi.core import Markup, escape
>>> Markup, escape
(<type 'genshi._speedups.Markup'>, <built-in method escape of type object at 0x7f5d42f5fb60>)
>>> escape('', False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unicode() argument 2 must be string, not bool
03:34:19 PM Trac[main] ERROR: Internal Server Error: <RequestWithSession "POST '/newticket'">, referrer 'http://27.local:3000/1.0-sqlite/newticket'
Traceback (most recent call last):
  File "<venv>/local/lib/python2.7/site-packages/trac/web/main.py", line 562, in _dispatch_request
    dispatcher.dispatch(req)
  File "<venv>/local/lib/python2.7/site-packages/trac/web/main.py", line 269, in dispatch
    iterable=chrome.use_chunked_encoding)
  File "<venv>/local/lib/python2.7/site-packages/trac/web/chrome.py", line 1115, in render_template
    encoding='utf-8')
  File "<venv>/local/lib/python2.7/site-packages/genshi/core.py", line 183, in render
    return encode(generator, method=method, encoding=encoding, out=out)
  File "<venv>/local/lib/python2.7/site-packages/genshi/output.py", line 58, in encode
    for chunk in iterator:
  File "<venv>/local/lib/python2.7/site-packages/genshi/output.py", line 339, in __call__
    for kind, data, pos in stream:
  File "<venv>/local/lib/python2.7/site-packages/genshi/output.py", line 826, in __call__
    for kind, data, pos in stream:
  File "<venv>/local/lib/python2.7/site-packages/genshi/output.py", line 670, in __call__
    for kind, data, pos in stream:
  File "<venv>/local/lib/python2.7/site-packages/genshi/output.py", line 771, in __call__
    for kind, data, pos in chain(stream, [(None, None, None)]):
  File "<venv>/local/lib/python2.7/site-packages/genshi/output.py", line 586, in __call__
    for ev in stream:
  File "<venv>/local/lib/python2.7/site-packages/genshi/core.py", line 288, in _ensure
    for event in stream:
  File "<venv>/local/lib/python2.7/site-packages/genshi/core.py", line 288, in _ensure
    for event in stream:
  File "<venv>/local/lib/python2.7/site-packages/trac/web/chrome.py", line 1310, in _generate
    for kind, data, pos in stream:
  File "<venv>/local/lib/python2.7/site-packages/genshi/core.py", line 288, in _ensure
    for event in stream:
  File "<venv>/local/lib/python2.7/site-packages/genshi/template/base.py", line 605, in _include
    for event in stream:
  File "<venv>/local/lib/python2.7/site-packages/genshi/template/markup.py", line 327, in _match
    for event in stream:
  File "<venv>/local/lib/python2.7/site-packages/genshi/template/base.py", line 565, in _flatten
    result = _eval_expr(data, ctxt, vars)
  File "<venv>/local/lib/python2.7/site-packages/genshi/template/base.py", line 277, in _eval_expr
    retval = expr.evaluate(ctxt)
  File "<venv>/local/lib/python2.7/site-packages/genshi/template/eval.py", line 178, in evaluate
    return eval(self.code, _globals, {'__data__': data})
  File "<venv>/local/lib/python2.7/site-packages/trac/ticket/templates/ticket_box.html", line 124, in <Expression u'wiki_to_html(context, ticket.description, escape_newlines=preserve_newlines)'>
    ${wiki_to_html(context, ticket.description, escape_newlines=preserve_newlines)}
  File "<venv>/local/lib/python2.7/site-packages/trac/wiki/formatter.py", line 1586, in format_to_html
    return HtmlFormatter(env, context, wikidom).generate(escape_newlines)
  File "<venv>/local/lib/python2.7/site-packages/trac/wiki/formatter.py", line 1541, in generate
    escape_newlines)
  File "<venv>/local/lib/python2.7/site-packages/trac/wiki/formatter.py", line 1323, in format
    result = re.sub(self.wikiparser.rules, self.replace, line)
  File "<venv>/lib/python2.7/re.py", line 151, in sub
    return _compile(pattern, flags).sub(repl, string, count)
  File "<venv>/local/lib/python2.7/site-packages/trac/wiki/formatter.py", line 1241, in replace
    replacement = self.handle_match(fullmatch)
  File "<venv>/local/lib/python2.7/site-packages/trac/wiki/formatter.py", line 1237, in handle_match
    return internal_handler(match, fullmatch)
  File "<venv>/local/lib/python2.7/site-packages/trac/wiki/formatter.py", line 782, in _macrolink_formatter
    return self._lhref_formatter(match, fullmatch)
  File "<venv>/local/lib/python2.7/site-packages/trac/wiki/formatter.py", line 618, in _lhref_formatter
    return self._make_lhref_link(match, fullmatch, rel, ns, target, label)
  File "<venv>/local/lib/python2.7/site-packages/trac/wiki/formatter.py", line 653, in _make_lhref_link
    fullmatch)
  File "<venv>/local/lib/python2.7/site-packages/trac/wiki/formatter.py", line 664, in _make_link
    return resolver(self, ns, target, escape(label, False))
TypeError: unicode() argument 2 must be string, not bool

The genshi.core.escape() crashes when first argument is None or an empty string. This issue has been filed in genshi:#439 and fixed on Genshi 0.7. However, we recommend Genshi 0.6 due to #11184 and the version still has it….

Then, I think we could detect this issue and fix up it like this. Thoughts?

  • trac/util/html.py

    diff --git a/trac/util/html.py b/trac/util/html.py
    index dd1357728..80d4b5a30 100644
    a b class TransposingElementFactory(ElementFactory):  
    291291html = TransposingElementFactory(str.lower)
    292292
    293293
     294try:
     295    escape('', False)  # detecting genshi:#439 on Genshi 0.6 with speedups...
     296except TypeError:
     297    _escape = escape
     298
     299    def escape(text, quotes=True):
     300        if text:
     301            return _escape(text, quotes=quotes)
     302        else:
     303            return Markup(u'')
     304
     305
    294306def plaintext(text, keeplinebreaks=True):
    295307    """Extract the text elements from (X)HTML content
    296308

Attachments (0)

Change History (9)

comment:1 by Jun Omae, 4 years ago

Description: modified (diff)

comment:2 by Ryan J Ollos, 4 years ago

That looks like a good workaround.

I noticed the error isn't seen when passing a keyword, so the following might be sufficient:

>>> try:
...   escape('', False)
... except TypeError:
...   _escape = escape
...   
...   def escape(text, quotes=True):
...     return _escape(text, quotes=quotes)
... 
>>> escape("''", False)
<Markup u"''">
>>> escape("''", True)
<Markup u"''">
>>> escape("", False)
<Markup u''>
>>> escape(None, True)
<Markup u'None'>
>>> escape(None, False)
<Markup u'None'>
>>> escape('"', True)
<Markup u'&#34;'>
>>> escape('"', False)
<Markup u'"'>
Last edited 4 years ago by Ryan J Ollos (previous) (diff)

comment:3 by Jun Omae, 4 years ago

Interested. However, escape(None, False) returns <Markup u''> on Genshi 0.7. I think work around should lead the same result for this, at least.

0.6 without speedups 0.6 with speedups 0.7 without speedups 0.7 with speedups
escape(None, False) <Markup u''> TypeError <Markup u''> <Markup u''>
escape('', False) <Markup u''> TypeError <Markup u''> <Markup u''>
escape(0, False) <Markup u''> TypeError <Markup u''> <Markup u''>
escape(1, False) AttributeError <Markup u'1'> AttributeError <Markup u'1'>
escape(False, False) <Markup u''> TypeError <Markup u''> <Markup u''>
escape(True, False) AttributeError <Markup u'True'> AttributeError <Markup u'True'>
escape('"', False) <Markup u'"'> <Markup u'"'> <Markup u'"'> <Markup u'"'>
escape('"', True) <Markup u'&#34;'> <Markup u'&#34;'> <Markup u'&#34;'> <Markup u'&#34;'>

comment:4 by Jun Omae, 4 years ago

Owner: set to Jun Omae
Status: newassigned

Proposed changes in jomae.git@t12536.

comment:5 by Ryan J Ollos, 4 years ago

The changes look good.

comment:6 by Jun Omae, 4 years ago

Thanks for the review. I'll push it after granting commit right to me for 1.2-stable branch.

comment:7 by Ryan J Ollos, 4 years ago

Okay, granted access rights just now.

comment:8 by Jun Omae, 4 years ago

Release Notes: modified (diff)
Resolution: fixed
Status: assignedclosed

Thanks. Committed in [14967] and merged to 1.2-stable and trunk in [14968:14969].

in reply to:  3 comment:9 by Christian Boos, 4 years ago

Replying to Jun Omae:

Interested. However, escape(None, False) returns <Markup u''> on Genshi 0.7.

Note how this also currently differs from escape in Trac/jinja2 branch:

Trac jinja2
escape(None, False) Markup(u'None')
escape('', False) Markup(u'')
escape(0, False) Markup(u'0')
escape(1, False) Markup(u'1')
escape(False, False) Markup(u'False')
escape(True, False) Markup(u'True')
escape('"', False) Markup(u'"')
escape('"', True) Markup(u'&#34;')

I somehow find that more consistent, but if we decide to favor backward compatibility, I could adapt it.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.