Edgewall Software
Modify ↓

Opened 13 years ago

Closed 13 years ago

Last modified 13 years ago

#10292 closed defect (fixed)

TypeError: unicode() argument 2 must be string, not bool

Reported by: trbs <trac@…> Owned by: Christian Boos
Priority: normal Milestone: 0.12.3
Component: timeline Version: 0.12-stable
Severity: normal Keywords:
Cc: Branch:
Release Notes:
API Changes:
Internal Changes:

Description

Error: 

TypeError: unicode() argument 2 must be string, not bool

The (most likely) cause of this was a commit in version control with the following text: 

#7777 zmid [[]]

Which trigged rendering a wiki link which failed.

Quick fix to have my team continue working and using Trac was to add a try-except in versioncontrol/web_ui/changeset.py:1011 .

This is not a proper solution but at least it keeps people working :)

Full traceback: 

Python Traceback
Most recent call last:

File "/usr/src/trac-0.12-stable/trac/timeline/templates/timeline.html", line 52, in <Expression u"event.render('description', context)">
  ${event.render('description', context)}
File "/usr/src/trac-0.12-stable/trac/timeline/web_ui.py", line 314, in <lambda>
  provider.render_timeline_event(context, field, event)
File "/usr/src/trac-0.12-stable/trac/versioncontrol/web_ui/changeset.py", line 1011, in render_timeline_event
  message)
File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1489, in format_to
  return format_to_oneliner(env, context, wikidom, **options)
File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1505, in format_to_oneliner
  return InlineHtmlFormatter(env, context, wikidom).generate(shorten)
File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1481, in generate
  shorten)
File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1342, in format
  result = re.sub(self.wikiparser.rules, self.replace, result)
File "/usr/lib/python2.6/re.py", line 151, in sub
  return _compile(pattern, 0).sub(repl, string, count)
File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1158, in replace
  replacement = self.handle_match(fullmatch)
File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1154, in handle_match
  return internal_handler(match, fullmatch)
File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 706, in _macrolink_formatter
  return self._lhref_formatter(match, fullmatch)
File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 540, in _lhref_formatter
  return self._make_lhref_link(match, fullmatch, rel, ns, target, label)
File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 581, in _make_lhref_link
  fullmatch)
File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 589, in _make_link
  return resolver(self, ns, target, escape(label, False),

System Information:

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0
Trac 	0.12.3dev-r10705
Babel 	0.9.6
Bitten 	0.7dev
Docutils 	0.7
Genshi 	0.6
Mercurial 	1.8.3
mod_wsgi 	3.3 (WSGIProcessGroup tracmbuyu WSGIApplicationGroup trac.mbuyu.nl|)
psycopg2 	2.2.1
Pygments 	1.3.1
Python 	2.6.6 (r266:84292, Dec 26 2010, 22:48:11) [GCC 4.4.5]
pytz 	2011h
RPC 	1.1.2-r9970
setuptools 	0.6
jQuery	1.4.4

Attachments (0)

Change History (16)

comment:1 by Christian Boos, 13 years ago

Weird, the stack trace doesn't seem to correspond to the exception. Can you double check in your trac.log? I'm curious to see where this wrong unicode() call is made, probably in TracMercurial plugin… but then, as I said already this morning at the occasion of another bug report, be sure to use the latest version of the plugin as most unicode errors there are now fixed.

in reply to:  1 ; comment:2 by Christian Boos, 13 years ago

Replying to cboos:

Weird, the stack trace doesn't seem to correspond to the exception.

… unless for very strange reasons, your escape() actually corresponds to unicode()? (seeing ... escape(label, False), ...).

escape here should be genshi.core.escape…

comment:3 by trbs <trac@…>, 13 years ago

This is the stacktrace from trac.log. 

2011-07-29 15:46:16,350 Trac[main] ERROR: Internal Server Error: 
Traceback (most recent call last):
  File "/usr/src/trac-0.12-stable/trac/web/main.py", line 522, in _dispatch_request
    dispatcher.dispatch(req)
  File "/usr/src/trac-0.12-stable/trac/web/main.py", line 264, in dispatch
    content_type)
  File "/usr/src/trac-0.12-stable/trac/web/chrome.py", line 870, in render_template
    encoding='utf-8')
  File "/usr/lib/pymodules/python2.6/genshi/core.py", line 183, in render
    return encode(generator, method=method, encoding=encoding, out=out)
  File "/usr/lib/pymodules/python2.6/genshi/output.py", line 58, in encode
    for chunk in iterator:
  File "/usr/lib/pymodules/python2.6/genshi/output.py", line 339, in __call__
    for kind, data, pos in stream:
  File "/usr/lib/pymodules/python2.6/genshi/output.py", line 826, in __call__
    for kind, data, pos in stream:
  File "/usr/lib/pymodules/python2.6/genshi/output.py", line 670, in __call__
    for kind, data, pos in stream:
  File "/usr/lib/pymodules/python2.6/genshi/output.py", line 771, in __call__
    for kind, data, pos in chain(stream, [(None, None, None)]):
  File "/usr/lib/pymodules/python2.6/genshi/output.py", line 586, in __call__
    for ev in stream:
  File "/usr/lib/pymodules/python2.6/genshi/core.py", line 288, in _ensure
    for event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/core.py", line 288, in _ensure
    for event in stream:
  File "/usr/src/trac-0.12-stable/trac/web/chrome.py", line 973, in _generate
    for kind, data, pos in stream:
  File "/usr/lib/pymodules/python2.6/genshi/core.py", line 288, in _ensure
    for event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/core.py", line 288, in _ensure
    for event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/filters/transform.py", line 686, in _unmark
    for mark, event in stream:
  File "/usr/src/timingandestimationplugin-0.12/timingandestimationplugin/tande_filters.py", line 95, in __call__
    for mark, event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/filters/transform.py", line 714, in __call__
    for mark, event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/filters/transform.py", line 682, in _mark
    for event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/template/base.py", line 605, in _include
    for event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/template/markup.py", line 378, in _match
    ctxt, start=idx + 1, **vars):
  File "/usr/lib/pymodules/python2.6/genshi/template/markup.py", line 378, in _match
    ctxt, start=idx + 1, **vars):
  File "/usr/lib/pymodules/python2.6/genshi/template/markup.py", line 327, in _match
    for event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/template/base.py", line 545, in _flatten
    for kind, data, pos in stream:
  File "/usr/lib/pymodules/python2.6/genshi/core.py", line 288, in _ensure
    for event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/path.py", line 588, in _generate
    subevent = next()
  File "/usr/lib/pymodules/python2.6/genshi/template/base.py", line 605, in _include
    for event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/template/markup.py", line 316, in _strip
    event = next()
  File "/usr/lib/pymodules/python2.6/genshi/template/base.py", line 545, in _flatten
    for kind, data, pos in stream:
  File "/usr/lib/pymodules/python2.6/genshi/core.py", line 288, in _ensure
    for event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/path.py", line 588, in _generate
    subevent = next()
  File "/usr/lib/pymodules/python2.6/genshi/template/base.py", line 605, in _include
    for event in stream:
  File "/usr/lib/pymodules/python2.6/genshi/template/markup.py", line 316, in _strip
    event = next()
  File "/usr/lib/pymodules/python2.6/genshi/template/base.py", line 565, in _flatten
    result = _eval_expr(data, ctxt, vars)
  File "/usr/lib/pymodules/python2.6/genshi/template/base.py", line 277, in _eval_expr
    retval = expr.evaluate(ctxt)
  File "/usr/lib/pymodules/python2.6/genshi/template/eval.py", line 178, in evaluate
    return eval(self.code, _globals, {'__data__': data})
  File "/usr/src/trac-0.12-stable/trac/timeline/templates/timeline.html", line 52, in <Expression u"event.render('description', context)">
    ${event.render('description', context)}
  File "/usr/src/trac-0.12-stable/trac/timeline/web_ui.py", line 314, in <lambda>
    provider.render_timeline_event(context, field, event)
  File "/usr/src/trac-0.12-stable/trac/versioncontrol/web_ui/changeset.py", line 1011, in render_timeline_event
    markup += format_to(self.env, None, context(cset_resource),
  File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1489, in format_to
    return format_to_oneliner(env, context, wikidom, **options)
  File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1505, in format_to_oneliner
    return InlineHtmlFormatter(env, context, wikidom).generate(shorten)
  File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1481, in generate
    shorten)
  File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1342, in format
    result = re.sub(self.wikiparser.rules, self.replace, result)
  File "/usr/lib/python2.6/re.py", line 151, in sub
    return _compile(pattern, 0).sub(repl, string, count)
  File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1158, in replace
    replacement = self.handle_match(fullmatch)
  File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 1154, in handle_match
    return internal_handler(match, fullmatch)
  File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 706, in _macrolink_formatter
    return self._lhref_formatter(match, fullmatch)
  File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 540, in _lhref_formatter
    return self._make_lhref_link(match, fullmatch, rel, ns, target, label)
  File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 581, in _make_lhref_link
    fullmatch)
  File "/usr/src/trac-0.12-stable/trac/wiki/formatter.py", line 589, in _make_link
    return resolver(self, ns, target, escape(label, False),
TypeError: unicode() argument 2 must be string, not bool

in reply to:  1 comment:4 by trbs <trac@…>, 13 years ago

Replying to cboos:

Weird, the stack trace doesn't seem to correspond to the exception. Can you double check in your trac.log? I'm curious to see where this wrong unicode() call is made, probably in TracMercurial plugin… but then, as I said already this morning at the occasion of another bug report, be sure to use the latest version of the plugin as most unicode errors there are now fixed.

After applying the work around I updated trac to latest 0.12-stable and I also updated the Mercurial Plugin.

When I remove the try-except clause it will throws the unicode() error.

This is while running: 

Powered by Trac 0.12.3dev-r10705
By Edgewall Software.

And 

TracMercurial 	0.12.0.28dev-r10748 	/usr/src/mercurial-plugin

in reply to:  2 comment:5 by trbs <trac@…>, 13 years ago

Replying to cboos:

Replying to cboos:

Weird, the stack trace doesn't seem to correspond to the exception.

… unless for very strange reasons, your escape() actually corresponds to unicode()? (seeing ... escape(label, False), ...).

escape here should be genshi.core.escape…

As far as I can see this is true. There are no modifications to the formatter.py file. And the only declaration of escape if from the genshi import. 

In [1]: import genshi.core

In [2]: import trac.wiki.formatter

In [3]: trac.wiki.formatter.escape == genshi.core.escape
Out[3]: True

comment:6 by Christian Boos, 13 years ago

… and what about genshi.core.escape('"<test>"', False)?

comment:7 by Christian Boos, 13 years ago

… and also, add a self.log.debug("escape: %r", escape) in your except: clause for example.

in reply to:  6 comment:8 by trbs <trac@…>, 13 years ago

Replying to cboos:

… and what about genshi.core.escape('"<test>"', False)?

In IPython: 

In [6]: genshi.core.escape('"<test>"', False)
Out[6]: <Markup u'"&lt;test&gt;"'>

And when I insert Raise Exception(escape('"<test>"', False)) in formatter.py:571 

Trac detected an internal error:

Exception: "&lt;test&gt;"

So that seems to work properly…

in reply to:  7 ; comment:9 by trbs <trac@…>, 13 years ago

Replying to cboos:

… and also, add a self.log.debug("escape: %r", escape) in your except: clause for example.

Output of self.env.log.debug("escape: %r", escape) : 

2011-07-29 17:01:50,906 Trac[formatter] DEBUG: escape: <built-in method escape of type object at 0x7fef24fe5ac0>

in reply to:  9 ; comment:10 by Christian Boos, 13 years ago

Replying to trbs <trac@…>:

Replying to cboos:

… and also, add a self.log.debug("escape: %r", escape) in your except: clause for example.

Output of self.env.log.debug("escape: %r", escape) : 

2011-07-29 17:01:50,906 Trac[formatter] DEBUG: escape: <built-in method escape of type object at 0x7fef24fe5ac0>

Hm, slowly getting clueless… And what if you'd do: 

self.env.log.debug("label: %r", label) 
self.env.log.debug("escape(label, False): %r", escape(label, False))

Do you use the Genshi _speedups by any chance? (check in your process /proc/…/maps file for a _speedups.so entry).

in reply to:  10 comment:11 by trbs <trac@…>, 13 years ago

Replying to cboos:

Replying to trbs <trac@…>:

Replying to cboos:

… and also, add a self.log.debug("escape: %r", escape) in your except: clause for example.

Output of self.env.log.debug("escape: %r", escape) : 

2011-07-29 17:01:50,906 Trac[formatter] DEBUG: escape: <built-in method escape of type object at 0x7fef24fe5ac0>

Hm, slowly getting clueless… And what if you'd do: 

self.env.log.debug("label: %r", label) 
self.env.log.debug("escape(label, False): %r", escape(label, False))

Do you use the Genshi _speedups by any chance? (check in your process /proc/…/maps file for a _speedups.so entry).

Yes I am using _speedups.so

And disabling that fixes the problem :)

(It's always terrible when stacktraces do not make sense)

The output of rendering now becomes: 

    2.11.6 #7777 zmid

I think this is still has a little problem. As the changeset log message was: 

    2.11.6 #7777 zmid [[]]

These brackets here do have meaning and it would make sense to me that if it's not renderable as a wiki link that it would be printed as-is.

comment:12 by trbs <trac@…>, 13 years ago

Looking at the _speedups.c to see where the underlying problem could be.

In the traceback page of trac it's clear that escape is called with a None value. So the test case is basically: 

# With _speedups.so:
>>> genshi.core.escape(None, False)
<Markup u''>

# Without _speedups.so:
>>> genshi.core.escape(None, False)
TypeError: unicode() argument 2 must be string, not bool

comment:13 by trbs <trac@…>, 13 years ago

Tracked it down to _speedups.c:Markup_escape() there it does type->tp_new(type, args, NULL);

But not exactly sure why that triggers the exception…

Might have something to do with the 'workaround for quird in visual studio' or something in creating a new PyUnicode_Type in general ?

in reply to:  13 ; comment:14 by Christian Boos, 13 years ago

Owner: set to Christian Boos

Replying to trbs <trac@…>:

Tracked it down to _speedups.c:Markup_escape() there it does type->tp_new(type, args, NULL);

You nailed it!

But not exactly sure why that triggers the exception…

It's simply that args shouldn't be used here, the arguments for escape shouldn't be propagated to the Markup new function but instead we should use a 0-tuple. I've filed ​#G439 for that, but we need also a workaround in Trac for coping with Genshi 0.6.

I agree that it doesn't make sense to interpret [[]] as a link, so I'll try to modify the regexp.

in reply to:  14 comment:15 by Christian Boos, 13 years ago

Resolution: → fixed
Status: new → closed

I agree that it doesn't make sense to interpret [[]] as a link, so I'll try to modify the regexp.

Done in r10776.

comment:16 by anonymous, 13 years ago

I have a page with 

[[]]

in it and I can't go to it or edit it (if I manually add ?action=edit to the url)

Is there a way I can fix this page?

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Christian Boos.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Christian Boos to the specified user.

Add Comment
E-mail address and name can be saved in the Preferences .
Attachments ↑ Description ↑
 
Note: See TracTickets for help on using tickets.