Edgewall Software
Modify

Opened 8 months ago

Closed 3 months ago

Last modified 3 months ago

#13253 closed defect (fixed)

UnicodeEncodeError raised while rendering timeline rss when author of event is unknown and unicode

Reported by: Jun Omae Owned by:
Priority: normal Milestone: 1.0.20
Component: timeline Version:
Severity: normal Keywords: genshi
Cc: Branch:
Release Notes:

Fix UnicodeEncodeError raised while rendering timeline rss when event's author is unknown and unicode.

API Changes:
Internal Changes:

Description

I noticed UnicodeEncodeError raised while rendering timeline rss on production environment.

2020-02-12 18:42:48,520 ERROR: Genshi UnicodeEncodeError error while rendering template (unknown template location)
Traceback (most recent call last):
  File "/venv/lib/python2.7/site-packages/trac/web/chrome.py", line 1180, in iterable_content
    for chunk in stream.serialize(method, **kwargs):
  File "/venv/lib/python2.7/site-packages/genshi/output.py", line 241, in __call__
    for kind, data, pos in stream:
  File "/venv/lib/python2.7/site-packages/genshi/output.py", line 671, in __call__
    for kind, data, pos in stream:
  File "/venv/lib/python2.7/site-packages/genshi/output.py", line 776, in __call__
    for kind, data, pos in chain(stream, [(None, None, None)]):
  File "/venv/lib/python2.7/site-packages/genshi/output.py", line 596, in __call__
    for ev in stream:
  File "/venv/lib/python2.7/site-packages/genshi/core.py", line 292, in _ensure
    for event in stream:
  File "/venv/lib/python2.7/site-packages/genshi/core.py", line 292, in _ensure
    for event in stream:
  File "/venv/lib/python2.7/site-packages/genshi/template/base.py", line 639, in _include
    for event in stream:
  File "/venv/lib/python2.7/site-packages/genshi/template/markup.py", line 327, in _match
    for event in stream:
  File "/venv/lib/python2.7/site-packages/genshi/template/base.py", line 579, in _flatten
    for kind, data, pos in stream:
  File "/venv/lib/python2.7/site-packages/genshi/template/directives.py", line 726, in __call__
    value = _eval_expr(expr, ctxt, vars)
  File "/venv/lib/python2.7/site-packages/genshi/template/base.py", line 289, in _eval_expr
    retval = expr.evaluate(ctxt)
  File "/venv/lib/python2.7/site-packages/genshi/template/eval.py", line 178, in evaluate
    return eval(self.code, _globals, {'__data__': data})
  File "/venv/lib/python2.7/site-packages/trac/templates/author_or_creator.rss", line 16, in <Expression '?'>
    <py:with vars="author = email_map[author] if show_email_addresses and email_map and '@' not in author else author">
  File "/venv/lib/python2.7/site-packages/genshi/template/eval.py", line 338, in lookup_item
    val = getattr(obj, key, UNDEFINED)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-15: ordinal not in range(128)

The issue occurs when email_map[author] expression is evaluated in genshi template and author is not existent in email_map dict.

How to reproduce same exception

>>> from genshi.template import MarkupTemplate
>>> t = MarkupTemplate('<x xmlns:py="http://genshi.edgewall.org/" py:with="v=d[k]" />')
>>> unicode(t.generate(d={'key': 42}, k=u'\u200b'*16))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/venv/lib/python2.7/site-packages/genshi/core.py", line 251, in __unicode__
    return self.render(encoding=None)
  File "/venv/lib/python2.7/site-packages/genshi/core.py", line 184, in render
    return encode(generator, method=method, encoding=encoding, out=out)
  File "/venv/lib/python2.7/site-packages/genshi/output.py", line 57, in encode
    return _encode(''.join(list(iterator)))
  File "/venv/lib/python2.7/site-packages/genshi/output.py", line 241, in __call__
    for kind, data, pos in stream:
  File "/venv/lib/python2.7/site-packages/genshi/output.py", line 671, in __call__
    for kind, data, pos in stream:
  File "/venv/lib/python2.7/site-packages/genshi/output.py", line 776, in __call__
    for kind, data, pos in chain(stream, [(None, None, None)]):
  File "/venv/lib/python2.7/site-packages/genshi/output.py", line 596, in __call__
    for ev in stream:
  File "/venv/lib/python2.7/site-packages/genshi/core.py", line 274, in _ensure
    event = stream.next()
  File "/venv/lib/python2.7/site-packages/genshi/template/base.py", line 639, in _include
    for event in stream:
  File "/venv/lib/python2.7/site-packages/genshi/template/markup.py", line 327, in _match
    for event in stream:
  File "/venv/lib/python2.7/site-packages/genshi/template/base.py", line 579, in _flatten
    for kind, data, pos in stream:
  File "/venv/lib/python2.7/site-packages/genshi/template/directives.py", line 726, in __call__
    value = _eval_expr(expr, ctxt, vars)
  File "/venv/lib/python2.7/site-packages/genshi/template/base.py", line 289, in _eval_expr
    retval = expr.evaluate(ctxt)
  File "/venv/lib/python2.7/site-packages/genshi/template/eval.py", line 178, in evaluate
    return eval(self.code, _globals, {'__data__': data})
  File "<string>", line 1, in <Expression '?'>
  File "/venv/lib/python2.7/site-packages/genshi/template/eval.py", line 338, in lookup_item
    val = getattr(obj, key, UNDEFINED)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-15: ordinal not in range(128)

Proposed changes

  • trac/templates/author_or_creator.rss

    diff --git a/trac/templates/author_or_creator.rss b/trac/templates/author_or_creator.rss
    index e0a21bf90..25af76675 100644
    a b Arguments:  
    1313     py:with="email_map = value_of('email_map', None)" py:strip="">
    1414  <py:if test="author">
    1515    <!--! Try our best to retrieve an email address if wanted and possible -->
    16     <py:with vars="author = email_map[author] if show_email_addresses and email_map and '@' not in author else author">
     16    <py:with vars="author = (email_map.get(author, author)
     17                             if show_email_addresses and email_map and '@' not in author
     18                             else author)">
    1719      <py:choose>
    1820        <author py:when="show_email_addresses and '@' in author">${format_author(author)}</author>
    1921        <dc:creator py:otherwise="">${format_author(author)}</dc:creator>
  • trac/timeline/tests/web_ui.py

    diff --git a/trac/timeline/tests/web_ui.py b/trac/timeline/tests/web_ui.py
    index 8325198f3..4d24dbdf2 100644
    a b class TimelineModuleTestCase(unittest.TestCase):  
    120120
    121121        self.assertIn('prev', req.chrome['links'])
    122122
     123    def test_render_rss_with_unicode_unknown_author(self):
     124        self.env.config.set('trac', 'show_email_addresses', 'enabled')
     125        self.env.known_users.append(('blah', 'Blah user', 'blah@example.org'))
     126        req = MockRequest(self.env, path_info='/timeline',
     127                          args={'format': 'rss'})
     128        rv = TimelineModule(self.env).process_request(req)
     129        self.assertEqual('timeline.rss', rv[0])
     130        event = ('mock', datetime_now(utc), u'jöé', None)
     131        rv[1]['events'] = [
     132            {'kind': event[0], 'date': event[1], 'author': event[2],
     133             'data': event[3], 'event': event, 'provider': None,
     134             'dateuid': '42', 'render': lambda field, context: 'mock'},
     135        ]
     136        output = Chrome(self.env).render_template(req, rv[0], rv[1], rv[2])
     137        self.assertIsNotNone(output)
     138
    123139
    124140def suite():
    125141    suite = unittest.TestSuite()

Attachments (0)

Change History (5)

comment:1 by Ryan J Ollos, 5 months ago

Milestone: 1.0.20next-stable-1.4.x

Please move back to 1.0.20 if you wish to fix for release 1.0.20 (#13294).

comment:2 by Ryan J Ollos, 4 months ago

Milestone: next-stable-1.4.x1.4.3

The change looks good to me. Feel free to move to 1.4.2 if you want to include the change in the upcoming release.

Last edited 4 months ago by Ryan J Ollos (previous) (diff)

comment:3 by Jun Omae, 3 months ago

Milestone: 1.4.31.0.20

Sorry for my delay.

Proposed fix is depended on Genshi. I'll push to 1.0-stable, also I'm going to confirm whether the same issue is existent on 1.4-stable.

comment:4 by Jun Omae, 3 months ago

Release Notes: modified (diff)
Resolution: fixed
Status: newclosed

Fixed in [17413] and merged in [17414] (only tests for 1.2-stable).

1.4-stable doesn't have the issue. Also, 1.2-stable doesn't have it because email_map[author] is not evaluated on author_or_creator.css.

comment:5 by Jun Omae, 3 months ago

The root cause is a Genshi issue. I've filed at https://github.com/edgewall/genshi/issues/28.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The ticket will remain with no owner.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from (none) to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.