Loading the timeline causes a Genshi UnicodeDecodeError without trace.

[Jérôme Poulin <jeromepoulin@…>]

When I click the Timeline on my project I get displayed: Genshi UnicodeDecodeError error while rendering template 'None'

I am a TRAC_ADMIN and do not get any backtrace.

The logfile isn't much useful neither;

2012-12-10 22:56:22,527 Trac[authz_policy] DEBUG: Checking BROWSER_VIEW on *:*@*
2012-12-10 22:56:22,533 Trac[authz_policy] DEBUG: Checking WIKI_VIEW on wiki:AssemblyInfo@*
2012-12-10 22:56:22,533 Trac[authz_policy] DEBUG: wiki:AssemblyInfo@* matched section wiki:*@* for user jeromepoulin
2012-12-10 22:56:22,533 Trac[authz_policy] DEBUG: Checking WIKI_VIEW on wiki:AssemblyInfo@*
2012-12-10 22:56:22,533 Trac[authz_policy] DEBUG: wiki:AssemblyInfo@* matched section wiki:*@* for user jeromepoulin
2012-12-10 22:56:22,535 Trac[authz_policy] DEBUG: Checking WIKI_CREATE on wiki:AssemblyInfo@*
2012-12-10 22:56:22,535 Trac[authz_policy] DEBUG: wiki:AssemblyInfo@* matched section wiki:*@* for user jeromepoulin
2012-12-10 22:56:22,536 Trac[authz_policy] DEBUG: Checking WIKI_CREATE on wiki:AssemblyInfo@*
2012-12-10 22:56:22,536 Trac[authz_policy] DEBUG: wiki:AssemblyInfo@* matched section wiki:*@* for user jeromepoulin
2012-12-10 22:56:22,577 Trac[main] WARNING: [2001:470:b010:1::10] HTTPInternalError: 500 Trac Error (Genshi UnicodeDecodeError error while rendering template 'None', line -1, char -1)
2012-12-10 22:56:23,163 Trac[env] INFO: Reloading environment due to configuration change
2012-12-10 22:56:23,179 Trac[env] INFO: -------------------------------- environment startup [Trac 1.0.1dev-r11480] --------------------------------
2012-12-10 22:56:23,187 Trac[main] DEBUG: Dispatching <RequestWithSession "GET '/chrome/tracwysiwyg/wysiwyg-load.js'">
2012-12-10 22:56:23,188 Trac[session] DEBUG: Retrieving session for ID 'jeromepoulin'
2012-12-10 22:56:23,199 Trac[loader] DEBUG: Loading datefield.filter from /usr/local/lib/python2.7/dist-packages/TracDateField-1.1.0_r12118-py2.7.egg
2012-12-10 22:56:23,208 Trac[loader] DEBUG: Loading ldapplugin.api from /var/trac/projects/cauca/plugins/LdapPlugin-0.7.0dev_r11019-py2.7.egg
2012-12-10 22:56:23,208 Trac[loader] DEBUG: Loading simplemultiproject from /var/trac/projects/cauca/plugins/SimpleMultiProject-0.0.3dev_r12068-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.about from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.admin.console from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.admin.web_ui from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.attachment from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.db.mysql from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.db.postgres from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,210 Trac[loader] DEBUG: Loading trac.db.sqlite from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg

Where can I look at to debug furthermore ?

[Jérôme Poulin <jeromepoulin@…>]

I removed a raise to get more infos:

Trac detected an internal error: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 26: ordinal not in range(128)

Most recent call last:

    File "/usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg/trac/web/main.py", line 497, in _dispatch_request
    File "/usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg/trac/web/main.py", line 233, in dispatch
    File "/usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg/trac/web/chrome.py", line 1001, in render_template
    File "/usr/lib/python2.7/dist-packages/genshi/core.py", line 183, in render
    File "/usr/lib/python2.7/dist-packages/genshi/output.py", line 58, in encode
    File "/usr/lib/python2.7/dist-packages/genshi/output.py", line 339, in __call__
    File "/usr/lib/python2.7/dist-packages/genshi/output.py", line 826, in __call__
    File "/usr/lib/python2.7/dist-packages/genshi/output.py", line 670, in __call__
    File "/usr/lib/python2.7/dist-packages/genshi/output.py", line 783, in __call__ 

System Information:

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:16.0) Gecko/20100101 Firefox/16.0
Trac 		1.0.1dev-r11480
Babel 		0.9.6
Docutils 	0.8.1
Genshi 		0.6 (with speedups)
GIT 		1.7.10.4
mod_wsgi 	3.3 (WSGIProcessGroup WSGIApplicationGroup %{GLOBAL})
MySQL 	server: "5.5.28-1", client: "5.5.28", thread-safe: 1
MySQLdb 	1.2.3
Pygments 	1.5
Python 		2.7.3rc2 (default, Apr 22 2012, 22:50:19) [GCC 4.6.3]
pytz 		2012c
setuptools 	0.6
Subversion 	1.6.17 (r1128011)
jQuery		1.7.2

Enabled Plugins:

authz-policy 	N/A 			/var/trac/projects/cauca/plugins/authz_policy.pyc
LdapPlugin 	0.7.0dev-r11019 	/var/trac/projects/cauca/plugins/LdapPlugin-0.7.0dev_r11019-py2.7.egg
MacrosCAUCA 	N/A 			/var/trac/projects/cauca/plugins/MacrosCAUCA.pyc
TracDateField 	1.1.0-r12118 		/usr/local/lib/python2.7/dist-packages/TracDateField-1.1.0_r12118-py2.7.egg
TracTocMacro 	11.0.0.3 		/usr/local/lib/python2.7/dist-packages/TracTocMacro-11.0.0.3-py2.7.egg
TracWysiwyg 	0.12.0.4-r11158 	/var/trac/projects/cauca/plugins/TracWysiwyg-0.12.0.4_r11158-py2.7.egg

[Jérôme Poulin <jeromepoulin@…>]

The main problem is that I didn't get the stack trace at first, here I found that the "dispatcher" variable contain \xc3\xa9 utf-8 character from a git branch name.

I guess this should not crash but it does.

Code fragment:

Line	
492	try:
493	if not env and env_error:
494	raise HTTPInternalError(env_error)
495	try:
496	dispatcher = RequestDispatcher(env)
497	dispatcher.dispatch(req)
498	except RequestDone:
499	pass
500	resp = req._response or []
501	except HTTPException, e:
502	_send_user_error(req, env, e)

Local variables:

Name		Value
dispatcher 	<trac.web.main.RequestDispatcher object at 0x7f5a9ae99450>
e 		UnicodeDecodeError('ascii', 'calcul_validation_coordonn\xc3\xa9', 26, 27, ...
env 		<trac.env.Environment object at 0x7f5a9a0a6e90>
env_error 	None
req 		<RequestWithSession "GET '/timeline'">
resp 		[]

[Jun Omae]

It seems an InstallationIssue.

MySQL database must be configured with utf8 charset and utf8_bin collation. Please see MySqlDb and check the configuration.

[Jérôme Poulin <jeromepoulin@…>]

This issue started as soon as someone published a GIT branch with a é in it.

The MySQL database has been setup according to the installation guide as well as Trac itself. All tables are utf8_bin.

This seems to be related to GIT or the GIT plugin, however trac.ini says git_fs_encoding = utf-8

There is also a problem in Trac as it does not display any backtrace when this error is encountered. This chunk of code is preventing a backtrace;

raise TracError(_("Genshi %(error)s error while rendering "
    "template %(location)s", 
    error=e.__class__.__name__, 
    location=location))

[Jun Omae]

Thanks for your investigation! I reproduced the original issue with unicode branches and tags. Would you please try t10980-unicode-branches-and-tags.diff​?

(We should add the unit tests.)

Also, I'll look into the error while rendering later.

[Jérôme Poulin <jeromepoulin@…>]

I confirm this patch fixes the problem, Thank you!

I didn't know where to look at in the code as it seems to take a bucket of data and dispatch it to multiple modules and I couldn't see the branch's name in the 'data' var.

[Ryan J Ollos <ryan.j.ollos@…>]

comment:1 indicates the user is running 1.0-stable, not 1.0dev.

[Remy Blank]

Jun, you don't need the iter() around the generator expressions. Parentheses are enough:

return ((self._fs_to_unicode(name), sha) 
        for name, sha in self.rev_cache.branch_dict)

[Jun Omae]

Sure! I just committed the patch with your suggestions in [11482-11483]. Thanks.