Edgewall Software
Modify

Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#10980 closed defect (fixed)

Loading the timeline causes a Genshi UnicodeDecodeError without trace.

Reported by: Jérôme Poulin <jeromepoulin@…> Owned by: Jun Omae
Priority: normal Milestone: 1.0.1
Component: version control Version: 1.0-stable
Severity: normal Keywords: git
Cc: Branch:
Release Notes:

Fixed unicode errors with non-ascii branches and tags in git repository.

API Changes:
Internal Changes:

Description

When I click the Timeline on my project I get displayed: Genshi UnicodeDecodeError error while rendering template 'None'

I am a TRAC_ADMIN and do not get any backtrace.

The logfile isn't much useful neither;

2012-12-10 22:56:22,527 Trac[authz_policy] DEBUG: Checking BROWSER_VIEW on *:*@*
2012-12-10 22:56:22,533 Trac[authz_policy] DEBUG: Checking WIKI_VIEW on wiki:AssemblyInfo@*
2012-12-10 22:56:22,533 Trac[authz_policy] DEBUG: wiki:AssemblyInfo@* matched section wiki:*@* for user jeromepoulin
2012-12-10 22:56:22,533 Trac[authz_policy] DEBUG: Checking WIKI_VIEW on wiki:AssemblyInfo@*
2012-12-10 22:56:22,533 Trac[authz_policy] DEBUG: wiki:AssemblyInfo@* matched section wiki:*@* for user jeromepoulin
2012-12-10 22:56:22,535 Trac[authz_policy] DEBUG: Checking WIKI_CREATE on wiki:AssemblyInfo@*
2012-12-10 22:56:22,535 Trac[authz_policy] DEBUG: wiki:AssemblyInfo@* matched section wiki:*@* for user jeromepoulin
2012-12-10 22:56:22,536 Trac[authz_policy] DEBUG: Checking WIKI_CREATE on wiki:AssemblyInfo@*
2012-12-10 22:56:22,536 Trac[authz_policy] DEBUG: wiki:AssemblyInfo@* matched section wiki:*@* for user jeromepoulin
2012-12-10 22:56:22,577 Trac[main] WARNING: [2001:470:b010:1::10] HTTPInternalError: 500 Trac Error (Genshi UnicodeDecodeError error while rendering template 'None', line -1, char -1)
2012-12-10 22:56:23,163 Trac[env] INFO: Reloading environment due to configuration change
2012-12-10 22:56:23,179 Trac[env] INFO: -------------------------------- environment startup [Trac 1.0.1dev-r11480] --------------------------------
2012-12-10 22:56:23,187 Trac[main] DEBUG: Dispatching <RequestWithSession "GET '/chrome/tracwysiwyg/wysiwyg-load.js'">
2012-12-10 22:56:23,188 Trac[session] DEBUG: Retrieving session for ID 'jeromepoulin'
2012-12-10 22:56:23,199 Trac[loader] DEBUG: Loading datefield.filter from /usr/local/lib/python2.7/dist-packages/TracDateField-1.1.0_r12118-py2.7.egg
2012-12-10 22:56:23,208 Trac[loader] DEBUG: Loading ldapplugin.api from /var/trac/projects/cauca/plugins/LdapPlugin-0.7.0dev_r11019-py2.7.egg
2012-12-10 22:56:23,208 Trac[loader] DEBUG: Loading simplemultiproject from /var/trac/projects/cauca/plugins/SimpleMultiProject-0.0.3dev_r12068-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.about from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.admin.console from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.admin.web_ui from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.attachment from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.db.mysql from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,209 Trac[loader] DEBUG: Loading trac.db.postgres from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg
2012-12-10 22:56:23,210 Trac[loader] DEBUG: Loading trac.db.sqlite from /usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg

Where can I look at to debug furthermore ?

Attachments (1)

t10980-unicode-branches-and-tags.diff (1.5 KB ) - added by Jun Omae 8 years ago.

Download all attachments as: .zip

Change History (12)

comment:1 by Jérôme Poulin <jeromepoulin@…>, 8 years ago

I removed a raise to get more infos:

Trac detected an internal error: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 26: ordinal not in range(128)

Most recent call last:

    File "/usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg/trac/web/main.py", line 497, in _dispatch_request
    File "/usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg/trac/web/main.py", line 233, in dispatch
    File "/usr/local/lib/python2.7/dist-packages/Trac-1.0.1dev_r11480-py2.7.egg/trac/web/chrome.py", line 1001, in render_template
    File "/usr/lib/python2.7/dist-packages/genshi/core.py", line 183, in render
    File "/usr/lib/python2.7/dist-packages/genshi/output.py", line 58, in encode
    File "/usr/lib/python2.7/dist-packages/genshi/output.py", line 339, in __call__
    File "/usr/lib/python2.7/dist-packages/genshi/output.py", line 826, in __call__
    File "/usr/lib/python2.7/dist-packages/genshi/output.py", line 670, in __call__
    File "/usr/lib/python2.7/dist-packages/genshi/output.py", line 783, in __call__ 

System Information:

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:16.0) Gecko/20100101 Firefox/16.0
Trac 		1.0.1dev-r11480
Babel 		0.9.6
Docutils 	0.8.1
Genshi 		0.6 (with speedups)
GIT 		1.7.10.4
mod_wsgi 	3.3 (WSGIProcessGroup WSGIApplicationGroup %{GLOBAL})
MySQL 	server: "5.5.28-1", client: "5.5.28", thread-safe: 1
MySQLdb 	1.2.3
Pygments 	1.5
Python 		2.7.3rc2 (default, Apr 22 2012, 22:50:19) [GCC 4.6.3]
pytz 		2012c
setuptools 	0.6
Subversion 	1.6.17 (r1128011)
jQuery		1.7.2

Enabled Plugins:

authz-policy 	N/A 			/var/trac/projects/cauca/plugins/authz_policy.pyc
LdapPlugin 	0.7.0dev-r11019 	/var/trac/projects/cauca/plugins/LdapPlugin-0.7.0dev_r11019-py2.7.egg
MacrosCAUCA 	N/A 			/var/trac/projects/cauca/plugins/MacrosCAUCA.pyc
TracDateField 	1.1.0-r12118 		/usr/local/lib/python2.7/dist-packages/TracDateField-1.1.0_r12118-py2.7.egg
TracTocMacro 	11.0.0.3 		/usr/local/lib/python2.7/dist-packages/TracTocMacro-11.0.0.3-py2.7.egg
TracWysiwyg 	0.12.0.4-r11158 	/var/trac/projects/cauca/plugins/TracWysiwyg-0.12.0.4_r11158-py2.7.egg

comment:2 by Jérôme Poulin <jeromepoulin@…>, 8 years ago

The main problem is that I didn't get the stack trace at first, here I found that the "dispatcher" variable contain \xc3\xa9 utf-8 character from a git branch name.

I guess this should not crash but it does.

Code fragment:

Line	
492	try:
493	if not env and env_error:
494	raise HTTPInternalError(env_error)
495	try:
496	dispatcher = RequestDispatcher(env)
497	dispatcher.dispatch(req)
498	except RequestDone:
499	pass
500	resp = req._response or []
501	except HTTPException, e:
502	_send_user_error(req, env, e)

Local variables:

Name		Value
dispatcher 	<trac.web.main.RequestDispatcher object at 0x7f5a9ae99450>
e 		UnicodeDecodeError('ascii', 'calcul_validation_coordonn\xc3\xa9', 26, 27, ...
env 		<trac.env.Environment object at 0x7f5a9a0a6e90>
env_error 	None
req 		<RequestWithSession "GET '/timeline'">
resp 		[]

comment:3 by Jun Omae, 8 years ago

It seems an InstallationIssue.

MySQL database must be configured with utf8 charset and utf8_bin collation. Please see MySqlDb and check the configuration.

comment:4 by Jérôme Poulin <jeromepoulin@…>, 8 years ago

This issue started as soon as someone published a GIT branch with a é in it.

The MySQL database has been setup according to the installation guide as well as Trac itself. All tables are utf8_bin.

This seems to be related to GIT or the GIT plugin, however trac.ini says git_fs_encoding = utf-8

There is also a problem in Trac as it does not display any backtrace when this error is encountered. This chunk of code is preventing a backtrace;

raise TracError(_("Genshi %(error)s error while rendering "
    "template %(location)s", 
    error=e.__class__.__name__, 
    location=location))

by Jun Omae, 8 years ago

comment:5 by Jun Omae, 8 years ago

Thanks for your investigation! I reproduced the original issue with unicode branches and tags. Would you please try t10980-unicode-branches-and-tags.diff?

(We should add the unit tests.)

Also, I'll look into the error while rendering later.

comment:6 by Jérôme Poulin <jeromepoulin@…>, 8 years ago

I confirm this patch fixes the problem, Thank you!

I didn't know where to look at in the code as it seems to take a bucket of data and dispatch it to multiple modules and I couldn't see the branch's name in the 'data' var.

comment:7 by Ryan J Ollos <ryan.j.ollos@…>, 8 years ago

Version: 1.0dev1.0-stable

comment:1 indicates the user is running 1.0-stable, not 1.0dev.

comment:8 by Remy Blank, 8 years ago

Jun, you don't need the iter() around the generator expressions. Parentheses are enough:

return ((self._fs_to_unicode(name), sha) 
        for name, sha in self.rev_cache.branch_dict)

comment:9 by Jun Omae, 8 years ago

Component: web frontendversion control
Keywords: git added
Milestone: 1.0.1
Release Notes: modified (diff)
Resolution: fixed
Status: newclosed

Sure! I just committed the patch with your suggestions in [11482-11483]. Thanks.

comment:10 by Jun Omae, 8 years ago

Owner: set to Jun Omae

comment:11 by Ryan J Ollos <ryan.j.ollos@…>, 8 years ago

Release Notes: modified (diff)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.