Edgewall Software
Modify

Opened 15 years ago

Closed 14 years ago

#7799 closed defect (duplicate)

listing files with utf8 filenames raises UnicodeDecodeError with certain characters

Reported by: shockwave107@… Owned by: Christian Boos
Priority: normal Milestone:
Component: plugin/mercurial Version: 0.12dev
Severity: normal Keywords: unicode
Cc: igor.dejanovic@… Branch:
Release Notes:
API Changes:
Internal Changes:

Description

How to Reproduce

  1. Requires: filesystem with utf8 filename support
  2. create mercurial repository
  3. create file with utf8 coded character "ö"

(["ö"="0xc3 0xb6"] or others with utf8 byte sequence containing "bytes out of range(128)" )

  1. add/commitpush, and navigate with repository browser to containing path

same procedure with subversion does work.

Original Query and Trace

While doing a GET operation on /browser/WuS/uebung, Trac issued an internal error.

Contained File

WuS/uebung/Lös4.doc

Request parameters:

{'path': u'/WuS/uebung'}

User Agent was: Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.9 (like Gecko) (Gentoo)

$&
Trac 0.11.1
Python 2.5.2 (r252, Oct 27 2008, 23:33:32)
[GCC 4.1.2 (Gentoo 4.1.2 p1.1)]
setuptools 0.6c8
psycopg2 2.0.2
Genshi 0.5.1
mod_python 3.3.1
Pygments 0.10
Mercurial 1.0.2
jQuery: 1.2.6 $&
Traceback (most recent call last):
  File "/usr/lib64/python2.5/site-packages/trac/web/main.py", line 423, in _dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib64/python2.5/site-packages/trac/web/main.py", line 197, in dispatch
    resp = chosen_handler.process_request(req)
  File "/usr/lib64/python2.5/site-packages/trac/versioncontrol/web_ui/browser.py", line 361, in process_request
    'dir': node.isdir and self._render_dir(req, repos, node, rev),
  File "/usr/lib64/python2.5/site-packages/trac/versioncontrol/web_ui/browser.py", line 407, in _render_dir
    entries = [entry(n) for n in node.get_entries()]
  File "build/bdist.linux-x86_64/egg/tracext/hg/backend.py", line 652, in get_entries
    entry = posixpath.join(self.path, entry)
  File "/usr/lib64/python2.5/posixpath.py", line 65, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)

Additional

TracMercurial from svn

http://svn.edgewall.com/repos/trac/sandbox/mercurial-plugin-0.11 
Revision: 7656

PS: same with Trac 0.11.0/0.10.5 and TracMercurial 0.10 svn version

Attachments (0)

Change History (6)

comment:1 by Remy Blank, 15 years ago

Milestone: not applicable

comment:2 by Christian Boos, 15 years ago

#8018 closed as duplicate.

Non-ascii file names are only one part of the problem. See also #7160.

comment:3 by Joseph Tate <jtate+trac@…>, 15 years ago

Version: 0.11.10.12dev

This still happens in 0.12 svn:

Most recent call last:
File "/usr/lib/python2.4/site-packages/Trac-0.12multirepos_r0-py2.4.egg/trac/web/main.py", line 467, in _dispatch_request
  dispatcher.dispatch(req)
File "/usr/lib/python2.4/site-packages/Trac-0.12multirepos_r0-py2.4.egg/trac/web/main.py", line 212, in dispatch
  resp = chosen_handler.process_request(req)
File "/usr/lib/python2.4/site-packages/Trac-0.12multirepos_r0-py2.4.egg/trac/versioncontrol/web_ui/browser.py", line 376, in process_request
  dir_data = self._render_dir(
File "/usr/lib/python2.4/site-packages/Trac-0.12multirepos_r0-py2.4.egg/trac/versioncontrol/web_ui/browser.py", line 493, in _render_dir
  entries = [entry(n) for n in node.get_entries()]
File "/home/admin/trac_stuff/mercurial-plugin-0.12/tracext/hg/backend.py", line 754, in get_entries
  dirnodes = self.findnode((n_rev > node_rev) and node_rev or n_rev, dirnames)
File "/home/admin/trac_stuff/mercurial-plugin-0.12/tracext/hg/backend.py", line 661, in findnode
  if f.startswith(d):

d=u'assets/help/' f='v_images/affiliate_0/affiliate_logos/BJ\x92s_Wholesale_Club_logo.gif'

Notice that f is a str. I tried wrapping that and a few other things in to_unicode(), but it ended up being a rat hole. I'm not sure where that wrapping needs to occur, but it appears to be partially applied at the moment.

comment:4 by Igor Dejanović <igor.dejanovic@…>, 14 years ago

Cc: igor.dejanovic@… added

Same issue here. Putting

import sys
sys.setdefaultencoding('utf-8')

in sitecustomize.py seems to fix it.

comment:5 by Christian Boos, 14 years ago

Keywords: unicode added
Milestone: not applicablemercurial-plugin

comment:6 by Christian Boos, 14 years ago

Milestone: mercurial-plugin
Resolution: duplicate
Status: newclosed

See #8538, which has the beginning of a patch…

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Christian Boos.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Christian Boos to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.