UnicodeDecodeError after upgrade

[anonymous]

Hello,

I just upgraded trac to trac-0.12.3-3.el6.noarch, the newest package from epel repository for centos.

After upgrade I saw this error:

---

Trac detected an internal error:

UnicodeDecodeError: 'utf8' codec can't decode byte 0xc4 in position 69: invalid continuation byte

Python Traceback Most recent call last:

File "/usr/lib/python2.6/site-packages/trac/web/main.py", line 525, in _dispatch_request
  dispatcher.dispatch(req)
File "/usr/lib/python2.6/site-packages/trac/web/main.py", line 246, in dispatch
  resp = chosen_handler.process_request(req)
File "/usr/lib/python2.6/site-packages/trac/wiki/web_ui.py", line 117, in process_request
  page = WikiPage(self.env, pagename)
File "/usr/lib/python2.6/site-packages/trac/wiki/model.py", line 44, in __init__
  self._fetch(name, version, db)
File "/usr/lib/python2.6/site-packages/trac/wiki/model.py", line 67, in _fetch
  row = cursor.fetchone()
File "/usr/lib/python2.6/site-packages/trac/db/mysql_backend.py", line 43, in fetchone
  return row and self._convert_row(row) or None
File "/usr/lib/python2.6/site-packages/trac/db/mysql_backend.py", line 40, in _convert_row
  for v in row])
File "/usr/lib64/python2.6/encodings/utf_8.py", line 16, in decode
  return codecs.utf_8_decode(input, errors, True)

System Information: User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:11.0) Gecko/20100101 Firefox/11.0

Trac	0.12.3
Babel	0.9.4
Genshi	0.6
GIT	1.7.1
mod_python	3.3.1
MySQL	server: "5.1.61-log", client: "5.1.61", thread-safe: 0
MySQLdb	1.2.3c1
Pygments	1.4
Python	2.6.6 (r266:84292, Dec 7 2011, 20:48:22) [GCC 4.4.6 20110731 (Red Hat 4.4.6-3)]
setuptools	0.6
Subversion	1.6.11 (r934486)
jQuery`	1.4.4

Enabled Plugins:

TracAccountManager	0.3.2	/usr/lib/python2.6/site-packages/TracAccountManager-0.3.2-py2.6.egg
TracAnnouncer	0.11.1-r10458	/usr/lib/python2.6/site-packages/TracAnnouncer-0.11.1_r10458-py2.6.egg
TracGit	0.12.0.5dev	/usr/lib/python2.6/site-packages/TracGit-0.12.0.5dev-py2.6.egg
TracTocMacro	11.0.0.3	/usr/lib/python2.6/site-packages/TracTocMacro-11.0.0.3-py2.6.egg
TracWysiwyg	0.12.0.4-r11158	/usr/lib/python2.6/site-packages/TracWysiwyg-0.12.0.4_r11158-py2.6.egg

---

I'm using trac with http, mod_python and mysql backend. I think there is any strange character in the database. Therefore I created a mysqldump and then a hexdump.

# grep --colour=auto -iE "\ c4\ [0-9a-f][0-9a-f]" trac.hex
00039070  3d 3d 3d 20 c4 8c 65 73  6b 79 20 3d 3d 3d 5c 6e  |=== ..esky ===\n|
00039080  c4 8c 65 c5 a1 74 69 6e  61 20 76 20 6b c3 b3 64  |..e..tina v k..d|
00039160  48 72 76 61 74 73 6b 69  20 2d 20 c4 8d c4 87 c5  |Hrvatski - .....|
00039170  be c5 a1 c4 91 20 c4 8c  c4 86 c5 bd c5 a0 c4 90  |..... ..........|
000394d0  64 61 20 61 72 c4 ab 20  73 74 72 c4 81 64 c4 81  |da ar.. str..d..|
00039520  c5 a1 6b 6f 73 20 72 61  69 64 c4 97 73 3f 20 c4  |..kos raid..s? .|
00039530  85 c4 8d c4 99 c4 97 c4  af c5 a1 c5 b3 c5 ab c5  |................|
00039540  be 20 c4 84 c4 8c c4 98  c4 96 c4 ae c5 a0 c5 b2  |. ..............|
000397b0  63 68 6e c4 85 c4 87 20  77 20 74 c4 99 20 c5 82  |chn.... w t.. ..|
00039800  69 65 6e 6e c4 85 20 70  72 7a 65 6b 75 6b 61 2e  |ienn.. przekuka.|
000399d0  6b 6f 73 20 c4 8d 69 6e  6a 65 6e 69 63 69 20 64  |kos ..injenici d|
00039a40  6b 61 66 65 63 20 70 75  c5 a1 c4 8d 61 20 76 6f  |kafec pu....a vo|
00039c30  e1 bb 87 74 20 63 c5 a9  6e 67 20 c4 91 c6 b0 e1  |...t c..ng .....|

Some grepping leads to the point that I only have these characters within the language descriptions of trac. I compared this grep output to the original mysqldump.

After that I started searching for an according debug mechanism to get detailed output in the trac log, but all I found was:

default_charset = iso-8859-15

Could that be the problem?

It would be really nice if you could give me an idea how to solve this issue.

greetings Andi

[Jun Omae]

UnicodeDecodeError: 'utf8' codec can't decode byte 0xc4 in position 69: invalid continuation byte … I'm using trac with http, mod_python and mysql backend. I think there is any strange character in the database. Therefore I created a mysqldump and then a hexdump.

It seems your database isn't configured with utf8 charset and utf8_bin collation. Please see MySqlDb and #10661.

Then, I think that the issue is an InstallationIssue.

[Remy Blank]

Replying to jomae:

Then, I think that the issue is an InstallationIssue.

Indeed.