Edgewall Software
Modify

Opened 13 years ago

Closed 12 years ago

Last modified 12 years ago

#9766 closed defect (fixed)

Supplemental Characters not supported

Reported by: Steven R. Loomis <srl@…> Owned by: Jun Omae
Priority: normal Milestone: 0.12.3
Component: database backend Version: 0.12.1
Severity: normal Keywords: mysql unicode documentation
Cc: Branch:
Release Notes:
API Changes:
Internal Changes:

Description

I tried to import (and later, just paste) the following into a ticket

http://www.goog𝓁e.com/>
may look like "www.google.com" if appropriate fonts are installed. The nameprep algorithm is supposed to translate '𝓁' into a lowercase L character but is not doing so.

mysql, trac 0.12.1

got this errr

/xsrl/src/trac/trac/db/util.py:65: Warning: Incorrect string value: '\xF0\x9D\x93\x81e….' for column 'newvalue' at row 1

return self.cursor.execute(sql_escape_percent(sql), args)

Attachments (0)

Change History (6)

comment:1 by Christian Boos, 13 years ago

What "kind" of mysql? (see MySqlDb#MySQL for the SHOW ... statement)

But this rather looks like a MySQL limitation, as on PostgreSQL it obviously works…

in reply to:  1 comment:2 by Steven R. Loomis <srl@…>, 13 years ago

Replying to cboos:

What "kind" of mysql? (see MySqlDb#MySQL for the SHOW ... statement)

But this rather looks like a MySQL limitation, as on PostgreSQL it obviously works…

InnoDB, utf8/utf8_bin (as per 'SHOW…'.

Well, maybe we're going to move to postgresql before we even try moving to mysql.

Also, why is this a 'warning'? An 'error' would be better, because it results in truncated inserts.

comment:3 by Steven R. Loomis <srl@…>, 13 years ago

Looks like it needs to be utf8mb4 not utf8: http://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html - only available in mysql 5.5

comment:4 by Christian Boos, 13 years ago

Component: generaldatabase backend
Keywords: mysql unicode documentation added
Milestone: unscheduled

For turning the warning into an error:

warnings.filterwarnings("error", "Incorrect string value",
                        MySQLdb.exceptions.Warning)

(untested)

But this looks more like a documentation issue to me, we should just make that clear in the MySqlDb page, once someone has verified that utf8mb4 works as expected with Trac. Btw, utf16 and utf32 should also work.

comment:5 by Jun Omae, 12 years ago

Milestone: unscheduled0.12.3
Resolution: fixed
Status: newclosed

Supported utf8mb4 charset in [10961].

For Supplemental Characters, it needs MySQL 5.5+, utf8mb4 charset and utf8mb4_bin collation.

CREATE DATABASE trac DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;

comment:6 by Jun Omae, 12 years ago

Owner: set to Jun Omae

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.