Edgewall Software
Modify

Opened 8 years ago

Closed 7 years ago

Last modified 7 years ago

#9766 closed defect (fixed)

Supplemental Characters not supported

Reported by: Steven R. Loomis <srl@…> Owned by: Jun Omae
Priority: normal Milestone: 0.12.3
Component: database backend Version: 0.12.1
Severity: normal Keywords: mysql unicode documentation
Cc:
Release Notes:
API Changes:

Description

I tried to import (and later, just paste) the following into a ticket

http://www.goog𝓁e.com/>
may look like "www.google.com" if appropriate fonts are installed. The nameprep algorithm is supposed to translate '𝓁' into a lowercase L character but is not doing so.

mysql, trac 0.12.1

got this errr

/xsrl/src/trac/trac/db/util.py:65: Warning: Incorrect string value: '\xF0\x9D\x93\x81e….' for column 'newvalue' at row 1

return self.cursor.execute(sql_escape_percent(sql), args)

Attachments (0)

Change History (6)

comment:1 Changed 8 years ago by Christian Boos

What "kind" of mysql? (see MySqlDb#MySQL for the SHOW ... statement)

But this rather looks like a MySQL limitation, as on PostgreSQL it obviously works…

comment:2 in reply to:  1 Changed 8 years ago by Steven R. Loomis <srl@…>

Replying to cboos:

What "kind" of mysql? (see MySqlDb#MySQL for the SHOW ... statement)

But this rather looks like a MySQL limitation, as on PostgreSQL it obviously works…

InnoDB, utf8/utf8_bin (as per 'SHOW…'.

Well, maybe we're going to move to postgresql before we even try moving to mysql.

Also, why is this a 'warning'? An 'error' would be better, because it results in truncated inserts.

comment:3 Changed 8 years ago by Steven R. Loomis <srl@…>

Looks like it needs to be utf8mb4 not utf8: ​http://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html - only available in mysql 5.5

comment:4 Changed 8 years ago by Christian Boos

Component: general → database backend
Keywords: mysql unicode documentation added
Milestone: → unscheduled

For turning the warning into an error:

warnings.filterwarnings("error", "Incorrect string value",
                        MySQLdb.exceptions.Warning)

(untested)

But this looks more like a documentation issue to me, we should just make that clear in the MySqlDb page, once someone has verified that utf8mb4 works as expected with Trac. Btw, ​utf16 and ​utf32 should also work.

comment:5 Changed 7 years ago by Jun Omae

Milestone: unscheduled → 0.12.3
Resolution: → fixed
Status: new → closed

Supported utf8mb4 charset in [10961].

For Supplemental Characters, it needs MySQL 5.5+, utf8mb4 charset and utf8mb4_bin collation.

CREATE DATABASE trac DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;

comment:6 Changed 7 years ago by Jun Omae

Owner: set to Jun Omae

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted.
to The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.