Supplemental Characters not supported

[Steven R. Loomis <srl@…>]

I tried to import (and later, just paste) the following into a ticket

http://www.goog𝓁e.com/>
may look like "www.google.com" if appropriate fonts are installed. The nameprep algorithm is supposed to translate '𝓁' into a lowercase L character but is not doing so.

mysql, trac 0.12.1

got this errr

/xsrl/src/trac/trac/db/util.py:65: Warning: Incorrect string value: '\xF0\x9D\x93\x81e….' for column 'newvalue' at row 1

return self.cursor.execute(sql_escape_percent(sql), args)

[Christian Boos]

What "kind" of mysql? (see MySqlDb#MySQL for the SHOW ... statement)

But this rather looks like a MySQL limitation, as on PostgreSQL it obviously works…

[Steven R. Loomis <srl@…>]

Replying to cboos:

What "kind" of mysql? (see MySqlDb#MySQL for the SHOW ... statement)

But this rather looks like a MySQL limitation, as on PostgreSQL it obviously works…

InnoDB, utf8/utf8_bin (as per 'SHOW…'.

Well, maybe we're going to move to postgresql before we even try moving to mysql.

Also, why is this a 'warning'? An 'error' would be better, because it results in truncated inserts.

[Steven R. Loomis <srl@…>]

Looks like it needs to be utf8mb4 not utf8: ​http://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html - only available in mysql 5.5

[Christian Boos]

For turning the warning into an error:

warnings.filterwarnings("error", "Incorrect string value",
                        MySQLdb.exceptions.Warning)

(untested)

But this looks more like a documentation issue to me, we should just make that clear in the MySqlDb page, once someone has verified that utf8mb4 works as expected with Trac. Btw, ​utf16 and ​utf32 should also work.

[Jun Omae]

Supported utf8mb4 charset in [10961].

For Supplemental Characters, it needs MySQL 5.5+, utf8mb4 charset and utf8mb4_bin collation.

CREATE DATABASE trac DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;