Changes between Initial Version and Version 1 of UnicodeEncodeError

Timestamp:

Mar 2, 2007, 1:07:32 PM (17 years ago)

Author:

Christian Boos

Comment:

Explain the often misunderstood UnicodeEncodeError

UnicodeEncodeError

@@ +1 @@
+= What does this `UnicodeEncodeError` mean? =
+
+It means that a given `unicode` object (i.e. the Python internal representation for a sequence of internationalized characters conforming to the Unicode standard) failed to be converted to a `str` object (i.e. a sequence of bytes). The failure means that there was a character which couldn't be represented by an appropriate sequence of bytes in the chosen output encoding.
+
+In practice, the default conversion being a "strict" one using the default encoding, which is 'ascii' most of the time, as soon as the `unicode` object contains characters outside of the ASCII range of characters (codepoint 0 to 127), the error will happen.
+
+This error was frequently seen during the transition to internal use of `unicode` that happened in Trac [milestone:0.10], and can still be seen now and then with Trac plugins that are not using the Trac API the way they should.
+
+Examples:
+
+{{{
+>>> str(u'chaîne de caractères')
+Traceback (most recent call last):
+  File "<stdin>", line 1, in ?
+UnicodeEncodeError: 'ascii' codec can't encode character u'\xee' in position 3: ordinal not in range(128)
+>>> 
+}}}
+
+The above is actually equivalent to the following, as `sys.getdefaultencoding()` is frequently 'ascii': 
+{{{
+>>> u'chaîne de caractères'.encode('ascii')
+Traceback (most recent call last):
+  File "<stdin>", line 1, in ?
+UnicodeEncodeError: 'ascii' codec can't encode character u'\xee' in position 3: ordinal not in range(128)
+}}}
+
+One possibility to force the conversion to happen would be to use a non-strict conversion:
+{{{
+>>> u'chaîne de caractères'.encode('ascii', 'ignore')
+'chane de caractres'
+>>> u'chaîne de caractères'.encode('ascii', 'replace')
+'cha?ne de caract?res'
+}}}
+
+----
+See also: TracDev/UnicodeGuidelines, UnicodeDecodeError