trac did not eat simple Umlaut!

[anonymous]

trac did not eat simple Umlaut!

Maybe it helps you, with your great work!

cu,

Klaus (kappel@…)

UnicodeDecodeError Python 2.4.2: /usr/bin/python Mon Nov 5 15:47:54 2007

A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.

 /usr/lib/python2.4/site-packages/trac/web/_fcgi.py in run(self=<trac.web._fcgi.Request object>)
  568         """Runs the handler, flushes the streams, and ends the request."""
  569         try:
  570             protocolStatus, appStatus = self.server.handler(self)
  571         except:
  572             traceback.print_exc(file=self.stderr)
protocolStatus undefined, appStatus undefined, self = <trac.web._fcgi.Request object>, self.server = <trac.web._fcgi.WSGIServer object>, self.server.handler = <bound method WSGIServer.handler of <trac.web._fcgi.WSGIServer object>>
 /usr/lib/python2.4/site-packages/trac/web/_fcgi.py in handler(self=<trac.web._fcgi.WSGIServer object>, req=<trac.web._fcgi.Request object>)
 1254         try:
 1255             try:
 1256                 result = self.application(environ, start_response)
 1257                 try:
 1258                     for data in result:
result = None, self = <trac.web._fcgi.WSGIServer object>, self.application = <function dispatch_request>, environ = {'CONTENT_LENGTH': '4536', 'CONTENT_TYPE': 'application/x-www-form-urlencoded', 'DOCUMENT_ROOT': '/srv/www/htdocs', 'GATEWAY_INTERFACE': 'CGI/1.1', 'HTTP_ACCEPT': 'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5', 'HTTP_ACCEPT_CHARSET': 'UTF-8,*', 'HTTP_ACCEPT_ENCODING': 'gzip,deflate', 'HTTP_ACCEPT_LANGUAGE': 'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'HTTP_CONNECTION': 'keep-alive', 'HTTP_COOKIE': 'trac_form_token=637c43a1c47557e0785eac02; trac_auth=7c1e46dfbd83e450478a06e226318355', ...}, start_response = <function start_response>
 /usr/lib/python2.4/site-packages/trac/web/main.py in dispatch_request(environ={'CONTENT_LENGTH': '4536', 'CONTENT_TYPE': 'application/x-www-form-urlencoded', 'DOCUMENT_ROOT': '/srv/www/htdocs', 'GATEWAY_INTERFACE': 'CGI/1.1', 'HTTP_ACCEPT': 'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5', 'HTTP_ACCEPT_CHARSET': 'UTF-8,*', 'HTTP_ACCEPT_ENCODING': 'gzip,deflate', 'HTTP_ACCEPT_LANGUAGE': 'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'HTTP_CONNECTION': 'keep-alive', 'HTTP_COOKIE': 'trac_form_token=637c43a1c47557e0785eac02; trac_auth=7c1e46dfbd83e450478a06e226318355', ...}, start_response=<function start_response>)
  395         env_error = e
  396 
  397     req = Request(environ, start_response)
  398     try:
  399         if not env and env_error:
req undefined, global Request = <class 'trac.web.api.Request'>, environ = {'CONTENT_LENGTH': '4536', 'CONTENT_TYPE': 'application/x-www-form-urlencoded', 'DOCUMENT_ROOT': '/srv/www/htdocs', 'GATEWAY_INTERFACE': 'CGI/1.1', 'HTTP_ACCEPT': 'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5', 'HTTP_ACCEPT_CHARSET': 'UTF-8,*', 'HTTP_ACCEPT_ENCODING': 'gzip,deflate', 'HTTP_ACCEPT_LANGUAGE': 'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'HTTP_CONNECTION': 'keep-alive', 'HTTP_COOKIE': 'trac_form_token=637c43a1c47557e0785eac02; trac_auth=7c1e46dfbd83e450478a06e226318355', ...}, start_response = <function start_response>
 /usr/lib/python2.4/site-packages/trac/web/api.py in __init__(self=<trac.web.api.Request object>, environ={'CONTENT_LENGTH': '4536', 'CONTENT_TYPE': 'application/x-www-form-urlencoded', 'DOCUMENT_ROOT': '/srv/www/htdocs', 'GATEWAY_INTERFACE': 'CGI/1.1', 'HTTP_ACCEPT': 'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5', 'HTTP_ACCEPT_CHARSET': 'UTF-8,*', 'HTTP_ACCEPT_ENCODING': 'gzip,deflate', 'HTTP_ACCEPT_LANGUAGE': 'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'HTTP_CONNECTION': 'keep-alive', 'HTTP_COOKIE': 'trac_form_token=637c43a1c47557e0785eac02; trac_auth=7c1e46dfbd83e450478a06e226318355', ...}, start_response=<function start_response>)
  165         self.abs_href = Href(self.base_url)
  166 
  167         self.args = self._parse_args()
  168 
  169     def _parse_args(self):
self = <trac.web.api.Request object>, self.args = None, self._parse_args = <bound method Request._parse_args of <trac.web.api.Request object>>
 /usr/lib/python2.4/site-packages/trac/web/api.py in _parse_args(self=<trac.web.api.Request object>)
  190                 for value in values:
  191                     if not value.filename:
  192                         value = unicode(value.value, 'utf-8')
  193                     if name in args:
  194                         if isinstance(args[name], list):
value = MiniFieldStorage('preview', "Previewbekannte Ink...n lange Recherchen n\xc3\xb6tig werden.\r\n\r\n"), builtin unicode = <type 'unicode'>, value.value = 'Previewbekannte Inkompatibilit\xc3\xa4ten) lassen sich...i R\xc3\xbcckfragen lange Recherchen n\xc3\xb6tig werden.\r\n\r\n'

UnicodeDecodeError: 'utf8' codec can't decode bytes in position 2816-2817: invalid data
      args = ('utf8', 'Previewbekannte Inkompatibilit\xc3\xa4ten) lassen sich...i R\xc3\xbcckfragen lange Recherchen n\xc3\xb6tig werden.\r\n\r\n', 2816, 2818, 'invalid data')
      encoding = 'utf8'
      end = 2818
      object = 'Previewbekannte Inkompatibilit\xc3\xa4ten) lassen sich...i R\xc3\xbcckfragen lange Recherchen n\xc3\xb6tig werden.\r\n\r\n'
      reason = 'invalid data'
      start = 2816 

[hyuga <hyugaricdeau@…>]

This looks to be a duplicate of #3339, but I'm not certain.

[Christian Boos]

No, it doesn't seem to be related to the SpamFilter.

It rather looks like invalid input, as it's the following statement which fails:

  unicode(value.value, 'utf-8')

with the (truncated) value.value being:

value.value == 'Previewbekannte Inkompatibilit\xc3\xa4ten) lassen sich...i R\xc3\xbcckfragen lange Recherchen n\xc3\xb6tig werden.\r\n\r\n'

The fragment of value.value displayed in the backtrace is valid utf-8, however the error message says that the error occurs at position 2816, i.e. somewhere in the truncated part.

It's a bit hard to say what's going on without additional details. Can you monitor the traffic and get a full dump of the HTTP request?

[Christian Boos]

Not much we can do here about that unreproducible issue.