Removing, viewing, editing of ticket with UTF8 (sent by mail) is impossible

[olaf@…]

Some of our tickets are created from emails. If there are encoding issues with this ticket, you cannot do anything with it. Not even delete it from the commandline with trac-admin…

Traceback (most recent call last):
  File "/var/lib/python-support/python2.5/trac/web/main.py", line 406, in dispatch_request
    dispatcher.dispatch(req)
  File "/var/lib/python-support/python2.5/trac/web/main.py", line 237, in dispatch
    resp = chosen_handler.process_request(req)
  File "/var/lib/python-support/python2.5/trac/ticket/web_ui.py", line 286, in process_request
    ticket = Ticket(self.env, id, db=db)
  File "/var/lib/python-support/python2.5/trac/ticket/model.py", line 40, in __init__
    self._fetch_ticket(tkt_id, db)
  File "/var/lib/python-support/python2.5/trac/ticket/model.py", line 83, in _fetch_ticket
    % ','.join(std_fields), (tkt_id,))
  File "/var/lib/python-support/python2.5/trac/db/util.py", line 50, in execute
    return self.cursor.execute(sql_escape_percent(sql), args)
  File "/var/lib/python-support/python2.5/trac/db/sqlite_backend.py", line 56, in execute
    args or [])
  File "/var/lib/python-support/python2.5/trac/db/sqlite_backend.py", line 48, in _rollback_on_error
    return function(self, *args, **kwargs)
OperationalError: Could not decode to UTF-8 column 'description' with text '-------- Original Message --------
Subject: [iDB] iDB
Date: Mon, 10 Mar 2008 09:33:23 +0100
From: <Voorzitter@ik.utwente.nl>
Reply-To: idb@i

[osimons]

Trac itself does not support new tickets made fom e-mails, so this is an issue with whatever script you use to create such tickets in that it does not store the data correctly in the database (encoding), causing Trac to choke when trying to read back the data for conversion to unicode.

And, yes you are right in that there is no Trac feature to support broken ticket records - or any other broken database records for that matter. You need to use SQL for that. Trac should (and does) ensure the integrity of data in and out using the Trac APIs, but if scripts use this incorrectly or even bypasses it and modifies database directly, there isn't much that can be done when the data isn't even readable.

Please report the issue with the developer/maintainer of that script - wherever it was found.