TracError: Genshi UnicodeEncodeError error while rendering template (fresh Trac install)

[dereks@…]

I have installed Trac like I have several times before, using 0.11 and 0.12, under Ubuntu 8.04 and Ubuntu 10.04. Today I installed 0.12.2 fresh, using this command:

sudo easy_install Trac

But when I visit my Trac site, I get this error in my web page:

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.2-py2.6.egg/trac/web/api.py", line 440, in send_error
    data, 'text/html')
  File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.2-py2.6.egg/trac/web/chrome.py", line 885, in render_template
    location=location))
TracError: Genshi UnicodeEncodeError error while rendering template '/usr/local/lib/python2.6/dist-packages/Trac-0.12.2-py2.6.egg/trac/templates/layout.html', line 12, char -1

I have several plugins installed, but the exception seems to be coming from the core Trac code, not one of the plugins. (Again, this all worked before.)

How do I troubleshoot this issue? There is no additional information in the Apache logs (I'm using mod_python).

A possibly related problem: When I visit my "/projects" URL, I get an internal server error and this Unicode error is in my Apache error.log:

[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10] mod_python (pid=4934, interpreter='main_interpreter', phase='PythonHandler', handler='trac.web.modpython_frontend'): Application error
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10] ServerName: 'derek.example.com'
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10] DocumentRoot: '/var/www'
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10] URI: '/projects/'
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10] Location: '/projects'
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10] Directory: None
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10] Filename: '/var/www/projects'
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10] PathInfo: '/'
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10] Traceback (most recent call last):
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10]   File "/usr/lib/python2.6/dist-packages/mod_python/importer.py", line 1537, in HandlerDispatch\n    default=default_handler, arg=req, silent=hlist.silent)
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10]   File "/usr/lib/python2.6/dist-packages/mod_python/importer.py", line 1229, in _process_target\n    result = _execute_target(config, req, object, arg)
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10]   File "/usr/lib/python2.6/dist-packages/mod_python/importer.py", line 1128, in _execute_target\n    result = object(arg)
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10]   File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.2-py2.6.egg/trac/web/modpython_frontend.py", line 149, in handler\n    gateway.run(dispatch_request)
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10]   File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.2-py2.6.egg/trac/web/wsgi.py", line 89, in run\n    response = application(self.environ, self._start_response)
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10]   File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.2-py2.6.egg/trac/web/main.py", line 411, in dispatch_request\n    env_paths)
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10]   File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.2-py2.6.egg/trac/web/main.py", line 703, in send_project_index\n    req.send(output, 'text/html')
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10]   File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.2-py2.6.egg/trac/web/api.py", line 416, in send\n    self.write(content)
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10]   File "/usr/local/lib/python2.6/dist-packages/Trac-0.12.2-py2.6.egg/trac/web/api.py", line 534, in write\n    raise ValueError("Can't send unicode content")
[Sun Apr 10 23:58:58 2011] [error] [client 192.168.2.10] ValueError: Can't send unicode content

[Remy Blank]

Does the name of one of your projects have any non-ASCII characters? Or possibly the content of the [project] name entry in trac.ini?

Can you please try again with all plugins disabled, just to make sure there's no influence?

[anonymous]

Thank you for your help!

I have exactly one (brand new, empty) project. It is called "My Project"… all ASCII chars. In trac.ini under [project], it says

[project]
name = My Project

I've used this exact project name before on other installs, so I don't think that's it.

I tried disabling all plugins in the [components] section of my trac.ini. The behaviour is unchanged. (I only had a few basic plugins configured:.)

[components]
#webadmin.* = enabled
#wikiprint.* = enabled
#tracfullblog.* = enabled
#privatewiki.api.privatewikisystem = enabled

I have no project-specific plugins installed at /var/lib/trac/projects/myproject/plugins/.

If I go to the template file and line number given in the traceback, I see there is a template variable there. If I remove it, then the error goes away, but instead I get a different Genshi UnicodeEncodeError on a different (core, default) template. It's like Genshi is broken.

I've played around with Unicode using the Python command prompt. I can encode Unicode utf-8 and ascii just fine using simple Python statements, so I don't think my Python install is broken.

This is on a virtual server (an LXC container). I'd be happy to give you full root on this box if you'd like to investigate… it is a fresh Ubuntu 10.04 with Trac, and nothing else. Feel free to post a throwaway public SSH key here… I will stick it into authorized_keys and post the login info.

Let me know if there is anything else I should post…

[Thijs Triemstra]

Replying to anonymous:

If I remove it, then the error goes away, but instead I get a different Genshi UnicodeEncodeError on a different (core, default) template. It's like Genshi is broken.

What version of Genshi are you using? Can you try different versions, and see if that makes a a difference?

[Christian Boos]

Replying to thijstriemstra:

Replying to anonymous:

If I remove it, then the error goes away, but instead I get a different Genshi UnicodeEncodeError on a different (core, default) template. It's like Genshi is broken.

What version of Genshi are you using? Can you try different versions, and see if that makes a a difference?

I also thought about this, but as he's using Trac 0.12, the only option is Genshi 0.6. Well, unless he got his version directly from the Genshi repository, 0.6 branch or trunk, in which case, sure, that information (which rev?) is very important.

What is that variable, title or project.name? What do you seen when you replace that var with ${repr(var)}?

[anonymous]

I got Trac to work by re-installing on a new server, but without installing the plugins. I believe the problem is caused by an old plugin or a plugin dependency.

Although I installed a few plugins on the broken server, I had none of them activated. But one of them seems to break Trac, even when not activated. I don't know which one.

There is a list of stuff I did not install this time (on the working server), but which I did install this on the broken server. Many of these are old .10 or .11 plugins (because there were no .12 versions available), but I confirmed them working on a Trac .12 release many months ago.

The list of candidate-problem-plugins can be found on the mailing list archive, here:

​http://groups.google.com/group/trac-users/browse_thread/thread/7d814cfc90c5681f/76bca836d7f028e7?lnk=raot

I don't understand how simply having a plugin on my hard drive can cause Genshi to fail. (I've seen plenty of broken Trac plugins before, but they cause an error w.r.t. the plugin, not a core Genshi template.)

—Derek

P.S.> I tried to post the list of plugins here on the ticket, but it was rejected as spam. (It gave me a Captcha, which I answered, but then rejected it as spam anyway. It said there were too many URLs in it.)

[Remy Blank]

Thanks for the feedback. So we can close this as an InstallationIssue.

[ScottW]

Replying to rblank:

Thanks for the feedback. So we can close this as an InstallationIssue.

How is this not an open bug? I get the cantfix bit. But, it seems there is a difference between can't fix and not a problem.

[Christian Boos]

cantfix means that unless proven otherwise, it's not something the Trac developers can fix, which doesn't mean it's not a real problem, of course, just beyond our reach.

[anonymous]

Hi,

got the same problem after fresh installed version of trac 0.11.7. Everything worked fine until I installed Genshi plugin using easy_install ​http://svn.edgewall.org/repos/genshi/trunk.

I downgraded it using easy_install ​http://svn.edgewall.org/repos/genshi/tags/0.6.0 and trac worked like it did before. Regards Sven

[Walter Jr.]

Replying to anonymous:

Hi,

got the same problem after fresh installed version of trac 0.11.7. Everything worked fine until I installed Genshi plugin using easy_install ​http://svn.edgewall.org/repos/genshi/trunk.

I downgraded it using easy_install ​http://svn.edgewall.org/repos/genshi/tags/0.6.0 and trac worked like it did before. Regards Sven

VERY GOOD MAN! NOW IT WORKS!!! THANKS A LOT!!'''

[Christian Boos]

Hm, did we document somewhere that Genshi trunk may not work anymore with 0.11.x and 0.12.x?

[anonymous]

Great! That was the solution, thanks a lot!

[Christian Boos]

Hm, verify this again.

It's probably not a big deal if 0.12.x doesn't support Genshi 0.7dev, but of course Trac 0.13dev must.

If indeed Genshi versions > 0.6 won't work with Trac, then we should at least make a more precise version requirement. Tentatively still for 0.12.3.

[osimons]

Running the Trac unit tests for current 0.12.3dev with current Genshi trunk results in 23 tests failing - all with the same error:

...
UnicodeError: source returned bytes, but no encoding specified
...
Ran 1179 tests in 17.932s
FAILED (errors=23)

[Christian Boos]

Ok, so Genshi trunk (0.7dev) won't work with 0.12-stable (now I remember we did some changes on Trac trunk to cope with the py3k related changes that happened there).

The remaining point to verify is whether 0.6dev (​genshi:source:/branches/stable/0.6.x) also works or not. I hope it will, as there's at least a bugfix which I'm interested in ;-) (​#G429).

Depending on the tests, we'll need to change the requirements from Genshi>=0.6 to either Genshi==0.6 or Genshi>=0.6,<0.7dev.

Ok, so the tests pass (Windows x86 with Python 2.5.4, 2.6.5 and 2.7.1, the latter with svn) and superficial testing with tracd seem to show that everything's fine with 0.6.1dev…

Would be nice to repeat with 2.4.x and also on Linux, just to be sure.

[osimons]

I'm always running latest Genshi 0.6-stable. No particular issues with it. I'm comfortable using Genshi>=0.6,<0.7dev for requirements.

[osimons]

Had a quick chat with Hodgestar on IRC, and we agreed that restricting the version is the best solution.

No doubt the Trac trunk fix can be backported to 0.12-stable, but there is no telling what various plugins may do or need - there are just too many ways plugins can fail with the unicode requirement of Genshi trunk.

[Christian Boos]

Ok then! Done in r10921.

[Christian Boos]

Replying to osimons:

[…] there is no telling what various plugins may do or need - there are just too many ways plugins can fail with the unicode requirement of Genshi trunk.

A cautionary note in TracDev/ReleaseNotes/0.13#ModifiedDependencies might be useful then.

[osimons]

Replying to cboos:

Ok then! Done in r10921.

Beat me to it. Thanks.

I'll add a note for future release documentation.

[Christian Boos]

Oops, sorry, I messed up the reference, I wanted to point to the TracDev/ApiChanges instead. Will fix.

[osimons]

Replying to cboos:

Oops, sorry, I messed up the reference, I wanted to point to the TracDev/ApiChanges instead. Will fix.

Well, both would be correct - I wrote my text as simple as possible to adress the needs of sysadmins installing Trac and plugins, and making them aware of this at the time where they actually decide which version of Genshi to install.

Anyway, I'll leave you to organize the docs - I have no strong feelings either way.

[Christian Boos]

There are indeed different audiences we need to address. The part for plugin authors and the details about str/unicode fit better with the TracDev/ApiChanges, where we already had a part about the other dependencies. We could also leave a note for people who simply install Trac, but then 0.13/TracInstall#MandatoryDependencies and 0.13/TracUpgrade would be also/more appropriate.

[anonymous]

On Ubuntu 10.4 installations with Trac 0.12, I got same error. Fixed like this:

easy_install --prefix=/usr --force-installation-into-system-dir --install-dir /usr/lib/python2.6/dist-packages  http://svn.edgewall.org/repos/genshi/tags/0.6.0

[anonymous]

Thanks a lot for the fix.

[anonymous]

I had the same problem with a fresh install of 1.0.1. and MySQL-DB. Like above, everything worked again after downgrading to genshi 0.6.0: easy_install ​http://svn.edgewall.org/repos/genshi/0.6.0

[anonymous]

I had the same problem after adding a comment to an existing ticket.

I solved it by executing:

pip install --upgrade genshi==0.6

It's not clear what triggered the error. There where no special characters in the text.

[anonymous]

see ​http://trac.edgewall.org/changeset/11683 - there appears to be an issue with text that contains e.g. URL: <​http://trac.edgewall.org/> with the angle brackets like that.