trac has locale problem

[alperkanat@…]

There is a general problem for Turkish for the following conversion:

i->İ
ı->I

---

MOD_PYTHON ERROR

ProcessId:      7382
Interpreter:    'main_interpreter'

ServerName:     'trac.isikun.edu.tr'
DocumentRoot:   '/srv/www'

URI:            '/ifsaet/login'
Location:       '/'
Directory:      None
Filename:       '/srv/www/ifsaet'
PathInfo:       '/login'

Phase:          'PythonHandler'
Handler:        'trac.web.modpython_frontend'

Traceback (most recent call last):

  File "/usr/lib/python2.5/site-packages/mod_python/importer.py", line 1537, in HandlerDispatch
    default=default_handler, arg=req, silent=hlist.silent)

  File "/usr/lib/python2.5/site-packages/mod_python/importer.py", line 1229, in _process_target
    result = _execute_target(config, req, object, arg)

  File "/usr/lib/python2.5/site-packages/mod_python/importer.py", line 1128, in _execute_target
    result = object(arg)

  File "/usr/lib/python2.5/site-packages/trac/web/modpython_frontend.py", line 87, in handler
    gateway.run(dispatch_request)

  File "/usr/lib/python2.5/site-packages/trac/web/wsgi.py", line 87, in run
    response = application(self.environ, self._start_response)

  File "/usr/lib/python2.5/site-packages/trac/web/main.py", line 391, in dispatch_request
    env = _open_environment(env_path, run_once=run_once)

  File "/usr/lib/python2.5/site-packages/trac/web/main.py", line 58, in _open_environment
    env_cache[env_path] = open_environment(env_path)

  File "/usr/lib/python2.5/site-packages/trac/env.py", line 462, in open_environment
    env = Environment(env_path)

  File "/usr/lib/python2.5/site-packages/trac/env.py", line 142, in __init__
    load_components(self)

  File "/usr/lib/python2.5/site-packages/trac/loader.py", line 140, in load_components
    __import__(module)

  File "/usr/lib/python2.5/site-packages/trac/ticket/web_ui.py", line 27, in <module>
    from trac.ticket.notification import TicketNotifyEmail

  File "/usr/lib/python2.5/site-packages/trac/ticket/notification.py", line 25, in <module>
    from trac.notification import NotifyEmail

  File "/usr/lib/python2.5/site-packages/trac/notification.py", line 17, in <module>
    import smtplib

  File "/usr/lib/python2.5/smtplib.py", line 49, in <module>
    from email.base64MIME import encode as encode_base64

  File "/usr/lib/python2.5/email/__init__.py", line 79, in __getattr__
    __import__(self.__name__)

ImportError: No module named base64mIme

[Tim Hatch]

I don't understand how this is Trac's problem. The way you describe it, it sounds like a filesystem charset issue. Or that you've edited the Trac source.

>>> from email.base64MIME import encode as encode_base64

Works fine for me… could you give more information please?

[alperkanat@…]

try writing

from email.base64mIme import encode as encode_base64

like in the error message, there's a problem with the conversion..

[Christian Boos]

Verify you Python installation, it's likely that you have a file named /usr/lib/python2.5/email/base64mIme.py. Either rename it to the expected name (/usr/lib/python2.5/email/base64MIME.py) or use export PYTHONCASEOK=1.

At any rate, this is a local installation issue, at worst a Python issue, but not a problem related to Trac itself.

[alperkanat@…]

-rw-r—r— 1 root root 5802 Şub 23 23:21 /usr/lib/python2.5/email/base64mime.py

Where can I use export PYTHONCASEOK=1 ?

On systems using Turkish locale, there might be a problem like this… I'm using en_US.UTF-8 system wide.. I use tr_TR.UTF-8 in my home.. Importing it through Python interpreter doesn't raise any problems like that, so I think it's not a Python problem nor a problem related with my system..

[02:13] (tunix@kulupler conf)$ python Python 2.5.2 (r252:60911, Feb 23 2008, 21:20:32) [GCC 4.2.3] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from email.base64mime import encode as encode_base64 dir()

['builtins', 'doc', 'name', 'encode_base64']

[Tim Hatch]

What do you get if you perform the line as Trac is trying to do (which should work right):

>>> from email.base64MIME import encode as encode_base64

[alperkanat@…]

[12:55][root@kulupler ~]# python Python 2.5.2 (r252:60911, Feb 23 2008, 21:20:32) [GCC 4.2.3] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from email.base64MIME import encode as encode_base64

[Christian Boos]

Replying to alperkanat@gmail.com:

-rw-r--r-- 1 root root 5802 Şub 23 23:21 /usr/lib/python2.5/email/base64mime.py

Ok, sorry I verified, that should be OK.

Where can I use export PYTHONCASEOK=1 ?

Forget it, my wrong assumption.

On systems using Turkish locale, there might be a problem like this… I'm using en_US.UTF-8 system wide.. I use tr_TR.UTF-8 in my home.. Importing it through Python interpreter doesn't raise any problems like that, so I think it's not a Python problem nor a problem related with my system..

Well, googling for similar weirdness gave me this: ​http://www.linuxforums.org/forum/linux-programming-scripting/110502-python-smtplib-module-error.html which is indeed another Turkish Python 2.5 user. The problem seems to be the call to 'base64MIME'.lower() in email/__init__.py, line 113. Any idea why that could sometimes return base64mIme?

But again, as demonstrated in the script you can find following the above link, this doesn't seem to be related to Trac itself. We can keep the ticket open for talking about the issue though, as it affects Trac.

[alperkanat@…]

Whenever I have to change things in httpd configuration files and therefore restart the server, I have to restart the whole server itself not just the apache server.. Because trac starts showing these error messages.. It's really annoying..

I know that Python can (generally) handle Turkish characters since I'm also a Python developer.. It's amazing that restarting the whole server works generally… But let's say after a few days or weeks, it suddenly stops working and starts printing those error messages… Seems pretty weird though..

By the way, I'm also using Arch Linux like the guy on the forum link.. I also have Python 2.5 and kernel 2.6.24.3..

[anonymous]

Replying to alperkanat@…:

Whenever I have to change things in httpd configuration files and therefore restart the server, I have to restart the whole server itself not just the apache server.. Because trac starts showing these error messages.. It's really annoying..

I know that Python can (generally) handle Turkish characters since I'm also a Python developer.. It's amazing that restarting the whole server works generally… But let's say after a few days or weeks, it suddenly stops working and starts printing those error messages… Seems pretty weird though..

By the way, I'm also using Arch Linux like the guy on the forum link.. I also have Python 2.5 and kernel 2.6.24.3..

[serkan@…]

Due to Python bug related to Turkish locale. I had the very same issue in bzr. See Gentoo bug[1] and my blog post[2] about the issue. 1: ​http://bugs.gentoo.org/show_bug.cgi?id=250075 2: ​http://tinyurl.com/6hxtzp

[Christian Boos]

Thanks for the feedback! However that doesn't explain why it starts working, then produce the errors after a while as reported in comment:8.

My take is that one should use the en_US.UTF-8 locale for Trac, or use a local workaround (like linking /usr/lib/python2.5/email/base64mIme.py to /usr/lib/python2.5/email/base64mime.py`).

That tr_TR bug also manifests itself in Trac in other places, see #7686.

Anyway, with i18n support and Trac 0.12, all the localizations should be done on a per-request basis, which means that relying on process-wide settings like the locale is the wrong approach. There was at least another ticket or a mail, for which I can't find the reference anymore, which discussed interferences with setlocale calls done inside some PHP code that happened to be running in the same Apache process…

So what to do next with this issue? I see two possibilities:

1. declare Trac as incompatible with the tr_TR locale, in the installation notes
2. find every usage of upper/lower and make sure they are called on unicode objects, along the line of suggested workaround for #7686 (note that this would still not fix the specific problem with base64mIme reported in this ticket)

[Thijs Triemstra]

Ticket summary is too vague..

[figaro]

An annoyance to a subset of users, but hardly critical.