Issues with docutils and console locale running the test suite

[Olemis Lang <olemis+trac@…>]

While working on #11585 I installed docutils (see comment:1:ticket:11585) and immediately after I got this error (platform=Mac OS X) .

$ python trac/tests/functional/__init__.py
Traceback (most recent call last):
  File "trac/tests/functional/__init__.py", line 69, in <module>
    from trac.tests.compat import rmtree
  File "/path/to/trac/trac/tests/__init__.py", line 16, in <module>
    from trac.tests import attachment, config, core, env, perm, notification, \
  File "/path/to/trac/trac/tests/wikisyntax.py", line 26, in <module>
    from trac.wiki.tests import formatter
  File "/path/to/trac/trac/wiki/tests/__init__.py", line 21, in <module>
    from trac.wiki.tests.functional import functionalSuite
  File "/path/to/trac/trac/wiki/tests/functional.py", line 16, in <module>
    from trac.mimeview.rst import has_docutils
  File "/path/to/trac/trac/mimeview/rst.py", line 30, in <module>
    from docutils.core import publish_parts
  File "/path/to/venv/lib/python2.7/site-packages/docutils/core.py", line 20, in <module>
    from docutils import frontend, io, utils, readers, writers
  File "/path/to/venv/lib/python2.7/site-packages/docutils/frontend.py", line 41, in <module>
    import docutils.utils
  File "/path/to/venv/lib/python2.7/site-packages/docutils/utils/__init__.py", line 20, in <module>
    import docutils.io

Everything was back to normal ( see comment:2:ticket:11585 ) after executing this command

$ export LC_ALL=en_US

[Jun Omae]

• An exception isn't in the error message. Please post entire of the error.
• Are you using which version of docutils?
• Please post locale settings of your environment when you got the error.
• It seems to be ​http://bugs.python.org/issue18378.9, that is a Python issue on Mac.

[Olemis Lang <olemis+trac@…>]

Replying to jomae:

• An exception isn't in the error message. Please post entire of the error.

sorry , did not notice that was missing …

$ python trac/tests/functional/__init__.py
Traceback (most recent call last):
  File "trac/tests/functional/__init__.py", line 69, in <module>
    from trac.tests.compat import rmtree
  File "/path/to/trac/trac/tests/__init__.py", line 16, in <module>
    from trac.tests import attachment, config, core, env, perm, notification, \
  File "/path/to/trac/trac/tests/wikisyntax.py", line 26, in <module>
    from trac.wiki.tests import formatter
  File "/path/to/trac/trac/wiki/tests/__init__.py", line 21, in <module>
    from trac.wiki.tests.functional import functionalSuite
  File "/path/to/trac/trac/wiki/tests/functional.py", line 16, in <module>
    from trac.mimeview.rst import has_docutils
  File "/path/to/trac/trac/mimeview/rst.py", line 30, in <module>
    from docutils.core import publish_parts
  File "/path/to/venv/lib/python2.7/site-packages/docutils/core.py", line 20, in <module>
    from docutils import frontend, io, utils, readers, writers
  File "/path/to/venv/lib/python2.7/site-packages/docutils/frontend.py", line 41, in <module>
    import docutils.utils
  File "/path/to/venv/lib/python2.7/site-packages/docutils/utils/__init__.py", line 20, in <module>
    import docutils.io
  File "/path/to/venv/lib/python2.7/site-packages/docutils/io.py", line 18, in <module>
    from docutils.utils.error_reporting import locale_encoding, ErrorString, ErrorOutput
  File "/path/to/venv/lib/python2.7/site-packages/docutils/utils/error_reporting.py", line 47, in <module>
    locale_encoding = locale.getlocale()[1] or locale.getdefaultlocale()[1]
  File "/path/to/venv/lib/python2.7/locale.py", line 511, in getdefaultlocale
    return _parse_localename(localename)
  File "/path/to/venv/lib/python2.7/locale.py", line 443, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

• Are you using which version of docutils?

$ python -c "import docutils ; print docutils.__version__"
0.11

• Please post locale settings of your environment when you got the error.

I'm just running the functional test suite , so I guess locale settings are the ones used by default to run the test suite and , if there was any issues with that , then I guess those should be fixed as well .

• It seems to be ​http://bugs.python.org/issue18378.9, that is a Python issue on Mac.

Maybe . Is there anything Trac can do to work around this (and run the test suite without major inconvenience) ?

[Olemis Lang <olemis+trac@…>]

Replying to Olemis Lang <olemis+trac@…>:

Replying to jomae:

[…]

• Please post locale settings of your environment when you got the error.

I'm just running the functional test suite , so I guess locale settings are the ones used by default to run the test suite and , if there was any issues with that , then I guess those should be fixed as well .

JFTR , I found no particular configuration , but in the log this is what I get

2014-04-15 20:11:48,194 Trac[main] DEBUG: Negotiated locale: None -> en_US

[…]

[Jun Omae]

Replying to Olemis Lang <olemis+trac@…>:

...
ValueError: unknown locale: UTF-8

Thanks.

• Please post locale settings of your environment when you got the error.

I'm just running the functional test suite , so I guess locale settings are the ones used by default to run the test suite and , if there was any issues with that , then I guess those should be fixed as well .

I don't know much about default settings in Terminal on Mac. Could you please post output of locale and export | egrep 'LC_|LANG'?

• It seems to be ​http://bugs.python.org/issue18378.9, that is a Python issue on Mac.

Maybe . Is there anything Trac can do to work around this (and run the test suite without major inconvenience) ?

If LANG environment is UTF-8, I think the settings is wrong. See ​'ValueError: unknown locale: UTF-8' problem in Mac OS X 10.5 Leoparad.

[Olemis Lang <olemis+trac@…>]

Replying to jomae:

Replying to Olemis Lang <olemis+trac@…>:

[…]

• Please post locale settings of your environment when you got the error.

I'm just running the functional test suite , so I guess locale settings are the ones used by default to run the test suite and , if there was any issues with that , then I guess those should be fixed as well .

I don't know much about default settings in Terminal on Mac.

I'm using Mac OS since less than 15 days … so I'm a neophyte in all aspects , even the keyboard … fwiw I'm using ​iTerm

Could you please post output of locale and export | egrep 'LC_|LANG'?

$ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

$ export | egrep 'LC_|LANG'
declare -x LC_CTYPE="UTF-8"

• It seems to be ​http://bugs.python.org/issue18378.9, that is a Python issue on Mac.

Maybe . Is there anything Trac can do to work around this (and run the test suite without major inconvenience) ?

If LANG environment is UTF-8

[…]

afaict it's unset .

[…]

[Jun Omae]

...
LC_CTYPE="UTF-8"
...

• It seems to be ​http://bugs.python.org/issue18378.9, that is a Python issue on Mac.

Maybe . Is there anything Trac can do to work around this (and run the test suite without major inconvenience) ?

If LANG environment is UTF-8

[…]

afaict it's unset .

I think the LC_CTYPE also should be complied the locale format [language[_territory][.codeset][@modifier]], e.g. es_ES.UTF-8, en_US.UTF-8, C.UTF-8. At least, Python and Perl interpreters expect that.

See also ​Issue 1667 - iterm2 - Locales go wrong - Development site for iTerm2.

I don't think we should do something. Also, we cannot know language and territory in the user's environment.

[Jun Omae]

The issue is leaded by LC_CTYPE=UTF-8 which is invalid settings. That is neither a Trac issue nor a docutils issue.

​google:"LC_CTYPE=UTF-8 Mac" would help you.