pytz interoperability: Unable to edit prefs timezone after installing pytz

[pkou@…]

Source: wiki:TracDev/ReleaseNotes/0.11#Timezonesupport

Environment: Python 2.4, Windows XP, Apache+mod_python

Scenario:

• Remove pytz if it is installed
• Install Trac 0.11dev from source:trunk@5121​
• Login and change timezone in preferences to GMT -2
• Install pytz
• Restart Trac and attempt to change timezone again

Expected:

• Current timezone is displayed. Timezone can be changed.

Actual:

• Exception IOError: [Errno 0] Error: 'pytz\\zoneinfo\\GMT -2:00' when there is an attempt to display timezone in preferences.

Details:

File "c:\python24\lib\site-packages\Trac-0.11dev_r5121-py2.4.egg\trac\web\main.py", line 428, in dispatch_request
  dispatcher.dispatch(req)
File "c:\python24\lib\site-packages\Trac-0.11dev_r5121-py2.4.egg\trac\web\main.py", line 230, in dispatch
  content_type)
File "c:\python24\lib\site-packages\Trac-0.11dev_r5121-py2.4.egg\trac\web\chrome.py", line 564, in render_template
  return stream.render(method, doctype=doctype)
File "build\bdist.win32\egg\genshi\core.py", line 146, in renderFile "build\bdist.win32\egg\genshi\output.py", line 178, in __call__File "build\bdist.win32\egg\genshi\output.py", line 421, in __call__File "build\bdist.win32\egg\genshi\output.py", line 578, in __call__File "build\bdist.win32\egg\genshi\output.py", line 363, in __call__File "build\bdist.win32\egg\genshi\core.py", line 207, in _ensureFile "build\bdist.win32\egg\genshi\core.py", line 207, in _ensureFile "c:\python24\lib\site-packages\Trac-0.11dev_r5121-py2.4.egg\trac\web\chrome.py", line 581, in _generate
  for kind, data, pos in stream:
File "build\bdist.win32\egg\genshi\template\markup.py", line 235, in _includeFile "build\bdist.win32\egg\genshi\template\markup.py", line 305, in _matchFile "build\bdist.win32\egg\genshi\template\markup.py", line 235, in _includeFile "build\bdist.win32\egg\genshi\template\markup.py", line 280, in _matchFile "build\bdist.win32\egg\genshi\template\markup.py", line 269, in _stripFile "build\bdist.win32\egg\genshi\template\markup.py", line 225, in _execFile "build\bdist.win32\egg\genshi\template\base.py", line 281, in _evalFile "build\bdist.win32\egg\genshi\template\base.py", line 331, in _flattenFile "build\bdist.win32\egg\genshi\template\base.py", line 325, in _flattenFile "build\bdist.win32\egg\genshi\template\directives.py", line 666, in __call__File "build\bdist.win32\egg\genshi\template\eval.py", line 125, in evaluateFile "prefs_datetime.html", line 12, in <Expression '?'>File "c:\python24\lib\site-packages\Trac-0.11dev_r5121-py2.4.egg\trac\util\datefmt.py", line 259, in get_timezone
  return timezone(tzname)
File "C:\Python24\lib\site-packages\pytz-2007c-py2.4.egg\pytz\__init__.py", line 92, in timezoneFile "C:\Python24\lib\site-packages\pytz-2007c-py2.4.egg\pytz\__init__.py", line 40, in open_resourceFile "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\pkg_resources.py", line 804, in resource_stream
  return get_provider(package_or_requirement).get_resource_stream(
File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\pkg_resources.py", line 1040, in get_resource_stream
  return StringIO(self.get_resource_string(manager, resource_name))
File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\pkg_resources.py", line 1043, in get_resource_string
  return self._get(self._fn(self.module_path, resource_name))
File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\pkg_resources.py", line 1112, in _get
  return self.loader.get_data(path)

The problem can be fixed by the following patch (but see below):

trac/util/datefmt.py

@@ -258 +258 @@
         """Fetch timezone instance by name or return `None`"""
         try:
             return timezone(tzname)
-        except KeyError:
+        except (KeyError, IOError):
             fixedzone = _tzmap.get(tzname)
             if fixedzone:
                 offset = fixedzone.utcoffset(None)

After the exception fixed, another exception ValueError: timestamp out of range for platform localtime()/gmtime() function is raised.

Details:

File "c:\python24\lib\site-packages\Trac-0.11dev_r5121-py2.4.egg\trac\web\main.py", line 428, in dispatch_request
  dispatcher.dispatch(req)
File "c:\python24\lib\site-packages\Trac-0.11dev_r5121-py2.4.egg\trac\web\main.py", line 230, in dispatch
  content_type)
File "c:\python24\lib\site-packages\Trac-0.11dev_r5121-py2.4.egg\trac\web\chrome.py", line 564, in render_template
  return stream.render(method, doctype=doctype)
File "build\bdist.win32\egg\genshi\core.py", line 146, in renderFile "build\bdist.win32\egg\genshi\output.py", line 178, in __call__File "build\bdist.win32\egg\genshi\output.py", line 421, in __call__File "build\bdist.win32\egg\genshi\output.py", line 578, in __call__File "build\bdist.win32\egg\genshi\output.py", line 363, in __call__File "build\bdist.win32\egg\genshi\core.py", line 207, in _ensureFile "build\bdist.win32\egg\genshi\core.py", line 207, in _ensureFile "c:\python24\lib\site-packages\Trac-0.11dev_r5121-py2.4.egg\trac\web\chrome.py", line 581, in _generate
  for kind, data, pos in stream:
File "build\bdist.win32\egg\genshi\template\markup.py", line 235, in _includeFile "build\bdist.win32\egg\genshi\template\markup.py", line 305, in _matchFile "build\bdist.win32\egg\genshi\template\markup.py", line 235, in _includeFile "build\bdist.win32\egg\genshi\template\markup.py", line 280, in _matchFile "build\bdist.win32\egg\genshi\template\markup.py", line 269, in _stripFile "build\bdist.win32\egg\genshi\template\markup.py", line 225, in _execFile "build\bdist.win32\egg\genshi\template\base.py", line 294, in _evalFile "build\bdist.win32\egg\genshi\template\base.py", line 304, in _evalFile "build\bdist.win32\egg\genshi\template\eval.py", line 125, in evaluateFile "C:\Python24\lib\site-packages\trac-0.11dev_r5121-py2.4.egg\trac\prefs\templates\prefs_datetime.html", line 19, in <Expression u"selected_tzname.startswith('Etc/') and                             selected_tz == timezone(tzname) or                             selected_tzname == tzname or None">
  <option py:for="tzname in timezones"
File "c:\python24\lib\site-packages\Trac-0.11dev_r5121-py2.4.egg\trac\util\datefmt.py", line 259, in get_timezone
  return timezone(tzname)
File "C:\Python24\lib\site-packages\pytz-2007c-py2.4.egg\pytz\__init__.py", line 92, in timezoneFile "C:\Python24\lib\site-packages\pytz-2007c-py2.4.egg\pytz\tzfile.py", line 31, in build_tzinfoFile "C:\Python24\lib\site-packages\pytz-2007c-py2.4.egg\pytz\tzinfo.py", line 26, in memorized_datetime

If, at last, add the exception ValueError in the ignored list, then the result is wrong anyway:

• Wrong timezone is displayed (Etc/GMT+2 instead of Etc/GMT-2);
• After changing timezone, it is saved as UTC regardless of new value.

[Matthew Good]

This is mostly fixed by r5130. However, I can't reproduce the ValueError exception you saw. Please try out the patch and let me know if the ValueError still occurs.

[Christian Boos]

Well, r5130 is wrong regarding the supposed bug in pytz: Etc/GMT-2 should actually be the same as GMT+2:00.

See #975 for more details about this, in particular comment:ticket:975:13.

So, if before pytz one has a timezone of GMT -2, after enabling pytz, one should indeed get Etc/GMT+2.

[Matthew Good]

Well, maybe we should have pulled out a separate ticket when this was discovered in #975 since this obviously is a source of confusion. So Trac follows the ISO convention while the Etc/ timezones in pytz are POSIX? Maybe we should filter out the Etc/ timezones from pytz and replace them with our own, since the ISO conventions are what most people would expect.

[Christian Boos]

Agreed, this is actually what I suggested at the bottom of the comment 13 referred to above, but never got around implementing it :-). Let's take this ticket for reminding us of doing that.

[Matthew Good]

Alright, r5132 swaps out the POSIX timezones from pytz with the ISO-based ones from Trac. I still don't get that ValueError though, so I think we need some more info on that.

[pkou@…]

Tested the source:trunk@r5133. It is better but it does not fix the problem.

Using scenario from description: After installing pytz, the timezone preferences display the timezone GMT-2 properly. But, an exception occurs after changing it to something like Europe/xxx. And the exception becomes regular when the user tries to display timezone.

Confirmed bug in pytz 2007c under Windows, the following test is failed:

>>> import pytz
>>> pytz.timezone('Europe/Amsterdam')
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
  File "C:\Python24\lib\site-packages\pytz-2007c-py2.4.egg\pytz\__init__.py", line 93, in timezone
  File "C:\Python24\lib\site-packages\pytz-2007c-py2.4.egg\pytz\tzfile.py", line 32, in build_tzinfo
  File "C:\Python24\lib\site-packages\pytz-2007c-py2.4.egg\pytz\tzinfo.py", line 27, in memorized_datetime
ValueError: timestamp out of range for platform localtime()/gmtime() function
>>> 

Reference issue for pytz: ​https://bugs.launchpad.net/pytz/+bug/90096 (2007c incompatible with windows).

Will retest after new release of 'pytz'.

[Matthew Good]

Replying to pkou@ua.fm:

Tested the source:trunk@r5133. It is better but it does not fix the problem.

Confirmed bug in pytz 2007c under Windows, the following test is failed:

…snip…

Reference issue for pytz: ​https://bugs.launchpad.net/pytz/+bug/90096 (2007c incompatible with windows).

Ok. This remaining issue is a bug in pytz I'm going to close this ticket since the issues on the Trac side have been resolved.

[pkou at ua.fm]

Retested with pytz 2007d. It crashes because timezone name should be ASCII string, not UNICODE.

Patch:

trac/util/datefmt.py

@@ -267 +267 @@
     def get_timezone(tzname):
         """Fetch timezone instance by name or return `None`"""
         try:
-            tz = pytz.timezone(tzname)
+            tz = pytz.timezone(str(tzname))
         except (KeyError, IOError):
             tz = _tzmap.get(tzname)
         if tz and tzname.startswith('Etc/'):

Traceback:

TypeError: type() argument 1 must be string, not unicode

File "c:\python24\lib\site-packages\Trac-0.11dev_r5145-py2.4.egg\trac\web\main.py", line 428, in dispatch_request
  dispatcher.dispatch(req)
File "c:\python24\lib\site-packages\Trac-0.11dev_r5145-py2.4.egg\trac\web\main.py", line 230, in dispatch
  content_type)
File "c:\python24\lib\site-packages\Trac-0.11dev_r5145-py2.4.egg\trac\web\chrome.py", line 564, in render_template
  return stream.render(method, doctype=doctype)
File "build\bdist.win32\egg\genshi\core.py", line 146, in renderFile "build\bdist.win32\egg\genshi\output.py", line 178, in __call__
File "build\bdist.win32\egg\genshi\output.py", line 421, in __call__
File "build\bdist.win32\egg\genshi\output.py", line 578, in __call__
File "build\bdist.win32\egg\genshi\output.py", line 363, in __call__
File "build\bdist.win32\egg\genshi\core.py", line 207, in _ensure
File "build\bdist.win32\egg\genshi\core.py", line 207, in _ensure
File "c:\python24\lib\site-packages\Trac-0.11dev_r5145-py2.4.egg\trac\web\chrome.py", line 581, in _generate
  for kind, data, pos in stream:
File "build\bdist.win32\egg\genshi\template\markup.py", line 235, in _include
File "build\bdist.win32\egg\genshi\template\markup.py", line 305, in _match
File "build\bdist.win32\egg\genshi\template\markup.py", line 235, in _include
File "build\bdist.win32\egg\genshi\template\markup.py", line 280, in _match
File "build\bdist.win32\egg\genshi\template\markup.py", line 269, in _strip
File "build\bdist.win32\egg\genshi\template\markup.py", line 225, in _exec
File "build\bdist.win32\egg\genshi\template\base.py", line 281, in _eval
File "build\bdist.win32\egg\genshi\template\base.py", line 331, in _flatten
File "build\bdist.win32\egg\genshi\template\base.py", line 325, in _flatten
File "build\bdist.win32\egg\genshi\template\directives.py", line 666, in __call__
File "build\bdist.win32\egg\genshi\template\eval.py", line 125, in evaluate
File "prefs_datetime.html", line 12, in <Expression '?'>
File "c:\python24\lib\site-packages\Trac-0.11dev_r5145-py2.4.egg\trac\util\datefmt.py", line 269, in get_timezone
  tz = pytz.timezone(tzname)
File "C:\Python24\lib\site-packages\pytz-2007d-py2.4.egg\pytz\__init__.py", line 125, in timezoneFile "C:\Python24\lib\site-packages\pytz-2007d-py2.4.egg\pytz\tzfile.py", line 98, in build_tzinfo

[Christian Boos]

str() is a bit fragile when given arbitrary unicode input. Other than that, thanks for the notice!

[Christian Boos]

Fixed in r5147.