ValueError: tzinfo.utcoffset() must return a whole number of minutes

[Ryan J Ollos]

I've seen a few instances of this error in the logs over the past few days.

2016-11-07 20:39:41,364 Trac[web_ui] ERROR: Timeline event provider failed: 
Traceback (most recent call last):
  File "/srv/trac-hacks.org/pve/lib/python2.7/site-packages/trac/timeline/web_ui.py", line 198, in process_request
    filters) or []:
  File "/srv/trac-hacks.org/pve/lib/python2.7/site-packages/trac/ticket/web_ui.py", line 249, in get_timeline_events
    ts_start = to_utimestamp(start)
  File "/srv/trac-hacks.org/pve/lib/python2.7/site-packages/trac/util/datefmt.py", line 193, in to_utimestamp
    diff = dt - _epoc
ValueError: tzinfo.utcoffset() must return a whole number of minutes

[Ryan J Ollos]

Moved ticket assigned to next-stable-1.0.x since maintenance of 1.0.x is coming to a close. Please move the ticket back if it's critical to fix on 1.0.x.

[Ryan J Ollos]

Possible hints in ​Python-issue:1447945, but I haven't been able to reproduce yet.

[Jun Omae]

Several time zones don't have a whole number of minutes as offset in the past. However, Python's tzinfo class requires a whole number of minutes as offset. Also, pytz library modifies the offset to avoid the error.

$ TZ=Europe/Dublin venv/trac/1.0.13/bin/python
>>> from datetime import datetime
>>> from trac.util.datefmt import localtz, to_datetime, to_utimestamp
>>> localtz.utcoffset(datetime(2017, 1, 2))
datetime.timedelta(0)
>>> localtz.utcoffset(datetime(2016, 8, 2))
datetime.timedelta(0, 3600)
>>> localtz.utcoffset(datetime(1900, 1, 2))
datetime.timedelta(-1, 84879)
>>> to_utimestamp(datetime(1900, 1, 2, tzinfo=localtz))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/venv/trac/1.0.13/lib/python2.5/site-packages/trac/util/datefmt.py", line 193, in to_utimestamp
    diff = dt - _epoc
ValueError: tzinfo.utcoffset() must return a whole number of minutes
>>> import pytz
>>> pytz.timezone('Europe/Dublin').utcoffset(datetime(1900, 1, 2))
datetime.timedelta(-1, 84900)

See also:

• ​https://en.wikipedia.org/wiki/UTC-00:25:21
• Time zone changes for "1900 - 1924" in ​https://www.timeanddate.com/time/zone/ireland/dublin

[Jun Omae]

We could correct such an offset of timezone with the same way of pytz, ​https://github.com/stub42/pytz/blob/release_2016.10/src/pytz/tzfile.py#L116:

trac/util/datefmt.py

@@ -1056 +1056 @@
         return False
 
     def utcoffset(self, dt):
-        return self._tzinfo(dt)._offset
+        offset = self._tzinfo(dt)._offset
+        seconds = offset.days * 86400 + offset.seconds
+        mod = seconds % 60
+        if mod != 0:
+            # Avoid "ValueError: tzinfo.utcoffset() must return a whole
+            # number of minutes" (#12617)
+            offset = timedelta(seconds=int((seconds + 30) // 60) * 60)
+        return offset
 
     def dst(self, dt):
         if self._is_dst(dt):

trac/util/tests/datefmt.py

@@ -1597 +1597 @@
                          datetime.datetime(2011, 10, 30, 2, 45, 42, 123456,
                                            datefmt.localtz).utcoffset())
 
+    def test_utcoffset_non_whole_number_of_minutes(self):
+        self._tzset('Europe/Dublin')
+        dt = datetime.datetime(1901, 1, 2)
+        self.assertEqual(datetime.timedelta(days=-1, seconds=84900),
+                         datefmt.localtz.utcoffset(dt))
+
     def test_localized_non_existent_time(self):
         self._tzset('Europe/Paris')
         dt = datetime.datetime(2012, 3, 25, 2, 15, 42, 123456)

[Ryan J Ollos]

That change sounds good. For time periods so far in the past I think it would also be fine to just raise a TracError, or otherwise handle the issue in any way that doesn't result in noisy logs. Since you have a fix though, we might as well just apply it.

It seems strange to me that the issue occurs on the trac-hacks.org server though, because pytz is installed. I can reproduce your terminal session on that server:

# TZ=Europe/Dublin pve/bin/python
Python 2.7.9 (default, Jun 29 2016, 13:08:31) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime
>>> from trac.util.datefmt import localtz, to_datetime, to_utimestamp
>>> localtz.utcoffset(datetime(2017, 1, 2))
datetime.timedelta(0)
>>> localtz.utcoffset(datetime(2016, 8, 2))
datetime.timedelta(0, 3600)
>>> localtz.utcoffset(datetime(1900, 1, 2))
datetime.timedelta(-1, 84879)
>>> to_utimestamp(datetime(1900, 1, 2, tzinfo=localtz))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/trac/util/datefmt.py", line 193, in to_utimestamp
    diff = dt - _epoc
ValueError: tzinfo.utcoffset() must return a whole number of minutes
>>> import pytz
>>> pytz.timezone('Europe/Dublin').utcoffset(datetime(1900, 1, 2))
datetime.timedelta(-1, 84900)

I cannot reproduce the issue on my local workstation, OSX with Python installed via homebrew.

(pve) ~$TZ=Europe/Dublin pve/bin/python
Python 2.7.13 (default, Dec 18 2016, 07:03:39) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime
>>> from trac.util.datefmt import localtz, to_datetime, to_utimestamp
>>> localtz.utcoffset(datetime(2017, 1, 2))
datetime.timedelta(0)
>>> localtz.utcoffset(datetime(2016, 8, 2))
datetime.timedelta(0, 3600)
>>> localtz.utcoffset(datetime(1900, 1, 2))
datetime.timedelta(0)
>>> to_utimestamp(datetime(1900, 1, 2, tzinfo=localtz))
-2208902400000000L
>>> import pytz
>>> pytz.timezone('Europe/Dublin').utcoffset(datetime(1900, 1, 2))
datetime.timedelta(-1, 84900)

Anyway, that's probably not too important. There may just be some difference in the implementation on OSX. I also get the following on my workstation:

$ zdump -v Europe/Helsinki | head -5
Europe/Helsinki  Fri Dec 13 20:45:52 1901 UTC = Fri Dec 13 22:25:41 1901 HMT isdst=0
Europe/Helsinki  Sat Dec 14 20:45:52 1901 UTC = Sat Dec 14 22:25:41 1901 HMT isdst=0
Europe/Helsinki  Sat Apr 30 22:20:10 1921 UTC = Sat Apr 30 23:59:59 1921 HMT isdst=0
Europe/Helsinki  Sat Apr 30 22:20:11 1921 UTC = Sun May  1 00:20:11 1921 EET isdst=0
Europe/Helsinki  Thu Apr  2 21:59:59 1942 UTC = Thu Apr  2 23:59:59 1942 EET isdst=0

No gmtoff in output?

whereas trac-hacks.org gives:

$ zdump -v Europe/Helsinki | head -5
Europe/Helsinki  -9223372036854775808 = NULL
Europe/Helsinki  -9223372036854689408 = NULL
Europe/Helsinki  Thu May 30 22:20:10 1878 UT = Thu May 30 23:59:59 1878 LMT isdst=0 gmtoff=5989
Europe/Helsinki  Thu May 30 22:20:11 1878 UT = Fri May 31 00:00:00 1878 HMT isdst=0 gmtoff=5989
Europe/Helsinki  Sat Apr 30 22:20:10 1921 UT = Sat Apr 30 23:59:59 1921 HMT isdst=0 gmtoff=5989

[Jun Omae]

Replying to Ryan J Ollos:

That change sounds good. For time periods so far in the past I think it would also be fine to just raise a TracError, or otherwise handle the issue in any way that doesn't result in noisy logs. Since you have a fix though, we might as well just apply it.

Thanks. I'll apply it.

It seems strange to me that the issue occurs on the trac-hacks.org server though, because pytz is installed. I can reproduce your terminal session on that server:

Yes. However, [trac] default_timezone is empty on trac-hacks.org. If the user doesn't configure timezone in preferences panel, localtz is used. I think it would be good to set GMT to [trac] default_timezone.

It can be reproduced by ​https://trac-hacks.org/timeline?from=1900-01-01 with default timezone.

I cannot reproduce the issue on my local workstation, OSX with Python installed via homebrew.

Hmm, it seems that is another issue about LocalTimezoe on OSX, though the same results from libc on both Ubuntu and OSX.

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.10.5
BuildVersion:   14F1909
$ TZ=GMT date -r -2208902400
Tue Jan  2 00:00:00 GMT 1900
$ TZ=Europe/Dublin date -r -2208902400
Mon Jan  1 23:34:39 DMT 1900
$ TZ=Europe/Dublin ~/venv/trac/1.0.12/bin/python
Python 2.7.10 (default, Jul 14 2015, 19:46:27)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.localtime(-2208902400)
time.struct_time(tm_year=1900, tm_mon=1, tm_mday=1, tm_hour=23, tm_min=34, tm_sec=39, tm_wday=0, tm_yday=1, tm_isdst=0)

$ TZ=Europe/Dublin python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.localtime(-2208902400)
time.struct_time(tm_year=1900, tm_mon=1, tm_mday=1, tm_hour=23, tm_min=34, tm_sec=39, tm_wday=0, tm_yday=1, tm_isdst=0)

Anyway, that's probably not too important. There may just be some difference in the implementation on OSX. I also get the following on my workstation:

[…]

No gmtoff in output?

It seems zdump on OSX is old version than on Ubuntu.

$ zdump --version
zdump: @(#)zdump.c      7.31

$ zdump --version
zdump (Ubuntu EGLIBC 2.19-0ubuntu6.9) 2.19

[Jun Omae]

Replying to Jun Omae:

Hmm, it seems that is another issue about LocalTimezoe on OSX, though the same results from libc on both Ubuntu and OSX.

LocalTimezone.utcoffset() uses time.mktime(), however OverflowError is raised on OSX if time old than 1901-12-13T20:20:31 is given.

>>> time.mktime((1901, 12, 13, 20, 20, 31, -1, 0, 0))
-2147483648.0
>>> time.mktime((1901, 12, 13, 20, 20, 30, -1, 0, 0))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: mktime argument out of range

I noticed another issue on OSX. I'll fix it.

>>> localtz.utcoffset(datetime(1901, 12, 13, 20, 20, 31))
datetime.timedelta(-1, 84879)
>>> localtz.utcoffset(datetime(1901, 12, 13, 20, 20, 30))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "trac/util/datefmt.py", line 1059, in utcoffset
    return self._tzinfo(dt)._offset
  File "trac/util/datefmt.py", line 1040, in _tzinfo
    tz_offset = timedelta(seconds=utc_ts - time.mktime(tt) - 6 * 3600)
OverflowError: mktime argument out of range
>>> localtz.utcoffset(datetime(1901, 12, 13))
datetime.timedelta(0)

[Ryan J Ollos]

Replying to Jun Omae:

Yes. However, [trac] default_timezone is empty on trac-hacks.org. If the user doesn't configure timezone in preferences panel, localtz is used. I think it would be good to set GMT to [trac] default_timezone.

Thanks, I made the change.

[Jun Omae]

Proposed changes in [ab1b040f1/jomae.git]. Unit tests pass on ​Linux, OSX and ​Windows.

[Jun Omae]

Committed in [15635] and merged in [15636].