Edgewall Software
Modify

Opened 3 years ago

Closed 3 years ago

#12617 closed defect (fixed)

ValueError: tzinfo.utcoffset() must return a whole number of minutes

Reported by: Ryan J Ollos Owned by: Jun Omae
Priority: normal Milestone: 1.2.1
Component: timeline Version: 1.0.13
Severity: normal Keywords:
Cc: Branch:
Release Notes:

Fix ValueError from datetime with local timezone when the offset isn't a whole number of minutes.

API Changes:

Description

I've seen a few instances of this error in the logs over the past few days.

2016-11-07 20:39:41,364 Trac[web_ui] ERROR: Timeline event provider failed: 
Traceback (most recent call last):
  File "/srv/trac-hacks.org/pve/lib/python2.7/site-packages/trac/timeline/web_ui.py", line 198, in process_request
    filters) or []:
  File "/srv/trac-hacks.org/pve/lib/python2.7/site-packages/trac/ticket/web_ui.py", line 249, in get_timeline_events
    ts_start = to_utimestamp(start)
  File "/srv/trac-hacks.org/pve/lib/python2.7/site-packages/trac/util/datefmt.py", line 193, in to_utimestamp
    diff = dt - _epoc
ValueError: tzinfo.utcoffset() must return a whole number of minutes

Attachments (0)

Change History (12)

comment:1 by Ryan J Ollos, 3 years ago

Milestone: next-stable-1.0.xnext-stable-1.2.x

Moved ticket assigned to next-stable-1.0.x since maintenance of 1.0.x is coming to a close. Please move the ticket back if it's critical to fix on 1.0.x.

comment:2 by Ryan J Ollos, 3 years ago

Milestone: next-stable-1.2.x1.2.1
Owner: set to Ryan J Ollos
Status: newassigned

comment:3 by Ryan J Ollos, 3 years ago

Milestone: 1.2.1next-dev-1.3.x

comment:4 by Ryan J Ollos, 3 years ago

Possible hints in Python-issue:1447945, but I haven't been able to reproduce yet.

comment:5 by Jun Omae, 3 years ago

Several time zones don't have a whole number of minutes as offset in the past. However, Python's tzinfo class requires a whole number of minutes as offset. Also, pytz library modifies the offset to avoid the error.

$ TZ=Europe/Dublin venv/trac/1.0.13/bin/python
>>> from datetime import datetime
>>> from trac.util.datefmt import localtz, to_datetime, to_utimestamp
>>> localtz.utcoffset(datetime(2017, 1, 2))
datetime.timedelta(0)
>>> localtz.utcoffset(datetime(2016, 8, 2))
datetime.timedelta(0, 3600)
>>> localtz.utcoffset(datetime(1900, 1, 2))
datetime.timedelta(-1, 84879)
>>> to_utimestamp(datetime(1900, 1, 2, tzinfo=localtz))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/venv/trac/1.0.13/lib/python2.5/site-packages/trac/util/datefmt.py", line 193, in to_utimestamp
    diff = dt - _epoc
ValueError: tzinfo.utcoffset() must return a whole number of minutes
>>> import pytz
>>> pytz.timezone('Europe/Dublin').utcoffset(datetime(1900, 1, 2))
datetime.timedelta(-1, 84900)

See also:

Last edited 3 years ago by Ryan J Ollos (previous) (diff)

comment:6 by Jun Omae, 3 years ago

We could correct such an offset of timezone with the same way of pytz, https://github.com/stub42/pytz/blob/release_2016.10/src/pytz/tzfile.py#L116:

  • trac/util/datefmt.py

    diff --git a/trac/util/datefmt.py b/trac/util/datefmt.py
    index a719637c4..cf4739b70 100644
    a b class LocalTimezone(tzinfo):  
    10561056        return False
    10571057
    10581058    def utcoffset(self, dt):
    1059         return self._tzinfo(dt)._offset
     1059        offset = self._tzinfo(dt)._offset
     1060        seconds = offset.days * 86400 + offset.seconds
     1061        mod = seconds % 60
     1062        if mod != 0:
     1063            # Avoid "ValueError: tzinfo.utcoffset() must return a whole
     1064            # number of minutes" (#12617)
     1065            offset = timedelta(seconds=int((seconds + 30) // 60) * 60)
     1066        return offset
    10601067
    10611068    def dst(self, dt):
    10621069        if self._is_dst(dt):
  • trac/util/tests/datefmt.py

    diff --git a/trac/util/tests/datefmt.py b/trac/util/tests/datefmt.py
    index ed102b14c..48aae7c13 100644
    a b class LocalTimezoneTestCase(unittest.TestCase):  
    15971597                         datetime.datetime(2011, 10, 30, 2, 45, 42, 123456,
    15981598                                           datefmt.localtz).utcoffset())
    15991599
     1600    def test_utcoffset_non_whole_number_of_minutes(self):
     1601        self._tzset('Europe/Dublin')
     1602        dt = datetime.datetime(1901, 1, 2)
     1603        self.assertEqual(datetime.timedelta(days=-1, seconds=84900),
     1604                         datefmt.localtz.utcoffset(dt))
     1605
    16001606    def test_localized_non_existent_time(self):
    16011607        self._tzset('Europe/Paris')
    16021608        dt = datetime.datetime(2012, 3, 25, 2, 15, 42, 123456)
Last edited 3 years ago by Jun Omae (previous) (diff)

comment:7 by Ryan J Ollos, 3 years ago

Milestone: next-dev-1.3.x1.2.1

That change sounds good. For time periods so far in the past I think it would also be fine to just raise a TracError, or otherwise handle the issue in any way that doesn't result in noisy logs. Since you have a fix though, we might as well just apply it.

It seems strange to me that the issue occurs on the trac-hacks.org server though, because pytz is installed. I can reproduce your terminal session on that server:

# TZ=Europe/Dublin pve/bin/python
Python 2.7.9 (default, Jun 29 2016, 13:08:31) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime
>>> from trac.util.datefmt import localtz, to_datetime, to_utimestamp
>>> localtz.utcoffset(datetime(2017, 1, 2))
datetime.timedelta(0)
>>> localtz.utcoffset(datetime(2016, 8, 2))
datetime.timedelta(0, 3600)
>>> localtz.utcoffset(datetime(1900, 1, 2))
datetime.timedelta(-1, 84879)
>>> to_utimestamp(datetime(1900, 1, 2, tzinfo=localtz))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/trac/util/datefmt.py", line 193, in to_utimestamp
    diff = dt - _epoc
ValueError: tzinfo.utcoffset() must return a whole number of minutes
>>> import pytz
>>> pytz.timezone('Europe/Dublin').utcoffset(datetime(1900, 1, 2))
datetime.timedelta(-1, 84900)

I cannot reproduce the issue on my local workstation, OSX with Python installed via homebrew.

(pve) ~$TZ=Europe/Dublin pve/bin/python
Python 2.7.13 (default, Dec 18 2016, 07:03:39) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime
>>> from trac.util.datefmt import localtz, to_datetime, to_utimestamp
>>> localtz.utcoffset(datetime(2017, 1, 2))
datetime.timedelta(0)
>>> localtz.utcoffset(datetime(2016, 8, 2))
datetime.timedelta(0, 3600)
>>> localtz.utcoffset(datetime(1900, 1, 2))
datetime.timedelta(0)
>>> to_utimestamp(datetime(1900, 1, 2, tzinfo=localtz))
-2208902400000000L
>>> import pytz
>>> pytz.timezone('Europe/Dublin').utcoffset(datetime(1900, 1, 2))
datetime.timedelta(-1, 84900)

Anyway, that's probably not too important. There may just be some difference in the implementation on OSX. I also get the following on my workstation:

$ zdump -v Europe/Helsinki | head -5
Europe/Helsinki  Fri Dec 13 20:45:52 1901 UTC = Fri Dec 13 22:25:41 1901 HMT isdst=0
Europe/Helsinki  Sat Dec 14 20:45:52 1901 UTC = Sat Dec 14 22:25:41 1901 HMT isdst=0
Europe/Helsinki  Sat Apr 30 22:20:10 1921 UTC = Sat Apr 30 23:59:59 1921 HMT isdst=0
Europe/Helsinki  Sat Apr 30 22:20:11 1921 UTC = Sun May  1 00:20:11 1921 EET isdst=0
Europe/Helsinki  Thu Apr  2 21:59:59 1942 UTC = Thu Apr  2 23:59:59 1942 EET isdst=0

No gmtoff in output?

whereas trac-hacks.org gives:

$ zdump -v Europe/Helsinki | head -5
Europe/Helsinki  -9223372036854775808 = NULL
Europe/Helsinki  -9223372036854689408 = NULL
Europe/Helsinki  Thu May 30 22:20:10 1878 UT = Thu May 30 23:59:59 1878 LMT isdst=0 gmtoff=5989
Europe/Helsinki  Thu May 30 22:20:11 1878 UT = Fri May 31 00:00:00 1878 HMT isdst=0 gmtoff=5989
Europe/Helsinki  Sat Apr 30 22:20:10 1921 UT = Sat Apr 30 23:59:59 1921 HMT isdst=0 gmtoff=5989

in reply to:  7 ; comment:8 by Jun Omae, 3 years ago

Replying to Ryan J Ollos:

That change sounds good. For time periods so far in the past I think it would also be fine to just raise a TracError, or otherwise handle the issue in any way that doesn't result in noisy logs. Since you have a fix though, we might as well just apply it.

Thanks. I'll apply it.

It seems strange to me that the issue occurs on the trac-hacks.org server though, because pytz is installed. I can reproduce your terminal session on that server:

Yes. However, [trac] default_timezone is empty on trac-hacks.org. If the user doesn't configure timezone in preferences panel, localtz is used. I think it would be good to set GMT to [trac] default_timezone.

It can be reproduced by https://trac-hacks.org/timeline?from=1900-01-01 with default timezone.

I cannot reproduce the issue on my local workstation, OSX with Python installed via homebrew.

Hmm, it seems that is another issue about LocalTimezoe on OSX, though the same results from libc on both Ubuntu and OSX.

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.10.5
BuildVersion:   14F1909
$ TZ=GMT date -r -2208902400
Tue Jan  2 00:00:00 GMT 1900
$ TZ=Europe/Dublin date -r -2208902400
Mon Jan  1 23:34:39 DMT 1900
$ TZ=Europe/Dublin ~/venv/trac/1.0.12/bin/python
Python 2.7.10 (default, Jul 14 2015, 19:46:27)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.localtime(-2208902400)
time.struct_time(tm_year=1900, tm_mon=1, tm_mday=1, tm_hour=23, tm_min=34, tm_sec=39, tm_wday=0, tm_yday=1, tm_isdst=0)
$ TZ=Europe/Dublin python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.localtime(-2208902400)
time.struct_time(tm_year=1900, tm_mon=1, tm_mday=1, tm_hour=23, tm_min=34, tm_sec=39, tm_wday=0, tm_yday=1, tm_isdst=0)

Anyway, that's probably not too important. There may just be some difference in the implementation on OSX. I also get the following on my workstation:

[…]

No gmtoff in output?

It seems zdump on OSX is old version than on Ubuntu.

$ zdump --version
zdump: @(#)zdump.c      7.31
$ zdump --version
zdump (Ubuntu EGLIBC 2.19-0ubuntu6.9) 2.19

in reply to:  8 comment:9 by Jun Omae, 3 years ago

Replying to Jun Omae:

Hmm, it seems that is another issue about LocalTimezoe on OSX, though the same results from libc on both Ubuntu and OSX.

LocalTimezone.utcoffset() uses time.mktime(), however OverflowError is raised on OSX if time old than 1901-12-13T20:20:31 is given.

>>> time.mktime((1901, 12, 13, 20, 20, 31, -1, 0, 0))
-2147483648.0
>>> time.mktime((1901, 12, 13, 20, 20, 30, -1, 0, 0))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: mktime argument out of range

I noticed another issue on OSX. I'll fix it.

>>> localtz.utcoffset(datetime(1901, 12, 13, 20, 20, 31))
datetime.timedelta(-1, 84879)
>>> localtz.utcoffset(datetime(1901, 12, 13, 20, 20, 30))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "trac/util/datefmt.py", line 1059, in utcoffset
    return self._tzinfo(dt)._offset
  File "trac/util/datefmt.py", line 1040, in _tzinfo
    tz_offset = timedelta(seconds=utc_ts - time.mktime(tt) - 6 * 3600)
OverflowError: mktime argument out of range
>>> localtz.utcoffset(datetime(1901, 12, 13))
datetime.timedelta(0)

in reply to:  8 comment:10 by Ryan J Ollos, 3 years ago

Replying to Jun Omae:

Yes. However, [trac] default_timezone is empty on trac-hacks.org. If the user doesn't configure timezone in preferences panel, localtz is used. I think it would be good to set GMT to [trac] default_timezone.

Thanks, I made the change.

comment:11 by Jun Omae, 3 years ago

Owner: changed from Ryan J Ollos to Jun Omae

Proposed changes in [ab1b040f1/jomae.git]. Unit tests pass on Linux, OSX and Windows.

comment:12 by Jun Omae, 3 years ago

Release Notes: modified (diff)
Resolution: fixed
Status: assignedclosed

Committed in [15635] and merged in [15636].

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted. Next status will be 'reopened'.
to as closed The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.