Edgewall Software
Modify

Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#10912 closed defect (fixed)

Wrong arithmetic datetime with LocalTimezone if across a DST boundary

Reported by: Jun Omae Owned by: Jun Omae
Priority: normal Milestone: 0.12.5
Component: general Version:
Severity: normal Keywords: datetime pytz timezone
Cc:
Release Notes:

Fix of datetime arithmetic with LocalTimezone across DST boundaries

API Changes:

localize and normalize of LocalTimezone now work like pytz.tzinfo when across DST boundaries

Description

If arithmetic datetime with LocalTimezone and timedelta across a DST boundary, the result have the lack of consistency.

$ TZ=Europe/Paris ~/venv/trac/0.12.4/bin/python
Python 2.4.3 (#1, Jun 18 2012, 08:55:31)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime, timedelta
>>> from trac.util.datefmt import to_datetime, localtz, to_timestamp, utc
>>> localtz
<LocalTimezone "CET" 1:00:00 "CEST" 2:00:00>
>>> t1 = datetime(2012, 3, 25, 1, 15, tzinfo=localtz)
>>> t2 = t1 + timedelta(hours=1)
>>> t1.isoformat()
'2012-03-25T01:15:00+01:00'
>>> t2.isoformat()
'2012-03-25T02:15:00+02:00'   # Expects 2012-03-25T02:15:00+01:00
                              #      or 2012-03-25T03:15:00+02:00
>>> t1 == t2
False
>>> t1 - t1.utcoffset() == t2 - t2.utcoffset()
True                          # Expects `False`
>>> to_timestamp(t2) - to_timestamp(t1)
0                             # Expects 3600

If arithmetic datetime with pytz across a DST boundary, the value of datetime.utcoffset() remains unchanged. tzinfo.normalize() method changes the utcoffset() value.

>>> from trac.util.datefmt import timezone
>>> tz = timezone('Europe/Paris')
>>> t1 = tz.normalize(tz.localize(datetime(2012, 3, 25, 1, 15)))
>>> t1
datetime.datetime(2012, 3, 25, 1, 15, tzinfo=<DstTzInfo 'Europe/Paris' CET+1:00:00 STD>)
>>> t2 = t1 + timedelta(hours=1)
>>> t1.isoformat()
'2012-03-25T01:15:00+01:00'
>>> t2.isoformat()
'2012-03-25T02:15:00+01:00'
>>> t3 = t2 + timedelta(hours=1)
>>> t3.isoformat()
'2012-03-25T03:15:00+01:00'
>>> tz.normalize(t2).isoformat()
'2012-03-25T03:15:00+02:00'

Attachments (0)

Change History (10)

comment:1 Changed 6 years ago by Christian Boos

On Windows, the first part works the way you expected:

>>> from datetime import datetime, timedelta
>>> from trac.util.datefmt import to_datetime, localtz, to_timestamp, utc
>>> localtz
<LocalTimezone "Romance Standard Time" 1:00:00 "Romance Daylight Time" 2:00:00>
>>> t1 = datetime(2012, 3, 25, 1, 15, tzinfo=localtz)
>>> t2 = t1 + timedelta(hours=1)
>>> t1.isoformat()
'2012-03-25T01:15:00+01:00'
>>> t2.isoformat()
'2012-03-25T02:15:00+01:00'
>>> t1 == t2
False
>>> t1 - t1.utcoffset() == t2 - t2.utcoffset()
False
>>> to_timestamp(t2) - to_timestamp(t1)
3600

However:

>>> t3 = t2 + timedelta(hours=1)
>>> t3.isoformat()
'2012-03-25T03:15:00+02:00'
>>> localtz.normalize(t2).isoformat()
'2012-03-25T02:15:00+01:00'
>>> localtz.normalize(t3).isoformat()
'2012-03-25T03:15:00+02:00'

So it also doesn't fully match what we get via pytz.

And it's therefore "buggy" as well, I think the criterion for correctness should definitely be localtz.normalize(t1 + timedelta(hours=1)).isoformat()'2012-03-25T03:15:00+02:00'.

comment:2 Changed 6 years ago by Genie

In paris, Daylight Saving Time started on 25th March 2012 at 2.00am.

in case t1 == '2012-03-25T01:15:00+01:00'

t1 + 1 hours == '2012-03-25T03:15:00+02:00'

comment:3 Changed 6 years ago by Genie

Keywords: pytz added

comment:4 Changed 6 years ago by Jun Omae

I worked in repos:jomae.git:ticket10912/localtz. After the changes, LocalTimezone behaves like pytz.tzinfo. I confirmed on CentOS 5.8 and Windows XP sp3 with timezone in Paris. It needs more testing….

comment:5 Changed 6 years ago by Christian Boos

[2ec1cf3e/jomae.git] could perhaps benefit from a LocalTimezone._localtime(dt) → localized time or None helper method, but other than that looks great! I'll test and let you know.

Last edited 6 years ago by Christian Boos (previous) (diff)

comment:6 Changed 6 years ago by Christian Boos

I just tested [63b4287d/jomae.git], redoing the steps from comment:1:

>>> localtz.normalize(t1 + timedelta(hours=0)).isoformat()
'2012-03-25T01:15:00+01:00'
>>> localtz.normalize(t1 + timedelta(hours=1)).isoformat()
'2012-03-25T03:15:00+02:00'
>>> localtz.normalize(t1 + timedelta(hours=2)).isoformat()
'2012-03-25T03:15:00+02:00'
>>> localtz.normalize(t1 + timedelta(hours=3)).isoformat()
'2012-03-25T04:15:00+02:00'

… so while the result for +1 hour is now fine, there's no differences between +1 and +2 in the above? I don't think that's correct.

And indeed, pytz gives:

>>> tz.normalize(t1 + timedelta(hours=0)).isoformat()
'2012-03-25T01:15:00+01:00'
>>> tz.normalize(t1 + timedelta(hours=1)).isoformat()
'2012-03-25T03:15:00+02:00'
>>> tz.normalize(t1 + timedelta(hours=2)).isoformat()
'2012-03-25T04:15:00+02:00'
>>> tz.normalize(t1 + timedelta(hours=3)).isoformat()
'2012-03-25T05:15:00+02:00'

But other than that, the new code looks nice, so I think we're on the right track ;-)

comment:7 in reply to:  6 ; Changed 6 years ago by Jun Omae

Thanks for the comments.

Replying to cboos:

I just tested [63b4287d/jomae.git], redoing the steps from comment:1:

>>> localtz.normalize(t1 + timedelta(hours=0)).isoformat()
'2012-03-25T01:15:00+01:00'
...
>>> localtz.normalize(t1 + timedelta(hours=2)).isoformat()
'2012-03-25T03:15:00+02:00'
...

… so while the result for +1 hour is now fine, there's no differences between +1 and +2 in the above? I don't think that's correct.

You're right. Ah, it seems difficult to fix without being localized, however, I'll try.

Also, time.mktime() is inconsistent with ambiguous times. The problem is caused by glibc. See time/mktime.c in glibc.git. I have yet to investigate the other C runtime.

$ PYTHONPATH=$PWD TZ=Europe/Paris ~/venv/py24/bin/python
Python 2.4.3 (#1, Jun 18 2012, 08:55:31)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.localtime(time.mktime((2001, 1, 1, 0, 0, 0, 0, 0, 0)))
(2001, 1, 1, 0, 0, 0, 0, 1, 0)
>>> time.localtime(time.mktime((2011, 10, 30, 2, 15, 0, 0, 0, -1)))
(2011, 10, 30, 2, 15, 0, 6, 303, 0)
>>> time.localtime(time.mktime((2001, 1, 1, 0, 0, 0, 0, 0, 1)))
(2000, 12, 31, 23, 0, 0, 6, 366, 0)
>>> time.localtime(time.mktime((2011, 10, 30, 2, 15, 0, 0, 0, -1)))
(2011, 10, 30, 2, 15, 0, 6, 303, 1)
>>>
Last edited 6 years ago by Jun Omae (previous) (diff)

comment:8 in reply to:  7 Changed 6 years ago by Jun Omae

Replying to jomae:

… so while the result for +1 hour is now fine, there's no differences between +1 and +2 in the above? I don't think that's correct.

You're right. Ah, it seems difficult to fix without being localized, however, I'll try.

I worked in [04eaf8aa/jomae.git]. After the changes, if the datetime object is not localized, LocalTimezone.normalize do nothing.

>>> t1 = datetime(2012, 3, 25, 1, 15, tzinfo=localtz)
>>> localtz.normalize(t1 + timedelta(hours=0)).isoformat()
'2012-03-25T01:15:00+01:00'
>>> localtz.normalize(t1 + timedelta(hours=1)).isoformat()
'2012-03-25T02:15:00+01:00'
>>> localtz.normalize(t1 + timedelta(hours=2)).isoformat()
'2012-03-25T03:15:00+02:00'
>>> localtz.normalize(t1 + timedelta(hours=3)).isoformat()
'2012-03-25T04:15:00+02:00'

Use to_datetime (call internally tz.localize() and tz.normalize()) to retrieve normalized datetime objects, in the same manner as pytz.tzinfo.

>>> t1 = to_datetime(datetime(2012, 3, 25, 1, 15), localtz)
>>> to_datetime(t1 + timedelta(hours=0), localtz).isoformat()
'2012-03-25T01:15:00+01:00'
>>> to_datetime(t1 + timedelta(hours=1), localtz).isoformat()
'2012-03-25T03:15:00+02:00'
>>> to_datetime(t1 + timedelta(hours=2), localtz).isoformat()
'2012-03-25T04:15:00+02:00'
>>> to_datetime(t1 + timedelta(hours=3), localtz).isoformat()
'2012-03-25T05:15:00+02:00'

Also, time.mktime() is inconsistent with ambiguous times. The problem is caused by glibc. See time/mktime.c in glibc.git. I have yet to investigate the other C runtime. ...

That issue is fixed in [c2ac69c7/jomae.git]. I confirmed with Python 2.5-2.7 on Linux 64-bits, Python 2.7 on FreeBSD 64-bits and Python 2.4-2.7 on Windows XP (32-bits).

comment:9 Changed 6 years ago by Jun Omae

API Changes: modified (diff)
Release Notes: modified (diff)
Resolution: fixed
Status: newclosed

The patch is applied in [11419-11421].

comment:10 Changed 6 years ago by Jun Omae

Owner: set to Jun Omae

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted.
to The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.