Edgewall Software
Modify

Opened 12 years ago

Closed 12 years ago

Last modified 12 years ago

#10912 closed defect (fixed)

Wrong arithmetic datetime with LocalTimezone if across a DST boundary

Reported by: Jun Omae Owned by: Jun Omae
Priority: normal Milestone: 0.12.5
Component: general Version:
Severity: normal Keywords: datetime pytz timezone
Cc: Branch:
Release Notes:

Fix of datetime arithmetic with LocalTimezone across DST boundaries

API Changes:

localize and normalize of LocalTimezone now work like pytz.tzinfo when across DST boundaries

Internal Changes:

Description

If arithmetic datetime with LocalTimezone and timedelta across a DST boundary, the result have the lack of consistency.

$ TZ=Europe/Paris ~/venv/trac/0.12.4/bin/python
Python 2.4.3 (#1, Jun 18 2012, 08:55:31)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime, timedelta
>>> from trac.util.datefmt import to_datetime, localtz, to_timestamp, utc
>>> localtz
<LocalTimezone "CET" 1:00:00 "CEST" 2:00:00>
>>> t1 = datetime(2012, 3, 25, 1, 15, tzinfo=localtz)
>>> t2 = t1 + timedelta(hours=1)
>>> t1.isoformat()
'2012-03-25T01:15:00+01:00'
>>> t2.isoformat()
'2012-03-25T02:15:00+02:00'   # Expects 2012-03-25T02:15:00+01:00
                              #      or 2012-03-25T03:15:00+02:00
>>> t1 == t2
False
>>> t1 - t1.utcoffset() == t2 - t2.utcoffset()
True                          # Expects `False`
>>> to_timestamp(t2) - to_timestamp(t1)
0                             # Expects 3600

If arithmetic datetime with pytz across a DST boundary, the value of datetime.utcoffset() remains unchanged. tzinfo.normalize() method changes the utcoffset() value.

>>> from trac.util.datefmt import timezone
>>> tz = timezone('Europe/Paris')
>>> t1 = tz.normalize(tz.localize(datetime(2012, 3, 25, 1, 15)))
>>> t1
datetime.datetime(2012, 3, 25, 1, 15, tzinfo=<DstTzInfo 'Europe/Paris' CET+1:00:00 STD>)
>>> t2 = t1 + timedelta(hours=1)
>>> t1.isoformat()
'2012-03-25T01:15:00+01:00'
>>> t2.isoformat()
'2012-03-25T02:15:00+01:00'
>>> t3 = t2 + timedelta(hours=1)
>>> t3.isoformat()
'2012-03-25T03:15:00+01:00'
>>> tz.normalize(t2).isoformat()
'2012-03-25T03:15:00+02:00'

Attachments (0)

Change History (10)

comment:1 by Christian Boos, 12 years ago

On Windows, the first part works the way you expected:

>>> from datetime import datetime, timedelta
>>> from trac.util.datefmt import to_datetime, localtz, to_timestamp, utc
>>> localtz
<LocalTimezone "Romance Standard Time" 1:00:00 "Romance Daylight Time" 2:00:00>
>>> t1 = datetime(2012, 3, 25, 1, 15, tzinfo=localtz)
>>> t2 = t1 + timedelta(hours=1)
>>> t1.isoformat()
'2012-03-25T01:15:00+01:00'
>>> t2.isoformat()
'2012-03-25T02:15:00+01:00'
>>> t1 == t2
False
>>> t1 - t1.utcoffset() == t2 - t2.utcoffset()
False
>>> to_timestamp(t2) - to_timestamp(t1)
3600

However:

>>> t3 = t2 + timedelta(hours=1)
>>> t3.isoformat()
'2012-03-25T03:15:00+02:00'
>>> localtz.normalize(t2).isoformat()
'2012-03-25T02:15:00+01:00'
>>> localtz.normalize(t3).isoformat()
'2012-03-25T03:15:00+02:00'

So it also doesn't fully match what we get via pytz.

And it's therefore "buggy" as well, I think the criterion for correctness should definitely be localtz.normalize(t1 + timedelta(hours=1)).isoformat()'2012-03-25T03:15:00+02:00'.

comment:2 by Genie, 12 years ago

In paris, Daylight Saving Time started on 25th March 2012 at 2.00am.

in case t1 == '2012-03-25T01:15:00+01:00'

t1 + 1 hours == '2012-03-25T03:15:00+02:00'

comment:3 by Genie, 12 years ago

Keywords: pytz added

comment:4 by Jun Omae, 12 years ago

I worked in repos:jomae.git:ticket10912/localtz. After the changes, LocalTimezone behaves like pytz.tzinfo. I confirmed on CentOS 5.8 and Windows XP sp3 with timezone in Paris. It needs more testing….

comment:5 by Christian Boos, 12 years ago

[2ec1cf3e/jomae.git] could perhaps benefit from a LocalTimezone._localtime(dt) → localized time or None helper method, but other than that looks great! I'll test and let you know.

Last edited 12 years ago by Christian Boos (previous) (diff)

comment:6 by Christian Boos, 12 years ago

I just tested [63b4287d/jomae.git], redoing the steps from comment:1:

>>> localtz.normalize(t1 + timedelta(hours=0)).isoformat()
'2012-03-25T01:15:00+01:00'
>>> localtz.normalize(t1 + timedelta(hours=1)).isoformat()
'2012-03-25T03:15:00+02:00'
>>> localtz.normalize(t1 + timedelta(hours=2)).isoformat()
'2012-03-25T03:15:00+02:00'
>>> localtz.normalize(t1 + timedelta(hours=3)).isoformat()
'2012-03-25T04:15:00+02:00'

… so while the result for +1 hour is now fine, there's no differences between +1 and +2 in the above? I don't think that's correct.

And indeed, pytz gives:

>>> tz.normalize(t1 + timedelta(hours=0)).isoformat()
'2012-03-25T01:15:00+01:00'
>>> tz.normalize(t1 + timedelta(hours=1)).isoformat()
'2012-03-25T03:15:00+02:00'
>>> tz.normalize(t1 + timedelta(hours=2)).isoformat()
'2012-03-25T04:15:00+02:00'
>>> tz.normalize(t1 + timedelta(hours=3)).isoformat()
'2012-03-25T05:15:00+02:00'

But other than that, the new code looks nice, so I think we're on the right track ;-)

in reply to:  6 ; comment:7 by Jun Omae, 12 years ago

Thanks for the comments.

Replying to cboos:

I just tested [63b4287d/jomae.git], redoing the steps from comment:1:

>>> localtz.normalize(t1 + timedelta(hours=0)).isoformat()
'2012-03-25T01:15:00+01:00'
...
>>> localtz.normalize(t1 + timedelta(hours=2)).isoformat()
'2012-03-25T03:15:00+02:00'
...

… so while the result for +1 hour is now fine, there's no differences between +1 and +2 in the above? I don't think that's correct.

You're right. Ah, it seems difficult to fix without being localized, however, I'll try.

Also, time.mktime() is inconsistent with ambiguous times. The problem is caused by glibc. See time/mktime.c in glibc.git. I have yet to investigate the other C runtime.

$ PYTHONPATH=$PWD TZ=Europe/Paris ~/venv/py24/bin/python
Python 2.4.3 (#1, Jun 18 2012, 08:55:31)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.localtime(time.mktime((2001, 1, 1, 0, 0, 0, 0, 0, 0)))
(2001, 1, 1, 0, 0, 0, 0, 1, 0)
>>> time.localtime(time.mktime((2011, 10, 30, 2, 15, 0, 0, 0, -1)))
(2011, 10, 30, 2, 15, 0, 6, 303, 0)
>>> time.localtime(time.mktime((2001, 1, 1, 0, 0, 0, 0, 0, 1)))
(2000, 12, 31, 23, 0, 0, 6, 366, 0)
>>> time.localtime(time.mktime((2011, 10, 30, 2, 15, 0, 0, 0, -1)))
(2011, 10, 30, 2, 15, 0, 6, 303, 1)
>>>
Last edited 12 years ago by Jun Omae (previous) (diff)

in reply to:  7 comment:8 by Jun Omae, 12 years ago

Replying to jomae:

… so while the result for +1 hour is now fine, there's no differences between +1 and +2 in the above? I don't think that's correct.

You're right. Ah, it seems difficult to fix without being localized, however, I'll try.

I worked in [04eaf8aa/jomae.git]. After the changes, if the datetime object is not localized, LocalTimezone.normalize do nothing.

>>> t1 = datetime(2012, 3, 25, 1, 15, tzinfo=localtz)
>>> localtz.normalize(t1 + timedelta(hours=0)).isoformat()
'2012-03-25T01:15:00+01:00'
>>> localtz.normalize(t1 + timedelta(hours=1)).isoformat()
'2012-03-25T02:15:00+01:00'
>>> localtz.normalize(t1 + timedelta(hours=2)).isoformat()
'2012-03-25T03:15:00+02:00'
>>> localtz.normalize(t1 + timedelta(hours=3)).isoformat()
'2012-03-25T04:15:00+02:00'

Use to_datetime (call internally tz.localize() and tz.normalize()) to retrieve normalized datetime objects, in the same manner as pytz.tzinfo.

>>> t1 = to_datetime(datetime(2012, 3, 25, 1, 15), localtz)
>>> to_datetime(t1 + timedelta(hours=0), localtz).isoformat()
'2012-03-25T01:15:00+01:00'
>>> to_datetime(t1 + timedelta(hours=1), localtz).isoformat()
'2012-03-25T03:15:00+02:00'
>>> to_datetime(t1 + timedelta(hours=2), localtz).isoformat()
'2012-03-25T04:15:00+02:00'
>>> to_datetime(t1 + timedelta(hours=3), localtz).isoformat()
'2012-03-25T05:15:00+02:00'

Also, time.mktime() is inconsistent with ambiguous times. The problem is caused by glibc. See time/mktime.c in glibc.git. I have yet to investigate the other C runtime. ...

That issue is fixed in [c2ac69c7/jomae.git]. I confirmed with Python 2.5-2.7 on Linux 64-bits, Python 2.7 on FreeBSD 64-bits and Python 2.4-2.7 on Windows XP (32-bits).

comment:9 by Jun Omae, 12 years ago

API Changes: modified (diff)
Release Notes: modified (diff)
Resolution: fixed
Status: newclosed

The patch is applied in [11419-11421].

comment:10 by Jun Omae, 12 years ago

Owner: set to Jun Omae

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.