6 failures in unit-test with Babel 2.10.1

[Jun Omae]

​Babel 2.10.1 has been released recently, which upgraded ​CLDR 41 from ​CLDR 37.

---

======================================================================
FAIL: test_i18n_datetime_hint (trac.util.tests.datefmt.I18nDateFormatTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/tracdev/git/trac/util/tests/datefmt.py", line 1250, in test_i18n_datetime_hint
    self.assertIn(datefmt.get_datetime_format_hint(fr),
AssertionError: 'd MMM y, HH:mm:ss' not found in ('d MMM yyyy HH:mm:ss', 'd MMM y HH:mm:ss', "d MMM y 'à' HH:mm:ss")

======================================================================
FAIL: test_i18n_format_datetime (trac.util.tests.datefmt.I18nDateFormatTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/tracdev/git/trac/util/tests/datefmt.py", line 1175, in test_i18n_format_datetime
    self.assertIn(datefmt.format_datetime(t, tzinfo=tz, locale=fr),
AssertionError: '28 août 2010, 13:45:56' not found in ('28 août 2010 13:45:56', '28 août 2010 à 13:45:56')

======================================================================
FAIL: test_i18n_format_time (trac.util.tests.datefmt.I18nDateFormatTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/tracdev/git/trac/util/tests/datefmt.py", line 1233, in test_i18n_format_time
    self.assertIn(datefmt.format_time(t, tzinfo=tz, locale=zh_CN),
AssertionError: '13:45:56' not found in ('下午01:45:56', '下午1:45:56')

======================================================================
FAIL: test_i18n_parse_date_datetime (trac.util.tests.datefmt.I18nDateFormatTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/tracdev/git/trac/util/tests/datefmt.py", line 1367, in test_i18n_parse_date_datetime
    self.assertEqual(expected,
AssertionError: datet[16 chars]0, 8, 28, 13, 45, 56, tzinfo=<FixedOffset "GMT +2:00" 2:00:00>) != datet[16 chars]0, 8, 28, 1, 45, 56, tzinfo=<FixedOffset "GMT +2:00" 2:00:00>)

======================================================================
FAIL: test_i18n_parse_date_datetime_meridiem (trac.util.tests.datefmt.I18nDateFormatTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/tracdev/git/trac/util/tests/datefmt.py", line 1398, in test_i18n_parse_date_datetime_meridiem
    self.assertEqual(expected_am,
AssertionError: datet[16 chars]1, 2, 22, 0, 45, 56, tzinfo=<FixedOffset "GMT +2:00" 2:00:00>) != datet[16 chars]1, 2, 22, 12, 45, 56, tzinfo=<FixedOffset "GMT +2:00" 2:00:00>)

======================================================================
FAIL: test_i18n_parse_date_roundtrip (trac.util.tests.datefmt.I18nDateFormatTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/tracdev/git/trac/util/tests/datefmt.py", line 1454, in test_i18n_parse_date_roundtrip
    self.assertEqual(tz_t, actual,
AssertionError: datet[16 chars]0, 8, 28, 13, 45, 56, tzinfo=<FixedOffset "GMT +2:00" 2:00:00>) != datet[16 chars]0, 8, 28, 1, 45, 56, tzinfo=<FixedOffset "GMT +2:00" 2:00:00>) : datetime.datetime(2010, 8, 28, 13, 45, 56, tzinfo=<FixedOffset "GMT +2:00" 2:00:00>) != datetime.datetime(2010, 8, 28, 1, 45, 56, tzinfo=<FixedOffset "GMT +2:00" 2:00:00>) ('2010年8月28日 B1:45:56' Locale('zh', territory='TW', script='Hant'))

----------------------------------------------------------------------
Ran 2762 tests in 41.802s

FAILED (failures=6, skipped=5)

[Jun Omae]

Hmm, babel.dates.format_datetime() in Babel 2.10.1 is broken with zh_TW locale. 1:00 and 13:00 are formatted to incorrectly same text with zh_TW locale.

>>> from datetime import datetime
>>> import pytz
>>> import babel
>>> from babel import dates
>>> babel.__version__
'2.10.1'
>>> dates.get_time_format('medium', 'zh_TW')
<DateTimePattern 'Bh:mm:ss'>
>>> t1 = datetime(2010, 8, 28, 1, 45, 56, 0, pytz.UTC)
>>> t2 = datetime(2010, 8, 28, 13, 45, 56, 0, pytz.UTC)
>>> s1 = dates.format_datetime(t2, 'medium', t1.tzinfo, 'zh_TW')
>>> s1
'2010年8月28日 B1:45:56'
>>> s2 = dates.format_datetime(t2, 'medium', t2.tzinfo, 'zh_TW')
>>> s2
'2010年8月28日 B1:45:56'
>>> t1 == t2
False
>>> s1 == s2
True

[Jun Omae]

I've filed at ​https://github.com/python-babel/babel/issues/866.

B symbol in date pattern is not supported, however it is used in the pattern for zh_TW locale….

>>> fmt = dates.get_time_format('medium', 'zh_TW')
>>> vars(fmt)
{'pattern': 'Bh:mm:ss', 'format': 'B%(h)s:%(mm)s:%(ss)s'}

[Jun Omae]

I posted ​babel PR#869 for supporting B symbol in time format.

Also, proposed changes in jomae.git@trunk:t13482 (diff), tested with the PR.

[Jun Omae]

I noticed the proposed changes are incomplete. parse_date() handles only am/pm periods, however other periods are used in formatting time with zh_TW locale.

>>> for h in range(24):
...   d = datetime(2029, 4, 29, h, tzinfo=datefmt.utc)
...   print(d.isoformat(), '→', dates.format_datetime(d, tzinfo=d.tzinfo, locale=zh_TW))
...
2029-04-29T00:00:00+00:00 → 2029年4月29日 午夜12:00:00
2029-04-29T01:00:00+00:00 → 2029年4月29日 凌晨1:00:00
2029-04-29T02:00:00+00:00 → 2029年4月29日 凌晨2:00:00
2029-04-29T03:00:00+00:00 → 2029年4月29日 凌晨3:00:00
2029-04-29T04:00:00+00:00 → 2029年4月29日 凌晨4:00:00
2029-04-29T05:00:00+00:00 → 2029年4月29日 清晨5:00:00
2029-04-29T06:00:00+00:00 → 2029年4月29日 清晨6:00:00
2029-04-29T07:00:00+00:00 → 2029年4月29日 清晨7:00:00
2029-04-29T08:00:00+00:00 → 2029年4月29日 上午8:00:00
2029-04-29T09:00:00+00:00 → 2029年4月29日 上午9:00:00
2029-04-29T10:00:00+00:00 → 2029年4月29日 上午10:00:00
2029-04-29T11:00:00+00:00 → 2029年4月29日 上午11:00:00
2029-04-29T12:00:00+00:00 → 2029年4月29日 中午12:00:00
2029-04-29T13:00:00+00:00 → 2029年4月29日 下午1:00:00
2029-04-29T14:00:00+00:00 → 2029年4月29日 下午2:00:00
2029-04-29T15:00:00+00:00 → 2029年4月29日 下午3:00:00
2029-04-29T16:00:00+00:00 → 2029年4月29日 下午4:00:00
2029-04-29T17:00:00+00:00 → 2029年4月29日 下午5:00:00
2029-04-29T18:00:00+00:00 → 2029年4月29日 下午6:00:00
2029-04-29T19:00:00+00:00 → 2029年4月29日 晚上7:00:00
2029-04-29T20:00:00+00:00 → 2029年4月29日 晚上8:00:00
2029-04-29T21:00:00+00:00 → 2029年4月29日 晚上9:00:00
2029-04-29T22:00:00+00:00 → 2029年4月29日 晚上10:00:00
2029-04-29T23:00:00+00:00 → 2029年4月29日 晚上11:00:00
>>> datefmt.parse_date('2029年4月29日 晚上8:00:00', tzinfo=datefmt.utc, locale=zh_TW).isoformat()
'2029-04-29T08:00:00+00:00'  # => should be '2029-04-29T20:00:00+00:00' 

[Jun Omae]

Another point, datetime picker must parse formatted datetime string by format_datetime().

[Jun Omae]

Babel 2.10.3 is released recently. The B symbol in time format for zh_TW locale is supported now. I'll try to fix parse_date in Trac core….

>>> import babel
>>> babel.__version__
'2.10.3'
>>> from datetime import datetime
>>> from babel import dates
>>> dates.format_time(datetime(2022, 6, 24, 12, 34, 56), locale='zh_TW')
'中午12:34:56'

[Jun Omae]

Pinned Babel version to <2.10 until fixing this in [17634].

[Jun Omae]

Another issue, KeyError is caused by the B symbol in date format for zh-TW locale since Babel 2.10+ when visiting page which uses date picker. It is reported on ​gmessage:trac-users:g1ODKMSSndk/m/R6C1xZYhAQAJ.

14:38:21 Trac[main] ERROR: [192.168.11.100] Internal Server Error: <RequestWithSession "GET '/newticket'">, referrer None
Traceback (most recent call last):
  File "/venv/trac-1.6/lib/python3.11/site-packages/trac/web/main.py", line 609, in dispatch_request
    dispatcher.dispatch(req)
  File "/venv/trac-1.6/lib/python3.11/site-packages/trac/web/main.py", line 301, in dispatch
    raise e
  File "/venv/trac-1.6/lib/python3.11/site-packages/trac/web/main.py", line 247, in dispatch
    resp = chosen_handler.process_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/trac-1.6/lib/python3.11/site-packages/trac/ticket/web_ui.py", line 162, in process_request
    return self._process_newticket_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/trac-1.6/lib/python3.11/site-packages/trac/ticket/web_ui.py", line 547, in _process_newticket_request
    chrome.add_jquery_ui(req)
  File "/venv/trac-1.6/lib/python3.11/site-packages/trac/web/chrome.py", line 1135, in add_jquery_ui
    'time_format': get_time_format_jquery_ui(req.lc_time),
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/trac-1.6/lib/python3.11/site-packages/trac/util/datefmt.py", line 450, in get_time_format_jquery_ui
    return f % values
           ~~^~~~~~~~
KeyError: 'B'

[Jun Omae]

Proposed changes in jomae.git@1.6-stable:t13482 (diff), again. In the changes, format datetime using a period instead of b and B periods, adapt unit tests to Babel 2.14.0, and unpin Babel<2.10 of requirements.

[Jun Omae]

Fixed in [17771:17772] and merged [17773].