Edgewall Software
Modify ↓

Opened 3 years ago

Closed 3 years ago

#13361 closed defect (fixed)

test_pretty_dateinfo failing when Babel is unavailable

Reported by: Jun Omae Owned by: Jun Omae
Priority: normal Milestone: 1.5.3
Component: general Version:
Severity: normal Keywords:
Cc: Branch:
Release Notes:

Workaround for test failures due to changing locale on call of tidylib.tidy_document.

API Changes:
Internal Changes:

Description

I noticed the following test fails while working tests with build=minimum on GitHub Actions. 

======================================================================
FAIL: test_pretty_dateinfo (trac.web.tests.chrome.ChromeTemplateRenderingTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/tracdev/git/trac/web/tests/chrome.py", line 1118, in test_pretty_dateinfo
    self.assertRegex(content, textwrap.dedent("""\
AssertionError: Regex didn't match: '<!DOCTYPE html>\n<html>\n<body>\n<ul>\n<li></li>\n<li><span title="07/01/(20)?07 12:34:56">\\d+ years ago</span></li>\n<li><span title="07/01/(20)?07 12:34:56">\\d+ years ago</span></li>\n<li><span title="\\d+ years ago">on 07/01/(20)?07</span></li>\n<li><span title="\\d+ years ago">on 07/01/(20)?07 at 12:34:56</span></li>\n<li><span title="\\d+ years ago">07/01/(20)?07</span></li>\n<li><span title="\\d+ years ago">07/01/(20)?07 12:34:56</span></li>\n</ul>\n</body>\n</html>' not found in '<!DOCTYPE html>\n<html>\n<body>\n<ul>\n<li></li>\n<li><span title="07/01/2007 12:34:56 PM">14 years ago</span></li>\n<li><span title="07/01/2007 12:34:56 PM">14 years ago</span></li>\n<li><span title="14 years ago">on 07/01/2007</span></li>\n<li><span title="14 years ago">on 07/01/2007 at 12:34:56 PM</span></li>\n<li><span title="14 years ago">07/01/2007</span></li>\n<li><span title="14 years ago">07/01/2007 12:34:56 PM</span></li>\n</ul>\n</body>\n</html>' : <!DOCTYPE html>
<html>
<body>
<ul>
<li></li>
<li><span title="07/01/2007 12:34:56 PM">14 years ago</span></li>
<li><span title="07/01/2007 12:34:56 PM">14 years ago</span></li>
<li><span title="14 years ago">on 07/01/2007</span></li>
<li><span title="14 years ago">on 07/01/2007 at 12:34:56 PM</span></li>
<li><span title="14 years ago">07/01/2007</span></li>
<li><span title="14 years ago">07/01/2007 12:34:56 PM</span></li>
</ul>
</body>
</html>

Work around is to add export LANC=C before make unit-test but the issue is not fixed on Windows. setlocale() in C runtime on Windows doesn't respect LANG, LC_* environments.

Attachments (0)

Change History (5)

comment:1 by Jun Omae, 3 years ago

I found the setlocale() is invoked by each call of tidylib.tidy_document.

refs. ​https://github.com/htacg/tidy-html5/blob/5.6.0/src/tidylib.c#L122 

$ ~/venv/py39/bin/python
Python 3.9.2 (default, Feb 20 2021, 20:59:40)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> from locale import LC_ALL, setlocale
>>> from datetime import datetime
>>> datetime.now().strftime('%x %X')
'03/16/21 20:26:03'
>>> curr = setlocale(LC_ALL, None)
>>> curr
'LC_CTYPE=en_US.UTF8;LC_NUMERIC=C;LC_TIME=C;LC_COLLATE=C;LC_MONETARY=C;LC_MESSAGES=C;LC_PAPER=C;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=C;LC_IDENTIFICATION=C'
>>>
>>> from tidylib import tidy_document
>>> datetime.now().strftime('%x %X')
'03/16/21 20:26:22'
>>> setlocale(LC_ALL, None)
'LC_CTYPE=en_US.UTF8;LC_NUMERIC=C;LC_TIME=C;LC_COLLATE=C;LC_MONETARY=C;LC_MESSAGES=C;LC_PAPER=C;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=C;LC_IDENTIFICATION=C'
>>>
>>> tidy_document('<html><body></body></html>')
('<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">\n<html>\n  <head>\n    <title></title>\n  </head>\n  <body>\n  </body>\n</html>\n', "line 1 column 1 - Warning: missing <!DOCTYPE> declaration\nline 1 column 7 - Warning: inserting missing 'title' element\n")
>>> datetime.now().strftime('%x %X')
'03/16/2021 08:27:00 PM'         # <== locale is changed
>>> setlocale(LC_ALL, None)
'en_US.UTF8'                     # <==
>>>
>>> setlocale(LC_ALL, curr)
'LC_CTYPE=en_US.UTF8;LC_NUMERIC=C;LC_TIME=C;LC_COLLATE=C;LC_MONETARY=C;LC_MESSAGES=C;LC_PAPER=C;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=C;LC_IDENTIFICATION=C'
>>> datetime.now().strftime('%x %X')
'03/16/21 20:30:47'
>>> setlocale(LC_ALL, None)
'LC_CTYPE=en_US.UTF8;LC_NUMERIC=C;LC_TIME=C;LC_COLLATE=C;LC_MONETARY=C;LC_MESSAGES=C;LC_PAPER=C;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=C;LC_IDENTIFICATION=C'
>>>
>>> tidy_document('<html>')
('<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">\n<html>\n  <head>\n    <title></title>\n  </head>\n  <body>\n  </body>\n</html>\n', "line 1 column 1 - Warning: missing <!DOCTYPE> declaration\nline 1 column 7 - Warning: inserting missing 'title' element\n")
>>> datetime.now().strftime('%x %X')
'03/16/2021 08:31:03 PM'
>>> setlocale(LC_ALL, None)
'en_US.UTF8'

Also, in tidy-html5 5.7.28, calls of setlocale() have been removed.

comment:2 by Jun Omae, 3 years ago

Component: i18n → general
Owner: set to Jun Omae
Status: new → assigned

Proposed changes restore the locale after call of tidylib.tidy_document.

  • trac/tests/functional/better_twill.py 

    diff --git a/trac/tests/functional/better_twill.py b/trac/tests/functional/better_twill.py
index eeb9fdb21..adbf25faa 100755
    a b It also handles twill's absense.  
    1919import hashlib
    2020import http.client
    2121import http.server
     22import locale
    2223import re
    2324import os.path
    2425import socketserver
    … … try:  
    3637except ImportError:
    3738    selenium = None
    3839
     40_curr = locale.setlocale(locale.LC_ALL, None)
    3941try:
    40     from tidylib import tidy_document
    41     tidy_document('<!DOCTYPE html><html><body></body></html>')
     42    import tidylib
     43    tidylib.tidy_document('<!DOCTYPE html><html><body></body></html>')
    4244except ImportError:
    4345    print("SKIP: validation of HTML output in functional tests"
    4446          " (no tidylib installed)")
    … … except OSError as e:  
    4749    print("SKIP: validation of HTML output in functional tests"
    4850          " (no tidy dynamic library installed: %s)" % e)
    4951    tidy_document = None
     52else:
     53    if _curr == locale.setlocale(locale.LC_ALL, None):
     54        tidy_document = tidylib.tidy_document
     55    else:
     56        def tidy_document(*args, **kwargs):
     57            curr = locale.setlocale(locale.LC_ALL, None)
     58            try:
     59                return tidylib.tidy_document(*args, **kwargs)
     60            finally:
     61                # Restore the locale because tidy-html5 library changes the
     62                # locale each call of tidy_document if 5.6.0 or early.
     63                locale.setlocale(locale.LC_ALL, curr)
     64finally:
     65    if _curr != locale.setlocale(locale.LC_ALL, None):
     66        locale.setlocale(locale.LC_ALL, _curr)
     67    del _curr
    5068

comment:3 by Jun Omae, 3 years ago

Proposed changes in log:jomae.git@trunk:t13361.

In addition, I think it would be good to show version and name of libtidy shared library. 

$ make python=39 status
...
  Docutils     : 0.16
  Selenium     : 3.141.0
  PyTidyLib    : 0.3.2 (5.6.0 libtidy.so.5deb1)
  LXML         : 4.6.2
  coverage     : 5.5
...

comment:4 by Ryan J Ollos, 3 years ago

The change looks good. Tested on macOS.

comment:5 by Jun Omae, 3 years ago

Release Notes: modified (diff)
Resolution: → fixed
Status: assigned → closed

Thanks for the reviewing. Committed in [17517:17518].

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Jun Omae to the specified user.

Add Comment
E-mail address and name can be saved in the Preferences .
Attachments ↑ Description ↑
 
Note: See TracTickets for help on using tickets.