ImportError: cannot import name 'soft_unicode' from 'jinja2.utils' with Jinja2 3.0.x

[Jun Omae]

======================================================================
ERROR: test (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test
Traceback (most recent call last):
  File "/opt/python/3.9.1/lib/python3.9/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/home/travis/build/edgewall/trac/trac/test.py", line 47, in <module>
    from trac.env import Environment
  File "/home/travis/build/edgewall/trac/trac/env.py", line 52, in <module>
    from trac.web.chrome import Chrome
  File "/home/travis/build/edgewall/trac/trac/web/chrome.py", line 40, in <module>
    from trac.mimeview.api import RenderingContext, get_mimetype
  File "/home/travis/build/edgewall/trac/trac/mimeview/__init__.py", line 14, in <module>
    from trac.mimeview.api import *
  File "/home/travis/build/edgewall/trac/trac/mimeview/api.py", line 70, in <module>
    from trac.resource import Resource
  File "/home/travis/build/edgewall/trac/trac/resource.py", line 21, in <module>
    from trac.util.presentation import classes
  File "/home/travis/build/edgewall/trac/trac/util/presentation.py", line 26, in <module>
    from jinja2.utils import soft_unicode
ImportError: cannot import name 'soft_unicode' from 'jinja2.utils' (/home/travis/virtualenv/python3.9.1/lib/python3.9/site-packages/jinja2/utils.py)

Jinja2 2.11.3 says the following warning when calling soft_unicode() but no warnings when importing it.

$ ~/venv/py39/bin/python
Python 3.9.4 (default, Apr  9 2021, 01:15:05)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import jinja2
>>> jinja2.__version__
'2.11.3'
>>> from jinja2.utils import soft_unicode
>>> soft_unicode('***')
<stdin>:1: DeprecationWarning: 'jinja2.utils.soft_unicode' will be removed in version 3.0. Use 'markupsafe.soft_unicode' instead.
'***'

[Jun Omae]

2 deprecated warnings are raised while running unit-test with Jinja2 3.0.0:

trac/util/presentation.py:133: DeprecationWarning: 'jinja2.Markup' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.Markup' instead.
  rv = Markup(rv)
/venv/py39/lib/python3.9/site-packages/jinja2/environment.py:119: DeprecationWarning: The 'with' extension is deprecated and will be removed in Jinja 3.1. This is built in now.
  result[extension.identifier] = extension(environment)

Also, 2 failures.

======================================================================
FAIL: test_timestamp_columns (trac.ticket.tests.report.ReportModuleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/tracdev/git/trac/ticket/tests/report.py", line 410, in test_timestamp_columns
    rendered = Chrome(self.env).render_template(req, rv[0], rv[1],
  File "/home/jun66j5/src/tracdev/git/trac/web/chrome.py", line 1377, in render_template
    template, data = self.prepare_template(req, filename, data, text,
...
  File "/venv/py39/lib/python3.9/site-packages/jinja2/compiler.py", line 812, in pop_assign_tracking
    ref = frame.symbols.ref(name)
  File "/venv/py39/lib/python3.9/site-packages/jinja2/idtracking.py", line 80, in ref
    raise AssertionError(
AssertionError: Tried to resolve a name to a reference that was unknown to the frame ('header_group')

======================================================================
FAIL: test_valid_html_for_report (trac.ticket.tests.report.ReportModuleTestCase)
----------------------------------------------------------------------
...
  File "/venv/py39/lib/python3.9/site-packages/jinja2/idtracking.py", line 80, in ref
    raise AssertionError(
AssertionError: Tried to resolve a name to a reference that was unknown to the frame ('header_group')

FAILED (failures=2, skipped=5)

[Jun Omae]

The AssertionError: Tried to resolve a name ... has been filed at ​https://github.com/pallets/jinja/issues/1427 and a workaround has been posted in the page.

Proposed changes in jomae.git@t13404, tested with Jinja 3.0.0 and 2.11.3.

[Ryan J Ollos]

Changes look good. Do you think we should create another release soon since this upgrading Jinja2 will cause breakage in production?

[Jun Omae]

Thanks for the reviewing. Committed in [17543].

Hmm, I don't think new release is not necessary for this issue. We don't recommend to use 1.5.x releasees in production because it is for development. It is easy to downgrade to Jinja2<3 for 1.5.x users.

[Tomasz Brzezina <tomasz@…>]

I'm trying to install trac with pip and i've got this message having Jinja2….

[Jun Omae]

Try to downgrade Jinja2 using pip install 'Jinja2<3'. See comment:4.

[Tomasz Brzezina <tomasz@…>]

I suggested 3.0.0 (I saw Jinja3.0.0 not Jinja2 3.0.0 :D)

[msiz]

I encountered this issue. Regarding with this issue, there is the following ​warning message in Markupsafe 2.0.1 source code.

'soft_unicode' has been renamed to 'soft_str'. The old name will be removed in MarkupSafe 2.1.

I applied the following patch to work around.

(_venv) PS C:\Users\...\trac> git diff
diff --git a/trac/util/presentation.py b/trac/util/presentation.py
index fe5824728..82c53c86c 100644
--- a/trac/util/presentation.py
+++ b/trac/util/presentation.py
@@ -23,7 +23,10 @@ import re

 from jinja2 import Markup, Undefined, contextfilter, evalcontextfilter
 from jinja2.filters import make_attrgetter
-from jinja2.utils import soft_unicode
+try:
+    from jinja2.utils import soft_unicode
+except:
+    from markupsafe import soft_str as soft_unicode

 from trac.core import TracError
 from .datefmt import to_utimestamp, utc

I use this patch on trac-1.5.3 tagged release in Windows10, Python 3.9.1 environment. I would appreciate if any Trac developer take a look at this patch.

[Jun Omae]

Thanks for the reporting. The markupsafe 2.1.0 is released yesterday (2022-02-18). Fixed in [17575].