0.11.3 ticket query error: NameError: global name 'set' is not defined

[dgaulke@…]

The python 2.3 incompatibility is back with 0.11.3. I get the following in the logs when going to "Custom Query". I wish I could upgrade python, but still not an option for me.

  File "/usr/lib/python2.3/site-packages/Trac-0.11.3-py2.3.egg/trac/ticket/query.py", line 577, in template_data
    wikify = set([f['name'] for f in self.fields
NameError: global name 'set' is not defined

[Christian Boos]

Hm, too bad, r7893 was not complete ;-)

[Christian Boos]

One of the problem is that none of the Trac developers seem to be using 2.3 anymore.

Jonas, you mentioned on IRC that you couldn't run a buildbot for 2.3 as it doesn't support virtualenv. I'm not familiar with buildbot, so I don't know if the requirement for virtualenv comes from buildbot usage or if you just used virtualenv to setup the test environments for 2.4, 2.5 and 2.6. If it's the latter, then I think you could nevertheless have a test environment for Python 2.3 without virtualenv, as we have to test Trac 0.11-stable only with that version.

[Christian Boos]

dgaulke: the problem you had should be fixed in r7903.

Can you please verify that it works now? Can you check for other possible Python 2.3 issues? If there are any, we need to know before releasing 0.11.3.1.

[dgaulke@…]

Replying to cboos: I have installed r7903 and it fixes the custom query problem. I have tried everything else I can think of and have not found any other Python 2.3 issues. Thanks for the help on this issue.

[Remy Blank]

Oh. So I broke 2.3. Again. Shesh…

Note to self: be much more careful when backporting stuff from trunk to 0.11-stable.

I agree with Christian, we need to improve our 2.3 testing. But I understand that not being able to use virtualenv is a major drawback. I'll try to set up a 2.3 environment on my new Mac mini, as it seems to be quite easy to install various Python versions using Macports (this is my first Mac, so it might take a little while). Maybe I can even set it up to build automatically for Trac's Bitten, but I'll have to catch up on reading the docs.

[Remy Blank]

First batch of Python 2.3 compatibility fixes

[Remy Blank]

I have managed to set up a 2.3 development environment, and there are a few other issues with 2.3 compatibility. The patch above addresses a few test errors, where the incompatible code is in the test suite. However:

• I still have two failing test cases, due to unicode objects not having a decode() method in 2.3. This might become tricky, because the missing method cannot be monkey-patched into unicode. OTOH, I'm not sure how likely we are to call decode() on a unicode object, maybe this is restricted to the test suite as well. Here's the traceback:

  ======================================================================
  ERROR: test_mixed_changeset (trac.versioncontrol.tests.svn_fs.NonSelfContainedScopedTestCase)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/home/joe/src/trac/0.11-stable-2.3/trac/versioncontrol/tests/svn_fs.py", line 711, in setUp
      logger_factory('test'))
    File "/home/joe/src/trac/0.11-stable-2.3/trac/versioncontrol/svn_fs.py", line 431, in __init__
      self.scope = path_utf8[len(root_path_utf8):].decode('utf-8')
  AttributeError: 'unicode' object has no attribute 'decode'
  
  ======================================================================
  ERROR: test_mixed_changeset_with_edit (trac.versioncontrol.tests.svn_fs.AnotherNonSelfContainedScopedTestCase)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/home/joe/src/trac/0.11-stable-2.3/trac/versioncontrol/tests/svn_fs.py", line 729, in setUp
      logger_factory('test'))
    File "/home/joe/src/trac/0.11-stable-2.3/trac/versioncontrol/svn_fs.py", line 431, in __init__
      self.scope = path_utf8[len(root_path_utf8):].decode('utf-8')
  AttributeError: 'unicode' object has no attribute 'decode'
  
  ----------------------------------------------------------------------
  Ran 719 tests in 24.709s
  
  FAILED (errors=2)
  

• The functional tests are currently disabled, due to subprocess missing in 2.3. I'll see if I can swap in popen() to get the functional tests to run.

IMO, we are getting pretty close to the point where we won't be able to support 2.3 anymore. Even installing Python 2.3 was a pain: I had to backport a fix from 2.4 to allow it to compile with a recent readline.

Do we really want to continue supporting 2.3 for the remaining of the 0.11 series?

[Christian Boos]

Replying to rblank:

I still have two failing test cases, due to unicode objects not having a decode() method in 2.3.

That's actually "good", as calling decode on an unicode object can be considered a bug.

Here:

self.scope = path_utf8[len(root_path_utf8):].decode('utf-8')

AttributeError: 'unicode' object has no attribute 'decode'

it's clearly a bug, as path_utf8 should be an utf-8 encoded str object.

I guess I really have to install 2.3 again as well. Anyway on Windows it's useful for building packages installers (bdist_wininst) which don't have extra runtime dependencies.

Do we really want to continue supporting 2.3 for the remaining of the 0.11 series?

Yes, I think so.

[Christian Boos]

Replying to rblank:

But I understand that not being able to use virtualenv is a major drawback.

How so, as Python 2.3 should be dedicated to testing one Trac branch and one set of support packages?

[Christian Boos]

I guess I really have to install 2.3 again as well.

Ok, so I did (2.3.5 for win32) and the first problem I had was that I hit a unittest incompatibility:

trac/test.py

@@ -127 +127 @@
         self.tearDown()
         return result
 
+    def __call__(self, result=None): # Python 2.3 compatibility
+        return self.run(result)
 
 class TestCaseSetup(unittest.TestCase):
     def setFixture(self, fixture):

Without that change, the SubversionRepositoryTestSetup.setup method is never called. So I'm a bit surprised it worked for you.

Then, once this is fixed, I get into further trouble with repos.svn_repos_create failing in the tests but not at the command line, followed by a problem with repos.svn_repos_load_fs2 failing at the command line as well… Well, I don't want to go down that dead-end more than that, so I'll stop there.

Anyway, the path_utf8 bug should be fixed in r7903.

[Christian Boos]

(r7904 actually)

[Christian Boos]

Replying to cboos:

… So I'm a bit surprised it worked for you.

Oops, I successfully managed to miss your patch until now :-) Sorry!

[Remy Blank]

Replying to cboos:

That's actually "good", as calling decode on an unicode object can be considered a bug.

I thought it was a "no operation", but looking at the Python code, it doesn't seem to be. I wonder what it does…

But I understand that not being able to use virtualenv is a major drawback.

How so, as Python 2.3 should be dedicated to testing one Trac branch and one set of support packages?

Well, having gone through this yesterday evening, one of the annoying things with setuptools is that it installs "non-versioned" scripts, i.e. in addition to e.g. /usr/bin/easy_install-2.3, it also installs /usr/bin/easy_install, and therefore overwrites the one installed with my package manager. The same applies to /usr/bin/pygmentize, /usr/bin/tracd, /usr/bin/trac-admin, …

With virtualenv, all scripts are installed under {venv}/bin, and as an additional benefit, you don't need superuser permissions. None of this is blocking, but virtualenv is just so much more convenient.

Oops, I successfully managed to miss your patch until now :-) Sorry!

Heh, took me about two hours to come up with that patch, so it might as well be useful ;-)

[kamil@…]

Just happened across this.

@rblank: You mentioned the tests don't run because subprocess isn't included in 2.3. You can drop in a copy of the subprocess module from a later Python version and it should work. It's pure python and supposed to be backwards compatible down to version 2.2

[Remy Blank]

Replying to kamil@…:

You can drop in a copy of the subprocess module from a later Python version and it should work. It's pure python and supposed to be backwards compatible down to version 2.2

Right, I remember having read that when 2.4 came out. That will save me quite some work. Thanks for the reminder!

Let's see…

Ran 813 tests in 279.681s

OK

All right!

[Christian Boos]

Replying to rblank:

Replying to cboos:

That's actually "good", as calling decode on an unicode object can be considered a bug.

I thought it was a "no operation", but looking at the Python code, it doesn't seem to be. I wonder what it does…

unicode.decode(encoding) does unicode.encode(sys.getdefaultencoding()).decode(encoding) … which breaks as soon as you have non-ascii characters in your unicode string, since the getdefaultencoding() is usually 'ascii'. Because of that, this call to decode will then fail with an UnicodeEncodeError…

But I understand that not being able to use virtualenv is a major drawback.

How so, as Python 2.3 should be dedicated to testing one Trac branch and one set of support packages?

Well, having gone through this yesterday evening, one of the annoying things with setuptools is that it installs "non-versioned" scripts, i.e. in addition to e.g. /usr/bin/easy_install-2.3, it also installs /usr/bin/easy_install, and therefore overwrites the one installed with my package manager. The same applies to /usr/bin/pygmentize, /usr/bin/tracd, /usr/bin/trac-admin, …

But that's only true if you have installed Python 2.3 itself at that location (/usr), and if you did that, then you probably also overwrote your previous default /usr/bin/python…

If instead you build and install Python with a configuration parameter of e.g. --prefix=/opt/python-2.3.5, then easy_install will install everything below /opt/python-2.3.5/bin.

[Remy Blank]

Patch applied in [7908]. All tests (except those requiring php, which I don't have at hand) pass with Python 2.3.

I'll close this ticket against 0.11.4 for now. We can move it back if we do a .1 release.