Wrong call of _tracadmin() in TestWikiReadonlyAttribute

[Jun Omae]

Call of _tracadmin() at tags/trac-1.2.5/trac/wiki/tests/functional.py@:198-199#L155​ should be set_config().

Also, the call must say the following output but we couldn't catch the error.

Error: Command not found
No documentation found for 'trac'. Use 'help' to see the list of commands.

[Ryan J Ollos]

My conclusion is that _tracadmin needs to be modified to check stderr. In one case we print a warning to stderr, so I modified that. All tests pass for me on OSX. Looking forward to your thoughts: [9aa60eeee/rjollos.git].

[Ryan J Ollos]

Replying to Ryan J Ollos:

In one case we print a warning to stderr, so I modified that.

Alternatively we could trap the exception from initenv when the functional test suite creates the environment:

$ make functional-test

[...]

Traceback (most recent call last):
  File "trac/tests/functional/__init__.py", line 224, in <module>
    unittest.main(defaultTest='test_suite')
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 95, in __init__
    self.runTests()
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 232, in runTests
    self.result = testRunner.run(self.test)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/runner.py", line 151, in run
    test(result)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in __call__
    return self.run(*args, **kwds)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 108, in run
    test(result)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in __call__
    return self.run(*args, **kwds)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/test.py", line 259, in run
    self.setUp()
  File "trac/tests/functional/__init__.py", line 144, in setUp
    self._testenv = self.env_class(env_path, port, baseurl)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/tests/functional/testenv.py", line 75, in __init__
    self.create()
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/tests/functional/testenv.py", line 143, in create
    self._tracadmin(*args)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/tests/functional/testenv.py", line 241, in _tracadmin
    % (args, proc.returncode, err))
Exception: Failed while running trac-admin with arguments ('initenv', '/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/testenv/trac', 'sqlite:db/trac.db', 'svn', "''").
Exitcode: 0

---------------------------------------------------------------------
Warning: couldn't index the default repository.

This can happen for a variety of reasons: wrong repository type,
no appropriate third party library for this repository type,
no repository at the specified repository path...

You can nevertheless start using your Trac environment, but
you'll need to check your `.type` and `.dir` option values in
the [repositories] section of your trac.ini file.


make: *** [functional-test] Error 1

Or change the arguments to initenv:

trac/tests/functional/testenv.py

@@ -136 +136 @@
         self.logfile = open(os.path.join(self.dirname, 'testing.log'), 'w')
         self.create_repo()
 
-        self._tracadmin('initenv', self.tracdir, self.dburi, self.repotype,
-                        self.repo_path_for_initenv())
+        repo_path = self.repo_path_for_initenv()
+        args = ['initenv', self.tracdir, self.dburi]
+        if repo_path:
+            args += [self.repotype, repo_path]
+        self._tracadmin(*args)
         if call([sys.executable,
                  os.path.join(self.trac_src, 'contrib', 'htpasswd.py'), "-c",
                  "-b", self.htpasswd, "admin", "admin"], close_fds=close_fds,
@@ -294 +297 @@
 
     def repo_path_for_initenv(self):
         """Default to no repository"""
-        return "''" # needed for Python 2.3 and 2.4 on win32
+        return None
 
     def call_in_dir(self, dir, args, environ=None):
         proc = Popen(args, stdout=PIPE, stderr=self.logfile,

The <repostype> <repospath> arguments were removed for Trac 1.3.3 (#12891)

[Ryan J Ollos]

Replying to Ryan J Ollos:

Or change the arguments to initenv:

I'm favoring this change: [24605cbb6/rjollos.git].

[Ryan J Ollos]

Should we cherry-pick [03fb919d/jomae.git] as well?: [6d4b9f669/rjollos.git]

[Jun Omae]

The commit cannot solve the issue when Python 2.x.

The args parameter of Popen() on Python 2.x requires a list of bytes instances, especially when Windows, encoded with ANSI code page. It means trac-admin command with unicode parameters leads a UnicodeError. In order to escape the limitation, stdin of trac-admin command is used to pass the parameters. In Python 3.x, args parameter of Popen() accepts a list of unicode instances.

See also r11947.

[Ryan J Ollos]

I see. How about [9834e866f/rjollos.git] to fix the problem for Python 2.6-2.7?

Windows tests are failing due to printing of stderr. I'll investigate.

[Ryan J Ollos]

Replying to Ryan J Ollos:

Windows tests are failing due to printing of stderr. I'll investigate.

Revised in [76274b3f3/rjollos.git]. With svn and git in tracopt, repositories can only be synchronized on environment creation if --inherit or --config is used to enable the components (comment:1:ticket:12891).

Tests pass on ​AppVeyor.

I'm unsure what is going on with ​Travis CI builds.

It runs make compile, but seems to skip the tests.

The command "if [ "$build" != minimum ]; then make compile; fi
if [ "$TRAVIS_BUILD_STAGE_NAME" = Test ]; then
  make Trac.egg-info unit-test functional-test;
fi
" exited with 0.

Investigating…

[Ryan J Ollos]

Replying to Ryan J Ollos:

Investigating…

Looks like $TRAVIS_BUILD_STAGE_NAME is test, not Test. I echoed it:

$ echo $TRAVIS_BUILD_STAGE_NAME
test

​Docs say:

TRAVIS_BUILD_STAGE_NAME: The build stage in capitalized form, e.g. Test or Deploy. If a build does not use build stages, this variable is empty ("").

Recent change to the platform? Maybe we should do a case-insensitive compare?

[Ryan J Ollos]

​Tests pass after [f591e3199/rjollos.git].

[Ryan J Ollos]

I was finally able to run tests against Subversion locally again. The latest Brew formulae installs 1.13.0_4, which results in:

(trac-2.7) trac-dev $ python -c "from svn import core"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/svn/core.py", line 26, in <module>
    from libsvn.core import *
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/libsvn/core.py", line 17, in <module>
    _core = swig_import_helper()
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/libsvn/core.py", line 16, in swig_import_helper
    return importlib.import_module('_core')
  File "/Users/rjollos/.pyenv/versions/2.7.17/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: dlopen(/usr/local/lib/svn-python/libsvn/_core.so, 2): Symbol not found: _PyBytes_AsString
  Referenced from: /usr/local/Cellar/subversion/1.13.0_4/lib/libsvn_swig_py-1.0.dylib
  Expected in: flat namespace
 in /usr/local/Cellar/subversion/1.13.0_4/lib/libsvn_swig_py-1.0.dylib

PyBytes_AsString is only in the ​Python 3 API.

Using the bindings with Python 3.8 also fails though:

(trac-py382) trac-dev $ python --version
Python 3.8.2
(trac-py382) trac-dev $ python -c "from svn import core"
Traceback (most recent call last):
  File "/Users/rjollos/.pyenv/versions/3.8.2/envs/trac-py382/lib/python3.8/site-packages/libsvn/core.py", line 14, in swig_import_helper
    return importlib.import_module(mname)
  File "/Users/rjollos/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 657, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 556, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1101, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: dlopen(/Users/rjollos/.pyenv/versions/3.8.2/envs/trac-py382/lib/python3.8/site-packages/libsvn/_core.so, 2): Symbol not found: _PyFile_AsFile
  Referenced from: /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib
  Expected in: flat namespace
 in /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/rjollos/.pyenv/versions/3.8.2/envs/trac-py382/lib/python3.8/site-packages/svn/core.py", line 26, in <module>
    from libsvn.core import *
  File "/Users/rjollos/.pyenv/versions/3.8.2/envs/trac-py382/lib/python3.8/site-packages/libsvn/core.py", line 17, in <module>
    _core = swig_import_helper()
  File "/Users/rjollos/.pyenv/versions/3.8.2/envs/trac-py382/lib/python3.8/site-packages/libsvn/core.py", line 16, in swig_import_helper
    return importlib.import_module('_core')
  File "/Users/rjollos/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: dlopen(/usr/local/lib/svn-python/libsvn/_core.so, 2): Symbol not found: _PyFile_AsFile
  Referenced from: /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib
  Expected in: flat namespace
 in /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib

I think the problem is probably due to the switch to Python 3.8 as a dependency in ​e073d0ba. I assume that causes the SWIG bindings to be built against 3.8, but I haven't traced that to be sure. I don't think Subversion 1.13 has Python bindings that are compatible with Python 3, but I also have not dug into that.

It works fine to force install a previous version (1.13.0_2) of the formulae:

$ brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/b309c2efabeb977aa97288efb85d7ee5ad2808b0/Formula/subversion.rb
$ brew pin subversion

(trac-py382) Formula (master)$ pwd
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula
(trac-py382) Formula (master)$ git log -2 bb6240cb4a -- subversion.rb
commit bb6240cb4af2fd9aef34aa765dca666c822ff871
Author: Michka Popoff <michkapopoff@gmail.com>
Date:   Tue Feb 18 12:38:14 2020 +0100

    subversion: use python@3.8

    Closes #50391.

    Signed-off-by: Michka Popoff <michkapopoff@gmail.com>

commit b309c2efabeb977aa97288efb85d7ee5ad2808b0
Author: Jonathan Chang <jchang641@gmail.com>
Date:   Tue Feb 18 09:43:55 2020 +1100

    subversion: add uses_from_macos dependencies (#49909)

As a proof that b309c2ef is the latest formulae that can be used, using the next revision (e073d0ba) fails:

$ brew unpin subversion
$ brew uninstall subversion 
$ brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/bb6240cb4af2fd9aef34aa765dca666c822ff871/Formula/subversion.rb
$ python -c "from svn import core"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/svn/core.py", line 26, in <module>
    from libsvn.core import *
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/libsvn/core.py", line 17, in <module>
    _core = swig_import_helper()
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/libsvn/core.py", line 16, in swig_import_helper
    return importlib.import_module('_core')
  File "/Users/rjollos/.pyenv/versions/2.7.17/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: dlopen(/usr/local/opt/subversion/lib/svn-python/libsvn/_core.so, 2): Symbol not found: _PyBytes_AsString
  Referenced from: /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib
  Expected in: flat namespace
 in /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib

I should probably just build Subversion and the bindings from source for my dev environment.

I might report this to Homebrew if I have time to investigate further.

[Jun Omae]

Replying to Ryan J Ollos:

I was finally able to run tests against Subversion locally again. The latest Brew formulae installs 1.13.0_4, which results in:

ImportError: dlopen(/usr/local/lib/svn-python/libsvn/_core.so, 2): Symbol not found: _PyBytes_AsString
                    ^^^^^^^^^^^^^^

I guess you've installed manually subversion with /usr/local as prefix. Delete /usr/local/lib/svn-python directory and retry.

Also, I recommend to pass site directory to swig_pydir and swig_pydir_extra variables on running make like ​https://github.com/jun66j5/trac/blob/1.5-py3-t11988/.travis.yml#L151.

[Ryan J Ollos]

Replying to Jun Omae:

I guess you've installed manually subversion with /usr/local as prefix. Delete /usr/local/lib/svn-python directory and retry.

That's the location that Brew symlinks:

$ ls -al /usr/local/lib/svn-python
lrwxr-xr-x  1 rjollos  admin  44 Apr  9 20:50 /usr/local/lib/svn-python -> ../Cellar/subversion/1.13.0_2/lib/svn-python

[Ryan J Ollos]

Replying to Jun Omae:

Also, I recommend to pass site directory to swig_pydir and swig_pydir_extra variables on running make like ​https://github.com/jun66j5/trac/blob/1.5-py3-t11988/.travis.yml#L151.

Thanks. When building from source I've been symlinking from the python environment to svn-python/svn and svn-python/libsvn, which I think is equivalent.

./configure --with-apr=$(brew --prefix apr) --with-apr-util=$(brew --prefix apr-util) --with-py3c=./py3c --prefix=$(pwd)/local
make all swig-py install install-swig-py

svn_python=$(pwd)/local/lib/svn-python
site_packages=$(find $(pyenv prefix)/ -name "site-packages" -print)
ln -sf "$svn_python/svn" "$site_packages/"
ln -sf "$svn_python/libsvn" "$site_packages/"
echo "$svn_python/libsvn" > "$site_packages/svn.pth"

[anonymous]

Just to summarize my previous findings:

• The subversion bindings installed from Homebrew fail with Python 2.7 for Subversion formulae 1.13.0_3 or later. The last revision that works with Python 2.7 is b309c2ef. These bindings also fail with Python 3.8.
• Subversion bindings built from source 1.14dev, and now 1.15dev, work for me with Python 3.5-3.8. I haven't tried them with Python 2.7, they ​claim compatibility (for now).

I've ​reported the issue to Homebrew.

[Ryan J Ollos]

Replying to anonymous:

I've ​reported the issue to Homebrew.

Issue fixed in ​e54983e4. The 1.13.0_5 formulae works.

[Ryan J Ollos]

Replying to Ryan J Ollos:

Replying to Ryan J Ollos:

Investigating…

Looks like $TRAVIS_BUILD_STAGE_NAME is test, not Test. I echoed it:

$ echo $TRAVIS_BUILD_STAGE_NAME
test

​Docs say:

TRAVIS_BUILD_STAGE_NAME: The build stage in capitalized form, e.g. Test or Deploy. If a build does not use build stages, this variable is empty ("").

Recent change to the platform? Maybe we should do a case-insensitive compare?

The builds don't fail for the Edgewall Travis CI account, but they do fail ​for my fork.

Here is a hint. The variables Test and Deploy are capitalized on Edgewall, but not for my fork:

I'm considering posting a question to the ​community.

[Ryan J Ollos]

Replying to Ryan J Ollos:

I'm considering posting a question to the ​community.

Someone already reported ​here. I added a reply.

We could use their workaround: ${TRAVIS_BUILD_STAGE_NAME} = [Tt]est

[Ryan J Ollos]

Replying to Ryan J Ollos:

Someone already reported ​here. I added a reply.

We could use their workaround: ${TRAVIS_BUILD_STAGE_NAME} = [Tt]est

I'm waiting a few days to take action on this, to see if there are follow-up on that forum topic. Seems like this should affect a good number of Travis CI users.

Committed changes to 1.2-stable in r17269, merged in r17270, r17271.

The changes in this ticket fit what I had in mind as "Internal Changes": ​gmessage:trac-dev:CNrnrXJ9A3o/rEayodNRBAAJ.

[Ryan J Ollos]

Replying to Ryan J Ollos:

I'm waiting a few days to take action on this, to see if there are follow-up on that forum topic. Seems like this should affect a good number of Travis CI users.

With Build Config Validation enabled I get test. With it disabled I get Test. So I disabled ​Build Config Validation on my fork. It's a ​beta feature.

[Ryan J Ollos]

Replying to Ryan J Ollos:

With Build Config Validation enabled I get test. With it disabled I get Test. So I disabled ​Build Config Validation on my fork. It's a ​beta feature.

⇒ #13274.