Edgewall Software
Modify

Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#13257 closed defect (fixed)

Wrong call of _tracadmin() in TestWikiReadonlyAttribute

Reported by: Jun Omae Owned by: Ryan J Ollos
Priority: normal Milestone: 1.2.6
Component: general Version:
Severity: normal Keywords:
Cc: Branch:
Release Notes:
API Changes:
Internal Changes:

Fixed uncaught failure in TestWikiReadonlyAttribute functional test case.

Improvements to function test environment:

  • Log stderr from FunctionalTestEnvironment._tracadmin and raise exception.
  • Add default repository through config file passed to intenv.

Description

Call of _tracadmin() at tags/trac-1.2.5/trac/wiki/tests/functional.py@:198-199#L155 should be set_config().

Also, the call must say the following output but we couldn't catch the error.

Error: Command not found
No documentation found for 'trac'. Use 'help' to see the list of commands.

Attachments (2)

Screen Shot 2020-04-23 at 12.53.31.jpg (17.9 KB ) - added by Ryan J Ollos 4 years ago.
Screen Shot 2020-04-23 at 12.53.49.jpg (25.7 KB ) - added by Ryan J Ollos 4 years ago.

Download all attachments as: .zip

Change History (22)

comment:1 by Ryan J Ollos, 4 years ago

Owner: set to Ryan J Ollos
Status: newassigned

My conclusion is that _tracadmin needs to be modified to check stderr. In one case we print a warning to stderr, so I modified that. All tests pass for me on OSX. Looking forward to your thoughts: [9aa60eeee/rjollos.git].

Last edited 4 years ago by Ryan J Ollos (previous) (diff)

in reply to:  1 ; comment:2 by Ryan J Ollos, 4 years ago

Replying to Ryan J Ollos:

In one case we print a warning to stderr, so I modified that.

Alternatively we could trap the exception from initenv when the functional test suite creates the environment:

$ make functional-test

[...]

Traceback (most recent call last):
  File "trac/tests/functional/__init__.py", line 224, in <module>
    unittest.main(defaultTest='test_suite')
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 95, in __init__
    self.runTests()
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 232, in runTests
    self.result = testRunner.run(self.test)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/runner.py", line 151, in run
    test(result)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in __call__
    return self.run(*args, **kwds)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 108, in run
    test(result)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in __call__
    return self.run(*args, **kwds)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/test.py", line 259, in run
    self.setUp()
  File "trac/tests/functional/__init__.py", line 144, in setUp
    self._testenv = self.env_class(env_path, port, baseurl)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/tests/functional/testenv.py", line 75, in __init__
    self.create()
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/tests/functional/testenv.py", line 143, in create
    self._tracadmin(*args)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/tests/functional/testenv.py", line 241, in _tracadmin
    % (args, proc.returncode, err))
Exception: Failed while running trac-admin with arguments ('initenv', '/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/testenv/trac', 'sqlite:db/trac.db', 'svn', "''").
Exitcode: 0

---------------------------------------------------------------------
Warning: couldn't index the default repository.

This can happen for a variety of reasons: wrong repository type,
no appropriate third party library for this repository type,
no repository at the specified repository path...

You can nevertheless start using your Trac environment, but
you'll need to check your `.type` and `.dir` option values in
the [repositories] section of your trac.ini file.


make: *** [functional-test] Error 1

Or change the arguments to initenv:

  • trac/tests/functional/testenv.py

    diff --git a/trac/tests/functional/testenv.py b/trac/tests/functional/testenv.py
    index 51fb93d8c..03c98f6b5 100755
    a b class FunctionalTestEnvironment(object):  
    136136        self.logfile = open(os.path.join(self.dirname, 'testing.log'), 'w')
    137137        self.create_repo()
    138138
    139         self._tracadmin('initenv', self.tracdir, self.dburi, self.repotype,
    140                         self.repo_path_for_initenv())
     139        repo_path = self.repo_path_for_initenv()
     140        args = ['initenv', self.tracdir, self.dburi]
     141        if repo_path:
     142            args += [self.repotype, repo_path]
     143        self._tracadmin(*args)
    141144        if call([sys.executable,
    142145                 os.path.join(self.trac_src, 'contrib', 'htpasswd.py'), "-c",
    143146                 "-b", self.htpasswd, "admin", "admin"], close_fds=close_fds,
    class FunctionalTestEnvironment(object):  
    294297
    295298    def repo_path_for_initenv(self):
    296299        """Default to no repository"""
    297         return "''" # needed for Python 2.3 and 2.4 on win32
     300        return None
    298301
    299302    def call_in_dir(self, dir, args, environ=None):
    300303        proc = Popen(args, stdout=PIPE, stderr=self.logfile,

The <repostype> <repospath> arguments were removed for Trac 1.3.3 (#12891)

in reply to:  2 comment:3 by Ryan J Ollos, 4 years ago

Replying to Ryan J Ollos:

Or change the arguments to initenv:

I'm favoring this change: [24605cbb6/rjollos.git].

comment:4 by Ryan J Ollos, 4 years ago

Should we cherry-pick [03fb919d/jomae.git] as well?: [6d4b9f669/rjollos.git]

comment:5 by Jun Omae, 4 years ago

The commit cannot solve the issue when Python 2.x.

The args parameter of Popen() on Python 2.x requires a list of bytes instances, especially when Windows, encoded with ANSI code page. It means trac-admin command with unicode parameters leads a UnicodeError. In order to escape the limitation, stdin of trac-admin command is used to pass the parameters. In Python 3.x, args parameter of Popen() accepts a list of unicode instances.

See also r11947.

comment:6 by Ryan J Ollos, 4 years ago

I see. How about [9834e866f/rjollos.git] to fix the problem for Python 2.6-2.7?

Windows tests are failing due to printing of stderr. I'll investigate.

Last edited 4 years ago by Ryan J Ollos (previous) (diff)

in reply to:  6 ; comment:7 by Ryan J Ollos, 4 years ago

Replying to Ryan J Ollos:

Windows tests are failing due to printing of stderr. I'll investigate.

Revised in [76274b3f3/rjollos.git]. With svn and git in tracopt, repositories can only be synchronized on environment creation if --inherit or --config is used to enable the components (comment:1:ticket:12891).

Tests pass on AppVeyor.

I'm unsure what is going on with Travis CI builds.

It runs make compile, but seems to skip the tests.

The command "if [ "$build" != minimum ]; then make compile; fi
if [ "$TRAVIS_BUILD_STAGE_NAME" = Test ]; then
  make Trac.egg-info unit-test functional-test;
fi
" exited with 0.

Investigating…

in reply to:  7 ; comment:8 by Ryan J Ollos, 4 years ago

Replying to Ryan J Ollos:

Investigating…

Looks like $TRAVIS_BUILD_STAGE_NAME is test, not Test. I echoed it:

$ echo $TRAVIS_BUILD_STAGE_NAME
test

Docs say:

TRAVIS_BUILD_STAGE_NAME: The build stage in capitalized form, e.g. Test or Deploy. If a build does not use build stages, this variable is empty ("").

Recent change to the platform? Maybe we should do a case-insensitive compare?

comment:10 by Ryan J Ollos, 4 years ago

I was finally able to run tests against Subversion locally again. The latest Brew formulae installs 1.13.0_4, which results in:

(trac-2.7) trac-dev $ python -c "from svn import core"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/svn/core.py", line 26, in <module>
    from libsvn.core import *
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/libsvn/core.py", line 17, in <module>
    _core = swig_import_helper()
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/libsvn/core.py", line 16, in swig_import_helper
    return importlib.import_module('_core')
  File "/Users/rjollos/.pyenv/versions/2.7.17/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: dlopen(/usr/local/lib/svn-python/libsvn/_core.so, 2): Symbol not found: _PyBytes_AsString
  Referenced from: /usr/local/Cellar/subversion/1.13.0_4/lib/libsvn_swig_py-1.0.dylib
  Expected in: flat namespace
 in /usr/local/Cellar/subversion/1.13.0_4/lib/libsvn_swig_py-1.0.dylib

PyBytes_AsString is only in the Python 3 API.

Using the bindings with Python 3.8 also fails though:

(trac-py382) trac-dev $ python --version
Python 3.8.2
(trac-py382) trac-dev $ python -c "from svn import core"
Traceback (most recent call last):
  File "/Users/rjollos/.pyenv/versions/3.8.2/envs/trac-py382/lib/python3.8/site-packages/libsvn/core.py", line 14, in swig_import_helper
    return importlib.import_module(mname)
  File "/Users/rjollos/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 657, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 556, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1101, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: dlopen(/Users/rjollos/.pyenv/versions/3.8.2/envs/trac-py382/lib/python3.8/site-packages/libsvn/_core.so, 2): Symbol not found: _PyFile_AsFile
  Referenced from: /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib
  Expected in: flat namespace
 in /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/rjollos/.pyenv/versions/3.8.2/envs/trac-py382/lib/python3.8/site-packages/svn/core.py", line 26, in <module>
    from libsvn.core import *
  File "/Users/rjollos/.pyenv/versions/3.8.2/envs/trac-py382/lib/python3.8/site-packages/libsvn/core.py", line 17, in <module>
    _core = swig_import_helper()
  File "/Users/rjollos/.pyenv/versions/3.8.2/envs/trac-py382/lib/python3.8/site-packages/libsvn/core.py", line 16, in swig_import_helper
    return importlib.import_module('_core')
  File "/Users/rjollos/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: dlopen(/usr/local/lib/svn-python/libsvn/_core.so, 2): Symbol not found: _PyFile_AsFile
  Referenced from: /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib
  Expected in: flat namespace
 in /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib

I think the problem is probably due to the switch to Python 3.8 as a dependency in e073d0ba. I assume that causes the SWIG bindings to be built against 3.8, but I haven't traced that to be sure. I don't think Subversion 1.13 has Python bindings that are compatible with Python 3, but I also have not dug into that.

It works fine to force install a previous version (1.13.0_2) of the formulae:

$ brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/b309c2efabeb977aa97288efb85d7ee5ad2808b0/Formula/subversion.rb
$ brew pin subversion
(trac-py382) Formula (master)$ pwd
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula
(trac-py382) Formula (master)$ git log -2 bb6240cb4a -- subversion.rb
commit bb6240cb4af2fd9aef34aa765dca666c822ff871
Author: Michka Popoff <michkapopoff@gmail.com>
Date:   Tue Feb 18 12:38:14 2020 +0100

    subversion: use python@3.8

    Closes #50391.

    Signed-off-by: Michka Popoff <michkapopoff@gmail.com>

commit b309c2efabeb977aa97288efb85d7ee5ad2808b0
Author: Jonathan Chang <jchang641@gmail.com>
Date:   Tue Feb 18 09:43:55 2020 +1100

    subversion: add uses_from_macos dependencies (#49909)

As a proof that b309c2ef is the latest formulae that can be used, using the next revision (e073d0ba) fails:

$ brew unpin subversion
$ brew uninstall subversion 
$ brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/bb6240cb4af2fd9aef34aa765dca666c822ff871/Formula/subversion.rb
$ python -c "from svn import core"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/svn/core.py", line 26, in <module>
    from libsvn.core import *
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/libsvn/core.py", line 17, in <module>
    _core = swig_import_helper()
  File "/Users/rjollos/.pyenv/versions/trac-2.7/lib/python2.7/site-packages/libsvn/core.py", line 16, in swig_import_helper
    return importlib.import_module('_core')
  File "/Users/rjollos/.pyenv/versions/2.7.17/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: dlopen(/usr/local/opt/subversion/lib/svn-python/libsvn/_core.so, 2): Symbol not found: _PyBytes_AsString
  Referenced from: /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib
  Expected in: flat namespace
 in /usr/local/Cellar/subversion/1.13.0_3/lib/libsvn_swig_py-1.0.dylib

I should probably just build Subversion and the bindings from source for my dev environment.

I might report this to Homebrew if I have time to investigate further.

Last edited 4 years ago by Ryan J Ollos (previous) (diff)

in reply to:  10 ; comment:11 by Jun Omae, 4 years ago

Replying to Ryan J Ollos:

I was finally able to run tests against Subversion locally again. The latest Brew formulae installs 1.13.0_4, which results in:

ImportError: dlopen(/usr/local/lib/svn-python/libsvn/_core.so, 2): Symbol not found: _PyBytes_AsString
                    ^^^^^^^^^^^^^^

I guess you've installed manually subversion with /usr/local as prefix. Delete /usr/local/lib/svn-python directory and retry.

Also, I recommend to pass site directory to swig_pydir and swig_pydir_extra variables on running make like https://github.com/jun66j5/trac/blob/1.5-py3-t11988/.travis.yml#L151.

in reply to:  11 comment:12 by Ryan J Ollos, 4 years ago

Replying to Jun Omae:

I guess you've installed manually subversion with /usr/local as prefix. Delete /usr/local/lib/svn-python directory and retry.

That's the location that Brew symlinks:

$ ls -al /usr/local/lib/svn-python
lrwxr-xr-x  1 rjollos  admin  44 Apr  9 20:50 /usr/local/lib/svn-python -> ../Cellar/subversion/1.13.0_2/lib/svn-python

in reply to:  11 comment:13 by Ryan J Ollos, 4 years ago

Replying to Jun Omae:

Also, I recommend to pass site directory to swig_pydir and swig_pydir_extra variables on running make like https://github.com/jun66j5/trac/blob/1.5-py3-t11988/.travis.yml#L151.

Thanks. When building from source I've been symlinking from the python environment to svn-python/svn and svn-python/libsvn, which I think is equivalent.

./configure --with-apr=$(brew --prefix apr) --with-apr-util=$(brew --prefix apr-util) --with-py3c=./py3c --prefix=$(pwd)/local
make all swig-py install install-swig-py

svn_python=$(pwd)/local/lib/svn-python
site_packages=$(find $(pyenv prefix)/ -name "site-packages" -print)
ln -sf "$svn_python/svn" "$site_packages/"
ln -sf "$svn_python/libsvn" "$site_packages/"
echo "$svn_python/libsvn" > "$site_packages/svn.pth"
Last edited 4 years ago by Ryan J Ollos (previous) (diff)

comment:14 by anonymous, 4 years ago

Just to summarize my previous findings:

  • The subversion bindings installed from Homebrew fail with Python 2.7 for Subversion formulae 1.13.0_3 or later. The last revision that works with Python 2.7 is b309c2ef. These bindings also fail with Python 3.8.
  • Subversion bindings built from source 1.14dev, and now 1.15dev, work for me with Python 3.5-3.8. I haven't tried them with Python 2.7, they claim compatibility (for now).

I've reported the issue to Homebrew.

in reply to:  14 comment:15 by Ryan J Ollos, 4 years ago

Replying to anonymous:

I've reported the issue to Homebrew.

Issue fixed in e54983e4. The 1.13.0_5 formulae works.

by Ryan J Ollos, 4 years ago

by Ryan J Ollos, 4 years ago

in reply to:  8 ; comment:16 by Ryan J Ollos, 4 years ago

Replying to Ryan J Ollos:

Replying to Ryan J Ollos:

Investigating…

Looks like $TRAVIS_BUILD_STAGE_NAME is test, not Test. I echoed it:

$ echo $TRAVIS_BUILD_STAGE_NAME
test

Docs say:

TRAVIS_BUILD_STAGE_NAME: The build stage in capitalized form, e.g. Test or Deploy. If a build does not use build stages, this variable is empty ("").

Recent change to the platform? Maybe we should do a case-insensitive compare?

The builds don't fail for the Edgewall Travis CI account, but they do fail for my fork.

Here is a hint. The variables Test and Deploy are capitalized on Edgewall, but not for my fork:

I'm considering posting a question to the community.

in reply to:  16 ; comment:17 by Ryan J Ollos, 4 years ago

Replying to Ryan J Ollos:

I'm considering posting a question to the community.

Someone already reported here. I added a reply.

Version 0, edited 4 years ago by Ryan J Ollos (next)

in reply to:  17 ; comment:18 by Ryan J Ollos, 4 years ago

Replying to Ryan J Ollos:

Someone already reported here. I added a reply.

We could use their workaround: ${TRAVIS_BUILD_STAGE_NAME} = [Tt]est

I'm waiting a few days to take action on this, to see if there are follow-up on that forum topic. Seems like this should affect a good number of Travis CI users.

Committed changes to 1.2-stable in r17269, merged in r17270, r17271.

The changes in this ticket fit what I had in mind as "Internal Changes": gmessage:trac-dev:CNrnrXJ9A3o/rEayodNRBAAJ.

in reply to:  18 ; comment:19 by Ryan J Ollos, 4 years ago

Resolution: fixed
Status: assignedclosed

Replying to Ryan J Ollos:

I'm waiting a few days to take action on this, to see if there are follow-up on that forum topic. Seems like this should affect a good number of Travis CI users.

With Build Config Validation enabled I get test. With it disabled I get Test. So I disabled Build Config Validation on my fork. It's a beta feature.

Last edited 4 years ago by Ryan J Ollos (previous) (diff)

in reply to:  19 comment:20 by Ryan J Ollos, 4 years ago

Internal Changes: modified (diff)

Replying to Ryan J Ollos:

With Build Config Validation enabled I get test. With it disabled I get Test. So I disabled Build Config Validation on my fork. It's a beta feature.

#13274.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Ryan J Ollos.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Ryan J Ollos to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.