Tests can't be run using `-m` flag to `python setup.py test`

[Ryan J Ollos]

There are several ways to run the unit tests in a module:

$ python ./tracopt/versioncontrol/git/tests/__init__.py
........................
----------------------------------------------------------------------
Ran 24 tests in 1.520s

OK

$ python -m tracopt.versioncontrol.git.tests.__init__
........................
----------------------------------------------------------------------
Ran 24 tests in 1.494s

OK

$ python setup.py test -m tracopt.versioncontrol.git.tests.__init__
running test
running egg_info
unrecognized .svn/entries format; skipping .
writing requirements to Trac.egg-info/requires.txt
writing Trac.egg-info/PKG-INFO
writing top-level names to Trac.egg-info/top_level.txt
writing dependency_links to Trac.egg-info/dependency_links.txt
writing entry points to Trac.egg-info/entry_points.txt
unrecognized .svn/entries format in /home/user/Workspace/t11500/teo-rjollos.git
reading manifest file 'Trac.egg-info/SOURCES.txt'
writing manifest file 'Trac.egg-info/SOURCES.txt'
running build_ext
test_git_version (tracopt.versioncontrol.git.tests.PyGIT.GitTestCase) ... ok
test_is_sha (tracopt.versioncontrol.git.tests.PyGIT.GitTestCase) ... ok
test_parse (tracopt.versioncontrol.git.tests.PyGIT.TestParseCommit) ... ok
test_control_files_detection (tracopt.versioncontrol.git.tests.PyGIT.NormalTestCase) ... ok
test_get_branches_with_cr_in_commitlog (tracopt.versioncontrol.git.tests.PyGIT.NormalTestCase) ... ok
test_node_get_history_with_empty_commit (tracopt.versioncontrol.git.tests.PyGIT.NormalTestCase) ... ok
test_rev_is_anchestor_of (tracopt.versioncontrol.git.tests.PyGIT.NormalTestCase) ... ok
test_sync_after_removing_branch (tracopt.versioncontrol.git.tests.PyGIT.NormalTestCase) ... ok
test_turn_off_persistent_cache (tracopt.versioncontrol.git.tests.PyGIT.NormalTestCase) ... ok
test_get_historian (tracopt.versioncontrol.git.tests.PyGIT.UnicodeNameTestCase) ... ok
test_ls_tree (tracopt.versioncontrol.git.tests.PyGIT.UnicodeNameTestCase) ... ok
test_unicode_branches (tracopt.versioncontrol.git.tests.PyGIT.UnicodeNameTestCase) ... ok
test_unicode_filename (tracopt.versioncontrol.git.tests.PyGIT.UnicodeNameTestCase) ... ok
test_unicode_tags (tracopt.versioncontrol.git.tests.PyGIT.UnicodeNameTestCase) ... ok
test_unicode_verifyrev (tracopt.versioncontrol.git.tests.PyGIT.UnicodeNameTestCase) ... ok
test_bare (tracopt.versioncontrol.git.tests.git_fs.SanityCheckingTestCase) ... ok
test_no_head_file (tracopt.versioncontrol.git.tests.git_fs.SanityCheckingTestCase) ... ok
test_no_objects_dir (tracopt.versioncontrol.git.tests.git_fs.SanityCheckingTestCase) ... ok
test_no_refs_dir (tracopt.versioncontrol.git.tests.git_fs.SanityCheckingTestCase) ... ok
test_non_bare (tracopt.versioncontrol.git.tests.git_fs.SanityCheckingTestCase) ... ok
test_non_persistent (tracopt.versioncontrol.git.tests.git_fs.PersistentCacheTestCase) ... ok
test_persistent (tracopt.versioncontrol.git.tests.git_fs.PersistentCacheTestCase) ... ok
test_with_cache (tracopt.versioncontrol.git.tests.git_fs.HistoryTimeRangeTestCase) ... ok
test_without_cache (tracopt.versioncontrol.git.tests.git_fs.HistoryTimeRangeTestCase) ... ok

----------------------------------------------------------------------
Ran 24 tests in 1.684s

OK
Exception TypeError: TypeError("'NoneType' object is not callable",) in <bound method Storage.__del__ of <tracopt.versioncontrol.git.PyGIT.Storage object at 0x2ba3d10>> ignored
Exception TypeError: TypeError("'NoneType' object is not callable",) in <bound method Storage.__del__ of <tracopt.versioncontrol.git.PyGIT.Storage object at 0x2ba3190>> ignored

The last example provides the most verbose output, which was useful in debugging the issue in #11505. However, that doesn't actually work without this change:

tracopt/versioncontrol/git/tests/__init__.py

@@ -16 +16 @@
 from tracopt.versioncontrol.git.tests import PyGIT, git_fs
 
 
-def suite():
+def test_suite():
     suite = unittest.TestSuite()
     suite.addTest(PyGIT.suite())
     suite.addTest(git_fs.suite())
@@ -24 +24 @@
 
 
 if __name__ == '__main__':
-    unittest.main(defaultTest='suite')
+    unittest.main(defaultTest='test_suite')

I haven't investigate yet whether that is a proper solution for this issue, or if there is some other change we could make.

[Jun Omae]

Otherwise, we can run tests using -s option like this.

$ ~/venv/py25/bin/python setup.py test -s tracopt.versioncontrol.git.tests.suite

[Ryan J Ollos]

That works. We could document that in trunk/doc/dev/testing-intro.rst​.

If we make the change in comment:description, the tests can be run using either the -m or -s flags. Maybe that is better for flexibility. Are there any downsides to making the change?

[Ryan J Ollos]

Replying to jomae:

Otherwise, we can run tests using -s option like this.

$ ~/venv/py25/bin/python setup.py test -s tracopt.versioncontrol.git.tests.suite

The following also seems to work:

$ ~/venv/py25/bin/python setup.py test -s tracopt.versioncontrol.git.tests

The -s flag seems to result in the detection of the suite in the module, regardless of whether it is a function named suite or test_suite.

[Jun Omae]

Are there any downsides to making the change?

I think that is no downsides. Fine by me.

[Jun Omae]

The -s flag seems to result in the detection of the suite in the module, regardless of whether it is a function named suite or test_suite.

The -s option's suite detection doesn't work in some test modules. I don't think we should use the detection.

$ ~/venv/py25/bin/python setup.py test -s tracopt.versioncontrol.svn.tests
....
Ran 0 tests in 0.000s

OK

$ ~/venv/py25/bin/python setup.py test -s tracopt.versioncontrol.svn.tests.suite
...
Ran 166 tests in 38.844s

OK
Exception exceptions.TypeError: "'NoneType' object is not callable" in <function <lambda> at 0x95fcd14> ignored

[Olemis Lang <olemis+trac@…>]

Replying to jomae:

Are there any downsides to making the change?

I think that is no downsides. Fine by me.

Be careful with functional tests . They are somehow special because test cases share a global fixture data . AFAICR in those cases there are two methods functionalSuite and suite (test_suite ?) see source:trunk/trac/tests/functional/__init__.py@12511:185-214#L185​ .

[Ryan J Ollos]

Proposed changes in log:rjollos.git:t11506_test_suite.

[Ryan J Ollos]

Committed to trunk in [14779,14780].

[Ryan J Ollos]

Equivalent change to SpamFilter in r14829. r14829 also fixes an import error when running test modules in tracspamfilter.filters.tests.

$python tracspamfilter/filters/tests/__init__.py
Traceback (most recent call last):
  File "tracspamfilter/filters/tests/__init__.py", line 16, in <module>
    from tracspamfilter.filters.tests import akismet, bayes, extlinks, regex, \
  File "/Users/rjollos/Documents/Workspace/trac-dev/spam-filter/tracspamfilter/filters/tests/__init__.py", line 16, in <module>
    from tracspamfilter.filters.tests import akismet, bayes, extlinks, regex, \
  File "/Users/rjollos/Documents/Workspace/trac-dev/spam-filter/tracspamfilter/filters/tests/bayes.py", line 21, in <module>
    from tracspamfilter.tests.model import drop_tables
  File "/Users/rjollos/Documents/Workspace/trac-dev/spam-filter/tracspamfilter/tests/__init__.py", line 17, in <module>
    from tracspamfilter.filters import tests as filters
ImportError: cannot import name tests