Test failures in trac.wiki.tests.formatter when executed in isolation

[Ryan J Ollos]

Executing all of the tests in trac.wiki.tests succeeds,

PYTHONPATH=. python -m trac.wiki.tests.__init__
...................................................................................................................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 355 tests in 1.417s

OK

However, executing just the tests in trac.wiki.tests.formatter fails:

PYTHONPATH=. python -m trac.wiki.tests.formatter
...........................................................................................F.F........................................................................................................................................................
======================================================================
FAIL: test (__main__.WikiTestCase)
Test Macro help shortcut (with content)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/Workspace/t10834/teo-rjollos.git/trac/wiki/tests/formatter.py", line 222, in test
    % (msg, self.file, self.line, self.title, formatter.flavor))
AssertionError: 
========== expected: ==========
<p>
</p><div class="trac-macrolist"><h3 id="HelloWorld-macro"><code>[[HelloWorld]]</code></h3><p>
A dummy macro used by the unit test.
</p>
</div><p>
</p>


========== actual: ==========
<p>
</p><div class="trac-macrolist"><h3 id="HelloWorld-macro"><code>[[HelloWorld]]</code></h3><p>
A dummy macro used by the unit test.
</p>
<h3 id="HelloWorld-macro"><code>[[HelloWorld]]</code></h3><p>
A dummy macro used by the unit test.
</p>
</div><p>
</p>


========== wiki: ==========
u'
[[HelloWorld?(hej hopp)]]
'
========== diff: ==========
--- expected
+++ actual
@@ -1,5 +1,8 @@
 <p>
 </p><div class="trac-macrolist"><h3 id="HelloWorld-macro"><code>[[HelloWorld]]</code></h3><p>
+A dummy macro used by the unit test.
+</p>
+<h3 id="HelloWorld-macro"><code>[[HelloWorld]]</code></h3><p>
 A dummy macro used by the unit test.
 </p>
 </div><p>


/home/user/Workspace/t10834/teo-rjollos.git/trac/wiki/tests/wiki-tests.txt:1006: "Macro help shortcut (with content)" (default flavor)

======================================================================
FAIL: test (__main__.WikiTestCase)
Test Macro help shortcut (no content)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/Workspace/t10834/teo-rjollos.git/trac/wiki/tests/formatter.py", line 222, in test
    % (msg, self.file, self.line, self.title, formatter.flavor))
AssertionError: 
========== expected: ==========
<p>
</p><div class="trac-macrolist"><h3 id="HelloWorld-macro"><code>[[HelloWorld]]</code></h3><p>
A dummy macro used by the unit test.
</p>
</div><p>
</p>


========== actual: ==========
<p>
</p><div class="trac-macrolist"><h3 id="HelloWorld-macro"><code>[[HelloWorld]]</code></h3><p>
A dummy macro used by the unit test.
</p>
<h3 id="HelloWorld-macro"><code>[[HelloWorld]]</code></h3><p>
A dummy macro used by the unit test.
</p>
</div><p>
</p>


========== wiki: ==========
u'
[[HelloWorld?]]
'
========== diff: ==========
--- expected
+++ actual
@@ -1,5 +1,8 @@
 <p>
 </p><div class="trac-macrolist"><h3 id="HelloWorld-macro"><code>[[HelloWorld]]</code></h3><p>
+A dummy macro used by the unit test.
+</p>
+<h3 id="HelloWorld-macro"><code>[[HelloWorld]]</code></h3><p>
 A dummy macro used by the unit test.
 </p>
 </div><p>


/home/user/Workspace/t10834/teo-rjollos.git/trac/wiki/tests/wiki-tests.txt:1017: "Macro help shortcut (no content)" (default flavor)

----------------------------------------------------------------------
Ran 246 tests in 0.710s

FAILED (failures=2)

[Ryan J Ollos]

The issue seems to be due to some module/import behaviour, possibly related to the explanation in comment:7:ticket:11312, but I don't understand it well enough to provide a fix yet.

My observation is that the HelloWorldMacro is being discovered twice at branches/1.0-stable/trac/wiki/macros.py@12046:689#L669​:

• <__main__.HelloWorldMacro object at 0x24afd10>
• <trac.wiki.tests.formatter.HelloWorldMacro object at 0x24afad0>

The test does not fail when running, PYTHONPATH=. python -m trac/wiki/tests/formatter.py, but in that case the <trac.wiki.tests.formatter.HelloWorldMacro object at 0x24afad0> is not returned by formatter.wiki.macro_providers. When running the tests as PYTHONPATH=. python -m trac.wiki.tests.formatter, the macro is discovered in two objects.

[Ryan J Ollos]

Fixed in [13851]. Merged to trunk in [13852]. See comment:21:ticket:11437.

[Ryan J Ollos]

The main method in functional test modules is functionalSuite (tags/trac-1.1.3/trac/versioncontrol/tests/functional.py@:367#L367​), not suite, and the result is that additional functional tests are executed when running make test=trac/versioncontrol/tests/functional.py. I'm not sure what the mechanism is yet, but there are only 17 tests in the module and the following results:

$ make test=trac/versioncontrol/tests/functional.py

python setup.py -q test -s trac.versioncontrol.tests.functional.suite 
....................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 212 tests in 602.593s

$ python -m trac.versioncontrol.tests.functional
.................
----------------------------------------------------------------------
Ran 17 tests in 47.758s

I'll investigate further.

[Jun Omae]

The suite symbol in trac/versioncontrol/tests/functional.py is imported by from trac.tests.functional import *. Should we add __all__ to trac/tests/functional.py or remove import *?

trac/versioncontrol/tests/functional.py

@@ -12 +12 @@
 # individuals. For the exact contribution history, see the revision
 # history and logs, available at http://trac.edgewall.org/log/.
 
+import os
 import tempfile
+import unittest
 
 from trac.admin.tests.functional import AuthorizationTestCaseSetup
-from trac.tests.functional import *
+from trac.tests.contentgen import random_page, random_word, random_unique_camel
+from trac.tests.functional import FunctionalTwillTestCaseSetup, has_svn, \
+                                  internal_error, tc
 
 
 class TestAdminRepositoryAuthorization(AuthorizationTestCaseSetup):
@@ -392 +396 @@
     return suite
 
 
+suite = functionalSuite
+
+
 if __name__ == '__main__':
-    unittest.main(defaultTest='functionalSuite')
+    unittest.main(defaultTest='suite')

[Ryan J Ollos]

Replying to jomae:

Should we add __all__ to trac/tests/functional.py or remove import *?

Your patch that avoids the * import looks good to me. I'll implement for the other functional.py modules. Thanks.

[Ryan J Ollos]

Committed to 1.0-stable in [13874,13877], merged to trunk in [13876,13878].