Improved functional test infrastructure

[Felix Schwarz <felix.schwarz@…>]

Trac has a nice functional test infrastructure. However from a plugin writer's point of view it leaves some things to be desired.

For our plugin ​Agilo for Scrum we wanted to have functional tests as well so we started adapting trac's existing code. The base design is still very similar to trac's current functional test infrastructure.

Goals / what we implemented:

• Use functional tests with different databases (basic implementation, (1))
• Provide a functional infrastructure which can be used by plugins (e.g. enable/disable components by default)
• Lift the limitation of a single environment (without changing the old behavior)
• Design clean up - the test suite should not know anything about the environment
• Even more design clean up - Test suite should not set attributes in test cases (run functional tests with nosetests)

What needs to be fixed before new functional tests can be included in trac:

• Skip functional tests if some libraries (like twill/svn/subprocess) are missing
• Adapt documentation

Due to the stuff left to do, the attached patch will need some rework. However I hope to get some feedback - if there is a chance that our changes will be included, we can do some more polishing.

(1) Support for MySQL+PostgreSQL is currenlty quite limited because you need to clear the database before starting the test manually.

[Felix Schwarz <felix.schwarz@…>]

For now I only converted trac.tests.functional.testcases

[Tim Hatch]

Hi Felix,

Your patch has an issue on line 230, we can't apply it with patch 2.5.8, 2.5.9. Could you please check and reupload?

We're trying to reconcile three sets of patches that all seem to solve the same problem during the pycon sprints, so we'll get this resolved soon.

[Tim Hatch]

Fixed version of previous patch

[Tim Hatch]

Spent a little time on it and managed to locate the patch error.

(a) new_functional_infrastructure_v1.patch vs. (b) new_diff.diff

@@ -2 +2 @@
 ===================================================================
 --- trac/tests/functional/api.py        (Revision 0)
 +++ trac/tests/functional/api.py        (Revision 0)
-@@ -0,0 +1,226 @@
+@@ -0,0 +1,224 @@
 +# -*- coding: utf-8 -*-
 +#   Copyright 2009 Agile42 GmbH, Berlin (Germany)
 +"""

[Christian Boos]

The first evolution of the source:sandbox/rework-testing branch has been merged in trunk in r8131, with 0.11-stable following.

[Christian Boos]

Since the rework-testing branch has completed a cycle of activity and that new related changes have been done directly on the main branches (e.g. r8416), further developments of the testing infrastructure should be rebased on a more recent version of trunk. So I've deleted the branch for now (r8476).

Feel free to recreate the branch whenever you have some new material.

[Remy Blank]

Tim, do you intend to integrate this before 0.12? If not, please move to either next-minor-0.12.x or next-major-0.1X. Thanks.

[figaro]

See also #11988.

[Ryan J Ollos]

If someone wants to create a new patch against the current trunk we could accept some more of these changes to the functional test suite. Otherwise, I don't see a reason to keep the ticket open any longer.