Retrieve trac test env path from global env var

["Olemis Lang" <olemis+trac@…>]

Support for configuring test env path via global OS env var e.g. TRAC_TEST_ENVPATH . Useful for e.g. those situations when FS does not support setting file execute permissions, thus making vcs hooks test to fail .

[Ryan J Ollos]

This sounds fine to me. Let's schedule it and see what others have to say.

The change is probably fairly trivial, but would you like to post a patch?

[Olemis Lang <olemis+trac@…>]

I was building ​this one since I needed it to test the patches submitted for #11440. Please review it.

[Ryan J Ollos]

There are two minor issue with the patch:

• Since the port is calculated when port is None, the second test if port is None will always be False.
• Further changes are needed to avoid two failing test cases:

  ======================================================================
  ERROR: runTest (trac.tests.functional.testcases.RegressionTestRev6017)
  Test for regression of the plugin reload fix in r6017
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "trac/tests/functional/testcases.py", line 41, in runTest
      'DeleteTicket.py')).read()
  IOError: [Errno 2] No such file or directory: '/tmp/sample-plugins/workflow/DeleteTicket.py'
  
  ======================================================================
  ERROR: runTest (trac.ticket.tests.functional.RegressionTestTicket6048)
  Test for regression of http://trac.edgewall.org/ticket/6048
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "trac/ticket/tests/functional.py", line 1796, in runTest
      'workflow', 'DeleteTicket.py')).read()
  IOError: [Errno 2] No such file or directory: '/tmp/sample-plugins/workflow/DeleteTicket.py'
  
  ----------------------------------------------------------------------
  Ran 173 tests in 204.576s
  

As for the naming of the variable, we have a few other environment variables that are prefixed with TRAC_TEST (TRAC_TEST_TRACD_OPTIONS, TRAC_TEST_DB_URI), so TRAC_TEST_ENV_PATH or TRAC_TEST_ENV seems like an appropriate name. The former is more descriptive, but the latter has better symmetry with TRAC_ENV. Side note: if we add this environment variable, it should be documented in branches/1.0-stable/doc/dev/testing-intro.rst​ (and possibly TracDev/FunctionalTests as well, but probably just the rst file is adequate).

If the source directory is not writable, wouldn't we also need to write functional-testing.log and testing.log to another location?

Since command_cwd​ is determined from the environment directory, change the location of the environment may affect some of the uses​ of the variable.

Now that I think about it more, I'm not sure I understand the use-case for these changes. Anyway, I've prepared some changes in log:rjollos.git:t11448.

Btw, do you have any idea how one would specify the port when running the tests? I don't see how a parameter would get passed to setUp​.

[Olemis Lang <olemis+trac@…>]

Replying to rjollos:

[…]

Now that I think about it more, I'm not sure I understand the use-case for these changes. Anyway, I've prepared some changes in log:rjollos.git:t11448.

NTFS filesystem mounted on Linux and it's not possible to chmod u+x ../repo/hooks/post_commit which leads to failures of test cases submitted for #11440 .

Btw, do you have any idea how one would specify the port when running the tests? I don't see how a parameter would get passed to setUp​.

That's up to the caller to decide (e.g. CI build job …)

[Ryan J Ollos]

Replying to Olemis Lang <olemis+trac@…>:

Btw, do you have any idea how one would specify the port when running the tests? I don't see how a parameter would get passed to setUp​.

That's up to the caller to decide (e.g. CI build job …)

There's doesn't seem to be a mechanism for the caller to pass the port from the command line.

[Olemis Lang <olemis+trac@…>]

Replying to rjollos:

Replying to Olemis Lang <olemis+trac@…>:

Btw, do you have any idea how one would specify the port when running the tests? I don't see how a parameter would get passed to setUp​.

That's up to the caller to decide (e.g. CI build job …)

There's doesn't seem to be a mechanism for the caller to pass the port from the command line.

No , you'll need a script for that or a custom test case sub-class . Should I add a patch for that too ?

[Olemis Lang <olemis+trac@…>]

Your changes look rellay nice . I've refreshed ​the patches based on your suggestion to make them work with /trunk .

$ python trac/tests/functional/testcases.py
............
----------------------------------------------------------------------
Ran 12 tests in 14.897s

OK

Nonetheless I cannot run ticket tests . See ​http://pastebin.com/CuLmKa24 .

[Ryan J Ollos]

Replying to Olemis Lang <olemis+trac@…>:

Nonetheless I cannot run ticket tests . See ​http://pastebin.com/CuLmKa24 .

Does the test execution hang indefinitely at TestTicketQueryLinks? I may have seen this issue before, discussed in #11249.

[Olemis Lang <olemis+trac@…>]

Replying to rjollos:

Replying to Olemis Lang <olemis+trac@…>:

Nonetheless I cannot run ticket tests . See ​http://pastebin.com/CuLmKa24 .

Does the test execution hang indefinitely at TestTicketQueryLinks? I may have seen this issue before, discussed in #11249.

thanks that fixed the issue in comment:8

Test results look ok after applying the patches against /trunk , see ​http://pastebin.com/zC3Jnxe4

[Ryan J Ollos]

Proposed changes can be found in log:rjollos.git:t11448.2. [29b59f25/rjollos.git] and [ec7d855c/rjollos.git] should be committed to the trunk since they change the API for plugins. The other changes are proposed for 1.0-stable.

[Olemis Lang <olemis+trac@…>]

Replying to rjollos:

Proposed changes

[…]

looks nice !

[ec7d855c/rjollos.git] should be committed to the trunk since they change the API for plugins.

what about writing functional test log in $env_path/trac/log/functional-testing.log ?

The other changes are proposed for 1.0-stable.

\o/

[Ryan J Ollos]

Replying to Olemis Lang <olemis+trac@…>:

what about writing functional test log in $env_path/trac/log/functional-testing.log ?

I considered this. The logs aren't part of the Trac environment but rather part of the testing environment, which led to putting them in the root of the testing environment directory. I don't have a strong preference to either choice, so we could go either way as far as I'm concerned.

[Ryan J Ollos]

Changes committed to 1.0-stable in [12509] and merged to trunk in [12510]. Functional test logs directory moved to test environment directory in [12511] (on trunk).