Edgewall Software
Modify

Opened 7 years ago

Closed 7 years ago

#11448 closed enhancement (fixed)

Retrieve trac test env path from global env var

Reported by: "Olemis Lang" <olemis+trac@…> Owned by: Ryan J Ollos
Priority: normal Milestone: 1.0.2
Component: general Version: 1.0dev
Severity: trivial Keywords:
Cc: Branch:
Release Notes:

Added environment variables for specifying the port (TRAC_TEST_PORT) and path (TRAC_TEST_ENV_PATH) of the functional test environment.

API Changes:
Internal Changes:

Description

Support for configuring test env path via global OS env var e.g. TRAC_TEST_ENVPATH . Useful for e.g. those situations when FS does not support setting file execute permissions, thus making vcs hooks test to fail .

Attachments (0)

Change History (14)

comment:1 by Ryan J Ollos, 7 years ago

Milestone: 1.0.3
Owner: set to Ryan J Ollos
Status: newassigned

This sounds fine to me. Let's schedule it and see what others have to say.

The change is probably fairly trivial, but would you like to post a patch?

comment:2 by Olemis Lang <olemis+trac@…>, 7 years ago

I was building this one since I needed it to test the patches submitted for #11440. Please review it.

comment:3 by Ryan J Ollos, 7 years ago

There are two minor issue with the patch:

  • Since the port is calculated when port is None, the second test if port is None will always be False.
  • Further changes are needed to avoid two failing test cases:
    ======================================================================
    ERROR: runTest (trac.tests.functional.testcases.RegressionTestRev6017)
    Test for regression of the plugin reload fix in r6017
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "trac/tests/functional/testcases.py", line 41, in runTest
        'DeleteTicket.py')).read()
    IOError: [Errno 2] No such file or directory: '/tmp/sample-plugins/workflow/DeleteTicket.py'
    
    ======================================================================
    ERROR: runTest (trac.ticket.tests.functional.RegressionTestTicket6048)
    Test for regression of http://trac.edgewall.org/ticket/6048
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "trac/ticket/tests/functional.py", line 1796, in runTest
        'workflow', 'DeleteTicket.py')).read()
    IOError: [Errno 2] No such file or directory: '/tmp/sample-plugins/workflow/DeleteTicket.py'
    
    ----------------------------------------------------------------------
    Ran 173 tests in 204.576s
    

As for the naming of the variable, we have a few other environment variables that are prefixed with TRAC_TEST (TRAC_TEST_TRACD_OPTIONS, TRAC_TEST_DB_URI), so TRAC_TEST_ENV_PATH or TRAC_TEST_ENV seems like an appropriate name. The former is more descriptive, but the latter has better symmetry with TRAC_ENV. Side note: if we add this environment variable, it should be documented in branches/1.0-stable/doc/dev/testing-intro.rst (and possibly TracDev/FunctionalTests as well, but probably just the rst file is adequate).

If the source directory is not writable, wouldn't we also need to write functional-testing.log and testing.log to another location?

Since command_cwd is determined from the environment directory, change the location of the environment may affect some of the uses of the variable.

Now that I think about it more, I'm not sure I understand the use-case for these changes. Anyway, I've prepared some changes in log:rjollos.git:t11448.

Btw, do you have any idea how one would specify the port when running the tests? I don't see how a parameter would get passed to setUp.

in reply to:  3 ; comment:4 by Olemis Lang <olemis+trac@…>, 7 years ago

Replying to rjollos:

[…]

Now that I think about it more, I'm not sure I understand the use-case for these changes. Anyway, I've prepared some changes in log:rjollos.git:t11448.

NTFS filesystem mounted on Linux and it's not possible to chmod u+x ../repo/hooks/post_commit which leads to failures of test cases submitted for #11440 .

Btw, do you have any idea how one would specify the port when running the tests? I don't see how a parameter would get passed to setUp.

That's up to the caller to decide (e.g. CI build job …)

in reply to:  4 ; comment:5 by Ryan J Ollos, 7 years ago

Replying to Olemis Lang <olemis+trac@…>:

Btw, do you have any idea how one would specify the port when running the tests? I don't see how a parameter would get passed to setUp.

That's up to the caller to decide (e.g. CI build job …)

There's doesn't seem to be a mechanism for the caller to pass the port from the command line.

in reply to:  5 comment:6 by Olemis Lang <olemis+trac@…>, 7 years ago

Replying to rjollos:

Replying to Olemis Lang <olemis+trac@…>:

Btw, do you have any idea how one would specify the port when running the tests? I don't see how a parameter would get passed to setUp.

That's up to the caller to decide (e.g. CI build job …)

There's doesn't seem to be a mechanism for the caller to pass the port from the command line.

No , you'll need a script for that or a custom test case sub-class . Should I add a patch for that too ?

comment:7 by Olemis Lang <olemis+trac@…>, 7 years ago

Your changes look rellay nice . I've refreshed the patches based on your suggestion to make them work with /trunk .

$ python trac/tests/functional/testcases.py
............
----------------------------------------------------------------------
Ran 12 tests in 14.897s

OK


Nonetheless I cannot run ticket tests . See http://pastebin.com/CuLmKa24 .

in reply to:  7 ; comment:8 by Ryan J Ollos, 7 years ago

Replying to Olemis Lang <olemis+trac@…>:

Nonetheless I cannot run ticket tests . See http://pastebin.com/CuLmKa24 .

Does the test execution hang indefinitely at TestTicketQueryLinks? I may have seen this issue before, discussed in #11249.

in reply to:  8 comment:9 by Olemis Lang <olemis+trac@…>, 7 years ago

Replying to rjollos:

Replying to Olemis Lang <olemis+trac@…>:

Nonetheless I cannot run ticket tests . See http://pastebin.com/CuLmKa24 .

Does the test execution hang indefinitely at TestTicketQueryLinks? I may have seen this issue before, discussed in #11249.

thanks that fixed the issue in comment:8

Test results look ok after applying the patches against /trunk , see http://pastebin.com/zC3Jnxe4

comment:10 by Ryan J Ollos, 7 years ago

Proposed changes can be found in log:rjollos.git:t11448.2. [29b59f25/rjollos.git] and [ec7d855c/rjollos.git] should be committed to the trunk since they change the API for plugins. The other changes are proposed for 1.0-stable.

in reply to:  10 ; comment:11 by Olemis Lang <olemis+trac@…>, 7 years ago

Replying to rjollos:

Proposed changes

[…]

looks nice !

[ec7d855c/rjollos.git] should be committed to the trunk since they change the API for plugins.

what about writing functional test log in $env_path/trac/log/functional-testing.log ?

The other changes are proposed for 1.0-stable.

\o/

in reply to:  11 comment:12 by Ryan J Ollos, 7 years ago

Replying to Olemis Lang <olemis+trac@…>:

what about writing functional test log in $env_path/trac/log/functional-testing.log ?

I considered this. The logs aren't part of the Trac environment but rather part of the testing environment, which led to putting them in the root of the testing environment directory. I don't have a strong preference to either choice, so we could go either way as far as I'm concerned.

comment:13 by Ryan J Ollos, 7 years ago

Milestone: 1.0.31.0.2

comment:14 by Ryan J Ollos, 7 years ago

Release Notes: modified (diff)
Resolution: fixed
Status: assignedclosed

Changes committed to 1.0-stable in [12509] and merged to trunk in [12510]. Functional test logs directory moved to test environment directory in [12511] (on trunk).

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Ryan J Ollos.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Ryan J Ollos to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.