[Errno 2] No such file or directory when running tracd on Windows with --auto-reload

[Ryan J Ollos]

Running tracd on windows with the -r switch:

(pve) C:\Users\Ryan Ollos\trac-dev>tracd -r -s -p 8000 tracenv
c:\users\ryanol~1\trac-dev\pve\scripts\python.exe: can't open file 'C:\Users
\RYANOL~1\trac-dev\pve\Scripts\tracd': [Errno 2] No such file or directory

Behavior seen on 1.2-stable, I need to check if behavior occurs on 1.0-stable.

It works fine without the -r switch, or when pointing to tracd-script.py rather than tracd:

trac-1.2-stable/trac/util/autoreload.py

@@ -63 +63 @@
 def _restart_with_reloader():
     while True:
         args = [sys.executable] + sys.argv
+        args[1] += '-script.py'
         if sys.platform == 'win32':
             args = ['"%s"' % arg for arg in args]
         new_environ = os.environ.copy()

That's not a proposed change, just a demonstration of what works.

[Jun Omae]

The same issue exists on Trac 1.0.13 but tracd-script.py isn't installed via wheel.

C:\>C:\usr\var\venv\trac-1.0.13\Scripts\python.exe -m pip install Trac==1.0.13
Collecting Trac==1.0.13
  Using cached Trac-1.0.13-py2-none-any.whl
Requirement already satisfied: setuptools>=0.6b1 in c:\usr\var\venv\trac-1.0.13\lib\site-packages (from Trac==1.0.13)
Requirement already satisfied: Genshi>=0.6 in c:\usr\var\venv\trac-1.0.13\lib\site-packages (from Trac==1.0.13)
Requirement already satisfied: six>=1.6.0 in c:\usr\var\venv\trac-1.0.13\lib\site-packages (from setuptools>=0.6b1->Trac==1.0.13)
Requirement already satisfied: packaging>=16.8 in c:\usr\var\venv\trac-1.0.13\lib\site-packages (from setuptools>=0.6b1->Trac==1.0.13)
Requirement already satisfied: appdirs>=1.4.0 in c:\usr\var\venv\trac-1.0.13\lib\site-packages (from setuptools>=0.6b1->Trac==1.0.13)
Requirement already satisfied: pyparsing in c:\usr\var\venv\trac-1.0.13\lib\site-packages (from packaging>=16.8->setuptools>=0.6b1->Trac==1.0.13)
Installing collected packages: Trac
Successfully installed Trac-1.0.13
C:\>C:\usr\var\venv\trac-1.0.13\Scripts\tracd -r -s -p 3000 C:\usr\var\trac\1.0
c:\usr\var\venv\trac-1.0.13\scripts\python.exe: can't open file 'C:\usr\var\venv\trac-1.0.13\Scripts\tracd': [Errno 2] No such file or directory

C:\>C:\usr\var\venv\trac-1.0.13\Scripts\python.exe
Python 2.7.10 (default, May 23 2015, 09:44:00) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> for _ in os.listdir(r'C:\usr\var\venv\trac-1.0.13\Scripts'): _
...
'activate'
'activate.bat'
'activate.ps1'
'activate_this.py'
'deactivate.bat'
'easy_install-2.7.exe'
'easy_install.exe'
'pip.exe'
'pip2.7.exe'
'pip2.exe'
'python.exe'
'pythonw.exe'
'trac-admin.exe'
'tracd.exe'
'wheel.exe'

C:\>C:\usr\var\venv\trac-1.0.13\Scripts\python.exe -m pip show -f Trac | findstr /C:"tracd"
  ..\..\Scripts\tracd.exe

C:\>C:\usr\var\venv\trac-1.0.13\Scripts\python.exe -m pip show -f Trac | findstr /C:"script"
  tracopt\mimeview\enscript.py
  tracopt\mimeview\enscript.pyc

tracd-script.py is installed if Trac is installed via egg package using easy_install.

C:\>C:\usr\var\venv\trac-1.0.13\Scripts\python.exe -m easy_install Trac==1.0.13
Searching for Trac==1.0.13
Reading https://pypi.python.org/simple/Trac/
Downloading https://pypi.python.org/packages/fa/07/2be60e9ac5f4dbd797105bc58d5dd95ce76d2f64071845a8ce035821c140/Trac-1.0.13.win-amd64.exe#md5=3ed8d0c5a9502d0dbb8b249c225d1a39
Best match: Trac 1.0.13
Processing Trac-1.0.13.win-amd64.exe
creating 'c:\users\omae\appdata\local\temp\easy_install-6flnnd\Trac-1.0.13-py2.7-win-amd64.egg' and adding 'c:\users\omae\appdata\local\temp\easy_install-6flnnd\Trac-1.0.13-py2.7-win-amd64.egg.tmp' to it
Moving Trac-1.0.13-py2.7-win-amd64.egg to c:\usr\var\venv\trac-1.0.13\lib\site-packages
Adding Trac 1.0.13 to easy-install.pth file
Installing tracd-script.py script to C:\usr\var\venv\trac-1.0.13\Scripts
Installing trac-admin-script.pyc script to C:\usr\var\venv\trac-1.0.13\Scripts
Installing trac-admin.exe script to C:\usr\var\venv\trac-1.0.13\Scripts
Installing tracd-script.pyc script to C:\usr\var\venv\trac-1.0.13\Scripts
Installing tracd.exe script to C:\usr\var\venv\trac-1.0.13\Scripts
Installing trac-admin-script.py script to C:\usr\var\venv\trac-1.0.13\Scripts
Installing trac-admin-script.py script to C:\usr\var\venv\trac-1.0.13\Scripts
Installing trac-admin.exe script to C:\usr\var\venv\trac-1.0.13\Scripts
Installing tracd-script.py script to C:\usr\var\venv\trac-1.0.13\Scripts
Installing tracd.exe script to C:\usr\var\venv\trac-1.0.13\Scripts

[Jun Omae]

According to ​http://wheel.readthedocs.io/en/latest/#setuptools-scripts-handling, script files can be installed using wheel install-files NAME.

C:\>C:\usr\var\venv\trac-1.0.13\Scripts\python.exe -m wheel install-scripts Trac==1.0.13

C:\>C:\usr\var\venv\trac-1.0.13\Scripts\python.exe
Python 2.7.10 (default, May 23 2015, 09:44:00) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> for _ in os.listdir(r'C:\usr\var\venv\trac-1.0.13\Scripts'):
...   if 'script' in _:
...     _
...
'trac-admin-script.py'
'tracd-script.py'
>>>

[Ryan J Ollos]

The issue I've reported is seen using develop mode with an egg, not wheel, and script files are installed. I've installed with python setup.py develop.

The issue I'm reporting is only seen when using -r/--auto-reload.

(pve) C:\Users\Ryan Ollos\trac-dev>pip list --format=columns
Package    Version    Location
---------- ---------- --------------------------------------------
appdirs    1.4.3
Genshi     0.7
packaging  16.8
pip        9.0.1
pyparsing  2.2.0
setuptools 34.3.3
six        1.10.0
Trac       1.2.2.dev0 c:\users\ryan ollos\trac-dev\trac-1.2-stable
wheel      0.29.0
(pve) C:\Users\Ryan Ollos\trac-dev>dir pve\Scripts
 Volume in drive C has no label.
 Volume Serial Number is 6C99-BF6F

 Directory of C:\Users\Ryan Ollos\trac-dev\pve\Scripts

04/04/2017  08:49 PM    <DIR>          .
04/04/2017  08:49 PM    <DIR>          ..
04/04/2017  05:31 PM             2,195 activate
04/04/2017  05:31 PM               765 activate.bat
04/04/2017  05:31 PM             8,325 activate.ps1
04/04/2017  05:31 PM             1,137 activate_this.py
04/04/2017  05:31 PM               508 deactivate.bat
04/04/2017  05:31 PM            98,180 easy_install-2.7.exe
04/04/2017  08:49 PM               433 easy_install-3.6-script.py
04/04/2017  08:49 PM            74,752 easy_install-3.6.exe
04/04/2017  08:49 PM               425 easy_install-script.py
04/04/2017  08:49 PM            74,752 easy_install.exe
04/04/2017  05:31 PM            98,152 pip.exe
04/04/2017  05:31 PM            98,152 pip2.7.exe
04/04/2017  05:31 PM            98,152 pip2.exe
04/04/2017  05:31 PM            28,160 python.exe
04/04/2017  05:31 PM            28,160 pythonw.exe
04/04/2017  08:49 PM               403 trac-admin-script.py
04/04/2017  08:49 PM            74,752 trac-admin.exe
04/04/2017  08:49 PM               393 tracd-script.py
04/04/2017  08:49 PM            74,752 tracd.exe
04/04/2017  05:31 PM            98,159 wheel.exe
              20 File(s)        860,707 bytes
               2 Dir(s)  10,092,253,184 bytes free
(pve) C:\Users\Ryan Ollos\trac-dev>tracd -r -s -p 8000 tracenv
C:\Users\RYANOL~1\trac-dev\pve\Scripts\python.exe: can't open file 'C:\Users\RYA
NOL~1\trac-dev\pve\Scripts\tracd': [Errno 2] No such file or directory
2

(pve) C:\Users\Ryan Ollos\trac-dev>tracd -s -p 8000 tracenv
Server starting in PID 4044.
Serving on 0.0.0.0:8000 view at http://127.0.0.1:8000/
Using HTTP/1.1 protocol version

Regarding the wheel behavior, see also ​gmessage:trac-dev:z6Vg8o0U5XA/zgJYzrREAAAJ.

[Jun Omae]

Sorry for my noise. I consider we intentionally use only for development and debugging with -r option of trac/web/standard.py. I propose wontfix, otherwise prevent uses of -r option in installed tracd script.

See also Automatic reloading: For development: … in TracStandalone#Pros.

[Jun Omae]

We could check whether sys.argv[0] is existent and executable, and whether sys.argv[0] + '.exe' is executable on Windows.

trac/util/autoreload.py

@@ -62 +62 @@
 
 def _restart_with_reloader():
     while True:
-        args = [sys.executable] + sys.argv
+        if os.path.isfile(sys.argv[0]):
+            args = sys.argv if os.access(sys.argv[0], os.X_OK) \
+                   else [sys.executable] + sys.argv
+        elif sys.platform == 'win32' and \
+                os.access(sys.argv[0] + '.exe', os.X_OK):
+            args = [sys.argv[0] + '.exe'] + sys.argv[1:]
+        else:
+            args = [sys.executable] + sys.argv
+        path = args[0]
         if sys.platform == 'win32':
             args = ['"%s"' % arg for arg in args]
         new_environ = os.environ.copy()
@@ -70 +78 @@
 
         # This call reinvokes ourself and goes into the other branch of main as
         # a new process.
-        exit_code = os.spawnve(os.P_WAIT, sys.executable,
-                               args, new_environ)
+        exit_code = os.spawnve(os.P_WAIT, path, args, new_environ)
         if exit_code != 3:
             return exit_code
 

[Ryan J Ollos]

Thanks, comment:6 change works well.

[Ryan J Ollos]

I tested on Windows7 and OSX. Committed to 1.0-stable in r15745, merged in r15746, r15747.

[Peter Suter]

Hm, before r15747 tracd.exe -r works fine for me (on Windows, with Trac installed via python setup.py develop in a virtualenv):

• <venv>\scripts\tracd.exe exists.
• In os.spawnve(..., sys.executable, args, ...):
  • sys.executable is <venv>\scripts\python.exe (which exists).
  • args is ['"<venv>\scripts\python.exe"', '"<venv>\scripts\tracd-script.py"', '"-r"', ...]

But after r15747 when trying to run tracd.exe -r I get OSError: [Errno 8] Exec format error from os.spawnve():

• <venv>\scripts\tracd.exe still exists.
• In os.spawnve(..., path, args, ...):
  • path is <venv>\scripts\tracd-script.py (which exists).
  • args is ['"<venv>\scripts\tracd-script.py"', '"-r"', ...]
• (sys.executable is still <venv>\scripts\python.exe.)
• sys.argv[0] is<venv>\scripts\tracd-script.py.
  • os.access(sys.argv[0], os.X_OK) returns True.

Any ideas?

[Jun Omae]

Thanks for the catching.

The issue is reproduced with:

Genshi (0.7)
pip (7.1.0)
setuptools (18.0.1)
Trac (1.2.2.dev0, c:\usr\src\trac\trac.git)
wheel (0.24.0)

Works fine with:

appdirs (1.4.3)
Genshi (0.7)
packaging (16.8)
pip (9.0.1)
pyparsing (2.2.0)
setuptools (34.4.1)
six (1.10.0)
Trac (1.2.2.dev0, c:\usr\src\trac\trac.git)
wheel (0.29.0)

[Jun Omae]

Ah. It seems to be unable to use os.X_OK on Windows….

>>> os.access(r'setup.py', os.X_OK)
True
>>> os.access(r'setup.cfg', os.X_OK)
True

Instead, check existent and suffix of the file is .exe.

trac/util/autoreload.py

@@ -61 +61 @@
         time.sleep(_SLEEP_TIME)
 
 def _restart_with_reloader():
-    while True:
-        if os.path.isfile(sys.argv[0]):
-            args = sys.argv if os.access(sys.argv[0], os.X_OK) \
-                   else [sys.executable] + sys.argv
-        elif sys.platform == 'win32' and \
-                os.access(sys.argv[0] + '.exe', os.X_OK):
-            args = [sys.argv[0] + '.exe'] + sys.argv[1:]
-        else:
-            args = [sys.executable] + sys.argv
-        path = args[0]
-        if sys.platform == 'win32':
-            args = ['"%s"' % arg for arg in args]
-        new_environ = os.environ.copy()
-        new_environ['RUN_MAIN'] = 'true'
+    is_win32 = sys.platform == 'win32'
+    if is_win32:
+        can_exec = lambda path: os.path.isfile(path) and \
+                                os.path.normpath(path).endswith('.exe')
+    else:
+        can_exec = lambda path: os.access(path, os.X_OK)
 
+    if os.path.isfile(sys.argv[0]):
+        args = sys.argv if can_exec(sys.argv[0]) else \
+               [sys.executable] + sys.argv
+    elif is_win32 and can_exec(sys.argv[0] + '.exe'):
+        args = [sys.argv[0] + '.exe'] + sys.argv[1:]
+    elif os.path.isfile(sys.argv[0] + '-script.py'):
+        args = [sys.executable, sys.argv[0] + '-script.py'] + sys.argv[1:]
+    else:
+        args = [sys.executable] + sys.argv
+    path = args[0]
+    if is_win32:
+        args = ['"%s"' % arg for arg in args]
+    new_environ = os.environ.copy()
+    new_environ['RUN_MAIN'] = 'true'
+
+    while True:
         # This call reinvokes ourself and goes into the other branch of main as
         # a new process.
         exit_code = os.spawnve(os.P_WAIT, path, args, new_environ)

[Peter Suter]

Ah. It seems to be unable to use os.X_OK on Windows…. Instead, check existent and suffix of the file is .exe.

This seems to work well for me, thanks!

[Jun Omae]

Thanks for the testing! Fixed in [15778-15780], again.