Unable to edit components with slash or backslash

[itamaro]

Components that are created with slashes or backslashes in the name are not editable via the admin web interface.

When trying to edit a component with backslash I get "(404) Not Found" error for the URL .../admin/ticket/components/Test \ Component

When trying to edit a component with slash I get a "Trac Error" with "Component Test/ Component does not exist." for the URL .../admin/ticket/components/Test / Component (note that a whitespace is missing between "Test" and "/" in the error report), and all styles are stripped from the result page.

Trying to modify the URL to .../admin/ticket/components/Test %2F Component results a 404 error, as in the backslash case.

I have set the severity to minor because I can still use the admin console.

[anonymous]

Works fine here on demo-0.12...

[cboos]

The URL is http://trac.edgewall.org/demo-0.12/admin/ticket/components/another%20/%20with%20/%20space (OK, just as an example as you probably can go to that page ;-) ), and I can modify the component without trouble.

[itamaro]

OK, further inspection reveals that the issue does not occur when I use tracd as front-end.

I don't know what's behind the demo site, but it does occur on my setup -- Windows 7, Apache with mod_wsgi, Python 2.6.5.

Can anyone confirm (or contradict) this with a similar setup?

[cboos]

Here it's lighty+fcgi. I just tested on Apache (both mod_wsgi and mod_python) and it also worked, but that was on Linux.

[Alexander Sokolov <iskander.sokol@…>]

I faced this problem on Windows XP when I used backslash in repository name.

Apache returns 404 error for links in repository browser if name of repository contains backslash.

Apache access.log

127.0.0.1 - - [19/Aug/2010:14:52:02 +0400] "GET /trac/sendbox/browser/e%3A%5CScratch%5Ctrac%5Cdata%5Csvn%5Csendbox HTTP/1.1" 404 251

Apache error.log

[Thu Aug 19 14:51:25 2010] [notice] Apache/2.2.13 (Win32) DAV/2 SVN/1.6.6 mod_wsgi/3.3 Python/2.6.4 configured -- resuming normal operations
...
[Thu Aug 19 14:52:02 2010] [info] [client 127.0.0.1] found %2f (encoded '/') in URI (decoded='/trac/sendbox/browser/e:\\Scratch\\trac\\data\\svn\\sendbox'), returning 404, referer: http://localhost/trac/sendbox/browser

tracd processes such link normally.

Besides unicode zero width spaces inserted in directory after slashes are rendered as squares (see attachment) in Opera 10.10.

[Alexander Sokolov <iskander.sokol@…>]

unicode ZWSP in repository directory name

[cboos]

Any suggestion about what should be done here?

[itamaro]

Not sure what to do, but I did manage to reproduce this on several setups, all on Windows (XP, 7, 2008), all with mod_wsgi.

Might be useful to note that when editing "Test \ Component" the apache log shows:

[Mon Sep 27 00:25:31 2010] [info] [client IP_ADDR] found %2f (encoded '/') in URI (decoded='/trac/Sandbox/admin/ticket/components/Test \\ Component'), returning 404, referer: http://my-server/trac/Sandbox/admin/ticket/components

and when editing "Test %2F Component" (by modifying in the address bar):

[Mon Sep 27 00:25:06 2010] [info] [client IP_ADDR] found %2f (encoded '/') in URI (decoded='/trac/Sandbox/admin/ticket/components/Test / Component'), returning 404

and when editing "Test / Component" the apache log is clean, but Trac log:

2010-09-27 00:30:42,829 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/ Component'">
2010-09-27 00:30:43,127 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/ Component does not exist.)
2010-09-27 00:30:43,174 Trac[chrome] DEBUG: Prepare chrome data for request
2010-09-27 00:30:43,440 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/Sandbox/chrome/common/css/trac.css'">
2010-09-27 00:30:43,440 Trac[session] DEBUG: Retrieving session for ID 'itamar'
2010-09-27 00:30:43,454 Trac[main] DEBUG: Negotiated locale: ['en-US', 'en'] -> en_US
2010-09-27 00:30:43,470 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/Sandbox/chrome/common/js/jquery.js'">
2010-09-27 00:30:43,470 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/Sandbox/chrome/common/js/babel.js'">
2010-09-27 00:30:43,470 Trac[session] DEBUG: Retrieving session for ID 'itamar'
2010-09-27 00:30:43,486 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/Sandbox/chrome/common/js/messages/en_US.js'">
2010-09-27 00:30:43,486 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/Sandbox/chrome/common/js/trac.js'">
2010-09-27 00:30:43,502 Trac[main] DEBUG: Negotiated locale: ['en-US', 'en'] -> en_US
2010-09-27 00:30:43,502 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/Sandbox/chrome/common/js/search.js'">
2010-09-27 00:30:43,502 Trac[session] DEBUG: Retrieving session for ID 'itamar'
2010-09-27 00:30:43,502 Trac[session] DEBUG: Retrieving session for ID 'itamar'
2010-09-27 00:30:43,502 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/Sandbox/chrome/common/css/trac.css does not exist.)
2010-09-27 00:30:43,502 Trac[session] DEBUG: Retrieving session for ID 'itamar'
2010-09-27 00:30:43,517 Trac[chrome] DEBUG: Prepare chrome data for request
2010-09-27 00:30:43,517 Trac[session] DEBUG: Retrieving session for ID 'itamar'
2010-09-27 00:30:43,517 Trac[main] DEBUG: Negotiated locale: ['en-US', 'en'] -> en_US
2010-09-27 00:30:43,704 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/Sandbox/chrome/common/js/jquery.js does not exist.)
2010-09-27 00:30:43,704 Trac[chrome] DEBUG: Prepare chrome data for request
2010-09-27 00:30:43,720 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/Sandbox/chrome/common/js/babel.js does not exist.)
2010-09-27 00:30:43,720 Trac[chrome] DEBUG: Prepare chrome data for request
2010-09-27 00:30:43,720 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/Sandbox/chrome/common/js/trac.js does not exist.)
2010-09-27 00:30:43,736 Trac[chrome] DEBUG: Prepare chrome data for request
2010-09-27 00:30:43,736 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/Sandbox/chrome/common/js/messages/en_US.js does not exist.)
2010-09-27 00:30:43,736 Trac[chrome] DEBUG: Prepare chrome data for request
2010-09-27 00:30:43,736 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/Sandbox/chrome/common/js/search.js does not exist.)
2010-09-27 00:30:43,752 Trac[chrome] DEBUG: Prepare chrome data for request
2010-09-27 00:30:44,190 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/Sandbox/chrome/common/css/code.css'">
2010-09-27 00:30:44,220 Trac[session] DEBUG: Retrieving session for ID 'itamar'
2010-09-27 00:30:44,236 Trac[main] DEBUG: Negotiated locale: ['en-US', 'en'] -> en_US
2010-09-27 00:30:44,345 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/Sandbox/chrome/common/css/code.css does not exist.)
2010-09-27 00:30:44,361 Trac[chrome] DEBUG: Prepare chrome data for request
2010-09-27 00:30:44,799 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/Sandbox/chrome/common/trac_logo_mini.png'">
2010-09-27 00:30:44,799 Trac[session] DEBUG: Retrieving session for ID 'itamar'
2010-09-27 00:30:44,815 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/Sandbox/chrome/site/your_project_logo.png'">
2010-09-27 00:30:44,815 Trac[main] DEBUG: Negotiated locale: ['en-US', 'en'] -> en_US
2010-09-27 00:30:44,815 Trac[session] DEBUG: Retrieving session for ID 'itamar'
2010-09-27 00:30:44,829 Trac[main] DEBUG: Negotiated locale: ['en-US', 'en'] -> en_US
2010-09-27 00:30:44,877 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/Sandbox/chrome/common/trac_logo_mini.png does not exist.)
2010-09-27 00:30:44,877 Trac[chrome] DEBUG: Prepare chrome data for request
2010-09-27 00:30:44,877 Trac[api] INFO: Synchronized '' repository in 0.06 seconds
2010-09-27 00:30:44,892 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/Sandbox/chrome/site/your_project_logo.png does not exist.)
2010-09-27 00:30:44,892 Trac[chrome] DEBUG: Prepare chrome data for request
2010-09-27 00:30:45,407 Trac[main] DEBUG: Dispatching <Request "GET '/admin/ticket/components/Test/Sandbox/chrome/common/trac.ico'">
2010-09-27 00:30:45,407 Trac[session] DEBUG: Retrieving session for ID 'itamar'
2010-09-27 00:30:45,407 Trac[main] DEBUG: Negotiated locale: ['en-US', 'en'] -> en_US
2010-09-27 00:30:45,440 Trac[api] INFO: Synchronized '' repository in 0.03 seconds
2010-09-27 00:30:45,454 Trac[main] WARNING: HTTPNotFound: 404 Trac Error (Component Test/Sandbox/chrome/common/trac.ico does not exist.)
2010-09-27 00:30:45,454 Trac[chrome] DEBUG: Prepare chrome data for request

[itamaro]

Following a multi-hour hard-core-debugging session of this issue,
here's my findings:

Tested the issue with mod_python, and it doesn't occur, so I decided to dive in and debug the wsgi handling code (trac.web.main).

First thing I did: ​Discover that I need to add AllowEncodedSlashes On to my Apache conf in order to prevent "fake 404" when request URL contains backslashes (relates to Alexander's comment:5).
Once I did this, Apache handed the requests over to wsgi, and the behavior was symmetrical for "Test / Component" and "Test \ Component".

Then I went to trac.web.main.dispatch_request and dumped the passed environ to a log file, in which I noticed the interesting variable are SCRIPT_NAME, PATH_INFO, and REQUEST_URI.

When trying to edit "Test / Component" (note that without whitespaces it works fine), the variables are:

'SCRIPT_NAME': '/trac/Sandbox/admin/ticket/components/Test ',
'PATH_INFO': '/Sandbox/admin/ticket/components/Test/ Component',
'REQUEST_URI': '/trac/Sandbox/admin/ticket/components/Test%20/%20Component',

And when trying to edit "Test\Component" (note this time- no whitespaces):

'SCRIPT_NAME': '/trac/Sandbox/admin/ticket/components/Test\\Component',
'PATH_INFO': '/Sandbox/admin/ticket/components/Test/Component',
'REQUEST_URI': '/trac/Sandbox/admin/ticket/components/Test%5CComponent',

So it seems that the REQUEST_URI variable is "reliable" in both cases (up to URL-encoding), while SCRIPT_NAME and PATH_INFO are not quite right, each in its own peculiar way. It also seems that the Trac code relies on the PATH_INFO variable to deduce the request details.

I think now we can think about "what to do", and I see several alternatives:

1. wontfix and forget about it (or put a warning next to mod_wsgi on Windows (I wonder what's going on with other platforms))
2. I would consider it a mod_wsgi bug, so if someone from core-devs agree, I can report it to mod_wsgi project.
3. Possibly, using REQUEST_URI as well, a workaround can be patched into trac.web.main to handle this correctly. I can also have a try at it, but am limited to w.r.t testing platforms (just Windows, Py-2.6).

[mrelbe]

I've never noticed this before... Editing any ticket element yields URLs according to the scheme .../admin/ticket/<element>/<name> which means that the same problem applies to all adminstrated ticket elements such as milestones, priorities etc.

This must be a design flaw in the Trac admin component, right?

When editing a ticket element, shouldn't the URL instead be something like .../admin/ticket/<element>?edit=<name>?

[rblank]

Replying to mrelbe:

When editing a ticket element, shouldn't the URL instead be something like .../admin/ticket/<element>?edit=<name>?

It could, but there is no fundamental rule about this. Note that using query arguments would also solve #7617.

[cboos]

Replying to itamaro:

When trying to edit "Test / Component" (note that without whitespaces it works fine), the variables are: [...]

You didn't say where you put those debug statements, we do quite a few manipulations on the path_info early on.

[itamaro]

Re comment:11: quite early on :-)

def dispatch_request(environ, start_response):
    """Main entry point for the Trac web interface.
    
    @param environ: the WSGI environment dict
    @param start_response: the WSGI callback for starting the response
    """

+    with open(r'...\debug.log', 'a') as f:
+        f.write(repr(environ) + '\n')
    # SCRIPT_URL is an Apache var containing the URL before URL rewriting
    # has been applied, so we can use it to reconstruct logical SCRIPT_NAME
    script_url = environ.get('SCRIPT_URL')

Re comment:9:

I checked environ for ../admin/ticket/component?edit=Test%20%5F%20Component, and it looks workable (similar result with %5C):

'PATH_INFO': '/Sandbox/admin/ticket/components',
'QUERY_STRING': 'edit=Test%20%2F%20Component',
'REQUEST_URI': '/trac/Sandbox/admin/ticket/components?edit=Test%20%2F%20Component',

So this is definitely +1 from me.

[cboos]

I'm also OK for the suggestion in comment:9, but for #7617 it's a different matter: there it's much like a Wiki page, we might one day add a semantic to milestone hierarchical names. In any case, it's better to keep the readable URL /milestone/<milestone-name> there.

[jomae]

I think the broken PATH_INFO is apache/win32 issue. When PATH_INFO contains %20/%20, the apache on Windows generates broken PATH_INFO. It also happens with mod_cgi.

wiki:"test / wiki"? link also occurs the same issue with apache on Windows.

The following text is the output of accessing http://localhost/cgi-bin/printenv.py/test%20/%20comp, printenv.py is like printenv.pl.

HTTP_COOKIE: 'jira.toggleblocks.cong.cookie=; jira.conglomerate.cookie=; AJS.conglomerate.cookie=||||||||||upm.tab=manage'
SERVER_SOFTWARE: 'Apache/2.2.14 (Win32) DAV/2 SVN/1.6.9 mod_wsgi/3.3 Python/2.6.6'
SCRIPT_NAME: '/cgi-bin/printenv.py/test '
SERVER_SIGNATURE: ''
REQUEST_METHOD: 'GET'
PATH_INFO: '/test/ comp'
SERVER_PROTOCOL: 'HTTP/1.1'
QUERY_STRING: ''
PATH: 'C:\\TracLight\\python;C:\\TracLight\\python\\Scripts;C:\\TracLight\\CollabNetSVN;C:\\TracLight\\CollabNetSVN\\httpd\\bin;C:\\TracLight\\Graphviz\\bin;C:\\oraclexe\\app\\oracle\\product\\10.2.0\\server\\bin;C:\\usr\\Ruby\\bin;C:\\Program Files\\Support Tools\\;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\Program Files\\Intel\\DMIX;C:\\Program Files\\Common Files\\Roxio Shared\\DLLShared\\;C:\\usr\\bin;C:\\WINDOWS\\system32\\kktools;c:\\Program Files\\Microsoft SQL Server\\90\\Tools\\binn\\;C:\\Program Files\\Common Files\\Ulead Systems\\MPEG;C:\\Program Files\\TortoiseSVN\\bin;C:\\TracLight\\python;C:\\TracLight\\python\\Scripts;C:\\TracLight\\CollabNetSVN;C:\\TracLight\\CollabNetSVN\\httpd\\bin;C:\\TracLight\\Graphviz\\bin;C:\\TracLight\\maven\\bin;C:\\TracLight\\bin'
HTTP_ACCEPT_CHARSET: 'UTF-8,*'
HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 5.1; rv:6.0.2) Gecko/20100101 Firefox/6.0.2'
HTTP_CONNECTION: 'keep-alive'
SERVER_NAME: 'localhost'
REMOTE_ADDR: '127.0.0.1'
PATH_TRANSLATED: 'C:\\TracLight\\CollabNetSVN\\httpd\\htdocs\\test\\ comp'
SERVER_PORT: '80'
SERVER_ADDR: '127.0.0.1'
DOCUMENT_ROOT: 'C:/TracLight/CollabNetSVN/httpd/htdocs'
SYSTEMROOT: 'C:\\WINDOWS'
COMSPEC: 'C:\\WINDOWS\\system32\\cmd.exe'
SCRIPT_FILENAME: 'C:/TracLight/CollabNetSVN/httpd/cgi-bin/printenv.py'
SERVER_ADMIN: 'admin@your-domain.com'
SCRIPT_URI: 'http://localhost/cgi-bin/printenv.py/test / comp'
HTTP_HOST: 'localhost'
SCRIPT_URL: '/cgi-bin/printenv.py/test / comp'
PATHEXT: '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.RB;.RBW'
REQUEST_URI: '/cgi-bin/printenv.py/test%20/%20comp'
HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
WINDIR: 'C:\\WINDOWS'
GATEWAY_INTERFACE: 'CGI/1.1'
REMOTE_PORT: '20623'
HTTP_ACCEPT_LANGUAGE: 'ja,en-us;q=0.7,en;q=0.3'
HTTP_ACCEPT_ENCODING: 'gzip, deflate'