Trac 1.5.3 with uWSGI returns empty responses when error occured.

[seino]

Trac 1.5.3 with uWSGI returns empty responses when error occured.

Reproduction steps:

1. run Trac 1.5.3 with a newly created environment.
2. try the uwsgi_curl 127.0.0.1:8080 /. This should be success.
3. try the uwsgi_curl 127.0.0.1:8080 /zzz. Trac returns an empty response.

Note that use uwsgi_curl to avoid the causes related to Nginx.

$ trac-admin test initenv
新規 Trac Environment /home/user/test の生成
..
$ cat > trac.ini << 'EOF'
[uwsgi]

master = true
processes = 2
plugins = python3
module = trac.web.main:dispatch_request
env = TRAC_ENV=/home/user/test
socket = 127.0.0.1:8080
EOF
$ uwsgi --ini trac.ini
[uWSGI] getting INI configuration from trac.ini
*** Starting uWSGI 2.0.18 (64bit) on [Fri Aug 27 07:19:29 2021] ***
compiled with version: 11.0.0 20210123 (Red Hat 11.0.0-0) on 27 January 2021 00:00:00
os: Linux-5.13.12-200.fc34.x86_64 #1 SMP Wed Aug 18 13:27:18 UTC 2021
nodename: hostname
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /home/user
detected binary path: /usr/sbin/uwsgi
your processes number limit is 3697
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 127.0.0.1:8080 fd 5
Python version: 3.9.6 (default, Jul 16 2021, 00:00:00)  [GCC 11.1.1 20210531 (Red Hat 11.1.1-3)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x199af50
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 218760 bytes (213 KB) for 2 cores
*** Operational MODE: preforking ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x199af50 pid: 100426 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 100426)
spawned uWSGI worker 1 (pid: 100427, cores: 1)
spawned uWSGI worker 2 (pid: 100428, cores: 1)
[pid: 100427|app: 0|req: 1/1]  () {16 vars in 159 bytes} [Fri Aug 27 07:19:32 2021] GET / => generated 9147 bytes in 759 msecs (HTTP/1.1 200) 5 headers in 310 bytes (0 switches on core 0)
[pid: 100428|app: 0|req: 1/2]  () {16 vars in 165 bytes} [Fri Aug 27 07:19:35 2021] GET /zzz => generated 0 bytes in 689 msecs (HTTP/1.1 404) 5 headers in 317 bytes (0 switches on core 0)

[Jun Omae]

I consider that is a uwsgi issue filed at ​SystemError when sys.exc_info() passed to start_response · Issue #2278 · unbit/uwsgi.

Should we add work around for that issue?

---

I applied temporarily the following changes.

trac/web/api.py

@@ -700 +700 @@
             self.send_header('X-XSS-Protection', 0)
         self._send_configurable_headers()
         self._send_cookie_headers()
-        self._write = self._start_response(self._status, self._outheaders,
-                                           exc_info)
+        try:
+            self._write = self._start_response(self._status, self._outheaders,
+                                               exc_info)
+        except Exception as e:
+            sys.stderr.write('%s\n' % exception_to_unicode(e, traceback=True))
+            raise
 
     def check_modified(self, datetime, extra=''):
         """Check the request "If-None-Match" header against an entity tag.

Then I get the following traceback.

Traceback (most recent call last):
  File "./trac/web/main.py", line 610, in dispatch_request
    dispatcher.dispatch(req)
  File "./trac/web/main.py", line 302, in dispatch
    raise e
  File "./trac/web/main.py", line 223, in dispatch
    raise HTTPNotFound('No handler matched request to %s',
abc.HTTPNotFound: 404 Not Found (No handler matched request to /zzz)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./trac/web/api.py", line 704, in end_headers
    self._write = self._start_response(self._status, self._outheaders,
SystemError: <built-in function uwsgi_spit> returned a result with an error set

[seino]

Thank you for your response. I understood where this defect derived from.

For the time being, I use my private patch such as followings.

trac/web/api.py

@@ -700 +700 @@
             self.send_header('X-XSS-Protection', 0)
         self._send_configurable_headers()
         self._send_cookie_headers()
-        self._write = self._start_response(self._status, self._outheaders,
-                                           exc_info)
+        self._write = self._start_response(self._status, self._outheaders)
 
     def check_modified(self, datetime, extra=''):
         """Check the request "If-None-Match" header against an entity tag.

I hope for an early release of Trac 1.6.

[michela]

This is still an issue with trac 1.6 , uwsgi 2.0.28 (ticket 2278 mentioned above still open), python 3.13.2

Could you merge seino's patch as a workaround? I can't see any issues from applying it.

[Jun Omae]

Why do you think ignoring the given exc_info is okay? I don't understand.

uWSGI should fix the issue rather than the following dirty workaround to Trac. At least, uWSGI didn't have this issue before uWSGI supporting Python 3.

trac/web/api.py

@@ -657 +657 @@
             evaluate attribute lookups
         """
         self.environ = environ
-        self._start_response = start_response
+        self._start_response = self._fixup_start_response(start_response)
         self._write = None
         self._status = '200 OK'
         self._response = None
@@ -1123 +1123 @@
         for cookie in cookies.splitlines():
             self._outheaders.append(('Set-Cookie', cookie.strip()))
 
+    def _fixup_start_response(self, fn):
+        if fn.__name__ == 'uwsgi_spit':
+            def start_response(status, response_headers, exc_info=None):
+                if exc_info and self._write:  # headers have been sent
+                    raise exc_info[1].with_traceback(exc_info[2])
+                return fn(status, response_headers)
+            return start_response
+        else:
+            return fn
+
 
 __no_apidoc__ = _HTTPException_subclass_names