Ability to send chunked response in Request.send

[Jun Omae]

Currently, Trac starts to response after entire of content is generated from genshi template. Users have a long wait for the response because the rendering is not fast. Especially, timeline and query pages are very slow when the page has many entries.

If Trac starts to response without waiting for entire of content, it would feel a bit fast to users.

In chunked-encoding.diff​, Request.send accepts an iterable instance. The patch for 0.12.x is used in our Trac hosting.

Drawback of the changes is that exception while rendering the template cannot be shown as internal error.

Also, it is less improved in wiki page. Because wiki formatter format_to_* methods return entire of generated content. It needs to modify the methods returning content as iterable.

[Jun Omae]

Proposed changes in jomae.git@t11802, applied the patch and added [trac] use_chunked_encoding option.

[Jun Omae]

I want to add the ability. Sending content as chunked-encoding would feel fast-response to users for large file in browser page, large diff in changeset page, many entries in timeline page and many tickets in query page.

jomae.git@t11802 includes the following changes.

• Add [trac] use_chunked_encoding option.
  • If disabled, Trac sends response with Content-Length header without chunked-encoding (default).
  • If enabled, Trac sends response with chunked-encoding.
• Request.write() and Request.send() accept an iterable instance.
• Add iterable parameter to Chrome.render_template() to retrieve content of genshi rendering as an iterable instance.
• Add Chrome.iterable_content() method which retrieves an iterable instance from a Genshi stream instance.
• Allow to return an iterable instance from IContentConverter.convert_content().

Thoughts?

[anonymous]

+1 Everything that makes trac feel faster is very welcomed

[Christian Boos]

A logical continuation of the work started in #717. Give me some time to review and test.

[Jun Omae]

I'll retest and push it at the start of 1.0.6 if no objections.

[anonymous]

Push it real good

[Jun Omae]

Updated jomae.git@t11802. Added unit tests and confirmed ontracd and Apache2 with mod_wsgi.

[Jun Omae]

I tested on tracd, Apache 2.2 with/without mod_deflate and Nginx with uWSGI. Committed in [14029] and merged to trunk in [14030].

[Jun Omae]

Functional tests use [trac] use_chunked_encoding = disabled because internal error while rendering templates cannot be handled.

[Ryan J Ollos]

While working on #12046, I encountered this error:

======================================================================
ERROR: test_rss_conversion (trac.ticket.tests.conversion.TicketConversionTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/ticket/tests/conversion.py", line 144, in test_rss_conversion
    self.req, 'trac.ticket.Ticket', ticket, 'rss')
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/api.py", line 725, in convert_content
    content = ''.join(content)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/web/chrome.py", line 1217, in iterable_content
    location = '%s (unknown template location)' % filename
NameError: global name 'filename' is not defined

It looks like filename isn't initialized: browser:tags/trac-1.0.6/trac/web/chrome.py@:1180#L1158​.

[Jun Omae]

Thanks for the reporting. Oh, I'm stupid….

The patch removes the filename variable bacause the file name of template is logged by exception_to_unicode(e, traceback=True). I'll apply this later.

trac/web/chrome.py

@@ -1180 +1180 @@
             if pos:
                 location = "'%s', line %s, char %s" % pos
             else:
-                location = '%s (unknown template location)' % filename
+                location = '(unknown template location)'
             self.log.error('Genshi %s error while rendering template %s%s',
                            e.__class__.__name__, location,
                            exception_to_unicode(e, traceback=True))

[Jun Omae]

Committed in [14132:14133].