Edgewall Software
Modify

Opened 4 years ago

Closed 4 years ago

Last modified 3 years ago

#11802 closed enhancement (fixed)

Ability to send chunked response in Request.send

Reported by: Jun Omae Owned by: Jun Omae
Priority: normal Milestone: 1.0.6
Component: rendering Version:
Severity: normal Keywords: chunked
Cc: leho@…
Release Notes:

Added [trac] use_chunked_encoding option. If enabled, send as chunked encoding from rendering Genshi templates and downloading in other formats.

API Changes:
  • trac.web.api:
    • Request.send() and Request.write() accept an iterable instance now.
    • Chrome.render_template: Added iterable keyword parameter.
    • Chrome.iterable_content: Added.
  • trac.mimeview.api:
    • Mimeview: Allow to return an iterable instance from IContentConverter.convert_content.
    • Mimeview.convert_content: added iterable keyword parameter.

Description

Currently, Trac starts to response after entire of content is generated from genshi template. Users have a long wait for the response because the rendering is not fast. Especially, timeline and query pages are very slow when the page has many entries.

If Trac starts to response without waiting for entire of content, it would feel a bit fast to users.

In chunked-encoding.diff, Request.send accepts an iterable instance. The patch for 0.12.x is used in our Trac hosting.

Drawback of the changes is that exception while rendering the template cannot be shown as internal error.

Also, it is less improved in wiki page. Because wiki formatter format_to_* methods return entire of generated content. It needs to modify the methods returning content as iterable.

Attachments (1)

chunked-encoding.diff (4.3 KB ) - added by Jun Omae 4 years ago.

Download all attachments as: .zip

Change History (17)

Changed 4 years ago by Jun Omae

Attachment: chunked-encoding.diff added

comment:1 Changed 4 years ago by lkraav <leho@…>

Cc: leho@… added

comment:2 Changed 4 years ago by Jun Omae

Proposed changes in jomae.git@t11802, applied the patch and added [trac] use_chunked_encoding option.

comment:3 Changed 4 years ago by Jun Omae

Milestone: next-stable-1.0.x1.0.3
Owner: set to Jun Omae
Status: newassigned

I want to add the ability. Sending content as chunked-encoding would feel fast-response to users for large file in browser page, large diff in changeset page, many entries in timeline page and many tickets in query page.

jomae.git@t11802 includes the following changes.

  • Add [trac] use_chunked_encoding option.
    • If disabled, Trac sends response with Content-Length header without chunked-encoding (default).
    • If enabled, Trac sends response with chunked-encoding.
  • Request.write() and Request.send() accept an iterable instance.
  • Add iterable parameter to Chrome.render_template() to retrieve content of genshi rendering as an iterable instance.
  • Add Chrome.iterable_content() method which retrieves an iterable instance from a Genshi stream instance.
  • Allow to return an iterable instance from IContentConverter.convert_content().

Thoughts?

Last edited 4 years ago by Ryan J Ollos (previous) (diff)

comment:4 Changed 4 years ago by anonymous

+1 Everything that makes trac feel faster is very welcomed

comment:5 Changed 4 years ago by Christian Boos

A logical continuation of the work started in #717. Give me some time to review and test.

comment:6 Changed 4 years ago by Jun Omae

Milestone: 1.0.31.0.4

comment:7 Changed 4 years ago by Ryan J Ollos

Milestone: 1.0.41.0.5

comment:8 Changed 4 years ago by Jun Omae

Milestone: 1.0.51.0.6

I'll retest and push it at the start of 1.0.6 if no objections.

comment:9 Changed 4 years ago by anonymous

Push it real good

comment:10 Changed 4 years ago by Jun Omae

API Changes: modified (diff)
Release Notes: modified (diff)

Updated jomae.git@t11802. Added unit tests and confirmed ontracd and Apache2 with mod_wsgi.

comment:11 Changed 4 years ago by Jun Omae

Resolution: fixed
Status: assignedclosed

I tested on tracd, Apache 2.2 with/without mod_deflate and Nginx with uWSGI. Committed in [14029] and merged to trunk in [14030].

comment:12 Changed 4 years ago by lkraav <leho@…>

http://i.imgur.com/RiJHn.gif

comment:13 Changed 4 years ago by Jun Omae

Release Notes: modified (diff)

Functional tests use [trac] use_chunked_encoding = disabled because internal error while rendering templates cannot be handled.

comment:14 Changed 3 years ago by Ryan J Ollos

While working on #12046, I encountered this error:

======================================================================
ERROR: test_rss_conversion (trac.ticket.tests.conversion.TicketConversionTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/ticket/tests/conversion.py", line 144, in test_rss_conversion
    self.req, 'trac.ticket.Ticket', ticket, 'rss')
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/api.py", line 725, in convert_content
    content = ''.join(content)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/web/chrome.py", line 1217, in iterable_content
    location = '%s (unknown template location)' % filename
NameError: global name 'filename' is not defined

It looks like filename isn't initialized: browser:tags/trac-1.0.6/trac/web/chrome.py@:1180#L1158.

comment:15 Changed 3 years ago by Jun Omae

Thanks for the reporting. Oh, I'm stupid….

The patch removes the filename variable bacause the file name of template is logged by exception_to_unicode(e, traceback=True). I'll apply this later.

  • trac/web/chrome.py

    diff --git a/trac/web/chrome.py b/trac/web/chrome.py
    index decf3c0d5..195935560 100644
    a b class Chrome(Component):  
    11801180            if pos:
    11811181                location = "'%s', line %s, char %s" % pos
    11821182            else:
    1183                 location = '%s (unknown template location)' % filename
     1183                location = '(unknown template location)'
    11841184            self.log.error('Genshi %s error while rendering template %s%s',
    11851185                           e.__class__.__name__, location,
    11861186                           exception_to_unicode(e, traceback=True))

comment:16 Changed 3 years ago by Jun Omae

Committed in [14132:14133].

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted.
to The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.