Edgewall Software
Modify

Opened 5 years ago

Closed 4 years ago

Last modified 4 years ago

#11802 closed enhancement (fixed)

Ability to send chunked response in Request.send

Reported by: Jun Omae Owned by: Jun Omae
Priority: normal Milestone: 1.0.6
Component: rendering Version:
Severity: normal Keywords: chunked
Cc: leho@… Branch:
Release Notes:

Added [trac] use_chunked_encoding option. If enabled, send as chunked encoding from rendering Genshi templates and downloading in other formats.

API Changes:
  • trac.web.api:
    • Request.send() and Request.write() accept an iterable instance now.
    • Chrome.render_template: Added iterable keyword parameter.
    • Chrome.iterable_content: Added.
  • trac.mimeview.api:
    • Mimeview: Allow to return an iterable instance from IContentConverter.convert_content.
    • Mimeview.convert_content: added iterable keyword parameter.

Description

Currently, Trac starts to response after entire of content is generated from genshi template. Users have a long wait for the response because the rendering is not fast. Especially, timeline and query pages are very slow when the page has many entries.

If Trac starts to response without waiting for entire of content, it would feel a bit fast to users.

In chunked-encoding.diff, Request.send accepts an iterable instance. The patch for 0.12.x is used in our Trac hosting.

Drawback of the changes is that exception while rendering the template cannot be shown as internal error.

Also, it is less improved in wiki page. Because wiki formatter format_to_* methods return entire of generated content. It needs to modify the methods returning content as iterable.

Attachments (1)

chunked-encoding.diff (4.3 KB ) - added by Jun Omae 5 years ago.

Download all attachments as: .zip

Change History (17)

by Jun Omae, 5 years ago

Attachment: chunked-encoding.diff added

comment:1 by lkraav <leho@…>, 5 years ago

Cc: leho@… added

comment:2 by Jun Omae, 5 years ago

Proposed changes in jomae.git@t11802, applied the patch and added [trac] use_chunked_encoding option.

comment:3 by Jun Omae, 5 years ago

Milestone: next-stable-1.0.x1.0.3
Owner: set to Jun Omae
Status: newassigned

I want to add the ability. Sending content as chunked-encoding would feel fast-response to users for large file in browser page, large diff in changeset page, many entries in timeline page and many tickets in query page.

jomae.git@t11802 includes the following changes.

  • Add [trac] use_chunked_encoding option.
    • If disabled, Trac sends response with Content-Length header without chunked-encoding (default).
    • If enabled, Trac sends response with chunked-encoding.
  • Request.write() and Request.send() accept an iterable instance.
  • Add iterable parameter to Chrome.render_template() to retrieve content of genshi rendering as an iterable instance.
  • Add Chrome.iterable_content() method which retrieves an iterable instance from a Genshi stream instance.
  • Allow to return an iterable instance from IContentConverter.convert_content().

Thoughts?

Last edited 4 years ago by Ryan J Ollos (previous) (diff)

comment:4 by anonymous, 5 years ago

+1 Everything that makes trac feel faster is very welcomed

comment:5 by Christian Boos, 5 years ago

A logical continuation of the work started in #717. Give me some time to review and test.

comment:6 by Jun Omae, 5 years ago

Milestone: 1.0.31.0.4

comment:7 by Ryan J Ollos, 5 years ago

Milestone: 1.0.41.0.5

comment:8 by Jun Omae, 5 years ago

Milestone: 1.0.51.0.6

I'll retest and push it at the start of 1.0.6 if no objections.

comment:9 by anonymous, 5 years ago

Push it real good

comment:10 by Jun Omae, 5 years ago

API Changes: modified (diff)
Release Notes: modified (diff)

Updated jomae.git@t11802. Added unit tests and confirmed ontracd and Apache2 with mod_wsgi.

comment:11 by Jun Omae, 4 years ago

Resolution: fixed
Status: assignedclosed

I tested on tracd, Apache 2.2 with/without mod_deflate and Nginx with uWSGI. Committed in [14029] and merged to trunk in [14030].

comment:12 by lkraav <leho@…>, 4 years ago

http://i.imgur.com/RiJHn.gif

comment:13 by Jun Omae, 4 years ago

Release Notes: modified (diff)

Functional tests use [trac] use_chunked_encoding = disabled because internal error while rendering templates cannot be handled.

comment:14 by Ryan J Ollos, 4 years ago

While working on #12046, I encountered this error:

======================================================================
ERROR: test_rss_conversion (trac.ticket.tests.conversion.TicketConversionTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/ticket/tests/conversion.py", line 144, in test_rss_conversion
    self.req, 'trac.ticket.Ticket', ticket, 'rss')
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/api.py", line 725, in convert_content
    content = ''.join(content)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/web/chrome.py", line 1217, in iterable_content
    location = '%s (unknown template location)' % filename
NameError: global name 'filename' is not defined

It looks like filename isn't initialized: browser:tags/trac-1.0.6/trac/web/chrome.py@:1180#L1158.

comment:15 by Jun Omae, 4 years ago

Thanks for the reporting. Oh, I'm stupid….

The patch removes the filename variable bacause the file name of template is logged by exception_to_unicode(e, traceback=True). I'll apply this later.

  • trac/web/chrome.py

    diff --git a/trac/web/chrome.py b/trac/web/chrome.py
    index decf3c0d5..195935560 100644
    a b class Chrome(Component):  
    11801180            if pos:
    11811181                location = "'%s', line %s, char %s" % pos
    11821182            else:
    1183                 location = '%s (unknown template location)' % filename
     1183                location = '(unknown template location)'
    11841184            self.log.error('Genshi %s error while rendering template %s%s',
    11851185                           e.__class__.__name__, location,
    11861186                           exception_to_unicode(e, traceback=True))

comment:16 by Jun Omae, 4 years ago

Committed in [14132:14133].

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted. Next status will be 'reopened'.
to as closed The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.