Changes between Version 24 and Version 25 of TracDev/Proposals/Jinja
- Timestamp:
- Mar 20, 2016, 1:15:06 PM (8 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
TracDev/Proposals/Jinja
v24 v25 1 [[PageOutline(2-3)]] 1 [[PageOutline(2-3,Contents)]] 2 2 3 = Switch to the Jinja2 Template Engine 3 4 4 We've decided some time ago to remove the legacy support for the ClearSilver template engine , for Trac 1.0 (r10570). Clearsilver had its share of inconveniences, enough that we decided to switch to the nicer [http://genshi.edgewall.org/ Genshi template engine] in 0.11, but to be honest ClearSilver was **very** fast and memory lenient. While we managed to keep Genshi memory usage somewhat in control (remember #6614?), the speed was never really adequate, especially for big changesets and for displaying source files over a few thousand lines of code (see TracDev/Performance#Genshi for details).5 We've decided some time ago to remove the legacy support for the ClearSilver template engine for Trac 1.0 (r10570). Clearsilver had its share of inconveniences, enough that we decided to switch to the nicer [http://genshi.edgewall.org/ Genshi template engine] in 0.11, but ClearSilver was **very** fast and memory lenient. While we managed to keep Genshi memory usage somewhat in control (#6614), the speed was never really adequate, especially for big changesets and for displaying source files over a few thousand lines of code (see TracDev/Performance#Genshi for details). 5 6 6 So one solution would be to switch once again, to a template engine that would combine the advantages of Genshi (pure Python, nice templates, flexible) and ClearSilver (speed!). Such a beast seems to exist now: **[http://jinja.pocoo.org/2/documentation/ Jinja2]**. 7 So one solution would be to switch template engine once again, to one that would combine the advantages of Genshi (pure Python, nice templates, flexible) and ClearSilver (speed!). Such a beast seems to exist now: **[http://jinja.pocoo.org/2/documentation/ Jinja2]**. 8 9 == Overview of activities and progress 7 10 8 11 There's an experimental branch which supports this proposal: [source:cboos.git@jinja2] \\ … … 12 15 13 16 [=#Status Status] of the branch (2016-03-17): 14 - ported **63%** of the Genshi templates, 2 unit-tests failures (with mysql and psql), 8 functional tests failures 15 - clarify upgrade path for plugins that came to rely on `ITemplateStreamFilter`s? 16 (127/898 plugins (14.1%) on trac-hacks.org use `filter_stream()`) 17 DONE -> see [PortingFromGenshiToJinja#ReplacingITemplateStreamFilter replacing ITemplateStreamFilter] 18 * clarify how to handle themeing? DONE -> see HtmlTemplates#Jinjaarchitecture 19 * rewrite tag builders ~~or use lightweight string templates~~? DONE -> [PortingFromGenshiToJinja#tag tag] `Fragment`/`Element` builder API has been reimplemented 20 - accesskey support TODO 21 - hack `ITemplateStreamFilter` support for Jinja2 templates TODO 22 - site.html replacement (e.g. try to reproduce t.e.o customizations) TODO 17 - ported **63%** of the Genshi templates, 2 unit-tests failures (with MySQL and PgSql), 8 functional tests failures 18 - DONE clarify upgrade path for plugins that came to rely on `ITemplateStreamFilter`s? 127/898 plugins (14.1%) on trac-hacks.org use `filter_stream()` -> see [PortingFromGenshiToJinja#ReplacingITemplateStreamFilter replacing ITemplateStreamFilter] 19 * DONE clarify how to handle themeing? -> see HtmlTemplates#Jinjaarchitecture 20 * DONE rewrite tag builders ~~or use lightweight string templates~~? -> [PortingFromGenshiToJinja#tag tag] `Fragment`/`Element` builder API has been reimplemented 21 - TODO accesskey support 22 - TODO hack `ITemplateStreamFilter` support for Jinja2 templates 23 - TODO site.html replacement, for example try to reproduce t.e.o customizations 23 24 * others? 24 25 25 See also [googlegroups:trac-dev:fc8d8c0447140110 this Trac-Dev discussion] from 2010, which is still pertinent. W ell, obviously we managed to release Genshi 0.6 since then, butthe issue is a recurring one, see this recent (2016-01)26 See also [googlegroups:trac-dev:fc8d8c0447140110 this Trac-Dev discussion] from 2010, which is still pertinent. While we managed to release Genshi 0.6 since then, the issue is a recurring one, see this recent (2016-01) 26 27 [gmessage:trac-users:PYqQ4UDRnl8/wg8lQzrGDAAJ Genshi question] on Trac-Users. 27 28 28 The topic is now (Feb / March 2016) again discussed on [gdiscussion:trac-dev:KqWPQWuZ63k 29 The topic is now (Feb / March 2016) again discussed on [gdiscussion:trac-dev:KqWPQWuZ63k trac-dev]. 29 30 31 The Jinja2 template engine has the peripheral benefit of being used by the Django community: [gdiscussion:trac-dev/KqWPQWuZ63k/GPfda0_PDgAJ]. 30 32 31 33 == Experimenting with Jinja2 (2.8) 32 34 33 Nothing like a few numbers to make a point ;-)35 Nothing like a few numbers to make a point. 34 36 35 37 These are the timings for rendering !r3871 (//don't try this one here, please//), with the diff options set to side-by-side, in place modifications, served by tracd on my development laptop. This generates a page weighing from 11.5MB (Genshi) to 10.3MB (Jinja2) in size. … … 50 52 - Jinja2 (2.8 with [http://www.pocoo.org/projects/markupsafe/ speedups]) 51 53 - ''generate'' means we use `Template.generate` and send chunks as we have them 52 - ''stream'' means we use the `TemplateBuffer` wrapper on the above, which groups a few chunks (given by the number in parenthesis) together before we send them; 53 for a chunk size of **100**, we get the best compromise: still a very low TTFB and a reduced Content download time; actually the sweet spot is probably between 54 10 and 100, and will most certainly depend on the actual content (I just tested 75 which gives 1160/2430 for example) 54 - ''stream'' means we use the `TemplateBuffer` wrapper on the above, which groups a few chunks (given by the number in parentheses) together before we send them; 55 for a chunk size of **100**, we get the best compromise: still a very low TTFB and a reduced Content download time; actually the sweet spot is probably between 10 and 100, and will most certainly depend on the actual content (I just tested 75 which gives 1160/2430 for example) 55 56 - ''blob'' means we first generate all the content in memory with `Template.render` 56 57 - both: … … 62 63 - CD (Content download), idem 63 64 - Rdr (template rendering time), mostly significant for the "blob" method otherwise it also takes the network latency into account 65 64 66 All values are given in milliseconds. 65 67 66 Note that even if the total "blob" time seems better than the total "stream" one, the lower TTFB is nevertheless a major benefit for the streaming variant, as this means the secondary requests can start earlier (and in this case, finish before the main request).68 Note that even if the total "blob" time seems better than the total "stream" one, the lower TTFB is nevertheless a major benefit for the streaming variant, as this means the secondary requests can start earlier. In this case, it can finish before the main request. 67 69 68 70 In addition, while I didn't measure precisely the memory usage, Genshi made the python.exe process jump from 109MB to 239MB while rendering the request (blob). The memory seems to be freed afterwards (there were no concurrent requests). By contrast, with Jinja2 the memory spike was 106MB to 126MB. 69 71 70 72 In another experiment, I used the [pypi:memory_profiler] on Windows, which provided the following results when rendering a big changeset, for a rendered page weighing 4.48MB (it was a side-by-side diff): 73 71 74 {{{ 72 75 Genshi xhtml: 293.612 total (load=0.017, generate=0.017, filter=0.000, render=293.578) … … 88 91 1294 158.1 MiB 0.0 MiB return gs 89 92 }}} 90 91 93 92 94 vs. … … 114 116 For smaller pages, the speed-up is between 5x to 10x as well. 115 117 116 117 118 == Genshi to Jinja2 Migration 118 119 … … 122 123 123 124 [=#jinjachecker] 124 In order to facilitate the creation of error-free Jinja2 templates for HTML (or XML), we also wrote a little utility called [PortingFromGenshiToJinja/Checker jinjachecker], which will help troubleshoot the most common nesting problems. 125 126 To facilitate the creation of error-free Jinja2 templates for HTML (or XML), we also wrote a utility called [PortingFromGenshiToJinja/Checker jinjachecker], which helps troubleshoot the most common nesting problems.
