[[PageOutline(2-3)]]
= Add Support for the Jinja2 Template Engine

We've decided some time ago to remove the legacy support for the ClearSilver template engine, for Trac 1.0 (r10570). Clearsilver had its share of inconveniences, enough that we decided to switch to the nicer [http://genshi.edgewall.org/ Genshi template engine] in 0.11, but to be honest ClearSilver was **very** fast and memory lenient. While we managed to keep Genshi memory usage somewhat in control (remember #6614?), the speed was never really adequate, especially for big changesets and for displaying source files over a few thousand lines of code (see TracDev/Performance#Genshi for details).

So one solution would be to switch once again, to a template engine that would combine the advantages of Genshi (pure Python, nice templates, flexible) and ClearSilver (speed!). Such a beast seems to exist now: **[http://jinja.pocoo.org/2/documentation/ Jinja2]**.


Several points remain to be clarified: 
 * what will be the upgrade path for plugins that came to rely on `ITemplateStreamFilter`s?
   - 127/898 plugins (14.1%) on trac-hacks.org use `filter_stream()`
   - -> [PortingFromGenshiToJinja#ReplacingITemplateStreamFilter replacing ITemplateStreamFilter]
 * how to handle themeing? -> see [#Themeing] below
 * should we rewrite tag builders or use lightweight string templates? -> [PortingFromGenshiToJinja#tag tag] `Fragment`/`Element` builder has been reimplemented
 * others?

See also [googlegroups:trac-dev:fc8d8c0447140110 this Trac-Dev discussion] from 2010, which is still pertinent. Well, obviously we managed to release Genshi 0.6 since then, but the issue is a recurring one, see this recent (2016-01)
[gmessage:trac-users:PYqQ4UDRnl8/wg8lQzrGDAAJ Genshi question] on Trac-Users.



== Experimenting with Jinja2 (2.8)

Nothing like a few numbers to make a point ;-)

These are the timings for rendering !r3871, with the diff options set to side-by-side, in place modifications, served by tracd on my development laptop. This generates a page weighing 11.5MB (Genshi) to 10.3MB (Jinja2) in size.

||       ||||||||= Genshi                             ||||||||||||||||||||||||= Jinja2                                                                                     ||
||       ||||= stream          ||||= blob             ||||= generate    ||||= stream (5) ||||= stream (10)     ||||= stream (100)   ||||= stream (1000)||||= blob          ||
||       ||= //1st//||= 2nd    ||= //1st// ||=   2nd  ||=//1st//||= 2nd ||=//1st//||= 2nd  ||=//1st//||= 2nd   ||=//1st//||= 2nd    ||=//1st//||= 2nd  ||=//1st//||=   2nd ||
||= TTFB ||//16600//||**15670**|| //25530//||    24460||//2020//||  1160||//2030//||   1160||//2070//||    1170||//2150//|| **1230**||//2280//||   1230||//3370//||    2450||
||= CD   ||//16090//||**16050**||   //387//||     1240||//2820//||  2720||//2730//||   2640||//2730//||    2680||//2470//|| **2390**||//2350//||   2250|| //488//||    1060||
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
||= Total||//32690//||**31720**|| //25917//||    25700||//4840//||  3880||//4760//||   3800||//4800//||    3850||//4620//|| **3620**||//4630//||   3480||//3850//||    3510||
||= Rdr  ||       --||       --|| //23533//||**23273**||      --||    --||      --||     --||      --||      --||      --||       --||      --||     --||//1477//||**1263**||

Some explanations:
 - Genshi (0.7 with speedups)
   - ''stream'' means we return content via `Stream.serialize` and send chunks as we have them
   - ''blob'' means we first generate all the content in memory with `Stream.render`, then send it at once
 - Jinja2 (2.8 with [http://www.pocoo.org/projects/markupsafe/ speedups])
   - ''generate'' means we use `Template.generate` and send chunks as we have them
   - ''stream'' means we use the `TemplateBuffer` wrapper on the above, which groups a few chunks (given by the number in parenthesis) together before we send them;
     for a chunk size of **100**, we get the best compromise: still a very low TTFB and a reduced Content download time; actually the sweet spot is probably between
     10 and 100, and will most certainly depend on the actual content (I just tested 75 which gives 1160/2430 for example)
   - ''blob'' means we first generate all the content in memory with `Template.render`
 - both:
   - ''1st'', is the time in ms for the first request, sent right after a server restart
   - 2nd, is the time in ms for the second request, sent just after the first (usually the 3rd and subsequent requests would show the same results as this 2nd request)

We measure:
 - TTFB (Time to first byte), as given by Chrome network panel in the developer window
 - CD (Content download), idem
 - Rdr (template rendering time), mostly significant for the "blob" method otherwise it also takes the network latency into account
All values are given in milliseconds.

Note that even if the total "blob" time seems better than the total "stream" one, the lower TTFB is nevertheless a major benefit for the streaming variant, as this means the secondary requests can start earlier (and in this case, finish before the main request).

In addition, while I didn't measure precisely the memory usage, Genshi made the python.exe process jump from 109MB to 239MB while rendering the request (blob). The memory seems to be freed afterwards (there were no concurrent requests). By contrast, with Jinja2 the memory spike was 106MB to 126MB.

In summary, this means that for the big problematic pages, we can easily have a 10x speedup and more, by migrating to Jinja2, and this with a much lighter memory footprint.
For smaller pages, the speed-up is between 5x to 10x as well.


== Implementation details

=== Themeing

We summarize the current template page architecture we use since the adoption of the Genshi template engine (i.e. since Trac 0.11), before describing the new but similar template page architecture that we'll use with the Jinja2 template engine.

==== Genshi theme
I've never really tried the TH:ThemeEnginePlugin plugin, or alternatives, so I can't be sure if I got it right, but from what I can see in Trac's code base itself, the idea with Genshi-based themeing (and page architecture in general) was to have a dynamically loaded "theme" template page that would primarily be in charge of the main structure of all HTML pages.

Let's take the example of a simple "end user" page, the search.html page.
What happens is that:
 - search.html includes
   - layout.html, which includes
     - $chrome.theme (typically "theme.html", the default theme page that ships with Trac)

In more details, the 
[source:tags/trac-1.0.9/trac/search/templates/search.html search.html] is structured like this:
{{{#!html+genshi
<html>
  <xi:include href="layout.html" />
  <head>
    <title>Search</title>
    ...
  </head>
  <body>
    <div id="content" class="search">
      <h1>Search</h1>
      ...
    </div>
  </body>
</html>
}}}
 - it starts by including the layout.html page (`<xi:include href="layout.html" />`)
 - it then provides the `<head>` and `<body>` elements specific to that search page; 
   these elements will be processed by the `<py:match>`es filters defined so far, 
   in the order in which they have been included

The [source:tags/trac-1.0.9/trac/templates/layout.html  layout.html] 
page in turn is structured like this:
{{{#!html+genshi
<html>
  <py:match path="head"><head>
    <title py:with="title = list(select('title/text()'))">
      ${title} – ${project.name or 'Trac'}  <!-- e.g.  "Search - Trac" -->
    </title>
    ...
    ${select("*[local-name() != 'title']|text()|comment()")}
  </head></py:match>

  <py:match path="body"><body>

    ${select('*|text()|comment()')}         <!-- e.g. "<h1>Search</h1>"
                                                       ... -->
    ...
    <div id="altlinks" py:if="'alternate' in chrome.links">
      ...
    </div>
  </body></py:match>

  <xi:include href="$chrome.theme"><xi:fallback /></xi:include>
</html>      
}}}
 - it defines `<py:match>` filters for transforming the content provided 
   in `<head>` and `<body>` elements in the including template (search.html);
   the head filter adds some content to the `<title>` and prepends some content
   in the <head>, the body filter appends some content in  the <body>
 - it then **dynamically** includes some "theme" page, `$chrome.theme`

By default, this theme page will be our
[source:tags/trac-1.0.9/trac/templates/theme.html theme.html] template:
{{{#!xml
<html>
  <body>
    <div id="banner">
      ...
    </div>
    <div id="main">
      ... 
      ${select('*|text()|comment()')}       <!-- e.g. "<h1>Search</h1>"
                                                       ... 
                                                       <div id="altlinks" ... </div> -->
    </div>
    <div id="footer">
      ...
    </div>
  </body>
</html>
}}}
 - it defines a `<py:match>` filter on the `<body>` tag 
   (the one that will be produced by the previously applied filters, 
   i.e. the output of the `<py:match>` from the layout.html)
   and it will **embed** that element into some predefined HTML
   structure (inside a div element), and 
   it will prepend and append other divs around it:

So this dynamic theme template has the last say, and can theoretically re-order 
the content generated by the previous filters any way it likes, although in practice
it simply inserts the body content produced by previous steps inside a predefined 
structure (the `<div id="main">`).

An example of another theme page: [https://github.com/chevah/trac-bootstrap-theme/blob/master/templates/theme.html trac-bootstrap-theme's theme.html].



Note that this scheme can be extended to additional intermediate levels, 
for example see what we do with the admin panels.

One such panel is
[source:tags/trac-1.0.9/trac/admin/templates/admin_basics.html admin_basics.html]:
{{{#!html+genshi
<html>
  <xi:include href="admin.html" />
  <head>
    <title>Basics</title>
  </head>

  <body>
    ...
  </body>
</html>
}}}
 - it starts by including the admin.html page
 - then it provides the `<head>` and `<body>` elements specific to that panel
    
The [source:tags/trac-1.0.9/trac/admin/templates/admin.html admin.html] page
is similar to the search.html page in that it includes the layout.html, but
it also first contains its own `<py:match>` templates to organize the content of
the admin panel which included it:
{{{#!html+genshi
<html>
  <py:match path="head" once="true"><head>
    <title>Administration: ${select('title/text()')}</title>
    ${select("*[local-name() != 'title']")}
  </head></py:match>

  <py:match path="body" once="true" buffer="false"><body>
    <div id="content" class="admin">
      <h1>Administration</h1>
      <div id="tabs">
      <div id="tabcontent">
        ${select("*|text()")}
        <br style="clear: right" />
      </div>
    </div>

  </body></py:match>

  <xi:include href="layout.html" />
</html>
}}}
 - defines `<py:match>` filters for `<head>` and `<body>` (of the including panel page), 
   which in turn will produce modified `<head>` and `<body>` elements
 - it then includes the layout.html page (see above)

Feel free to brush up your Genshi craft by reading ([G:GenshiTutorial#AddingaLayoutTemplate]),
as I just did ;-)


==== Jinja2 theme

Fortunately Jinja2 can do dynamic includes as well, or more precisely, dynamic ''extends''.
Therefore the same idea can be transposed: have the end user page extend the layout page, 
then have the layout page extend whatever has been defined to be the theme page.

The differences with Genshi are subtle: while in both case the control of the
output is delegated to the more generic page, with Jinja2 the parent only controls
what it puts around //blocks//. It can put some default content in these blocks,
but the end user page has the final say about what to do with this default content, 
as it can reuse it inside its block (by calling `super()`) or not.

Let's transpose the previous example of the search.html template.
For as long as we'll have both template engines coexisting, 
we'll prefix the new Jinja2 templates with a `j`.
So we're now discussing:
 - jsearch.html, which extends
   - jlayout.html, which extends
     - jtheme.html (as this is our default theme page)

In more details, we describe what happens with the
[source:cboos.git/trac/search/templates/jsearch.html jsearch.html] page:
{{{#!html+jinja
# extends 'jlayout.html'

<!DOCTYPE html>
<html>
  <head>
    <title>
    #   block title
    ${_("Search")}    ${ super() }
    #   endblock title
    </title>

    # block head
    ${ super() }
    ...
    # endblock head
  </head>

  <body>
    # block content
    <div id="content" class="search">
      <h1>${_("Search")}</h1>
      ...
    </div>
  </body>
</html>
}}}
 - it starts by //extending// the jlayout.html page (`# extends 'jlayout.html')
 - then it redefines the ''title'', ''head'' and ''content'' blocks, 
   and has to place a `${ super() }` expression in order to insert the default 
   content proposed by the extended template at the right place;
   note that the presence of the `<html>`, `<head>` and `<body>` tags here
   is strictly "decorative", it will be ignored in the final output. As we're
   in a template extending another, only what's in the redefined blocks matters.

These blocks are first defined in the extended template, jlayout.html.

The [source:cboos.git/trac/templates/jlayout.html jlayout.html] page looks like this:
{{{#!html+jinja
# extends ('j' + chrome.theme)

<!DOCTYPE html>
<html>
  <head>
    # block head

    <title>
    #  block title
    – ${project.name or 'Trac'}
    #  endblock title
    </title>

    ...
    # endblock head
  </head>

  <body>
    # block content
    # endblock content

    ...
    <div id="altlinks">
      ...
    </div>
  </body>
</html>
}}}
 - first **dynamically** //extends// in turn some "theme" page (`# extends ('j' + chrome.theme)`)
 - then the jlayout.html template defines a few blocks:
    - the ''head'' block and its ''title'' sub-block; here we understand why we've put
      the ''title'' block **outside** of the ''head'' block in the jsearch.html template:
      the jlayout.html's ''head'' block contains among other things a `<title>` element, 
      and we're reusing  that default content in the inheriting ''head'' block; if in 
      jsearch.html we had defined the <title> element in the ''head'' block as well,
      we would have had two of these <title> elements in that block
    - the ''content'' block which is filled with some predefined, generic content,
      mostly the same stuff that could be found in the corresponding layout.html,
      in `<py:match>` filters 

By default, this theme template will be our 
[source:cboos.git/trac/templates/jtheme.html jtheme.html] page:
{{{#!html+jinja

<!DOCTYPE html>
<html>
  <head>
    # block head
    # endblock head
  </head>

  <body>
    # block body
    <div id="banner">
      ...
    </div>
    <div id="main">
      ... 
      # block content
      (here goes the content of the content block produced by layout.html)
      # endblock content
    </div>
    <div id="footer">
      ...
    </div>
    # block body
  </body>
</html>
}}}
 - it defines a ''head'' block inside an otherwise empty `<head>` element;
   this means this is simply a "slot" that will be filled by the content
   of the `head` block in the extending templates (in this case, jlayout.html)
 - it contains a `<body>` element; 
   as we want to replicate what the original theme.html did,
   what we want to achieve here is to provide a ''slot''
   at the place where we want to insert the content
   produced by the jlayout.html template;
   I didn't name that inner block "body", as this could be confusing:
   we're not in control of the `<body>` element there, just of
   a fraction of it, the bottom part of the main div.

Note that //if// we wanted to be in full control of the body in the extending
template, we could (as opposed to what you can do in Genshi): we would simply 
have to redefine the ''body'' block which contains all of the default structure
(possibly reusing the content of that block by a call to `${ super() }`). 
In our case, neither jsearch.html nor jlayout.html redefine the `body` block, 
as they're happy with what jtheme does with it.


Depending how one looks at it, it seems this approach is even more flexible than
what we had in Genshi, as the end user template can decide which bits of the
parent template it wants or not ("bottom-up" control), something that was not 
readily doable with Genshi ("top-down" control).


Like what we did with Genshi, this scheme can be extended to support additional intermediate levels. We did that for the admin and the preference panels.

Let's take for example the Logging admin panel, [source:cboos.git/trac/admin/templates/jadmin_logging.html jadmin_logging.html]:
{{{#!html+jinja
# extends "jadmin.html"
<html>
  <head>
    <title>
      # block admintitle
      Logging
      # endblock admintitle
    </title>
  </head>

  <body>
    # block adminpanel
    ...
    # block adminpanel
  </body>
</html>
}}}
 - it starts by extending the jadmin.html page
 - then it provides the `<head>` and `<body>` elements specific to that panel,
   more precisely the //admintitle// and //adminpanel// blocks (if some JavaScript
   or other resources are needed, the //head// could be redefined as well)
    
The [source:cboos.git/trac/admin/templates/jadmin.html@jinja2 jadmin.html] page
is similar to the jsearch.html page in that it extends the jlayout.html:
{{{#!html+jinja
# extends "jlayout.html"
<html>
  <head>
    <title>
      # block title
      Administration:
      #   block admintitle
      #   endblock admintitle 
      ${ super() }
      # endblock
    </title>
  </head>

  <body>
    # block content
    <div id="content" class="admin">
      <h1>Administration</h1>
      <div id="tabs">
      <div id="tabcontent">
        # block adminpanel
        # endblock adminpanel
        <br style="clear: right" />
      </div>
    </div>
    # endblock content
  </body>
</html>
}}}

//See [1df4e05c/cboos.git] for the full conversion of admin.html -> jadmin.html and admin_logging.html -> jadmin_logging.html.//


I omitted the discussion of the 
replacement for the `<xi:include href="site.html>` template 
(`# include site_head.html` and `# include site_body.html`).


== Genshi to Jinja2 Migration

Some systematic comparison of the Genshi and Jinja2 template syntax can be seen in PortingFromGenshiToJinja#Changesinthetemplatesyntax (modeled after the old PortingFromClearSilverToGenshi page). 

=== Conversion example

Here we detail the conversion process of a sample Genshi template ([source:tags/trac-1.0.9/trac/wiki/templates/wiki_view.html wiki_view.html]) into the corresponding Jinja2 template ([source:cboos.git/trac/wiki/templates/jwiki_view.html jwiki_view.html]).

As will become apparent, we mainly use [http://jinja.pocoo.org/docs/dev/templates/#line-statements line statements] for the Jinja2 control structures.
Note that we have reformatted the Genshi template to use shorter line widths, so that both columns of the table below stay visible (a wide screen also helps).

{{{#!th style="max-width: 200px"
wiki_view.html (Genshi template) 
}}}
{{{#!th style="max-width: 200px"
jwiki_view.html (Jinja2 template) 
}}}
|------
||= ||= ||
{{{#!td
{{{#!html+genshi
<!--!  Copyright (C) 2006-2014 Edgewall Software

  This software is licensed as described in the file COPYING, which
  you should have received as part of this distribution. The terms
  are also available at http://trac.edgewall.com/license.html.

  This software consists of voluntary contributions made by many
  individuals. For the exact contribution history, see the revision
  history and logs, available at http://trac.edgewall.org/.
-->
}}}
}}}
{{{#!td
{{{#!html+jinja
{# Copyright (C) 2006-2014 Edgewall Software

  This software is licensed as described in the file COPYING, which
  you should have received as part of this distribution. The terms
  are also available at http://trac.edgewall.com/license.html.

  This software consists of voluntary contributions made by many
  individuals. For the exact contribution history, see the revision
  history and logs, available at http://trac.edgewall.org/.
#}
}}}
}}}
|-----------------------------------------------------------------------------
|||| Block comments ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
}}}
}}}
{{{#!td
{{{#!html+jinja
# extends "jlayout.html"
}}}
}}}
|-----------------------------------------------------------------------------
|||| This is the equivalent of Genshi's `<py:include href="layout.html" />`. We have to do the extends before outputing any content on our own, otherwise it would show up in the final result. Once we made the `extends`, only the content written within //blocks// will matter. ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
<!DOCTYPE html
    PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
}}}
}}}
{{{#!td
{{{#!html+jinja
<!DOCTYPE html>
}}}
}}}
|-----------------------------------------------------------------------------
|||| Let's directly make the jump to HTML5, while we're at it (I'm not sure what is the current state of the HTML5 support in Genshi, but it no longer matters) ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/"
      xmlns:i18n="http://genshi.edgewall.org/i18n"
      xmlns:xi="http://www.w3.org/2001/XInclude"
      py:with="modify_perm = 'WIKI_MODIFY' in perm(page.resource);
               create_perm = 'WIKI_CREATE' in perm(page.resource);
               admin_perm = 'WIKI_ADMIN' in perm(page.resource);
               is_not_latest = page.exists and page.version != latest_version">
}}}
}}}
{{{#!td
{{{#!html+jinja
<html>

}}}
}}}
|-----------------------------------------------------------------------------
{{{#!td
No need for exotic namespace declarations. 

Note that we can't use a `with` declaration that would encompass the ''head'' and ''content'' blocks we'll define in a moment, as this would force the generation of content twice, once by having the with block present in the template, and a second time because of the extends and the call of the blocks.
}}}
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
  <xi:include href="layout.html" />
}}}
}}}
{{{#!td
{{{#!html+jinja
}}}
}}}
|-----------------------------------------------------------------------------
|||| See above, this will be converted to an initial `extends` statement. ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
  <head>
    <title py:if="title">$title</title>
}}}
}}}
{{{#!td
{{{#!html+jinja
  <head>
    <title>
      # block title
      #   if title:
      ${title} ${ super() }
      #   endif
      # endblock title
    </title>

    # block head
    #  set modify_perm = 'WIKI_MODIFY' in perm(page.resource)
    #  set is_not_latest = page.exists and page.version != latest_version

    ${ super() }
}}}
}}}
|-----------------------------------------------------------------------------
{{{#!td colspan=2
The content of the <title> element is placed in the ''title'' block,
followed by the content of the parent ''title'' block (`${ super() }`).
When closing the block, we reuse the name of the block.

Note that while the only thing that really matters in an extending template 
is the content of the blocks, we try hard to keep a correct HTML structure
for the whole template page, so that its HTML content can be validated in a
standalone way (cf. [#jinjachecker]). This is why we add the <html>, <head> and
<title> tags.

We explained earlier why the title block has to be outside of the ''head'' block:
to avoid appearing twice, once as part of the head block, and the second time
when we call `${ super() }` in order to retrieve the content of the parent ''head''
block.

Finally, we define some variables at the beginning of the block (corresponding to
some which were part of the `py:with` attribute in the `<html>` tag, in the Genshi
template).
}}}
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
    <meta py:if="version or page.author == 'trac'" 
          name="ROBOTS" content="NOINDEX, NOFOLLOW" />
    <link py:if="modify_perm" rel="alternate" 
          type="application/x-wiki"
          href="${href.wiki(page.name, action='edit', 
                  version=page.version if is_not_latest else None)}"
          title="${_('Revert page to this version') if 
                     is_not_latest else _('Edit this page')}"/>
}}}
}}}
{{{#!td
{{{#!html+jinja
    #   if version or page.author == 'trac':
    <meta name="ROBOTS" content="NOINDEX, NOFOLLOW" />
    #   endif
    #   if modify_perm:
    <link rel="alternate" type="application/x-wiki"
          href="${href.wiki(page.name, action='edit',
                version=page.version if is_not_latest)}"
          title="${_("Revert page to this version") if is_not_latest else
                                                  _("Edit this page")}"/>
    #   endif
}}}
}}}
|-----------------------------------------------------------------------------
|||| With Jinja2, the logic is now clearly distinct from the content. ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
    <script type="text/javascript">
      jQuery(document).ready(function($) {
        $("#content").find("h1,h2,h3,h4,h5,h6")
          .addAnchor(_("Link to this section"));
        $("#content").find(".wikianchor").each(function() {
          $(this).addAnchor(babel.format(_("Link to #%(id)s"), 
                                         {id: $(this).attr('id')}));
        });
        $(".foldable").enableFolding(true, true);
      });
    </script>
}}}
}}}
{{{#!td
{{{#!html+jinja
    <script type="text/javascript">
      jQuery(document).ready(function($) {
        $("#content").find("h1,h2,h3,h4,h5,h6")
          .addAnchor(_("Link to this section"));
        $("#content").find(".wikianchor").each(function() {
        $(this).addAnchor(babel.format(_("Link to #%(id)s"), {
                                       id: $(this).attr('id')}));
        });
        $(".foldable").enableFolding(true, true);
      });
    </script>
}}}
}}}
|-----------------------------------------------------------------------------
|||| No changes here, except for some reformatting (we try to stay below 80 chars for nicer future side-by-side diffs). ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
  </head>

}}}
}}}
{{{#!td
{{{#!html+jinja
    # endblock head
  </head>

}}}
}}}
|-----------------------------------------------------------------------------
|||| We close the ''head'' block. ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
  <body>
    <div id="content" class="${classes('wiki', create=not page.exists)}">

}}}
}}}
{{{#!td
{{{#!html+jinja
  <body>
    # block content
    #   set modify_perm = 'WIKI_MODIFY' in perm(page.resource)
    #   set create_perm = 'WIKI_CREATE' in perm(page.resource)
    #   set admin_perm = 'WIKI_ADMIN' in perm(page.resource)
    #   set is_not_latest = page.exists and page.version != latest_version

    <div id="content" class="${classes('wiki', create=not page.exists)}">

}}}
}}}
|-----------------------------------------------------------------------------
|||| We start the ''content'' block. As before with the ''header'' block, we define some variables at the beginning of the block. ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
      <py:if test="version">
        <br />
        <table id="info" summary="Revision info">
          <tr><th scope="row" i18n:msg="version, author, date">
             Version $page.version (modified by ${authorinfo(page.author)},
                                            ${pretty_dateinfo(page.time)})
             (<a href="${href.wiki(page.name, action='diff', 
                                   version=page.version)}">diff</a>)
          </th></tr>
          <tr><td class="message" xml:space="preserve">
            ${wiki_to_html(context, page.comment or '--')}
          </td></tr>
        </table>
      </py:if>
}}}
}}}
{{{#!td
{{{#!html+jinja
      # if version:
      <br />
      <table id="info" summary="${_("Revision info")}">
        <tr><th scope="row">
            # with
            #   set version = page.version
            #   set author = authorinfo(page.author)
            #   set date = pretty_dateinfo(page.time)
            #   set hef = href.wiki(page.name, action='diff', version=page.version)
            #   trans version, author, date, href

            Version ${version} (modified by ${author}, ${date})
            (<a href="${href}">diff</a>)

            #   endtrans
            # endwith
        </th></tr>
        <tr><td class="message">
            ${wiki_to_html(context, page.comment or '--')}
        </td></tr>
      </table>
      # endif

}}}
}}}
|-----------------------------------------------------------------------------
{{{#!td colspan=2
Here we illustrate the i18n changes.

First, any sentence that corresponds to visible end-user text that should be
translated has to be marked somehow. 

One way is to use the standard i18n calls,
`_`, `ngettext()`, etc. 

The other way is to use `trans` blocks.
There are two big differences with their Genshi i18n equivalent:
 - variable substitutions can only be those of direct variables,
   no kind of expression is allowed, even as simple as attribute 
   lookup
 - Jinja2, markup neutral as it is, will not do any substitutions
   on the markup found in a trans block; what would have ended
   in a Genshi bracketed expression `... [1:diff]` in the catalogs
   will now remain HTML markup: `... <a href="${href}">diff</a>`.
Note that one can use a `with` statement for breaking up the
assignments needed on multiple separate lines.
Smaller lists of variables can be placed on the `trans` line directly.
}}}
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
      <div class="wikipage searchable" py:choose="" xml:space="preserve">
        <py:when test="page.exists">
          <div id="wikipage" class="trac-content"
               py:content="wiki_to_html(context, text)" />
          <?python
            last_modification = (page.comment and
                 _('Version %(version)s by %(author)s: %(comment)s',
                   version=page.version, author=format_author(page.author),
                   comment=page.comment) or
                 _('Version %(version)s by %(author)s',
                   version=page.version, author=format_author(page.author)))
          ?>
          <div py:if="not version" class="trac-modifiedby">
            <span i18n:msg="reldate">
              <a href="${href.wiki(page.name, action='diff', 
                                   version=page.version)}"
                 title="$last_modification">Last modified</a>
                 ${pretty_dateinfo(page.time)}
            </span>
            <span class="trac-print" i18n:msg="date">Last modified on
              ${format_datetime(page.time)}</span>
          </div>
        </py:when>
        <py:otherwise>
          <p i18n:msg="name">The page <strong>${name_of(page.resource)}</strong>
          does not exist. You can create it here.</p>
        </py:otherwise>
      </div>
}}}
}}}
{{{#!td
{{{#!html+jinja
      <div class="wikipage searchable">
        # if page.exists:
        <div id="wikipage" class="trac-content">${
          wiki_to_html(context, text)
          }</div>
        #   set last_modification = (page.comment and
        _('Version %(version)s by %(author)s: %(comment)s',
        version=page.version, author=format_author(page.author),
        comment=page.comment) or
        _('Version %(version)s by %(author)s',
        version=page.version, author=format_author(page.author)))
        #   if not version:
        <div class="trac-modifiedby">
          <span>
            # with
            #   set href = href.wiki(page.name, action='diff',
                                     version=page.version),
            #   set date = pretty_dateinfo(page.time)
            #   trans href, last_modification, date

            <a href="${href}"
               title="${last_modification}">Last modified</a> ${date}

            #   endtrans
            # endwith
          </span>
          <span class="trac-print">
            ${_("Last modified on %(date)s", date=format_datetime(page.time))}
          </span>
        </div>
        #   endif
        # else:
        <p>
          # trans name = name_of(page.resource)

          The page <strong>${name}</strong> does not exist.
          You can create it here.

          # endtrans
        </p>
        # endif
      </div>
}}}
}}}
|-----------------------------------------------------------------------------
|||| ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
      <xi:include href="list_of_attachments.html"
                  py:with="alist = attachments; compact = True; 
                           foldable = True"/>
}}}
}}}
{{{#!td
{{{#!html+jinja
      # with
      #   set alist = attachments
      #   set compact = True
      #   set foldable = True
      #   include "jlist_of_attachments.html"
      # endwith
}}}
}}}
|-----------------------------------------------------------------------------
|||| Jinja2 includes also know about their context, so that make them kind of parametric. ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
      <py:with vars="delete_perm = 'WIKI_DELETE' in perm(page.resource);
                     rename_perm = 'WIKI_RENAME' in perm(page.resource)">
        <py:if test="modify_perm or create_perm or delete_perm">
          <div class="buttons">
            <py:if test="modify_perm or create_perm">
}}}
}}}
{{{#!td
{{{#!html+jinja
      # with
      #   set delete_perm = 'WIKI_DELETE' in perm(page.resource)
      #   set rename_perm = 'WIKI_RENAME' in perm(page.resource)
      #   if modify_perm or create_perm or delete_perm:
      <div class="buttons">
        #   if modify_perm or create_perm:
}}}
}}}
|-----------------------------------------------------------------------------
|||| Again, even when there's a lot of control lines, it's a bit easier to immediately spot the actual content in Jinja2 templates, due to the clearer syntactic difference. ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
              <form method="get" action="${href.wiki(page.name)}" id="modifypage">
                <div>
                  <input type="hidden" name="action" value="edit" />
                  <py:choose>
                    <py:when test="is_not_latest and modify_perm">
                      <input type="hidden" name="version" value="${page.version}"/>
                      <input type="submit" value="${_('Revert to this version')}"/>
                    </py:when>
                    <py:when test="page.exists and modify_perm">
                      <input type="submit" value="${_('Edit this page')}" 
                             accesskey="e" />
                    </py:when>
                    <py:when test="not page.exists and create_perm">
                      <input type="submit" value="${_('Create this page')}" 
                             accesskey="e" />
                      <div py:if="templates" id="template">
                        <label for="template">using the template:</label>
                        <select name="template">
                          <option selected="${not default_template in templates
                                              or None}"
                                  value="">(blank page)</option>
                          <option py:for="t in sorted(templates)" value="$t"
                                  selected="${t == default_template or None}"
                                  >$t</option>
                        </select>
                      </div>
                    </py:when>
                  </py:choose>
                </div>
              </form>
}}}
}}}
{{{#!td
{{{#!html+jinja
        <form method="get" action="${href.wiki(page.name)}" id="modifypage">
          <div>
            <input type="hidden" name="action" value="edit" />
            # if is_not_latest and modify_perm:
            <input type="hidden" name="version" value="${page.version}"/>
            <input type="submit" value="${_('Revert to this version')}"/>
            # elif page.exists and modify_perm:
            <input type="submit" value="${_('Edit this page')}"
                   accesskey="e" />
            # elif not page.exists and create_perm:
            <input type="submit" value="${_('Create this page')}"
                   accesskey="e" />
            #   if templates:
            <div id="template">
              <label for="template">${_("using the template:")}</label>
              <select name="template">
                <option ${{'selected': not default_template in templates
                        }|htmlattr}
                        value="">${_("(blank page)")}</option>
                # for t in sorted(templates):
                <option value="${t}"
                        ${{'selected': t == default_template
                        }|htmlattr}>${t}</option>
                # endfor
              </select>
            </div>
            #   endif
            # endif
          </div>
        </form>

}}}
}}}
|-----------------------------------------------------------------------------
{{{#!td
A `<py:choose>` and its series of `<py:when>` translates very smoothly into a sequence of `if / elif` statements.

Special care should be taken when producing attributes dynamically, as it's the
case here for `selected`. We use the `htmlattr` filter that takes care of
producing the right value for the attribute depending on its content: with 
a truth value, we'll output `selected="selected"` otherwise we'll just omit
the parameter.
}}}
|-----------------------------------------------------------------------------
|||| ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
        <py:if test="page.exists">
          <xi:include href="attach_file_form.html"
                      py:with="alist = attachments"/>
        </py:if>
      </py:if>
}}}
}}}
{{{#!td
{{{#!html+jinja
        #   if page.exists:
        #     with alist = attachments
        #       include "jattach_file_form.html"
        #     endwith
        #   endif
        # endif
}}}
}}}
|-----------------------------------------------------------------------------
|||| We pass `attachments` as `alist` to the included template (a `j...` template, obviously). ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
            <form method="get" action="${href.wiki(page.name)}" 
                  id="rename" py:if="page.exists and rename_perm">
              <div>
                <input type="hidden" name="action" value="rename" />
                <input type="submit" value="${_('Rename page')}" />
              </div>
            </form>
            <form method="get" action="${href.wiki(page.name)}" 
                  id="delete" py:if="page.exists and delete_perm">
              <div>
                <input type="hidden" name="action" value="delete" />
                <input type="hidden" name="version" value="$page.version" />
                <py:if test="page.version == latest_version">
                  <input type="submit" name="delete_version" 
                         value="${_('Delete this version')}" />
                </py:if>
                <input type="submit" value="${_('Delete page')}" />
              </div>
            </form>
          </div>
        </py:if>
      </py:with>
}}}
}}}
{{{#!td
{{{#!html+jinja
        # if page.exists and rename_perm:
        <form method="get" action="${href.wiki(page.name)}" id="rename">
          <div>
            <input type="hidden" name="action" value="rename" />
            <input type="submit" value="${_('Rename page')}" />
          </div>
        </form>
        # endif
        # if page.exists and delete_perm:
        <form method="get" action="${href.wiki(page.name)}" id="delete">
          <div>
            <input type="hidden" name="action" value="delete" />
            <input type="hidden" name="version" value="${page.version}" />
            # if page.version == latest_version:
            <input type="submit" name="delete_version" value="${_('Delete this version')}" />
            # endif
            <input type="submit" value="${_('Delete page')}" />
          </div>
        </form>
        # endif
      </div>
      # endif
      # endwith

}}}
}}}
|-----------------------------------------------------------------------------
|||| ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
      <div class="wikipage searchable" py:if="not page.exists and higher">
        <p>You could also create the same page higher in the hierarchy:</p>
        <ul>
          <li py:for="markup in higher">${markup}</li>
        </ul>
      </div>
}}}
}}}
{{{#!td
{{{#!html+jinja
      # if not page.exists and higher:
      <div class="wikipage searchable">
        <p>You could also create the same page higher in the hierarchy:</p>
        <ul>
          # for markup in higher:
          <li>${markup}</li>
          # endfor
        </ul>
      </div>
      # endif

}}}
}}}
|-----------------------------------------------------------------------------
|||| Here's the first use of the `for` statement. Straightforward. ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
      <div class="wikipage searchable" py:if="not page.exists and related">
        <p>The following pages have a name similar to this page,
           and may be related:</p>
        <ul>
          <li py:for="markup in related">${markup}</li>
        </ul>
      </div>
}}}
}}}
{{{#!td
{{{#!html+jinja
      # if not page.exists and related:
      <div class="wikipage searchable">
        <p>${_("The following pages have a name similar to this page, and may be related:")}</p>
        <ul>
          # for markup in related:
          <li>${markup}</li>
          # endif
        </ul>
      </div>
      # endif
}}}
}}}
|-----------------------------------------------------------------------------
|||| ||
|-----------------------------------------------------------------------------
{{{#!td
{{{#!html+genshi
    </div>
  </body>
</html>
}}}
}}}
{{{#!td
{{{#!html+jinja
    </div>

    ${ super() }

    # endblock content
  </body>
</html>
}}}
}}}
|-----------------------------------------------------------------------------
|||| Once we're done with the `<div id="content">`, we also call `${ super() }` to re-insert the parent content of the ''content'' block, i.e. the altlinks div and the late_links/late_scripts logic. ||
|-----------------------------------------------------------------------------

=== `jinjachecker`

Jinja2 is very helpful when it detects any kind of error, as you always end up with a meaningful backtrace. Nevertheless, it can be tedious to get the nesting of control structures right.

To help with that, there's the [source:cboos.git/contrib/jinjachecker.py contrib/jinjachecker.py] tool.

{{{
python contrib/jinjachecker.py trac/wiki/templates/jwiki_view.html
}}}

It first analyzes the Jinja2 control structures, and tries to provide some helpful diagnostics in case of nesting or stylistic errors.
It also adds curly braces to the statements, so if you have an editor which can do matching of brace pairs, you can quickly spot the origin o a nesting problem.

Finally, while the indentation of the statements is free in Jinja2 templates, being consistent with it also helps to ensure a proper nesting.
{{{
# -- Jinja2 check for 'trac/wiki/templates/jwiki_view.html'
   12    EXTENDS "jlayout.html"
   18          {BLOCK title
   19            {IF title:
   21            }IF
   22          }BLOCK title
   25        {BLOCK head
   26          SET modify_perm = 'WIKI_MODIFY' in perm(page.resource)
   27          SET is_not_latest = page.exists and page.version != latest_version
   31          {IF version or page.author == 'trac':
   33          }IF
   34          {IF modify_perm:
   40          }IF
   52        }BLOCK head
   56        {BLOCK content
   57          SET modify_perm = 'WIKI_MODIFY' in perm(page.resource)
   58          SET create_perm = 'WIKI_CREATE' in perm(page.resource)
   59          SET admin_perm = 'WIKI_ADMIN' in perm(page.resource)
   60          SET is_not_latest = page.exists and page.version != latest_version
   63          {IF version:
   67                {WITH
   68                  SET version = page.version
   69                  SET author = authorinfo(page.author)
   70                  SET date = pretty_dateinfo(page.time)
   71                  SET hef = href.wiki(page.name, action='diff', version=page.version)
   72                  {TRANS version, author, date, href
   77                  }TRANS
   78                }WITH
   84          }IF
   87            {IF page.exists:
   91              SET last_modification = (page.comment and
   97              {IF not version:
  100                {WITH
  101                  SET href = href.wiki(page.name, action='diff',
  103                  SET date = pretty_dateinfo(page.time)
  104                  {TRANS href, last_modification, date
  109                  }TRANS
  110                }WITH
  116              }IF
  117            ELSE:
  119              {TRANS name = name_of(page.resource)
  124              }TRANS
  126            }IF
  129          {WITH
  130            SET alist = attachments
  131            SET compact = True
  132            SET foldable = True
  133            INCLUDE "jlist_of_attachments.html"
  134          }WITH
  136          {WITH
  137            SET delete_perm = 'WIKI_DELETE' in perm(page.resource)
  138            SET rename_perm = 'WIKI_RENAME' in perm(page.resource)
  139            {IF modify_perm or create_perm or delete_perm:
  141              {IF modify_perm or create_perm:
  145                {IF is_not_latest and modify_perm:
  148                ELIF page.exists and modify_perm:
  151                ELIF not page.exists and create_perm:
  154                  {IF templates:
  161                    {FOR t in sorted(templates):
  165                    }FOR
  168                  }IF
  169                }IF
  173              {IF page.exists:
  174                {WITH alist = attachments
  175                  INCLUDE "jattach_file_form.html"
  176                }WITH
  177              }IF
  178            }IF
  180            {IF page.exists and rename_perm:
  187            }IF
  188            {IF page.exists and delete_perm:
  193                {IF page.version == latest_version:
  195                }IF
  199            }IF
  201          }IF
  202          }WITH
  204          {IF not page.exists and higher:
  208              {FOR markup in higher:
  210              }FOR
  213          }IF
  215          {IF not page.exists and related:
  219              {FOR markup in related:
  221              }FOR
  224          }IF
  230        }BLOCK content
# -- Jinja2 OK
}}}

As a second step, jinjachecker removes the Jinja2 markup and performs a validation of the document, using lxml.
{{{
# -- HTML check for 'trac/wiki/templates/jwiki_view.html'
    1 
...
   13 
   14 <!DOCTYPE html>
   15 <html>
   16   <head>
   17     <title>
   18 
   19 
   20       ${title} ${ super() }
   21 
   22 
   23     </title>
   24 
   25 
   26 
   27 
   28 
   29     ${ super() }
   30 
...
   52 
   53   </head>
   54 
   55   <body>
   56 
   57 
   58 
   59 
   60 
   61     <div id="content" class="${classes('wiki', create=not page.exists)}">
   62 
...
  213 
  214 
  215     </div>
  216 
  217     ${ super() }
  218 
  219 
  220   </body>
  221 </html>
# -- HTML OK
}}}