{{{#!div class="important" style="text-align: center; padding: 0em 2em; margin: 2em 0" ** Warning ** The following documentation corresponds to the [Proposals/Jinja Jinja] development proposal; it will be in Trac 1.3.x if all goes well \\ (Git branch: [log:cboos.git@jinja2] - [https://github.com/cboos/trac.git github mirror]). Track the integration progress in #12639. }}} [[PageOutline(2-4)]] = Porting Templates from Genshi to Jinja2 The following documentation is primarily targeted at plugin developers who wish to adapt their Genshi templates to the Jinja2 template engine that will be used in Trac 1.4. For migrating your own templates, a good way to start is to learn from examples. Compare the 'j...' Jinja templates found in source:cboos.git/trac/templates@jinja2 with their corresponding Genshi ones. In the first part of this document, we try to cover all the Genshi features used by Trac and present their Jinja2 equivalent. Whenever possible, we tried to minimize these differences by customizing the Jinja2 syntax. For example, we use `${...}` for variable expansion, like Genshi does, instead of `{{...}}`. Another aspect of our usage convention is that we favor [#ifthenelse line statements] over `{% ... %}`. So even someone familiar with the "default" Jinja2 syntax should glance through this document to see how "we" use Jinja2, as summarized in the table below. The last part of the document describes the Python code changes, focusing notably on how to replace the deprecated `ITemplateStreamFilter` interface. Note that Genshi will be supported concurrently with Jinja2 only for a short while, including the deprecated `ITemplateStreamFilter` interface, but probably only during the 1.3.x development period, and at most for the 1.4-stable period. If for some reason you're stuck to having to support Genshi templates, you'll have to stick to Trac 1.2.x or 1.3.x. But you really should make the transition effort as Jinja2 templates are 5-10x faster than their Genshi equivalent, for only a 1/5th of the cost in memory usage. == The Jinja2 syntax The Jinja2 template engine is quite flexible and its syntax can be customized to some extent. We took this opportunity to make it as close as possible to the Genshi template syntax, in particular for the variable expansion. We use the following configuration: || key || value || example / explanation || extensions || jinja2.ext.with_ jinja2.ext.i18n || `with` directive can be used ([#with more]) || block_start_string || {% || \ {{{#!td rowspan=2 `{% if cond %} value {% endif %}` possible (but discouraged) }}} |-- || block_end_string || %} || || variable_start_string || ${ || \ {{{#!td rowspan=2 `${the_variable}` (but not `$the_variable`) ([#expandavariable more]) }}} |-- || variable_end_string || } || || line_statement_prefix || # || \ {{{#!td {{{ # if cond: value # endif }}} (preferred form) ([#if more]) }}} |-- || line_comment_prefix || ## || `## comments are good` || trim_blocks || yes || whitespace removal after a block || lstrip_blocks || yes || whitespace removal before a block || newstyle_gettext || yes || i.e. like the Trac `gettext` == Changes in the HTML We took the opportunity to switch to HTML5 while performing this template overhaul. You should probably do the same. This usually won't change anything for your templates, except for a few details. The doctype and the element should be changed from Genshi's: {{{#!html+genshi }}} to the somewhat simpler: {{{#!xml }}} Special care should be taken when dealing with //void elements//. In XHTML, it's fine to write: {{{#!xml
}}} However, when going to HTML5, you should use instead: {{{#!xml
}}} The valid [https://www.w3.org/TR/html-markup/syntax.html#void-elements void elements] in HTML5 are limited to the following list: - area, base, br, col, command, embed, hr, img, input, keygen, link, meta, param, source, track, wbr == Changes in the template syntax Most of the time, the porting is a straightforward operation. === expand a variable - Genshi [[html+genshi($the_variable)]] - Jinja2 [[html+jinja(${the_variable})]] Note that Jinja2 doesn't support the `$the_variable` style, the curly braces are mandatory. Tip: {{{#!shell sed -i -e 's,\$\([a-z_][a-z._]*\),${\1},g' template.html }}} === expand a simple computation - Genshi [[html+genshi(${the_variable + 1})]] - Jinja2 [[html+jinja(${the_variable + 1})]] However, Jinja2 expressions are only //similar// to Python expressions, there are a few differences and limitations, see [http://jinja.pocoo.org/docs/dev/templates/#expressions expressions] doc. See also [#set set complex variables] below for more involved examples. Another customization we made to Jinja2 is to avoid having a Python `None` value be expanded to the `"None"` string. Instead, we make it produce an empty string, like Genshi did. === include another template #include - Genshi [[xml()]] - Jinja2 [[xml(# include "the_file.html" ignore missing)]] See [http://jinja.pocoo.org/docs/dev/templates/#include include] doc. Note that the `ignore missing` part is mandatory no templates are given to the `include` directive. This includes the following situation: {{{#!html+jinja # include "sometemplate.html" if false }}} This alone would raise an error. The correct thing to write is: {{{#!html+jinja # include "sometemplate.html" if false ignore missing }}} It is also possible to pass "parameters" to included templates. Those are not actually declared parameters, simply variables expected to be available in the template that can be set for the scope of one specific include. This works exactly like in Genshi, with a `with` statement: - Genshi {{{#!html+genshi }}} - Jinja {{{#!html+jinja # with # set can_append = false # set preview_mode = false # include "jticket_box.html" # endwith }}} See [http://jinja.pocoo.org/docs/dev/extensions/#with-statement with] doc. In passing, note how the boolean constants slightly differ from Python, `False` becomes `false`, `True` becomes `true` and `None` is `none`. Caveat: contrary to the Genshi way, one shouldn't include the `"layout.html"` template in order to inherit the default page layout. There's a big difference in the way template "inheritance" works between Genshi and Jinja2, see HtmlTemplates#Jinjaarchitecture for all the details. Despite these differences, we kept the same spirit and there's actually also a `"layout.html"` template that you can "inherit" from (as well as a `"theme.html"` template that the layout inherits in turn). But instead of "including" it in your template, you "extend" it: {{{#!html+jinja # extends "layout.html" }}} But this is not the place to go further in the details about how extending works, refer to the [http://jinja.pocoo.org/docs/dev/templates/#extends extends] documentation and to the link above for how this applies to Trac. === simple if...then (no else) #if - Genshi [[xml(OK)]] or simply: [[xml(OK)]] - Jinja2 {{{#!html+jinja # if flag: OK # endif }}} See [http://jinja.pocoo.org/docs/dev/templates/#if if] doc. === if...then...else - Genshi {{{#!html+genshi OK !!! }}} or simply: {{{#!html+genshi OK !!! }}} - Jinja2 {{{#!html+jinja # if flag: OK # else: !!! # endif }}} If you really have to, you can also use the block style: {{{#!html+jinja {{ if flag }}OK{{ else }}!!!{{ endif }} }}} However this goes against readability and processing via the [./Checker jinjachecker] tool, so we really advise that you stick to the use of //[http://jinja.pocoo.org/docs/dev/templates/#line-statements line statements]//. === iterate over a collection #for - Genshi {{{#!html+genshi }}} or simply: {{{#!html+genshi }}} - Jinja2 {{{#!html+jinja }}} See [http://jinja.pocoo.org/docs/dev/templates/#for for] doc. ==== No need for `enumerate` - Genshi: {{{#!html+genshi ${section.name} }}} - Jinja2: {{{#!html+jinja # for option in section.options: # if loop.first: ${section.name} }}} All common uses (and more) for such an `idx` variable are addressed by the special `loop` variable. See [http://jinja.pocoo.org/docs/dev/templates/#for loop] doc. === define a macro #macro - Genshi {{{#!html+genshi
$key
$val
}}} - Jinja2 {{{#!html+jinja # macro entry(key, val='--')
${key}
${val}
# endmacro }}} See [http://jinja.pocoo.org/docs/dev/templates/#macros macros] doc. === set a variable - Genshi {{{#!html+genshi We have ${count > 10 and 'too much' or count} elements. }}} Note that we had to use `>` in Genshi, we can't use `>` directly. - Jinja2 {{{#!html+jinja # set count = len(collection) We have ${'too much' if count is greaterthan(10) else count} elements. }}} Note that we avoid using `>` in Jinja2 expressions as well, but simply to avoid that XML/HTML text editors get confused. We added a few custom tests for that (`greaterthan`, `greaterthanorequal`, `lessthan`, `lessthanorequal`). See [http://jinja.pocoo.org/docs/dev/templates/#tests tests] doc. === set several variables in a scope #with - Genshi {{{#!html+genshi ... }}} The variables are set for the scope of the element in which they are set (here , so the whole document) - Jinja2 {{{#!html+jinja # with # set is_query = report.sql.startswith('query:') # set new_report = action == 'new' and not is_query ... # endwith }}} But actually you will only use `with` in specific situations, like for wrapping an include directive (see [#include]). If you're already within a `for`, a `block` or a `macro`, the scope of the assignment is already limited to that of this directive. See [http://jinja.pocoo.org/docs/dev/templates/#assignments set] and [http://jinja.pocoo.org/docs/dev/templates/#with-statement with] docs. In addition, `with` can also be used to better control how the successive `set` on a given variable are being applied (see for example [1e25c852/cboos.git] and [cc1b959e/cboos.git]). Finally be careful when using `with`: don't wrap a `block` directive within a `with`. If you want to set a global scope for the document (like in our example above), it's tempting to use a single `with` statement, for clarity. But that would wrap all blocks defined in the template and strange results would ensue. === set HTML attributes #htmlattr In Genshi, an attribute with a `None` value wouldn't be output. However, Jinja2 doesn't know what an attribute is (or anything else about HTML, XML for that matter), so we have to use a special filter, `htmlattr`, to reproduce this behavior: - Genshi: {{{#!html+genshi }}} - Jinja2: {{{#!html+jinja # set components = plugin.modules|map(attribute='components')|flatten }}} (the `htmlattr` filter will add a space if needed; that way, if the condition is true, you end up with ``, otherwise with just ``) If you wonder why the `if all(...)` expression morphed into `if not components|...`, it's because Jinja2 expressions are similar to Python expressions, but not quite the same. === set complex variables #set Note that Jinja2 expressions are a subset of Python expressions, and for the sake of simplicity the generator expressions are not part of that subset. This limitation often requires one to make creative use of [http://jinja.pocoo.org/docs/dev/templates/#filters filters], [http://jinja.pocoo.org/docs/dev/templates/#builtin-filters built-in] or custom (`min`, `max`, `trim`, `flatten`). A few more examples: ||= Genshi ||= Jinja2 || || `${', '.join([p.name for p in faulty_plugins])}` || \ || `${faulty_plugins|map('name')|join(', ')}` || || `sum(1 for change in changes if 'cnum' in change)` || \ || `changes|selectattr('cnum')|list|count` || || `sum(1 for change in changes if 'cnum' not in change)` || \ || `changes|rejectattr('cnum')|list|count` || || ... || ... || === conditional wrappers There's no direct equivalent to the `py:strip`, but it can often be emulated with an `if/else/endif`. - Genshi {{{#!html+genshi $plugin.name }}} - Jinja2 {{{#!html+jinja # if url: ${plugin.name} # else: ${plugin.name} # endif }}} If the repeated content is complex, one can use a //block assignment// (see below). === i18n #trans Genshi had a pretty good notion of what was a piece of translatable text within an HTML template, but Jinja2 doesn't, so there's no "guessing" and no i18n will happen unless explicitly asked. This can be done in two ways. First, with translation expressions, using the familiar `_()` gettext function (`gettext` and `ngettext` also supported). - Genshi: [[xml(Trac detected an internal error:)]] - Jinja2: [[xml(${_("Trac detected an internal error:")})]] Second, using `trans` directives. - Genshi: {{{#!html+genshi

Otherwise, please ${create_ticket(tracker)} a new bug report describing the problem and explain how to reproduce it.

}}} - Jinja2: {{{#!html+jinja

# trans create = create_ticket(tracker) Otherwise, please ${create} a new bug report describing the problem and explain how to reproduce it. # endtrans

}}} another example, without assignment: {{{#!html+jinja # trans formatted In the default time zone, this would be displayed as ${formatted}. # endtrans }}} last example, two expanded variables, with and without assignment: {{{#!html+jinja # trans tz = nowtz.tzname(), formatted In your time zone ${tz}, this would be displayed as ${formatted}. # endtrans }}} See [http://jinja.pocoo.org/docs/dev/templates/#i18n i18n] doc. Note that only direct variable expansions are supported in `trans` blocks, nothing more complex. So one way to deal with complex translatable content is to factor out the complex parts in variable blocks. See [http://jinja.pocoo.org/docs/dev/templates/#block-assignments block assignments] doc. - Genshi: {{{#!html+genshi

There was an internal error in Trac. It is recommended that you notify your local Trac administrator with the information needed to reproduce the issue.

}}} - Jinja2: {{{#!html+jinja

# set trac_admin = _("Trac administrator") # set project_admin # if project.admin: ${trac_admin} # else: ${trac_admin} # endif # endset # trans project_admin = project_admin|safe There was an internal error in Trac. It is recommended that you notify your local ${project_admin} with the information needed to reproduce the issue. # endtrans

}}} (the need for the `|safe` filter in the above might go away once the following issue //[https://github.com/mitsuhiko/jinja2/issues/490 Should set blocks be safe by default]// gets fixed) Note that another tricky case is when you want to use `gettext` and one of the variables is Markup. Using `|safe` is also needed, but that's not enough, as currently `gettext()` doesn't support Markup, you need to use `tgettext()` which is available with the `tag_` shortcut: {{{#!html+jinja # set preferences_link ${ _("Preferences")} # endset ${tag_("Set your email in %(preferences_link)s", preferences_link=preferences_link|safe)} }}} == Examples === Standalone template Let's first take a simple full-contained example from the Trac source, the simple index.html / jindex.html templates. - Genshi [source:sandbox/genshi/templates/index.html@3728 index.html]: {{{#!html+genshi Available Projects

Available Projects

}}} - Jinja [source:cboos.git/trac/templates/jindex.html@jinja2 jindex.html]: {{{#!html+jinja ${_("Available Projects")}

${_("Available Projects")}

    # for project in projects:
  • # if 'href' in project: ${project.name} # else: ${project.name}: ${_("Error")}
    (${project.description})
    # endif
  • # endfor
}}} In this small example, there's no common Trac layout used (as the index is a bit special). For how a "normal" template looks like, see for example [source:trac/templates/jdiff_form.html@jinja2 jdiff_form.html], another small template. Note that a Jinja2 .html template can usually be rendered directly in the browser, to have a rough taste of how it will look like: {{{#!div style="border: 2px solid #ccc; margin: 1em; padding: 0 2em" {{{#!html (html mode on purpose here!)

${_("Available Projects")}

    # for project in projects:
  • # if 'href' in project: ${project.name} # else: ${project.name}: ${_("Error")}
    (${project.description})
    # endif
  • # endfor
}}} }}} Though there's absolutely no constraints on what text a Jinja2 template may contain, for templates that will produce HTML (or XML), it will be useful if the template is itself already '''a well-formed XML document'''. **Never** go back to the bad old habits from the ClearSilver time, were sometimes the logic in those templates took advantage of the lack of well-formedness constraints, e.g. by conditionally inserting end/start pairs of tags to split sequences. Such templates were hard to maintain, and you always have cleaner alternatives. The [./Checker jinjachecker] tool should also help you maintain well-formed templates by stripping off Jinja2 expressions and line statements before attempting to XML validate the document ([pypi:lxml] should be installed for this feature). === "Standard" templates By "standard", we mean templates that follow the standard Trac layout, and even adapt to the currently selected theme. Instead of the Genshi way of including a template containing filters, the Jinja2 way follows an "object oriented" approach, with inheritance and overriders. Consider that some named sections (or "blocks") of the base template are similar to "methods", imagine that you only have to "subclass" this base template and "reimplement" the overridable methods with your specific content, and there you have it. More specifically, you'll have to "extend" the [source:cboos.git/trac/templates/jlayout.html@jinja2 jlayout.html] template, and redefine the "head", and "content" blocks if needed. All the details are available in HtmlTemplates#Jinjaarchitecture, including a walkthrough for the specific example of the [source:cboos.git/trac/search/templates/jsearch.html@jinja2 jsearch.html] template. For the jsearch.html example we focus on the //structure// of the templates, the //include// relationship and the decomposition in //blocks//. But we also have a complete conversion [./Example example], which displays the Genshi wiki_view.html template and the Jinja2 jwiki_view.html template side-by-side, along with comments explaining the conversion choices. === Tips and tricks - when you need to output a plain "#" character at the beginning of a line, this will be parsed as a line statement; the trick here is to use an empty inline comment as a prefix: `{##}# ...` (see [61e8d9ec/cboos.git]) - when you need to output indented text, this can be made difficult due to our `lstrip_block` configuration setting; you can work around this by embedding Python strings with the exact whitespace content you need in variable expressions: ` ${"\t "}` (see [0fdef480/cboos.git]) == Changes in the controllers == === Implementing the `IRequestHandler` interface With Genshi, the data for the template is basically a `dict`, which has to be returned by `process_request` at the same time as the template name. This hasn't changed with Jinja2. In fact, no changes to the `IRequestHandler` interface were needed. === Replacing the `ITemplateStreamFilter` interface #ReplacingITemplateStreamFilter One of the strengths of Genshi was its ability to transform the normal HTML content and, for example, to inject arbitrary content at any point in the HTML, thanks to the use of the Transform stream filter and its API. With Jinja2, the content is produced in one step, with no kind of post-processing. Hence the content should either be produced right away, or if it really has to be produced as an extra step, it should be produced dynamically on client-side using JavaScript. ==== Producing the correct content directly There were two post-processing steps from which plugin writers did benefit, possibly unknowingly: 1. the addition of the `__FORM_TOKEN` hidden parameter to
elements, necessary for successful POST operations 2. accessibility key enabling/disabling (TODO) As this no longer happen, it's now the responsibility of plugin writers to add this in their content. This is simple enough: {{{#!html+jinja ...
}}} This gets even simpler thanks to a default macro: {{{#!html+jinja
${jmacros.form_token_input()} ...
}}} The `jmacros` corresponds to the [source:cboos.git/trac/templates/jmacros.html@jinja2 trac/templates/jmacros.html] default macros, and this file is included by default (in `jlayout.html`), so you don't have to bother to include it yourself (as most of the templates will extend `jlayout.html`). ==== Hooking into the HTML content produced by other templates On this day, 127/898 plugins (14.1%) on trac-hacks.org make use of `filter_stream()` from the `ITemplateStreamFilter` interface. So this means this specific step of the migration, perhaps the less straightforward, will be of interest for most plugin developers. Note that though it wouldn't harm to leave the code for `ITemplateStreamFilter` around for use by Trac < 1.4, the new suggested way also works great with earlier versions of Trac (1.0 and 1.2, perhaps even 0.12), so there's really no reason to maintain both versions once you did the switch. The steps for replacing `filter_stream()` are the following: 1. implement `ITemplateProvider` if you haven't done so already, as you'll need to provide a JavaScript file 2. implement `IRequestFilter` if you haven't done so already, as you'll need to add the