| Version 17 (modified by , 8 years ago) ( diff ) |
|---|
Warning
The following documentation corresponds to an experimental branch (Proposals/Jinja; Git branch: cboos.git@jinja2 - github mirror).
Porting Templates from Genshi to Jinja2
The following documentation is primarily targeted at plugin developers who wish to adapt their Genshi templates to the Jinja2 template engine that will be used in Trac 1.4.
For migrating your own templates, a good way to start is to learn from examples. Compare the 'j…' Jinja templates found in source:cboos.git/trac/templates@jinja2 with their corresponding Genshi ones.
Note that Genshi will not be supported concurrently with Jinja2. If for some reason you're stuck to having to support Genshi templates, you'll have to stick to Trac 1.2.x. But you really should make the transition effort as Jinja2 templates are 5-10x faster than their Genshi equivalent, for a 1/5th of the cost in memory usage.
Changes in the template syntax
Most of the time, the porting is a straightforward operation.
expand a variable
- Genshi
<b>$the_variable</b>
- Jinja2
<b>${the_variable}</b>
Tip:
sed -ie 's,\$\([a-z_][a-z._]*\),${\1},g' template.html
expand a simple computation
- Genshi
<b>${the_variable + 1}</b>
- Jinja2
<b>${the_variable + 1}</b>
However, Jinja2 expressions are only similar to Python expressions, there are a few differences and limitations, see expressions doc.
See also set complex variables below for more involved examples.
include another template
- Genshi
<xi:include href="the_file.html"><xi:fallback/></xi:include>
- Jinja2
# include "the_file.html" ignore missing
See include doc.
It is also possible to pass "parameters" to included templates. Those are not actually declared parameters, simply variables expected to be available in the template that can be set for the scope of one specific include. This works exactly like in Genshi, with a with statement:
- Genshi
<xi:include href="ticket_box.html" py:with="can_append = False; preview_mode = False"/>
- Jinja
# with # set can_append = false # set preview_mode = false # include "jticket_box.html" # endwith
See with doc.
Caveat: note how the boolean constants slightly differ from Python, False becomes false, True becomes true and None is none.
simple if…then (no else)
- Genshi or simply:
<py:if test="flag"><b>OK</b></py:if>
<b py:if="flag">OK</b>
- Jinja2
# if flag: <b>OK</b> # endif
See if doc.
if…then…else
- Genshi
or simply:
<py:choose test="flag"> <py:when test="True"> <b>OK</b> </py:when> <py:otherwise> <i>!!!</i> </py:otherwise> </py:choose>
<py:choose> <b py:when="flag">OK</b> <i py:otherwise="">!!!</i> </py:choose>
- Jinja2
# if flag: <b>OK</b> # else: <i>!!!</i> # endif
If you really have to, you can also use the block style:
{% if flag %}<b>OK</b>{% else %}<i>!!!</i>{% endif %}
However this goes against readability and processing via the jinjachecker tool, so we really advise that you stick to the use of line statements.
iterate over a collection
- Genshi
or simply:
<ul> <py:for each="element in list"> <li>$element</li> </py:for> </ul>
<ul> <li py:for="element in list">$element</li> </ul>
- Jinja2
<ul> # for element in list: <li>${element}</li> # endfor </ul>
See for doc.
No need for enumerate
- Genshi:
<tr py:for="idx,option in enumerate(section.options)" class="${'modified' if option.modified else None}"> <th py:if="idx == 0" class="section" rowspan="${len(section.options)}">${section.name}</th>
- Jinja2:
# for option in section.options: <tr ${{'class': 'modified' if option.modified}|htmlattr}> # if loop.first: <th class="section" rowspan="${len(section.options)}">${section.name}</th>
All common uses (and more) for such an idx variable are addressed by the special loop variable.
See loop doc.
define a macro
- Genshi
<py:def function="entry(key, val='--')"> <dt>$key</dt><dd>$val</dd> </py:def>
- Jinja2
# macro entry(key, val='--') <dt>${key}</dt><dd>${val}</dd> # endmacro
See macros doc.
set a variable
- Genshi
Note that we had to use
<py:with vars="count = len(collection)"> We have ${count > 10 and 'too much' or count} elements. </py:with>
>in Genshi, we can't use>directly. - Jinja2
Note that we avoid using
# set count = len(collection) We have ${'too much' if count is greaterthan(10) else count} elements.>in Jinja2 expressions as well, but simply to avoid that XML/HTML text editors get confused. We added a few custom tests for that (greaterthan,greaterthanorequal,lessthan,lessthanorequal).
See tests doc.
set HTML attributes
In Genshi, an attribute with a None value wouldn't be output. However, Jinja2 doesn't know what an attribute is (or anything else about HTML, XML for that matter), so we have to use a special filter, htmlattr, to reproduce this behavior:
- Genshi:
<tr class="${'disabled' if all(not component.enabled for module in plugin.modules.itervalues() for component in module.components.itervalues()) else None}">
- Jinja2:
# set components = plugin.modules|map(attribute='components')|flatten <tr ${{'class': 'disabled' if not components|selectattr('enabled') }|htmlattr}>
If you wonder why the if all(...) expression morphed into if not components|..., it's because Jinja2 expressions are similar to Python expressions, but not quite the same.
set complex variables
Note that Jinja2 expressions are a subset of Python expressions, and for the sake of simplicity the generator expressions are not part of that subset. This limitation often requires one to make creative use of filters, built-in or custom (min, max, trim, flatten).
A few more examples:
| Genshi | Jinja2 |
|---|---|
${', '.join([p.name for p in faulty_plugins])}
| ${faulty_plugins|map('name')|join(', ')}
|
sum(1 for change in changes if 'cnum' in change)
| changes|selectattr('cnum')|list|count
|
sum(1 for change in changes if 'cnum' not in change)
| changes|rejectattr('cnum')|list|count
|
| … | … |
conditional wrappers
There's no direct equivalent to the py:strip, but it can often be emulated with an if/else/endif.
- Genshi
<a py:strip="not url" href="$url">$plugin.name</a>
- Jinja2
# if url: <a href="${url}">${plugin.name}</a> # else: ${plugin.name} # endif
If the repeated content is complex, one can use a block assignment (see below).
i18n
Genshi had a pretty good notion of what was a piece of translatable text within an HTML template, but Jinja2 doesn't, so there's no "guessing" and no i18n will happen unless explicitly asked.
This can be done in two ways. First, with translation expressions, using the familiar _() gettext function (gettext and ngettext also supported).
- Genshi:
<strong>Trac detected an internal error:</strong>
- Jinja2:
<strong>${_("Trac detected an internal error:")}</strong>
Second, using trans directives.
- Genshi:
<p i18n:msg="create">Otherwise, please ${create_ticket(tracker)} a new bug report describing the problem and explain how to reproduce it.</p>
- Jinja2:
<p> # trans create = create_ticket(tracker) Otherwise, please ${create} a new bug report describing the problem and explain how to reproduce it. # endtrans </p>
See i18n doc.
Note that only direct variable expansions are supported in trans blocks, nothing more complex.
So one way to deal with complex translatable content is to factor out the complex parts in variable blocks. See block assignments doc.
- Genshi:
<p i18n:msg="">There was an internal error in Trac. It is recommended that you notify your local <a py:strip="not project.admin" href="mailto:${project.admin}"> Trac administrator</a> with the information needed to reproduce the issue. </p>
- Jinja2:
(the need for the
<p> # set trac_admin = _("Trac administrator") # set project_admin # if project.admin: <a href="mailto:${project.admin}">${trac_admin}</a> # else: ${trac_admin} # endif # endset # trans project_admin = project_admin|safe There was an internal error in Trac. It is recommended that you notify your local ${project_admin} with the information needed to reproduce the issue. # endtrans </p>
|safefilter in the above might go away once the following issue Should set blocks be safe by default gets fixed)
Note that another tricky case is when you want to use gettext and one of the variables is Markup. Using |safe is also needed, but that's not enough, as currently gettext() doesn't support Markup, you need to use tgettext() which is available with the tag_ shortcut:
<em> # set preferences_link <a href="${href.prefs()}" class="trac-target-new">${ _("Preferences")}</a> # endset ${tag_("Set your email in %(preferences_link)s", preferences_link=preferences_link|safe)} </em>
Examples
Standalone template
Let's first take a simple full-contained example from the Trac source, the simple index.html / jindex.html templates.
- Genshi index.html:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xmlns:py="http://genshi.edgewall.org/" xmlns:xi="http://www.w3.org/2001/XInclude"> <head><title>Available Projects</title></head> <body> <h1>Available Projects</h1> <ul> <li py:for="project in projects" py:choose=""> <a py:when="project.href" href="$project.href" title="$project.description"> $project.name</a> <py:otherwise> <small>$project.name: <em>Error</em> <br /> ($project.description)</small> </py:otherwise> </li> </ul> </body> </html>
- Jinja jindex.html:
<!DOCTYPE html> <html> <head> <title>${_("Available Projects")}</title> </head> <body> <h1>${_("Available Projects")}</h1> <ul> # for project in projects: <li> # if 'href' in project: <a href="${project.href}" title="${project.description}">${project.name}</a> # else: <small>${project.name}: <em>${_("Error")}</em> <br /> (${project.description})</small> # endif </li> # endfor </ul> </body> </html>
In this small example, there's no common Trac layout used (as the index is a bit special). For how a "normal" template looks like, see for example jdiff_form.html, another small template.
Note that a Jinja2 .html template can usually be rendered directly in the browser, to have a rough taste of how it will look like:
${_("Available Projects")}
-
# for project in projects:
-
# if 'href' in project:
${project.name}
# else:
${project.name}: ${_("Error")}
(${project.description}) # endif
# endfor
Though there's no absolutely no constraints on a Jinja2 template, it helps to have an .xml or .html template be itself a well-formed XML document.
Never go back to the bad old Clearsilver habits were sometimes the logic in those templates took advantage of the lack of well-formedness constraints, e.g. by conditionally inserting end/start pairs of tags. Such templates were hard to maintain, and you always have cleaner alternatives.
The jinjachecker tool should also help you maintain well-formed templates.
"Standard" templates
By "standard", we mean templates that follow the standard Trac layout, and even adapt to the currently selected theme.
Instead of the Genshi way of including a template containing filters, the Jinja2 way follows an "object oriented" approach, with inheritance and overriders. Consider that some named sections (or "blocks") of the base template are similar to "methods", imagine that you only have to "subclass" this base template and "reimplement" the overridable methods with your specific content, and there you have it.
More specifically, you'll have to "extend" the jlayout.html template, and redefine the "head", and "content" blocks if needed.
All the details are available in Proposals/Jinja#Jinja2theme, including a walkthrough for the specific example of the jsearch.html template.
For the jsearch.html example we focus on the structure of the templates, the include relationship and the decomposition in blocks.
But we also have a complete conversion example, which displays the Genshi wiki_view.html template and the Jinja2 jwiki_view.html template side-by-side, along with comments explaining the conversion choices.
Changes in the controllers
Implementing the IRequestHandler interface
With Genshi, the data for the template is basically a dict, which has to be returned by process_request
at the same time as the template name. This hasn't changed with Jinja2.
In fact, no changes to the IRequestHandler interface were needed.
Replacing the ITemplateStreamFilter interface
One of the strengths of Genshi was its ability to transform the normal HTML content and, for example, to inject arbitrary content at any point in the HTML, thanks to the use of the Transform stream filter and its API.
With Jinja2, the content is produced in one step, with no kind of post-processing. Hence the content should either be produced right away, or if it really has to be produced as an extra step, it should be produced dynamically on client-side using JavaScript.
Producing the correct content directly
There were two post-processing steps from which plugin writers did benefit, possibly unknowingly:
- the addition of the
__FORM_TOKENhidden parameter to <form> elements, necessary for successful POST operations - accessibility key enabling/disabling TODO
As this no longer happen, it's now the responsibility of plugin writers to add this <input> in their content. This is simple enough:
<form action="#" method="post"> <input type="hidden" name="__FORM_TOKEN" value="${form_token}" /> ... </form>
This gets even simpler thanks to a default macro:
<form action="#" method="post"> ${jmacros.form_token_input()} ... </form>
The jmacros corresponds to the trac/templates/jmacros.html default macros, and this file is included by default (in jlayout.html), so you don't have to bother to include it yourself (as most of the templates will extend jlayout.html).
Hooking into the HTML content produced by other templates
On this day, 127/898 plugins (14.1%) on trac-hacks.org make use of filter_stream() from the ITemplateStreamFilter interface.
So this means this specific step of the migration, perhaps the less straightforward, will be of interest for most plugin developers.
Note that though it wouldn't harm to leave the code for ITemplateStreamFilter around for use by Trac < 1.4, the new suggested way also works great with earlier versions of Trac (1.0 and 1.2, perhaps even 0.12), so there's really no reason to maintain both versions once you did the switch.
The steps for replacing filter_stream() are the following:
- implement
ITemplateProviderif you haven't done so already, as you'll need to provide a JavaScript file - implement
IRequestFilterif you haven't done so already, as you'll need to add the <script> tag for that JavaScript file, under the same circumstances in which you'd have injected your HTML code infilter_stream() - translate the Genshi Transform filter manipulations into JavaScript:
- instead of building content with the
tagbuilder, use jQuery's facilities for creating HTML elements - instead of using the Transform filter API to append/prepend the content to some place in the input stream identified by XPath expressions, use jQuery DOM manipulation API
- instead of building content with the
We'll discuss the specific example of the ticket deleter.
- this component already implemented
ITemplateProvider(for providing theticket_delete.htmltemplate), but theget_htdocs_dirdidn't yet return a location. We now have to return the localhtdocsdirectory, as we'll put our JavaScript file there:Adding an-
tracopt/ticket/deleter.py
diff --git a/tracopt/ticket/deleter.py b/tracopt/ticket/deleter.py index bd38c77..9a537bd 100644
a b 47 47 48 48 # ITemplateProvider methods 49 49 50 50 def get_htdocs_dirs(self): 51 return [] 51 from pkg_resources import resource_filename 52 yield 'ticketopt', resource_filename(__name__, 'htdocs') 52 53 53 54 def get_templates_dirs(self): 54 55 from pkg_resources import resource_filename 55 56 return [resource_filename(__name__, 'templates')] 56 57
ITemplateProviderimplementation from scratch is not more complicated (if there are no templates provided by the plugin,get_template_dirs()can simplyreturn []). -
- we need to transfer the logic at the beginning of
filter_stream()intopost_process_request(), i.e. the condition for which we decided to either let the content pass through unmodified or to modify it, now becomes the condition for which we decide to either add or not add our extra bit of JavaScript code.
So we had:
def filter_stream(self, req, method, filename, stream, data): if filename not in ('ticket.html', 'ticket_preview.html'): return stream ticket = data.get('ticket') if not (ticket and ticket.exists and 'TICKET_ADMIN' in req.perm(ticket.resource)): return stream # modify the stream! ...
which becomes now:
def post_process_request(self, req, template, data, content_type): if template in ('ticket.html', 'ticket_preview.html'): ticket = data.get('ticket') if (ticket and ticket.exists and 'TICKET_ADMIN' in req.perm(ticket.resource)): add_script(req, 'ticketopt/ticketdeleter.js') add_script_data(req, ui={'use_symbols': req.session.get('ui.use_symbols')}) return template, data, content_type
i.e. the condition remains the same: the
filename(/template) is either "ticket.html" or "ticket_preview.html", and we have a ticket in thedata, that ticket exists and we have admin perm on that ticket; if true, we would have altered the stream infilter_stream(), now inpost_process_request()we'll calladd_script.
Note that we also call
add_script_data. Here we do it for some piece of session information which is not yet available in the default JavaScript data, but you'll probably have to do that for any piece of the templatedatayou'll need to use in the JavaScript code. Don't pass the wholedatadictionary though, that would be overkill and it's quite likely some bits won't convert readily to JSON. Pass only the information you'll need.
- now the "juicy" part: do in JavaScript what the Transform filter did in Python.
Well, actually the browser needs JavaScript, but you can use whatever you want in order to produce that JavaScript code. I personally recommend using CoffeeScript as it's well suited for producing the HTML snippets we'll need.
- producing the content
The first Python helper method:becomes:# Insert "Delete" buttons for ticket description and each comment def delete_ticket(): return tag.form( tag.div( tag.input(type='hidden', name='action', value='delete'), tag.input(type='submit', value=captioned_button(req, u'–', # 'EN DASH' _("Delete")), title=_('Delete ticket'), class_="trac-delete"), class_="inlinebuttons"), action='#', method='get')
(captionedButton = (symbol, text) -> if ui.use_symbols then symbol else "#{symbol} #{text}" deleteTicket = () -> $ """ <form action="#" method="get"> <div class="inlinebuttons"> <input type="hidden" name="action" value="delete"> <input type="submit" value="#{captionedButton '–', _("Delete")}" title="#{_("Delete ticket")}" class="trac-delete"> <input type="hidden" name="__FORM_TOKEN" value="#{form_token}"> </div> </form> """
captioned_button(req, symbol, text)is a small Python utility function, which is trivial to adapt in JavaScript; note however that it's for this part of the logic that we needed to passreq.session.get('ui.use_symbols')from Python to JavaScript'sui.use_symbolsvia the call toadd_script_datain 2.)
The second Python helper method:becomes:def delete_comment(): for event in buffer: cnum, cdate = event[1][1].get('id')[12:].split('-', 1) return tag.form( tag.div( tag.input(type='hidden', name='action', value='delete-comment'), tag.input(type='hidden', name='cnum', value=cnum), tag.input(type='hidden', name='cdate', value=cdate), tag.input(type='submit', value=captioned_button(req, u'–', # 'EN DASH' _("Delete")), title=_('Delete comment %(num)s', num=cnum), class_="trac-delete"), class_="inlinebuttons"), action='#', method='get')
Not really more complex, quite the opposite. And don't ask me whatdeleteComment = (c) -> # c.id == "trac-change-3-1347886395121000" # 0123456789012 [cnum, cdate] = c.id.substr(12).split('-') $ """ <form action="#" method="get"> <div class="inlinebuttons"> <input type="hidden" name="action" value="delete-comment"> <input type="hidden" name="cnum", value="#{cnum}"> <input type="hidden" name="cdate" value="#{cdate}"> <input type="submit" value="#{captionedButton '–', _("Delete")}" title="#{_("Delete comment %(num)s", num: cnum)}" class="trac-delete"> <input type="hidden" name="__FORM_TOKEN" value="#{form_token}"> </div> </form> """
event[1][1]was ;-)
- inserting the content at the right place
Now this bit of Python magic:becomes a more straightforward sequence of jQuery calls:buffer = StreamBuffer() return stream | Transformer('//div[@class="description"]' '/h3[@id="comment:description"]') \ .after(delete_ticket).end() \ .select('//div[starts-with(@class, "change")]/@id') \ .copy(buffer).end() \ .select('//div[starts-with(@class, "change") and @id]' '//div[@class="trac-ticket-buttons"]') \ .append(delete_comment)
For those of you not so familiar with CoffeeScript, here is the corresponding plain JavaScript (assuming we're already in the document ready() callback):$(document).ready () -> # Insert "Delete" buttons for ticket description and each comment $('#ticket .description h3').after(deleteTicket()) $('#changelog div.change').each () -> $('.trac-ticket-buttons', this).prepend deleteComment this
$('#ticket .description h3').after(deleteTicket()); $('#changelog div.change').each(function() { $('.trac-ticket-buttons', this).prepend(deleteComment(this)); });
See the full changeset, [bf49f871/cboos.git] (and [be0ff94a/cboos.git] for a look to the generated .js).
Generating content
Rendering template "fragments"
When one wants to directly render a template, the Chrome component facilities can be used, as before:
return Chrome(self.env).render_template( req, 'query_results.html', data, None, fragment=True)
implementing Ticket Query macro (table mode)
A newer alternative is to use Chrome.generate_template_fragment, as in:
if req.is_xhr: # render and return the content only stream = Chrome(self.env).generate_template_fragment( req, 'changeset_content.html', data) req.send(stream)
This automatically retrieves the use_chunked_encoding TracIni setting and uses it to return an iterable. In any case, the returned value can be sent directly from the Request object.
The tag builder
Genshi provided a nice Python API for programmatically building (X)HTML elements. This consisted of the Fragment, Element and the tag builder, all from the genshi.builder module:
from genshi import Fragment, Element, tag
This has now been replaced by an equivalent API which lives in trac.util.html, so the above import should be replaced with:
from trac.util.html import Fragment, Element, tag
Note that the html symbol from trac.util.html which used to be a alias to genshi.builder.tag is now naturally an alias to trac.util.html.tag.
One way to write "portable" code would be:
from trac.util.html import html as tag
You can then use the tag builder API regardless of its origin.
Likewise, if you want to use the Markup class, you should write:
from trac.util.html import Markup
In "old" versions of Trac, you'll get genshi.core.Markup, whereas now you'll get markupsafe.Markup: as we're using Jinja2, we're also making use of its direct dependency MarkupSafe.
from trac.util.html import escape
Note that in a similar way to Markup, escape now also comes from markupsafe, with some slight adaptations, as markupsafe.escape always escapes the quotes, which is something we don't do by default. Hence always import escape from trac.util.html, never directly from markupsafe or Jinja2, unless you really know what you're doing.
