Edgewall Software
Home
Trac
Trac Hacks
Genshi
Babel
Bitten
Home
Download
Documentation
Mailing Lists
License
FAQ
Search:
Login
Preferences
Help/Guide
About Trac
Wiki
Timeline
Roadmap
Browse Source
View Tickets
New Ticket
Search
Context Navigation
+0
Start Page
Index
History
Editing TracDev/PortingFromGenshiToJinja
Adjust edit area height:
8
12
16
20
24
28
32
36
40
Edit side-by-side
{{{#!div class="important" style="padding: 0em 2em; margin: 2em 0" ** Warning ** The following documentation corresponds to an experimental branch ([[Proposals/Jinja]] / [log:cboos.git@jinja2]). }}} [[PageOutline(2-3)]] = Porting Templates from Genshi to Jinja2 The following documentation is primarily targeted at plugin developers who wish to adapt their Genshi templates to the Jinja2 template engine that will be used in Trac 1.4. For migrating your own templates, a good way to start is to learn from examples. Compare the 'j...' Jinja templates found in source:cboos.git/trac/templates@jinja2 with their corresponding Genshi ones. Note that Genshi will **not** be supported concurrently with Jinja2. If for some reason you're stuck to having to support Genshi templates, you'll have to stick to Trac 1.2.x. But you really should make the transition effort as Jinja2 templates are 5-10x faster than their Genshi equivalent, for a 1/5th of the cost in memory usage. == Changes in the template syntax Most of the time, the porting is a straightforward operation. === expand a variable - Genshi [[xml(<b>$the_variable</b>)]] - Jinja2 [[xml(<b>${the_variable}</b>)]] Tip: {{{#!shell sed -ie 's,\$\([a-z_][a-z._]*\),${\1},g' template.html }}} === expand a simple computation - Genshi [[xml(<b>${the_variable + 1}</b>)]] - Jinja2 [[xml(<b>${the_variable + 1}</b>)]] However, Jinja2 expressions are only //similar// to Python expressions, there are a few differences and limitations, see [http://jinja.pocoo.org/docs/dev/templates/#expressions expressions] doc. See also [#setcomplexvariabels set complex variables] below for more involved examples. === include another template - Genshi [[xml(<xi:include href="the_file.html"><xi:fallback/></xi:include>)]] - Jinja2 [[xml(# include "the_file.html" ignore missing)]] See [http://jinja.pocoo.org/docs/dev/templates/#include include] doc. It is also possible to pass "parameters" to included templates. Those are not actually declared parameters, simply variables expected to be available in the template that can be set for the scope of one specific include. This works exactly like in Genshi, with a with statement: - Genshi {{{#!html+genshi <xi:include href="ticket_box.html" py:with="can_append = False; preview_mode = False"/> }}} - Jinja {{{#!html+jinja # with # set can_append = false # set preview_mode = false # include "jticket_box.html" # endwith }}} See [http://jinja.pocoo.org/docs/dev/extensions/#with-statement with] doc. Caveat: note how the boolean constants slightly differ from Python, `False` becomes `false`, `True` becomes `true` and `None` is `none`. === simple if...then (no else) - Genshi [[xml(<py:if test="flag"><b>OK</b></py:if>)]] or simply: [[xml(<b py:if="flag">OK</b>)]] - Jinja2 {{{#!html+jinja # if flag: <b>OK</b> # endif }}} See [http://jinja.pocoo.org/docs/dev/templates/#if if] doc. === if...then...else - Genshi {{{ #!xml <py:choose test="flag"> <py:when test="True"> <b>OK</b> </py:when> <py:otherwise> <i>!!!</i> </py:otherwise> </py:choose> }}} or simply: {{{ #!xml <py:choose> <b py:when="flag">OK</b> <i py:otherwise="">!!!</i> </py:choose> }}} - Jinja2 {{{ #!xml # if flag: <b>OK</b> # else: <i>!!!</i> # endif }}} If you really have to, you can also use the block style: {{{#!html+jinja {% if flag %}<b>OK</b>{% else %}<i>!!!</i>{% endif %} }}} However this goes against readability and processing via the [#jinjachecker] tool, so we really advise that you stick to the use of //[http://jinja.pocoo.org/docs/dev/templates/#line-statements line statements]//. === iterate over a collection - Genshi {{{ #!xml <ul> <py:for each="element in list"> <li>$element</li> </py:for> </ul> }}} or simply: {{{ #!xml <ul> <li py:for="element in list">$element</li> </ul> }}} - Jinja2 {{{ #!xml <ul> # for element in list: <li>${element}</li> # endfor </ul> }}} See [http://jinja.pocoo.org/docs/dev/templates/#for for] doc. ==== No need for `enumerate` - Genshi: {{{#!html+genshi <tr py:for="idx,option in enumerate(section.options)" class="${'modified' if option.modified else None}"> <th py:if="idx == 0" class="section" rowspan="${len(section.options)}">${section.name}</th> }}} - Jinja2: {{{#!html+jinja # for option in section.options: <tr ${{'class': 'modified' if option.modified}|htmlattr}> # if loop.first: <th class="section" rowspan="${len(section.options)}">${section.name}</th> }}} All common uses (and more) for such an `idx` variable are addressed by the special `loop` variable. See [http://jinja.pocoo.org/docs/dev/templates/#for loop] doc. === define a macro - Genshi {{{#!html+genshi <py:def function="entry(key, val='--')"> <dt>$key</dt><dd>$val</dd> </py:def> }}} - Jinja2 {{{#!html+jinja # macro entry(key, val='--') <dt>${key}</dt><dd>${val}</dd> # endmacro }}} See [http://jinja.pocoo.org/docs/dev/templates/#macros macros] doc. === set a variable - Genshi {{{#!html+genshi <py:with vars="count = len(collection)"> We have ${count > 10 and 'too much' or count} elements. </py:with> }}} Note that we had to use `>` in Genshi, we can't use `>` directly. - Jinja2 {{{#!html+jinja # set count = len(collection) We have ${'too much' if count is greaterthan(10) else count} elements. }}} Note that we avoid using `>` in Jinja2 expressions as well, but simply to avoid that XML/HTML text editors get confused. We added a few custom tests for that (`greaterthan`, `greaterthanorequal`, `lessthan`, `lessthanorequal`). See [http://jinja.pocoo.org/docs/dev/templates/#tests tests] doc. === set HTML attributes In Genshi, an attribute with a `None` value wouldn't be output. However, Jinja2 doesn't know what an attribute is (or anything else about HTML, XML for that matter), so we have to use a special filter, `htmlattr`, to reproduce this behavior: - Genshi: {{{#!html+genshi <tr class="${'disabled' if all(not component.enabled for module in plugin.modules.itervalues() for component in module.components.itervalues()) else None}"> }}} - Jinja2: {{{#!html+jinja # set components = plugin.modules|map(attribute='components')|flatten <tr ${{'class': 'disabled' if not components|selectattr('enabled') }|htmlattr}> }}} If you wonder why the `if all(...)` expression morphed into `if not components|...`, it's because Jinja2 expressions are similar to Python expressions, but not quite the same. === set complex variables Note that Jinja2 expressions are a subset of Python expressions, and for the sake of simplicity the generator expressions are not part of that subset. This limitation often requires one to make creative use of [http://jinja.pocoo.org/docs/dev/templates/#filters filters], [http://jinja.pocoo.org/docs/dev/templates/#builtin-filters built-in] or custom (`min`, `max`, `trim`, `flatten`). A few more examples: ||= Genshi ||= Jinja2 || || `${', '.join([p.name for p in faulty_plugins])}` || \ || `${faulty_plugins|map('name')|join(', ')}` || || `sum(1 for change in changes if 'cnum' in change)` || \ || `changes|selectattr('cnum')|list|count` || || `sum(1 for change in changes if 'cnum' not in change)` || \ || `changes|rejectattr('cnum')|list|count` || || ... || ... || === conditional wrappers There's no direct equivalent to the `py:strip`, but it can often be emulated with an `if/else/endif`. - Genshi {{{#!html+genshi <a py:strip="not url" href="$url">$plugin.name</a> }}} - Jinja2 {{{#!html+jinja # if url: <a href="${url}">${plugin.name}</a> # else: ${plugin.name} # endif }}} If the repeated content is complex, one can use a //block assignment// (see below). === i18n Genshi had a pretty good notion of what was a piece of translatable text within an HTML template, but Jinja2 doesn't, so there's no "guessing" and no i18n will happen unless explicitly asked. This can be done in two ways. First, with translation expressions, using the familiar `_()` gettext function (`gettext` and `ngettext` also supported). - Genshi: [[xml(<strong>Trac detected an internal error:</strong>)]] - Jinja2: [[xml(<strong>${_("Trac detected an internal error:")}</strong>)]] Second, using `trans` directives. - Genshi: {{{#!html+genshi <p i18n:msg="create">Otherwise, please ${create_ticket(tracker)} a new bug report describing the problem and explain how to reproduce it.</p> }}} - Jinja2: {{{#!html+jinja <p> # trans create = create_ticket(tracker) Otherwise, please ${create} a new bug report describing the problem and explain how to reproduce it. # endtrans </p> }}} Note that only direct variable expansions are supported in `trans` blocks, nothing more complex. So one way to deal with complex translatable content is to factor out the complex parts in variable blocks. See [http://jinja.pocoo.org/docs/dev/templates/#block-assignments block assignments] doc. - Genshi: {{{#!html+genshi <p i18n:msg="">There was an internal error in Trac. It is recommended that you notify your local <a py:strip="not project.admin" href="mailto:${project.admin}"> Trac administrator</a> with the information needed to reproduce the issue. </p> }}} - Jinja2: {{{#!html+jinja <p> # set trac_admin = _("Trac administrator") # set project_admin # if project.admin: <a href="mailto:${project.admin}">${trac_admin}</a> # else: ${trac_admin} # endif # endset # trans project_admin = project_admin|safe There was an internal error in Trac. It is recommended that you notify your local ${project_admin} with the information needed to reproduce the issue. # endtrans </p> }}} (the need for the `|safe` filter in the above might go away once the following issue //[https://github.com/mitsuhiko/jinja2/issues/490 Should set blocks be safe by default]// gets fixed) == Examples Let's first take a simple full-contained example from the Trac source, the simple index.html / jindex.html templates. - Genshi [source:sandbox/genshi/templates/index.html@3728 index.html]: {{{#!html+genshi <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xmlns:py="http://genshi.edgewall.org/" xmlns:xi="http://www.w3.org/2001/XInclude"> <head><title>Available Projects</title></head> <body> <h1>Available Projects</h1> <ul> <li py:for="project in projects" py:choose=""> <a py:when="project.href" href="$project.href" title="$project.description"> $project.name</a> <py:otherwise> <small>$project.name: <em>Error</em> <br /> ($project.description)</small> </py:otherwise> </li> </ul> </body> </html> }}} - Jinja [source:cboos.git/trac/templates/jindex.html@jinja2 jindex.html]: {{{#!html+jinja <!DOCTYPE html> <html> <head> <title>${_("Available Projects")}</title> </head> <body> <h1>${_("Available Projects")}</h1> <ul> # for project in projects: <li> # if 'href' in project: <a href="${project.href}" title="${project.description}">${project.name}</a> # else: <small>${project.name}: <em>${_("Error")}</em> <br /> (${project.description})</small> # endif </li> # endfor </ul> </body> </html> }}} In this small example, there's no common Trac layout used (as the index is a bit special). For how a "normal" template looks like, see for example [source:trac/templates/jdiff_form.html@jinja2 jdiff_form.html], another small template. Note that a Jinja2 .html template can usually be rendered directly in the browser, to have a rough taste of how it will look like: {{{#!div style="border: 1px solid #999; margin: 1em" {{{#!html (html mode on purpose here!) <h1>${_("Available Projects")}</h1> <ul> # for project in projects: <li> # if 'href' in project: <a href="${project.href}" title="${project.description}">${project.name}</a> # else: <small>${project.name}: <em>${_("Error")}</em> <br /> (${project.description})</small> # endif </li> # endfor </ul> }}} }}} Though there's no absolutely no constraints on a Jinja2 template, it helps to have an .xml or .html template be itself '''a well-formed XML document'''. **Never** go back to the bad old Clearsilver habits were sometimes the logic in those templates took advantage of the lack of well-formedness constraints, e.g. by conditionally inserting end/start pairs of tags. Such templates were hard to maintain, and you always have cleaner alternatives. The [#jinjachecker] tool should also help you maintain well-formed templates. == Changes in the controllers == === Implementing the `IRequestHandler` interface With Genshi, the data for the template is basically a `dict`, which has to be returned by `process_request` at the same time as the template name. This hasn't changed with Jinja2. In fact, no changes to the `IRequestHandler` interface were needed. === Replacing the `ITemplateStreamFilter` interface #ReplacingITemplateStreamFilter One of the strengths of Genshi was its ability to transform the normal HTML content and, for example, to inject arbitrary content at any point in the HTML, thanks to the use of the Transform stream filter and its API. With Jinja2, the content is produced in one step, with no kind of post-processing. Hence the content should either be produced right away, or if it really has to be produced as an extra step, it should be produced dynamically on client-side using JavaScript. ==== Producing the correct content directly There were two post-processing steps from which plugin writers did benefit, possibly unknowingly: 1. the addition of the `__FORM_TOKEN` hidden parameter to <form> elements, necessary for successful POST operations 2. accessibility key enabling/disabling (TODO) As this no longer happen, it's now the responsibility of plugin writers to add this <input> in their content. This is simple enough: {{{#!html+jinja <form action="#" method="post"> <input type="hidden" name="__FORM_TOKEN" value="${form_token}" /> ... </form> }}} This gets even simpler thanks to a default macro: {{{#!html+jinja <form action="#" method="post"> ${jmacros.form_token_input()} ... </form> }}} The `jmacros` corresponds to the [source:cboos.git/trac/templates/jmacros.html@jinja2 trac/templates/jmacros.html] default macros, and this file is included by default (in `jlayout.html`), so you don't have to bother to include it yourself (as most of the templates will extend `jlayout.html`). ==== Hooking into the HTML content produced by other templates On this day, 127/898 plugins (14.1%) on trac-hacks.org make use of `filter_stream()` from the `ITemplateStreamFilter` interface. So this means this specific step of the migration, perhaps the less straightforward, will be of interest for most plugin developers. Note that though it wouldn't harm to leave the code for `ITemplateStreamFilter` around for use by Trac < 1.4, the new suggested way also works great with earlier versions of Trac (1.0 and 1.2, perhaps even 0.12), so there's really no reason to maintain both versions once you did the switch. The steps for replacing `filter_stream()` are the following: 1. implement `ITemplateProvider` if you haven't done so already, as you'll need to provide a JavaScript file 2. implement `IRequestFilter` if you haven't done so already, as you'll need to add the <script> tag for that JavaScript file, under the same circumstances in which you'd have injected your HTML code in `filter_stream()` 3. translate the Genshi Transform filter manipulations into JavaScript: - instead of building content with the `tag` builder, use jQuery's facilities for creating HTML elements - instead of using the Transform filter API to append/prepend the content to some place in the input stream identified by XPath expressions, use jQuery DOM manipulation API We'll discuss the specific example of the ticket [source:cboos.git/tracopt/ticket/deleter.py deleter]. 1. this component already implemented `ITemplateProvider` (for providing the `ticket_delete.html` template), but the `get_htdocs_dir` didn't yet return a location. We now have to return the local `htdocs` directory, as we'll put our JavaScript file there: {{{#!diff diff --git a/tracopt/ticket/deleter.py b/tracopt/ticket/deleter.py index bd38c77..9a537bd 100644 --- a/tracopt/ticket/deleter.py +++ b/tracopt/ticket/deleter.py @@ -47,10 +47,11 @@ # ITemplateProvider methods def get_htdocs_dirs(self): - return [] + from pkg_resources import resource_filename + yield 'ticketopt', resource_filename(__name__, 'htdocs') def get_templates_dirs(self): from pkg_resources import resource_filename return [resource_filename(__name__, 'templates')] }}} Adding an `ITemplateProvider` implementation from scratch is not more complicated (if there are no templates provided by the plugin, `get_template_dirs()` can simply `return []`). 2. we need to transfer the logic at the beginning of `filter_stream()` into `post_process_request()`, i.e. the condition at which we decided to let through the content unmodified or to modify it becomes the condition at which we decide to add or not our extra piece of JavaScript. So we had: {{{#!python def filter_stream(self, req, method, filename, stream, data): if filename not in ('ticket.html', 'ticket_preview.html'): return stream ticket = data.get('ticket') if not (ticket and ticket.exists and 'TICKET_ADMIN' in req.perm(ticket.resource)): return stream # modify the stream! ... }}} which becomes now: {{{#!python def post_process_request(self, req, template, data, content_type): if template in ('ticket.html', 'ticket_preview.html'): ticket = data.get('ticket') if (ticket and ticket.exists and 'TICKET_ADMIN' in req.perm(ticket.resource)): add_script(req, 'ticketopt/ticketdeleter.js') add_script_data(req, ui={'use_symbols': req.session.get('ui.use_symbols')}) return template, data, content_type }}} i.e. the condition remains "filename is either ticket.html or ticket_preview.html, and we have a ticket in the data, that ticket exists and we have admin perm on that ticket"; previously if true we would alter the stream, now we call `add_script`. Note that we also call `add_script_data`. Here we do it for some piece of session information which is not yet available in the default JavaScript data, but you'll probably have to do that for any piece of the template `data` you'll need to use in the JavaScript. Don't pass the whole `data` dictionary though, that would be overkill and it's quite likely some bits won't convert readily to JSON. Pass only the information you'll need. 3. now the "juicy" part, do what the Transform filter did in JavaScript. Well, actually the //browser// needs JavaScript, but you can use whatever you want in order to produce that JavaScript code. I personally recommend using CoffeeScript as it's well suited here for producing HTML snippets a. **producing the content** {{{#!python # Insert "Delete" buttons for ticket description and each comment def delete_ticket(): return tag.form( tag.div( tag.input(type='hidden', name='action', value='delete'), tag.input(type='submit', value=captioned_button(req, u'–', # 'EN DASH' _("Delete")), title=_('Delete ticket'), class_="trac-delete"), class_="inlinebuttons"), action='#', method='get') }}} becomes: {{{#!coffeescript captionedButton = (symbol, text) -> if ui.use_symbols then symbol else "#{symbol} #{text}" deleteTicket = () -> $ """ <form action="#" method="get"> <div class="inlinebuttons"> <input type="hidden" name="action" value="delete"> <input type="submit" value="#{captionedButton '–', _("Delete")}" title="#{_("Delete ticket")}" class="trac-delete"> <input type="hidden" name="__FORM_TOKEN" value="#{form_token}"> </div> </form> """ }}} (`captioned_button(req, symbol, text)` is a small Python utility function, which is nevertheless trivial to adapt in JavaScript; note however that it's for this part of the logic that we needed to pass `req.session.get('ui.use_symbols')` from Python to JavaScript's `ui.use_symbols` via the call to `add_script_data` in 2.) \\Also, {{{#!python def delete_comment(): for event in buffer: cnum, cdate = event[1][1].get('id')[12:].split('-', 1) return tag.form( tag.div( tag.input(type='hidden', name='action', value='delete-comment'), tag.input(type='hidden', name='cnum', value=cnum), tag.input(type='hidden', name='cdate', value=cdate), tag.input(type='submit', value=captioned_button(req, u'–', # 'EN DASH' _("Delete")), title=_('Delete comment %(num)s', num=cnum), class_="trac-delete"), class_="inlinebuttons"), action='#', method='get') }}} becomes: {{{#!coffeescript deleteComment = (c) -> # c.id == "trac-change-3-1347886395121000" # 0123456789012 [cnum, cdate] = c.id.substr(12).split('-') $ """ <form action="#" method="get"> <div class="inlinebuttons"> <input type="hidden" name="action" value="delete-comment"> <input type="hidden" name="cnum", value="#{cnum}"> <input type="hidden" name="cdate" value="#{cdate}"> <input type="submit" value="#{captionedButton '–', _("Delete")}" title="#{_("Delete comment %(num)s", num: cnum)}" class="trac-delete"> <input type="hidden" name="__FORM_TOKEN" value="#{form_token}"> </div> </form> """ }}} Not really more complex, quite the opposite. And don't ask me what `event[1][1]` was ;-) b. **inserting the content at the right place** \\Now this bit of Python magic: {{{#!python buffer = StreamBuffer() return stream | Transformer('//div[@class="description"]' '/h3[@id="comment:description"]') \ .after(delete_ticket).end() \ .select('//div[starts-with(@class, "change")]/@id') \ .copy(buffer).end() \ .select('//div[starts-with(@class, "change") and @id]' '//div[@class="trac-ticket-buttons"]') \ .append(delete_comment) }}} becomes a more straightforward sequence of jQuery calls: {{{#!coffeescript $(document).ready () -> # Insert "Delete" buttons for ticket description and each comment $('#ticket .description h3').after(deleteTicket()) $('#changelog div.change').each () -> $('.trac-ticket-buttons', this).prepend deleteComment this }}} For those of you not so familiar with CoffeeScript, here is the corresponding plain JavaScript (assuming we're already in the document ready() callback): {{{#!javascript $('#ticket .description h3').after(deleteTicket()); $('#changelog div.change').each(function() { $('.trac-ticket-buttons', this).prepend(deleteComment(this)); }); }}} See the full changeset, [bf49f871/cboos.git] (and [be0ff94a/cboos.git] for a look to the generated .js). === Generating content ==== Rendering template "fragments" When one wants to directly render a template, the Chrome component facilities can be used, as before: {{{#!py return Chrome(self.env).render_template( req, 'query_results.html', data, None, fragment=True) }}} //implementing [source:cboos.git/trac/ticket/query.py@jinja2#L1403 Ticket Query] macro (''table'' mode)// A newer alternative is to use `Chrome.generate_template_fragment`, as in: {{{#!python if req.is_xhr: # render and return the content only stream = Chrome(self.env).generate_template_fragment( req, 'changeset_content.html', data) req.send(stream) }}} This automatically retrieves the `use_chunked_encoding` TracIni setting and uses it to return an iterable. In any case, the returned value can be sent directly from the `Request` object. ==== The `tag` builder #tag Genshi provided a nice Python API for programmatically building (X)HTML elements. This consisted of the `Fragment`, `Element` and the `tag` builder, all from the `genshi.builder` module: {{{#!python from genshi import Fragment, Element, tag }}} This has now been replaced by an equivalent API which lives in `trac.util.html`, so the above import should be replaced with: {{{#!python from trac.util.html import Fragment, Element, tag }}} Note that the `html` symbol from `trac.util.html` which used to be a alias to `genshi.builder.tag` is now naturally an alias to `trac.util.html.tag`. One way to write "portable" code would be: {{{#!python from trac.util.html import html as tag }}} You can then use the `tag` builder API regardless of its origin. Likewise, if you want to use the `Markup` class, you should write: {{{#!python from trac.util.html import Markup }}} In "old" versions of Trac, you'll get `genshi.core.Markup`, whereas now you'll get `markupsafe.Markup`: as we're using Jinja2, we're also making use of its direct dependency [PyPI:MarkupSafe]. {{{#!python from trac.util.html import escape }}} Note that in a similar way to `Markup`, `escape` now also comes from `markupsafe`. It has a slightly different API from Genshi in that it will //always// escape the quotes. Though a bit unfortunate (for our unit-tests compatibility...), it's not a big deal and we adopt the new behavior, keeping the old API for backward compatibility but ignoring the `quotes` flag. [[comment(//Sending notification e-mails//)]]
Note:
See
WikiFormatting
and
TracWiki
for help on editing wiki content.
Change information
Your email or username:
E-mail address and name can be saved in the
Preferences
Comment about this change (optional):
Note:
See
TracWiki
for help on using the wiki.